City interest point representation learning semantic enhancement system based on large language model

Abstract

The invention discloses a city interest point representation learning semantic enhancement system based on a large language model, in the system, three types of different cue words are designed for the task of interest point representation learning, each type of cue word is composed of three different parts, each part provides information in different aspects for the large language model, and the large language model can be used for providing information in different aspects for the city interest point representation learning semantic enhancement system. The quality of a result generated by the large language model is improved; besides, the last hidden layer output of the large language model is used as a feature vector of the output, a cross attention mechanism is used to align information extracted from different types of cue words, and then a multi-layer perceptron and a weighted sum are used to fuse the aligned feature vectors, so that the accuracy of the cue word fusion is improved. And finally, enhancing the representation of the interest point based on a cross attention mechanism by using the fusion result. By performing comparative learning from three angles, a representation vector with higher representation quality than that of the original point of interest can be finally obtained, so that the purpose of enhancing the representation of the point of interest is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and intelligent transportation systems, and more specifically to a city interest point representation learning semantic enhancement system based on a large language model. Background Art

[0002] With the widespread use of smart electronic devices, the amount of location-based mobile data such as user interest point check-in data and user trajectory data has increased dramatically. These data have high commercial value, and downstream tasks such as user destination prediction and access traffic prediction based on these data have also been widely used. As one of the basic supports for these tasks, interest point representation largely determines the performance of these downstream tasks. How to obtain better interest point representation is one of the current research focuses in this field.

[0003] Large language models have been one of the research focuses in the field of artificial intelligence in recent years. Large language models trained based on massive Internet text data not only show extremely amazing capabilities in natural language processing, but also have great potential in the field of space and time. However, so far, no work has applied large language models to the field of interest point representation. Learning interest point representation with the help of large language models has the potential to further improve the performance of various downstream tasks, but at present, this technical route still has the following problems:

[0004] First, it is difficult to effectively extract geographical knowledge that is highly relevant to POIs from large language models. Although many studies have explored how to design prompt words to extract geographical knowledge from large language models, the current large language models are unable to grasp the geographical information of POIs that contain precise numbers such as GPS coordinates, due to the weakness of large language models in that they have poor numerical recognition and cannot accurately process numbers.

[0005] Second, it is difficult to integrate the text information output by the large language model into the system for learning the representation of interest points. Many existing studies use the technical route of one-hot encoding the category features of interest points and directly concatenating the encoding with the representation vector of the interest point to generate the final result. However, the use of one-hot encoding will reduce the quality of the semantic information contained in the vector, and the concatenation operation will result in a rigid connection between different parts of the vector, bringing more difficulties to downstream tasks. Summary of the invention

[0006] In view of this, the present invention aims at the needs and shortcomings of current technological development and provides a semantic enhancement system for learning urban POI representation based on a large language model. The system can make full use of the text generated by the large language model based on geographical knowledge to enhance the POI representation, thereby improving the performance of downstream tasks related to POI.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] The embodiment of the present invention provides a city interest point representation learning semantic enhancement system based on a large language model, which uses the text generated by the large language model based on geographical knowledge to enhance the interest point representation. The system includes: a prompt word generation and feature extraction module, an interest point representation enhancement module and a multi-view comparison learning module, wherein:

[0009] The prompt word generation and feature extraction module includes three types of prompt words: POI access mode, POI address and POI surrounding. Each type of prompt word consists of three parts: role introduction, POI information and question, providing different aspects of information for the large language model. In the feature extraction process, the three types of prompt words are sequentially passed into the large language model, and the text finally outputted is ignored, and the feature vector is extracted from the last hidden layer of the large language model.

[0010] The interest point representation enhancement module uses a cross attention mechanism to align feature vectors extracted from different types of prompt words, then uses a multi-layer perceptron and a weighted sum to fuse the aligned feature vectors, and finally uses the fusion result to enhance the representation of the interest point based on the cross attention mechanism;

[0011] The multi-view contrastive learning module adopts different views to select positive samples and randomly samples to generate negative samples; and uses the loss function to perform contrastive learning, thereby reducing the distance between representation vectors of the same type in the latent space and increasing the distance between representation vectors of different types.

[0012] Furthermore, in the role import part, the prompt word instructs the large language model as a role familiar with geographic information; in the point of interest information part, according to the different types of prompt words, the corresponding point of interest information is passed into the large language model in the form of natural language; in the question part, different questions are designed for different types of prompt words.

[0013] Furthermore, the multi-view comparative learning module adopts three different views to select positive samples, and the three different views include: check-in record sequence, geographical perspective and functional perspective.

[0014] Furthermore, a check-in record is represented by a triple r = (u, p, t), which means that user u visited point of interest p at time t. The user's behavior over a period of time is represented by a sequence of check-in records R = {r1, r2, ..., r L}, where each sign-in record r1,…,r L Arranged in chronological order, L represents the length of this sign-in record sequence.

[0015] Furthermore, the loss function used by the multi-view contrast learning module includes: an InfoNCE loss function and a loss function based on cosine similarity, and the expression is:

[0016]

[0017] In the formula, represents the InfoNCE loss function, represents the loss function based on cosine similarity; p i , They represent the original sample, positive sample and negative sample in contrastive learning respectively, sim is the similarity function, γ is the temperature parameter, m is the total number of interest points in the training batch; cos is the cosine similarity function; E FUSE Represents the enhanced interest point representation, E POI Represents the original interest point representation vector before enhancement.

[0018] It can be seen from the above technical solutions that, compared with the prior art, the present invention has at least the following technical advantages:

[0019] The present invention provides a semantic enhancement system for learning and representing urban interest points based on a large language model. In view of the difficulty in extracting geographic information from a large language model, the present invention specially designs three different types of prompt words for the task of learning interest point representation, and each type of prompt word is composed of three different parts, each part providing different aspects of information for the large language model to improve the quality of the result generated by the large language model; in view of the difficulty in integrating the output of the large language model with the interest point representation vector, the present invention uses the last hidden layer output of the large language model as its output feature vector, and uses a cross attention mechanism to align the information extracted by different types of prompt words, and then uses a multi-layer perceptron and a weighted sum to fuse the aligned feature vectors, and finally uses this fusion result to enhance the representation of interest points based on the cross attention mechanism. By carrying out comparative learning from three perspectives, the system of the present invention can finally obtain a representation vector with higher quality than the original interest point representation, thereby achieving the purpose of enhancing the interest point representation; the use of the system framework of the present invention can significantly improve the performance of the interest point representation method on various downstream tasks.

[0020] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0021] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0024] Figure 1 A schematic diagram of the overall architecture of a city POI representation learning and semantic enhancement system based on a large language model provided in an embodiment of the present invention.

[0025] Figure 2 Schematic diagram of ablation experiment results under different downstream tasks provided by an embodiment of the present invention.

[0026] Figure 3 A schematic diagram of the impact of different numbers of model layers on overall performance provided in an embodiment of the present invention.

[0027] Figure 4 A performance diagram of POI clustering tasks on different data sets provided by an embodiment of the present invention.

[0028] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0030] In the description of the present invention, it should be noted that: in some processes described in the specification and drawings of this application, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be performed in the order in which they appear in this document or may be performed in parallel. In addition, various serial numbers are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0031] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0032] See also Figure 1 As shown, the present invention provides a POI-Enhancer framework (Point of Interest, POI), that is, a city point of interest representation learning semantic enhancement system based on a large language model. This system framework can make full use of the text generated by the large language model based on geographical knowledge to enhance the point of interest representation, thereby improving the performance of many downstream tasks related to points of interest.

[0033] The problem studied by the present invention can be expressed by the following definitions:

[0034] Definition 1 (POI): A POI is a geographical location represented by a five-tuple (id, pn, c, lon, lat), where id represents the node number, pn represents the name of the POI, c represents the category to which the POI belongs, and lon and lat represent the longitude and latitude of the point, respectively. In addition to this five-tuple, each POI can also carry certain additional attributes, such as access mode, address of the POI, and environment around the POI. The typical composition of POIs in a city dataset is shown in Table 1.

[0035] Table 1 Information contained in a typical point of interest in the New York dataset

[0036] Fields value Point of interest number 22337 name NewYorkStockExchange longitude 74.011154 latitude 40.706806 Point of interest categories StockExchange Street Name WallStreet House Number 11 post code 10005 Points of Interest Nearby Office, Building and Road Access Mode Between 6am and 9am, Weekday

[0037] Definition 2 (check-in record): A check-in record is represented by a triple r = (u, p, t), which means that user u visited the point of interest p at time t. The behavior of a user in a period of time can be represented by a sequence of check-in records R = {r1, r2, …, r L}, where each sign-in record r1,…,r L Arranged in chronological order, L represents the length of this sign-in record sequence. The present invention additionally defines the set of sign-in record sequences of all users as S.

[0038] Definition 3 (Point of Interest Representation): For the set of all points of interest P = {p1, p2, ..., p N}, the present invention maps each point of interest to p through a mapping function f i Mapped to a fixed vector This vector That is the point of interest p i Representation of, where N is the total number of interest points.

[0039] Problem statement: For any interest point representation mapping function The set of points of interest P = {p1, p2, ..., p N}, the check-in sequence S and other related data, the system of the present invention learns a mapping function that can enhance the representation of interest points The function g makes the enhanced representation vector Able to achieve better performance on downstream tasks.

[0040] Combine the following Figure 1-Figure 4 As shown, the architecture, working principle and specific implementation methods of the system of the present invention are described in detail:

[0041] 1. System framework:

[0042] In a specific embodiment, the framework of the system proposed in the present invention can be subdivided into three modules: prompt word generation and feature extraction, interest point representation enhancement and multi-view contrast learning, corresponding to Figure 1 There are three parts (a), (b) and (c) in the above. Among them:

[0043] In the prompt word generation and feature extraction part, such as Figure 1 As shown in part (a), the present invention designs three types of prompt words, namely, point of interest access mode, point of interest address, and point of interest surrounding, for the task of enhancing the representation of points of interest using the geographical knowledge in the large language model, and each type of prompt word consists of three parts: role introduction, point of interest information, and question. In the role introduction part, the prompt word instructs the large language model to play the role of a resident who is familiar with local geographical information; in the point of interest information part, according to the different types of prompt words, the present invention passes the corresponding point of interest information into the large language model in the form of natural language; and in the question part, the present invention designs different questions for different types of prompt words, fully mobilizes the geographical knowledge within the large language, and improves the quality of the large language model output. In the feature extraction process, the present invention passes the three types of prompt words into the large language model in turn, ignores the text finally output, extracts the vector from the last hidden layer of the large language model and performs further processing. The three types of prompt words, namely, point of interest access mode, point of interest address, and point of interest surrounding, are respectively Then the present invention can obtain three corresponding eigenvectors:

[0044]

[0045] in, It is a large language model that ignores all structures after the last hidden layer.

[0046] In the embodiment of the present invention, the interest point representation enhancement part includes three step modules: dual feature alignment, semantic feature fusion and cross attention fusion. The dual feature alignment step involves aligning feature vectors with a certain relationship to further improve the quality of the feature vectors. In this step, the three feature vectors obtained above will be converted into a higher dimensional latent vector

[0047]

[0048] in, d is the dimension of the latent space, W V′ ,W A ,W S All three matrices can be learned. After the transformation is completed, This pair of vectors, the present invention uses the cross attention mechanism to align and merge them into one vector. First, the present invention defines

[0049]

[0050] in, is a learnable parameter, φ is the Softmax function, d h represents the dimension size of each head in the multi-head attention mechanism, || is the concatenation operation of the vector, It is also a learnable weight parameter, and H represents the number of heads in the multi-head attention mechanism. Suppose the output of the first layer of this multi-layer network is Z'1, then

[0051]

[0052] Z′1=LN(Z+FFN(Z))

[0053] Among them, FFN is a feedforward neural network based on FFN. After finding the output Z'1 of the first layer network, this vector will be combined with E A The output of the last network layer is recorded as For E A-V ; E A-S The calculation process is similar to E A-V , I will not go into details here.

[0054] In the semantic feature fusion part, the present invention combines the two feature vectors E obtained previously A-V ,E A-S Combined together by weighted sum. First, the present invention calculates

[0055] θ A-V =W2·LeakyReLU([W1E A-V ||W1EA-S ])

[0056] θ A-S =W2·LeakyReLU([w1E A-S ||W1E A-V ])

[0057] Among them, θ A-V ,θ A-S For E A-V ,E A-S The attention score, The role of is to project the feature vector into the latent space or convert it into an attention score.

[0058] After calculating the score, the present invention can calculate the weight in the weighted sum

[0059] [ω A-V ,ω A-S ]=φ([θ A-V ,θ A-S ])

[0060] Among them, ω A-V ,ω A-S Respectively represent E A-V ,E A-S The corresponding weight of the weighted combination.

[0061] And calculate the final output

[0062] E LLM =ω A-V ·E A-V +ω A-S ·E A-S

[0063] in,

[0064] In the last step of cross attention fusion, the present invention uses the cross attention mechanism to obtain E LLM To represent the output E of the interest point representation model POI Enhance. Let the output of the first layer of this multi-layer model be X'1, then

[0065] X=LN(E POI +MQA(E LLM ,E POI ))

[0066] X′1=LN(X+FFN(X))

[0067] This process is repeated until the L2 layer network completes the calculation, and the output of the last layer is recorded in the present invention. For E FUSE .

[0068] In a specific embodiment, in the multi-view comparative learning part, the present invention preferably uses three different views to select positive samples. Specifically: from the user's check-in record sequence, the various points of interest visited by the user on the same day are likely to have similarities, so the present invention stipulates that a certain point of interest visited by the user on the same day and the two points of interest visited before and after the user are positive samples of each other. From a geographical point of view, points of interest with similar geographical locations should have similarities, so for any point of interest, the present invention constructs a square area of ​​fixed size on the map with it as the center, and stipulates that all points of interest in the area are positive samples of the point of interest. From a functional point of view, points of interest belonging to the same point of interest category and with similar access patterns should be semantically similar, so the present invention sets the points of interest that meet these two conditions as positive samples of each other; and negative samples are generated by random sampling.

[0069] In a specific embodiment, the loss function of the contrastive learning process of the present invention is composed of two parts. The first part is the InfoNCE loss function, which is expressed as:

[0070]

[0071] Among them, p i , Respectively represent the original sample, positive sample and negative sample in contrastive learning, sim is the similarity function, γ is the temperature parameter, and m is the total number of interest points in the training batch; and in order to ensure that the similarity between the interest point representations before and after enhancement is not lost, the present invention adds another loss function based on cosine similarity, which is expressed as follows:

[0072]

[0073] Among them, cos is the cosine similarity function;

[0074] The loss function used in the training process of the present invention is the sum of two parts, namely

[0075] 2. Experimental verification:

[0076] In the system evaluation phase, the present invention uses the user check-in sequence data sets Foursquare-NY, Foursquare-SG, and FourSquare-TKY, which are collected in New York, Singapore, and Tokyo respectively. In order to reduce the noise in the data, the present invention removes the points of interest with less than 5 check-ins and the user check-in sequences with a length of less than 10. The statistical information of the processed data set is shown in Table 2. In all experiments, the present invention shuffles the data in the data set and divides the training set, verification set, and test set in a ratio of 7:1:2.

[0077] Table 2 Statistics of the preprocessed dataset

[0078] Dataset Number of users Points of Interest Number of sign-in records FourSquare-NY 15,171 24,118 641,005 FourSquare-SG 10,909 20,154 696,306 FourSquare-TKY 2,293 15,164 496,459

[0079] In order to fully demonstrate the capabilities of the system proposed in the present invention, six different POI representation models Skip-Gram, POI2Vec, Geo-Teaser, TALE, Hier and CTLE were selected for experiments, and evaluated on three different downstream tasks: POI recommendation, check-in sequence classification, and POI access traffic prediction. For the POI recommendation task, the present invention uses Hit@k, that is, the frequency of the first k POIs recommended by the model as the evaluation index; for the check-in sequence classification task, the present invention uses accuracy (ACC) and Macro-F1 as evaluation indicators; for the POI access traffic prediction, the present invention uses mean absolute error (MSE) and root mean square error (RMSE) as evaluation indicators. In this experiment, the present invention uses Llama-2-7B as the backbone large model, sets the latent space size d to 256, L1 and L2 to 4 and 2 respectively, the temperature parameter γ to 0.1, trains for 100 epochs, and uses the optimizer AdamW, with a learning rate of 0.001 and a weight decay coefficient of 0.001.

[0080] The results of this experiment are shown in Table 3. In the task of recommending points of interest, the model of the present invention improves the performance of POI2VEC on the New York dataset by more than 20%, which is most likely because the POI2Vec model focuses on capturing the relationship between each sign-in information in the sequence and ignores information from other modalities. . In the task of classifying the sign-in data sequence, the POI2Vec model also achieved a very high improvement, while the Skip-Gram algorithm improved less, which may be because the Skip-Gram model pays more attention to modeling user trajectories, and its optimization space is less than that of other algorithms. In the prediction of point of interest visit traffic, CTLE, as a SOTA model, has achieved a significant improvement in performance after being strengthened, which further illustrates the effectiveness of the model of the present invention in strengthening POI representation. At the same time, the present invention has been improved in almost all datasets, models and downstream tasks, which also illustrates the feasibility and effectiveness of this method of extracting additional geographic knowledge from a large language model and integrating it into the representation of points of interest.

[0081] Table 3 Improvements of POI-Enhancer framework on various models, datasets and downstream tasks

[0082]

[0083]

[0084] Continued

[0085]

[0086] Continued

[0087]

[0088] Furthermore, in addition to the main experiment, the present invention also conducted additional ablation experiments, parameter analysis, and characterization quality analysis. The experimental results of the ablation experiment are shown in Figure 2As shown, / P represents that this system removes the specially designed role introduction and question part in the prompt word, / D represents that this system removes the dual feature alignment and semantic feature fusion module, / F represents that this system uses direct splicing instead of the cross attention fusion module, and / C represents that this system only selects positive samples from a geographical perspective when performing comparative learning. The experimental results show that when the system model is deleted according to the above steps, the quality of the interest point representation generated by it will decline; this shows that the special prompt word design of the system of the present invention can better extract information from the large language model, dual feature alignment and semantic fusion can ensure the quality of the obtained feature vector, the cross attention fusion module can better combine the information in the large language model into the existing interest point representation, and the comparative learning under multi-view can bring more useful information to the model than the single view.

[0089] The experimental results of parameter analysis are as follows Figure 3 As shown, this experiment was completed using the Hier model on the New York dataset. It can be found that, with other conditions unchanged, as L1 or L2 increases, the performance of the model first increases and then decreases, and the experimental performance is best when L1=4 and L2=2. This is because when the number of layers of the model of the present invention is too low, it cannot correctly complete the task of integrating features with features, and when the number of layers of the model is too high, problems such as overfitting will cause the results of the model on the test set to drop. Therefore, in the main experiment, the present invention sets L1 and L2 to 4 and 2 respectively.

[0090] The experimental results of mass analysis are as follows Figure 4 As shown, in order to further measure the quality of the interest point representation enhanced by this framework, the present invention uses the K-means algorithm to test the clustering task on three data sets. In this process, the present invention uses the K-means algorithm to cluster the interest point representation vectors before and after enhancement, and uses the normalized mutual information (NMI) indicator to evaluate the clustering results. From the experimental results, it can be found that compared with the results before enhancement, the quality of the interest point representation after enhancement is significantly improved. This illustrates the following three points: (1) The present invention successfully extracts high-quality features from the large language model, and this feature plays a key role in improving the quality of interest point representation; (2) The present invention efficiently integrates the extracted features into the unenhanced interest point representation, so that its properties in all aspects are improved; (3) The multi-view comparative learning method used in the present invention can shorten the distance between representation vectors of the same type in the latent space, while increasing the distance between representation vectors of different types.

[0091] From the description of the above embodiments, those skilled in the art can know that: the present invention innovatively introduces a system framework called POI-Enhancer, which can enhance the representation of points of interest using a large language model. In order to achieve this goal, on the one hand, the present invention overcomes the challenge that large language models are insensitive to numbers and it is difficult to extract information from large language models by designing three types of special prompt words; on the other hand, in order to better align and fuse the obtained information, the present invention designs three steps of dual feature alignment, feature fusion, and cross-attention fusion to replace the relatively crude splicing method. In addition, the present invention also designs a training method of multi-view comparative learning for the training of this system framework. Experiments have shown that the system framework of the present invention can significantly improve the performance of almost all point of interest representation methods on various downstream tasks.

[0092] Further, refer to Figure 5 As shown, an embodiment of the present invention further provides an electronic device, which may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, the processor executes the computer program to implement the above-mentioned city interest point representation learning semantic enhancement system based on a large language model, making full use of the text generated by the large language model based on geographic knowledge to enhance the interest point representation, thereby improving the performance of numerous downstream tasks related to the interest points.

[0093] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing programs or modules stored in the memory 11, and calling data stored in the memory 11.

[0094] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as computer program products or electronic devices. Therefore, the present invention may take the form of a complete software embodiment, or a combination of software and hardware embodiments. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0095] It should be noted that the word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several distinct components, and by means of a suitably programmed computer.

[0096] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0097] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A city interest point representation learning semantic enhancement system based on a large language model, characterized by: The text generated by the large language model based on geographical knowledge is used to enhance the representation of points of interest. The system includes: a prompt word generation and feature extraction module, a point of interest representation enhancement module and a multi-view comparative learning module, wherein: The prompt word generation and feature extraction module includes three types of prompt words: POI access mode, POI address and POI surrounding. Each type of prompt word consists of three parts: role introduction, POI information and question, providing different aspects of information for the large language model. In the feature extraction process, the three types of prompt words are sequentially passed into the large language model, and the text finally outputted is ignored, and the feature vector is extracted from the last hidden layer of the large language model. The interest point representation enhancement module uses a cross attention mechanism to align feature vectors extracted from different types of prompt words, then uses a multi-layer perceptron and a weighted sum to fuse the aligned feature vectors, and finally uses the fusion result to enhance the representation of the interest point based on the cross attention mechanism; The multi-view contrastive learning module adopts different views to select positive samples and randomly samples to generate negative samples; and uses the loss function to perform contrastive learning, thereby reducing the distance between representation vectors of the same type in the latent space and increasing the distance between representation vectors of different types.

2. According to the large language model-based city interest point representation learning semantic enhancement system of claim 1, it is characterized in that: In the role import part, the prompt word instructs the large language model as a role familiar with geographic information; in the POI information part, according to the different types of prompt words, the corresponding POI information is passed into the large language model in the form of natural language; in the question part, different questions are designed for different types of prompt words.

3. According to the large language model-based city interest point representation learning semantic enhancement system of claim 1, it is characterized in that: The multi-view comparative learning module adopts three different views to select positive samples, and the three different views include: check-in record sequence, geographical perspective and functional perspective.

4. According to claim 3, a city interest point representation learning semantic enhancement system based on a large language model is characterized in that: A check-in record is represented by a triple r = (u, p, t), which means that user u visited the point of interest p at time t. The user's behavior over a period of time is represented by a sequence of check-in records R = {r1, r2, ..., r L }, where each sign-in record r1,…,r L Arranged in chronological order, L represents the length of this sign-in record sequence.

5. According to the large language model-based city interest point representation learning semantic enhancement system of claim 1, it is characterized in that: The loss function used by the multi-view contrast learning module includes: InfoNCE loss function and cosine similarity-based loss function, expressed as: In the formula, represents the InfoNCE loss function, represents the loss function based on cosine similarity; p i , They represent the original sample, positive sample and negative sample in contrastive learning respectively, sim is the similarity function, γ is the temperature parameter, m is the total number of interest points in the training batch; cos is the cosine similarity function; E FUSE Represents the enhanced interest point representation, E POI Represents the original interest point representation vector before enhancement.

6. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement a city interest point representation learning semantic enhancement system based on a large language model as described in any one of claims 1 to 5.

Images