A personalized scenic spot recommendation method based on large model picture content feature enhancement

Abstract

The application discloses a kind of personalized scenic spot recommendation methods based on large model picture content feature enhancement, comprising:1, data set acquisition and processing: obtain the tourism data set with geographic tagged picture, and the geographic information of picture is decoded, obtain user-scenic spot interaction matrix;2, picture content feature extraction: data enhancement is carried out using large model, and picture is converted into interpretable text semantic information;3, establish prediction preference model: using the recommendation model based on Wide &Deep variant, capture the direct relationship between features;4, obtain the preference value of all scenic spots from the prediction preference model, sort the preference value in descending order, and select the scenic spot corresponding to the top preference value and recommend it to the user.The application integrates geographic information, time information and picture content attribute information extracted by large model, and designs the recommendation model based on Wide &Deep variant, thereby effectively improving the accuracy and personalization level of scenic spot recommendation.

Description

Technical Field

[0001] The present invention belongs to the field of image processing based on computer vision technology, and mainly involves a personalized scenic spot recommendation method based on reverse geocoding, large model data enhancement, graph neural network and feature intersection. Background Art

[0002] In recent years, with the rapid development of the internet, many social media platforms, such as Flickr, have launched image sharing services. Users can upload location-linked images, resulting in a large number of images with text tags, timestamps, and geographic references. These images not only record users' daily lives but also provide a rich data resource for researchers. Unlike other online media, geotagged images combine geographic, temporal, and visual information, becoming the photographer's digital footprint. By analyzing the time and geographic information of image uploads, the user's spatiotemporal movement trajectory can be derived. This effectively alleviates the data sparsity problem common in traditional geotagging-based POI (Point of Interest) recommendations and provides important contextual information for predicting user visits to points of interest.

[0003] Existing POI recommendation methods based on geotagged images typically only use the image's visual features and geotag information as auxiliary information for the recommendation task, without fully leveraging the image's deeper entity and atmospheric emotional information. They also fail to fully consider customers' preferences for tourist attraction content features implicit in social media data. This reduces the diversity and accuracy of recommendations, which in turn impacts user experience. For example, for a single user, those who post many rock climbing and diving photos are more likely to visit adventurous and exciting attractions. Different users may visit different attractions, but comparing their photos may reveal extremely similar content. This suggests that even in different locations, users may be attracted to similar experiences.

[0004] At the same time, existing image visual feature extraction methods are mostly based on deep neural networks (DNNs). DNNs are often viewed as "black box" models, making it difficult to intuitively understand their internal workings and decision-making processes. Features are not interpretable, and correlations between different features cannot be captured. This is a significant limitation for POI recommendation applications that require transparency and credibility. Furthermore, many geotagged images on social platforms currently contain a large amount of redundant information, such as low-quality images and repetitive content. Directly extracting representations of these images places a significant strain on the model and makes it difficult to identify specific user preferences. This redundant information can lead to model overfitting, affecting the accuracy and personalization of POI recommendations. Summary of the Invention

[0005] In response to the many problems existing in the above-mentioned prior art, the present invention proposes a personalized scenic spot recommendation method based on large model image content feature enhancement, in order to effectively improve the accuracy and personalization level of scenic spot recommendations, thereby improving the recommendation accuracy and interpretability.

[0006] The present invention adopts the following technical solutions to solve the technical problems:

[0007] The personalized scenic spot recommendation method based on large model image content feature enhancement of the present invention is characterized in that it is carried out according to the following steps:

[0008] Step 1: Get the i-th user A collection of geotagged images taken ,in, represents the i-th user The jth picture taken, represents the i-th user The total number of pictures taken, , Represents a user collection;

[0009] Step 2: The geographic coordinates of the jth picture are inversely encoded to obtain the scenic spot , thus obtaining the i-th user A collection of attractions , , L represents a collection of scenic spots;

[0010] Step 3: Use the large model to and the kth attraction Data augmentation is performed on the images to convert the image information into interpretable multi-attribute textual semantic information;

[0011] Step 4: Build a wide-end preference prediction network, perform feature cross-talk on the attribute-level information extracted from the image, capture the relationship between different features, and obtain the i-th user For the kth attraction Preference ;

[0012] Step 5: Build a Deep preference prediction network and as well as and Process and get the i-th user For the kth attraction Preference ;

[0013] Step 6: Use formula (13) to establish the i-th user of the Wide & Deep end joint prediction For the kth attraction Preference :

[0014] (13)

[0015] In formula (13), represents the weight that controls the relative importance between the two model outputs, is the deviation term, Convert the output to Functions in the interval ;

[0016] Step 7: Use formula (14) to construct the logistic regression loss function L, and train the wide preference prediction network and the deep preference prediction network until the loss function converges, thereby obtaining the optimal preference prediction model;

[0017] (14)

[0018] In formula (14), represents the i-th user Whether you have visited the kth attraction The true label of

[0019] Step 8: Output of the optimal preference prediction model for the i-th user Sort the preference values ​​in the optimal preference set of all attractions in descending order, and recommend the attractions corresponding to the top preference values ​​to the i-th user .

[0020] The personalized scenic spot recommendation method based on large model image content feature enhancement according to the present invention is also characterized in that step 2 is performed as follows:

[0021] Step 2.1: Calculate using formula (1) and attractions The geographical coordinate distance between :

[0022] (1)

[0023] In formula (1), represents the radius of the Earth, and They are pictures and The latitude of the attraction, and are the longitudes of the pictures and attractions respectively;

[0024] Step 2.2, set the distance threshold Δ, if , then Add attractions Otherwise, assign j+1 to j and return to repeat step 2.1 to get the i-th user A collection of attractions , , Represents the collection of attractions for all users.

[0025] Furthermore, step 3 is performed as follows:

[0026] Step 3.1, using the multimodal visual model GPT4V Perform data enhancement to get the i-th user Entity text collection and atmosphere emotion text collection ,in, represents the i-th user The jth picture taken The entity description text extracted from represents the i-th user The jth picture taken The atmosphere and emotional description text extracted from

[0027] Step 3.2: Use the multimodal visual model GPT4V to identify the kth scenic spot Picture collection Perform data enhancement to obtain the kth scenic spot Entity text collection and atmosphere emotion text collection ,in, represents the kth scenic spot The qth picture The entity description text extracted from represents the kth scenic spot The qth picture The atmosphere and emotional description text extracted from represents the kth scenic spot Total number of pictures;

[0028] Step 3.3: If the i-th user Photographed the kth scenic spot , then it means the i-th user Visited the kth attraction , then let the i-th user Interaction vector with attractions The element in row i and column k of Otherwise, let , thus obtaining the interaction matrix , for The number of users in for The number of scenic spots in the ; thus constructing the adjacency matrix , where T represents transpose;

[0029] Step 3.4: Set the i-th user and the kth attraction The ID of the i-th user is converted into a continuous feature vector to obtain the ID feature embedding vector and the ID feature embedding vector of the k-th scenic spot ; For the kth scenic spot The longitude and latitude information is normalized and embedded to obtain the geographic feature embedding vector of the k-th scenic spot .

[0030] Furthermore, step 4 is performed as follows:

[0031] Step 4.1: Use equations (2) and (3) to get the i-th user Entity feature embedding vector and the i-th user The atmosphere emotional feature embedding vector :

[0032] (2)

[0033] (3)

[0034] In formula (2) and formula (3), represents the attention mechanism network;

[0035] Step 4.2: Use equations (4) and (5) to get the k-th scenic spot Entity feature embedding vector and the kth attraction The atmosphere emotional feature embedding vector :

[0036] (4)

[0037] (5)

[0038] Step 4.3: Use formula (6) to establish the prediction preference function of the wide end :

[0039] (6)

[0040] In formula (6), represents the global bias term, Represents the i-th user The two first-order feature weights of Represents the kth scenic spot The two first-order feature weights of represents the i-th user The second-order cross-feature weights of Indicates the kth scenic spot The second-order cross-feature weights of Represents the i-th user The two second-order feature weights of Represents the kth scenic spot The two second-order feature weights of .

[0041] Furthermore, step 5 is performed as follows:

[0042] Step 5.1. Define the current layer of the graph neural network as the nth layer, the total number of layers is N, and initialize n=1. Define and initialize the i-th user Embedding vector at layer n-1 , the kth attraction The embedding vector at layer n-1 is ;

[0043] Step 5.2: Use equations (8) and (9) to get the i-th user In the Embedding vector of the layer and the kth attraction In the Embedding vector of the layer :

[0044] (8)

[0045] (9)

[0046] In formula (8) and formula (9), and Represents the i-th user The number of visited attractions and the kth attraction The number of users accessing the

[0047] Step 5.3: Use equations (10) and (11) to get the i-th user The final embedding vector and the kth attraction The final embedding vector :

[0048] (10)

[0049] (11)

[0050] In formula (10) and formula (11), Indicates the The weight coefficient of the layer;

[0051] Step 5.4: Use formula (12) to establish the i-th user predicted by the Deep side For the kth attraction Preference function :

[0052] (12).

[0053] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the personalized scenic spot recommendation method, and the processor is configured to execute the program stored in the memory.

[0054] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program executes the steps of the personalized scenic spot recommendation method when the computer program is executed by a processor.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] 1. This invention combines geographic tag information, shooting time information, and visual attribute information extracted from images. It uses a large model to convert images into interpretable textual semantic information. It then establishes a predictive preference model based on a Wide & Deep architecture variant to better capture the characteristics of tourist destinations and users' personal interests, thereby providing more accurate and personalized tourist attraction recommendation services.

[0057] 2. The present invention uses inverse coding technology to map the user's geo-tagged image information into the interaction information between the user and the scenic spot, fully mining the user's historical behavior data to establish a more accurate user portrait, effectively alleviating the data sparsity problem existing in traditional tourist attraction recommendations, and improving the accuracy of the recommendation system and user experience. It not only enhances the connection between users and service providers, but also promotes the digital transformation of related industries and the improvement of service quality.

[0058] 3. This invention uses a large model to extract deep entity information and atmospheric and emotional information from images, converting image information into more interpretable text information. It deeply explores the entity information and atmospheric and emotional information in images, effectively solving the "black box problem" existing in traditional image visual feature extraction methods, thereby more accurately understanding user needs and providing personalized scenic spot recommendation service suggestions, improving user experience.

[0059] 4. This paper uses a Wide&Deep architecture variant to establish a predictive preference model. The Wide side uses a feature intersection algorithm to capture the association between different attribute features, and the Deep side uses graph neural network technology to extract the representation of users and attractions. It effectively captures the interaction information between users and attractions while providing explainable POI recommendations, thus providing a theoretical basis for tourist attraction recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is the overall flow chart of the present invention;

[0061] Figure 2 This is the overall flow chart of the prediction preference model of the present invention. DETAILED DESCRIPTION

[0062] In this embodiment, a personalized scenic spot recommendation method based on large model image content feature enhancement includes: 1. obtaining a tourism dataset with geo-tagged images, inversely encoding the user's image geographic information, and obtaining a user-attraction interaction matrix; 2. based on the data enhancement of the large model, converting the image into interpretable textual semantic information, and obtaining different attribute feature matrices of users and attractions respectively; 3. establishing a prediction preference model based on Wide & Deep variants to capture the direct relationship between features, and using the logistic regression loss function to update the prediction preference model, thereby obtaining the user's preference value for all attractions in the final prediction preference model, and after sorting the preference values ​​in descending order, selecting the attractions corresponding to the top preference values ​​and recommending them to the user. Specifically, if Figure 1 As shown, the method is performed in the following steps:

[0063] Step 1: Get the i-th user A collection of geotagged images taken ,in, represents the i-th user The jth picture taken, represents the i-th user The total number of pictures taken, , Represents a user collection;

[0064] Step 1.1, initialize the URL list;

[0065] Step 1.2, call API to obtain a large amount of product information stored in XML format;

[0066] Step 1.3, parse the obtained XML file to get the seed list and store the parsed results;

[0067] Step 1.4, after obtaining the seed list of product names, the obtained list is then filtered and de-duplicated;

[0068] Step 1.5, if the URL list needs to be expanded, continue to execute step 1.2, otherwise obtain the user set , image set and image geographic coordinate set .

[0069] Step 2, inverse encoding of geographic coordinates to get the jth picture of the scenic spot , L represents the scenic spot set;

[0070] Step 2.1, calculate the geographic coordinate distance between and scenic spot using formula (1):

[0071] (1)

[0072] In formula (1), represents the radius of the earth, and are the latitudes of picture and scenic spot, and are the longitudes of picture and scenic spot;

[0073] Step 2.2, set the distance threshold Δ, if , then is added to the set where the scenic spot is located, otherwise j+1 is assigned to j, and then the step 2.1 is repeated to get the scenic spot set of the ith user , , represent the scenic spot sets of all users.

[0074] Step 3, use large model for data augmentation to convert picture information into interpretable multi-attribute text semantic information;

[0075] Step 3.1, use multi-modal visual model GPT4V to ​Perform data enhancement to obtain the i-th user Entity text collection and atmosphere emotion text collection ,in, represents the i-th user The jth picture taken The entity description text extracted from represents the i-th user The jth picture taken The atmosphere and emotional description text extracted from

[0076] Step 3.2: Use the multimodal visual model GPT4V to identify the kth scenic spot Picture collection Perform data enhancement to obtain the kth scenic spot Entity text collection and atmosphere emotional text collection ,in, Indicates the kth scenic spot The qth picture The entity description text extracted from Indicates the kth scenic spot The qth picture The atmosphere and emotional description text extracted from Indicates the kth scenic spot The total number of pictures.

[0077] Step 3.3: If the i-th user Photographed the kth scenic spot , then it means the i-th user Visited the kth attraction , then let the i-th user Interaction vector with attractions The element in row i and column k of Otherwise, let , thus obtaining the interaction matrix , for The number of users in for The number of scenic spots in the ; thus constructing the adjacency matrix , where T represents transpose;

[0078] Step 3.4: Set the i-th user and the kth attraction The ID of the i-th user is converted into a continuous feature vector to obtain the ID feature embedding vector and the ID feature embedding vector of the k-th scenic spot ; For the kth scenic spot The latitude and longitude information is normalized and embedded to obtain the geographic feature embedding vector of the kth scenic spot .

[0079] Step 4: Build a wide-end preference prediction network, perform feature cross-talk on the attribute-level information extracted from the image, capture the relationship between different features, and obtain the i-th user For the kth attraction Preference ;

[0080] Step 4.1: Use equations (2) and (3) to get the i-th user Entity feature embedding vector and the i-th user The atmosphere emotional feature embedding vector :

[0081] (2)

[0082] (3)

[0083] In formula (2) and formula (3), Represents the attention mechanism network.

[0084] Step 4.2: Use equations (4) and (5) to get the k-th scenic spot Entity feature embedding vector and the kth attraction The atmosphere emotional feature embedding vector :

[0085] (4)

[0086] (5)

[0087] Step 4.3: Use formula (6) to establish the prediction preference function of the wide end :

[0088] (6)

[0089] In formula (6), represents the global bias term, Represents users and attractions The first-order feature weights of Represents a user The second-order cross-feature weights of Indicates attractions The second-order cross-feature weights of Represents users and attractions The second-order feature weights of .

[0090] Step 5: Build a Deep preference prediction network and as well as and Process and get the i-th user For the kth attraction Preference ;

[0091] Step 5.1, define the current layer of the graph neural network as the nth layer, the total number of layers is N, and initialize n=1, let the i-th user The initial embedding vector , the kth attraction The initial embedding vector of ;

[0092] Step 5.2: Use equations (8) and (9) to get the i-th user In the Embedding vector of the layer and the kth attraction In the Embedding vector of the layer :

[0093] (8)

[0094] (9)

[0095] In formula (8) and formula (9), and Represents the i-th user The number of visited attractions and the kth attraction The number of users accessed, Represents a user The embedding vector of the n-1th layer, Indicates attractions Embedding vector at layer n-1.

[0096] Step 5.3: Use equations (10) and (11) to get the i-th user The final embedding vector and the kth attraction The final embedding vector :

[0097] (10)

[0098] (11)

[0099] In formula (10) and formula (11), Represents a user No. Embedding vector of the layer and attractions No. Embedding vector of the layer The weight coefficient of

[0100] Step 5.4: Use formula (12) to establish the i-th user predicted by the Deep side For the kth attraction Preference function :

[0101] (12).

[0102] Step 6: Use formula (13) to establish the i-th user of the Wide & Deep end joint prediction For the kth attraction Preference :

[0103] (13)

[0104] In formula (13), represents the weight that controls the relative importance between the two model outputs, is the deviation term, Convert the output to Function in the interval .

[0105] Step 7: Use formula (14) to construct the logistic regression loss function L, and train the wide preference prediction network and the deep preference prediction network until the loss function converges, thereby obtaining the optimal preference prediction model;

[0106] (14)

[0107] In formula (14), represents the i-th user Whether you have visited the kth attraction The true label.

[0108] Step 8: Output of the optimal preference prediction model for the i-th user Sort the preference values ​​in the optimal preference set of all attractions in descending order, and recommend the attractions corresponding to the top preference values ​​to the i-th user .

[0109] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0110] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

Claims

1. A personalized scenic spot recommendation method based on large model image content feature enhancement, characterized in that: The steps are as follows: Step 1: Get the i-th user A collection of geotagged images taken ,in, represents the i-th user The jth picture taken, represents the i-th user The total number of pictures taken, , Represents a user collection; Step 2: The geographic coordinates of the jth picture are inversely encoded to obtain the scenic spot , thus getting the i-th user A collection of attractions , , L represents a collection of scenic spots; Step 3: Use the large model to and the kth attraction Data augmentation is performed on the images to convert the image information into interpretable multi-attribute textual semantic information; Step 4: Build a wide-end preference prediction network, perform feature cross-talk on the attribute-level information extracted from the image, capture the relationship between different features, and obtain the i-th user For the kth attraction Preference ; Step 5: Build a Deep preference prediction network and as well as and Process and get the i-th user For the kth attraction Preference ; Step 6: Use formula (13) to establish the i-th user of the Wide & Deep end joint prediction For the kth attraction Preference : (13) In formula (13), represents the weight that controls the relative importance between the two model outputs, is the deviation term, Convert the output to Functions in the interval ; Step 7: Use formula (14) to construct the logistic regression loss function L, and train the wide preference prediction network and the deep preference prediction network until the loss function converges, thereby obtaining the optimal preference prediction model; (14) In formula (14), represents the i-th user Whether you have visited the kth attraction The true label of Step 8: Output of the optimal preference prediction model for the i-th user Sort the preference values ​​in the optimal preference set of all attractions in descending order, and recommend the attractions corresponding to the top preference values ​​to the i-th user .

2. The personalized scenic spot recommendation method based on large model image content feature enhancement according to claim 1 is characterized in that: Described step 2 is carried out as follows: Step 2.1: Calculate using formula (1) and attractions The geographical coordinate distance between : (1) In formula (1), represents the radius of the Earth, and They are pictures and The latitude of the attraction, and are the longitudes of the pictures and attractions respectively; Step 2.2, set the distance threshold Δ, if , then Add attractions Otherwise, assign j+1 to j and return to repeat step 2.1 to get the i-th user A collection of attractions , , Represents the collection of attractions for all users.

3. The personalized scenic spot recommendation method based on large model image content feature enhancement according to claim 2 is characterized in that: Described step 3 is carried out as follows: Step 3.1, using the multimodal visual model GPT4V Perform data enhancement to get the i-th user Entity text collection and atmosphere emotion text collection ,in, represents the i-th user The jth picture taken The entity description text extracted from represents the i-th user The jth picture taken The atmosphere and emotional description text extracted from Step 3.2: Use the multimodal visual model GPT4V to identify the kth scenic spot Picture collection Perform data enhancement to obtain the kth scenic spot Entity text collection and atmosphere emotion text collection ,in, represents the kth scenic spot The qth picture The entity description text extracted from represents the kth scenic spot The qth picture The atmosphere and emotional description text extracted from represents the kth scenic spot Total number of pictures; Step 3.3: If the i-th user Photographed the kth scenic spot , then it means the i-th user Visited the kth attraction , then let the i-th user Interaction vector with attractions The element in row i and column k of Otherwise, let , thus obtaining the interaction matrix , for The number of users in for The number of scenic spots in the ; thus constructing the adjacency matrix , where T represents transpose; Step 3.4: Set the i-th user and the kth attraction The ID of the i-th user is converted into a continuous feature vector to obtain the ID feature embedding vector and the ID feature embedding vector of the k-th scenic spot ; For the kth scenic spot The longitude and latitude information is normalized and embedded to obtain the geographic feature embedding vector of the kth scenic spot .

4. The personalized scenic spot recommendation method based on large model image content feature enhancement according to claim 3 is characterized in that: Described step 4 is carried out as follows: Step 4.1: Use equations (2) and (3) to get the i-th user Entity feature embedding vector and the i-th user The atmosphere emotional feature embedding vector : (2) (3) In formula (2) and formula (3), represents the attention mechanism network; Step 4.2: Use equations (4) and (5) to get the k-th scenic spot Entity feature embedding vector and the kth attraction The atmosphere emotional feature embedding vector : (4) (5) Step 4.3: Use formula (6) to establish the prediction preference function of the wide end : (6) In formula (6), represents the global bias term, Represents the i-th user The two first-order feature weights of Represents the kth scenic spot The two first-order feature weights of represents the i-th user The second-order cross-feature weights of represents the kth scenic spot The second-order cross-feature weights of Represents the i-th user The two second-order feature weights of Represents the kth scenic spot The two second-order feature weights of .

5. The personalized scenic spot recommendation method based on large model image content feature enhancement according to claim 4 is characterized in that: Described step 5 is carried out as follows: Step 5.

1. Define the current layer of the graph neural network as the nth layer, the total number of layers is N, and initialize n=1. Define and initialize the i-th user Embedding vector at layer n-1 , the kth attraction The embedding vector at layer n-1 is ; Step 5.2: Use equations (8) and (9) to get the i-th user In the Embedding vector of the layer and the kth attraction In the Embedding vector of the layer : (8) (9) In formula (8) and formula (9), and Represents the i-th user The number of visited attractions and the kth attraction The number of users accessing the Step 5.3: Use equations (10) and (11) to get the i-th user The final embedding vector and the kth attraction The final embedding vector : (10) (11) In formula (10) and formula (11), Indicates the The weight coefficient of the layer; Step 5.4: Use formula (12) to establish the i-th user predicted by the Deep side For the kth attraction Preference function : (12)。 6. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the personalized scenic spot recommendation method according to any one of claims 1 to 5, and the processor is configured to execute the program stored in the memory.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the personalized scenic spot recommendation method according to any one of claims 1 to 5 are executed.

Images