Intelligent customizing method and system for text and travel products based on multi-modal fusion

Abstract

The invention relates to the field of travel product customization, in particular to an intelligent travel product customization method and system based on multi-modal fusion, and the method comprises the following steps: S10, building a plurality of AI models; s20, establishing a keyword database; s30, generating a comprehensive user demand vector through a multi-modal fusion technology; s40, based on the user demand vector, matching a tourism keyword combination, according to a matching result, screening a preliminarily consistent tourism product A from a tourism product library, and extracting information; s50, calculating traffic time by using the geographic data and the vehicle information, if the traffic time is less than set time, displaying the traffic time on a recommendation page, and if the traffic time is greater than the set time, splitting the tourism product A into a plurality of pieces; s60, the split product is optimized; and S70, recommending the split tourism products B and C to the user, and dividing the long-distance product into a plurality of sub-products by splitting the journey, thereby avoiding fatigue of the user caused by overlong travel time, and improving comfort and experience of travel.

Description

Technical Field

[0001] The present invention relates to the field of tourism product customization, and particularly to an intelligent customization method and system for cultural and tourism products based on multimodal fusion. Background Art

[0002] Customers generally search for tourism products in tourism software or websites. There are usually several options on the website or software shelves, such as recommendations and hot products. Recommendations are generally based on collaborative algorithms, and a recommendation page is formed according to the keywords provided by users, their accessible basic information, and the sales volume of products. Hot products are generally sorted according to sales volume to increase product exposure and ensure stable profits.

[0003] Most online users are not from the same city. For users who choose the same product, the means of transportation and travel duration for departure and return are often diverse. To achieve granular alignment, a unified gathering place and gathering time are usually set, and the transportation time and means are arranged for each user according to the gathering place and time.

[0004] In the above recommendation method, the products on the first few pages of the recommendation page generally have higher sales volumes. However, due to different departure locations of users, some users may only need 1-2 hours of transportation time and fewer means of transportation to reach the gathering place, while some users may need more than 10 hours. When gathering, the vitality of multiple user groups is different, and even their work and rest schedules are different, resulting in different fatigue levels and activity willingness among users. The behaviors of user groups affect each other, and it may lead to poor experiences for multiple groups in the initial stage. Summary of the Invention

[0005] One of the purposes of the present invention is to provide an intelligent customization method and system for cultural and tourism products based on multimodal fusion, so as to avoid the problems of different transportation consumption times among user groups, resulting in different fatigue levels and activity willingness, and improve the tourism experience of users from the product customization stage.

[0006] An intelligent customization method for cultural and tourism products based on multimodal fusion provided by the present invention includes the following steps:

[0007] S10. Establish multiple AI models for processing different types of data by using the API of the publicly available AI model;

[0008] S20. Establish a keyword database according to the label data of the tourism product library;

[0009] S30. Through multimodal fusion technology, uniformly model various data features provided by users to generate a comprehensive user demand vector;

[0010] S40. Based on the user demand vector and combined with the user's geographical location, screen out the travel keyword combination that best matches the user's needs from the keyword database. According to the matching results, screen out the initially matching travel product A from the travel product library, and extract the scenic spots and route information it contains;

[0011] S50. For each user, combine their departure location and the gathering location of travel product A, and use geographical data and transportation vehicle information to calculate the transportation time. If the transportation time is less than the set time, display it on the recommendation page. If the transportation time is greater than the set time, split travel product A into multiple ones;

[0012] S60. Use multi-modal data to further optimize the split products;

[0013] S70. Recommend the split travel products B and C to the user, and display the scenic spot list, transportation time, and route plan.

[0014] Preferably, step S10 includes:

[0015] S11. Configure a text model based on the API of BERT, configure an image model based on the API of ResNet, and configure a voice model based on the API of Whisper; obtain the time information of transportation modes based on the transportation data interface;

[0016] Preferably, step S30 includes:

[0017] S31. Analyze the relationship between the user's geographical location and the transportation time to the destination through the KNN or geographical data regression model;

[0018] S32. Map the features extracted by each sub-task to a unified feature space, and use a shared network layer to achieve interaction between modalities;

[0019] S33. Input the fused features into the Task Head to generate the final user demand vector.

[0020] Preferably, step S50 includes:

[0021] S51. Import user features, travel product data, and actual transportation data;

[0022] S52. Calculate the preliminary transportation plan through the path planning algorithm;

[0023] S53. Input the preliminary plan and run the multi-objective optimization model to generate a set of Pareto optimal solutions;

[0024] S54. Dynamically adjust the weights according to user features and select the optimal recommendation plan.

[0025] A second object of the present invention is to provide an intelligent customization system for cultural and tourism products based on multimodal fusion, including:

[0026] A user interaction module that receives data input by the user (text, voice, pictures, geographical location), sends a request to obtain location information, and the multimodal feature processing module sends a user demand vector and displays recommended cultural and tourism products and their details (including scenic spots, transportation routes, time arrangements, etc.).

[0027] A multimodal feature processing module that extracts and fuses various data input by the user, including three sub-modules:

[0028] A text processing sub-module that uses natural language processing (NLP) technology to extract tourism keywords in the user's requirements;

[0029] An image processing sub-module that uses computer vision technology (CNN) to analyze pictures uploaded by the user and extract visual features;

[0030] A geographical location processing sub-module that calculates the transportation time to the gathering location based on the user's current geographical location;

[0031] Then, the multimodal feature processing module outputs a comprehensive user demand vector.

[0032] A keyword database module that stores and manages a set of keywords related to cultural and tourism products.

[0033] A tourism product library management module that stores and manages all cultural and tourism product information, including scenic spot lists, transportation plans, time arrangements, recommended groups, etc.

[0034] A matching and splitting module that, based on the user demand vector and the keyword database, screens out initially matched cultural and tourism products, and intelligently splits cultural and tourism products according to the user's transportation time and requirements, including three sub-modules:

[0035] A keyword matching sub-module that uses the keywords in the user demand vector to match the keywords in the tourism product library;

[0036] A transportation time calculation sub-module that calculates the transportation time based on the user's geographical location and the gathering location and determines whether the product needs to be split;

[0037] A product splitting sub-module that, if the transportation time > N hours, splits the initially matched cultural and tourism product into two sub-products.

[0038] Recommendation and Feedback Module: Displays a recommendation window to the user, showing products with a travel time <N and product packages composed of two sub-products after splitting, providing diverse recommendation displays combining graphics and text. Below the window, keywords are displayed, allowing users to delete or add keywords. Based on user feedback, it returns to the matching and splitting module to adjust the recommendation plan.

[0039] Order and Itinerary Management Module: Automatically generates the user's travel itinerary and payment link, supporting the user to independently screen and confirm orders and itinerary plans.

[0040] The beneficial effects achieved by the present invention using the above methods and systems are as follows:

[0041] 1. When the travel time between the user's departure location and the meeting point is too long, by splitting the itinerary, the system can automatically split travel products, dividing long-distance products into multiple sub-products, avoiding user fatigue caused by long travel times, and improving the comfort and experience of the trip;

[0042] 2. Through the time planning in the sub-products, before multiple groups of users meet, their work and rest, fatigue levels, etc. can reach a basic consistency, reducing the mutual influence among multiple user groups after they are combined. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Method step diagram for product customization of the present invention;

[0044] Figure 2 System structure diagram of the product customization system of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0046] Although in some instances the terms first, second, etc. are used herein to denote various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, the first interface and the second interface, etc. are indicated. Furthermore, as used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms unless the context clearly dictates otherwise. It should be further understood that the terms "comprising", "including" indicate the presence of the features, steps, operations, elements, components, items, kinds, and / or groups, but do not preclude the presence, occurrence or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are to be construed as inclusive, or meaning any one or any combination. Thus, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C".

[0047] As Figure 1 shown, the present invention proposes an intelligent customization method for cultural and tourism products based on multi-modal fusion, including:

[0048] S10, establishing multiple AI models for processing different types of data by using the API of the publicly available AI models;

[0049] S20, establishing a keyword database according to the label data of the tourism product library;

[0050] S30, through multi-modal fusion technology, unifying the modeling of various data features provided by the user to generate a comprehensive user demand vector;

[0051] S40, based on the user demand vector, combined with the user's geographical location, screening out the tourism keyword combination that best matches the user's needs from the keyword database, and according to the matching result, screening out the initially conforming tourism product A from the tourism product library, and extracting the scenic spots and route information it contains;

[0052] S50, for each user, combining the departure place and the gathering place of tourism product A, calculating the transportation time by using geographical data and transportation tool information. If the transportation time is less than the set time, it is displayed on the recommendation page. If the transportation time is greater than the set time, then tourism product A is split into multiple;

[0053] S60, further optimizing the split products by using multi-modal data;

[0054] S70, recommending the split tourism products B and C to the user, and displaying the scenic spot list, transportation time and route plan.

[0055] Preferably, step S10 includes:

[0056] S11, Configure a text model based on the BERT API, an image model based on the ResNet API, and a voice model based on the Whisper API; obtain the time information of the transportation mode based on the traffic data interface;

[0057] Preferably, step S30 includes:

[0058] S31, Analyze the traffic time relationship between the user's geographical location and the destination through KNN;

[0059] S32, Map the features extracted by each subtask to a unified feature space and use a shared network layer to achieve interaction between modalities;

[0060] S33, Input the fused features into the task head to generate the final user demand vector;

[0061] In another embodiment, the geographical features are corrected based on the user's historical behavior patterns;

[0062] Obtain the historical departure and destination, transportation mode preference, historical time preference, and current geographical features;

[0063] First, compare the user's current departure and destination with the historical data, find the record most similar to the current situation, and use the KNN algorithm to match the geographical location features in the historical records;

[0064] Feature vector [departure longitude and latitude, destination longitude and latitude, distance].

[0065] Calculate the Euclidean distance in two-dimensional space;

[0066] Correct the weight of the current geographical features and add the historical preference influence factor:

[0067] w′ geo = α·w geo + β·w hist

[0068] where w′ geo is the weight of the current geographical features; w hist is the weight of the geographical features in the historical preference; α, β are adjustment parameters (default α = 0.6, β = 0.4).

[0069] If the user has a historical preference for short-distance high-speed rail trips, the priority of the current long-distance flight will be reduced; according to the traffic time range accepted by the user, the transportation modes that are significantly time-consuming will be automatically excluded.

[0070] Integrate the current geographical features with the corrected historical weights to generate a new geographical feature vector:

[0071] Fg ′ eo = [Distance, Transportation mode priority, Time acceptance range]

[0072] Integrate the corrected geographical feature vector F g ′ eo into the user demand vector U final ;

[0073] U final = F text , F image , F g ′ eo , F other

[0074] Preferably, step S50 includes:

[0075] S51, Import user characteristics, tourism product data, and actual transportation data;

[0076] S52, Calculate a preliminary transportation plan through a path planning algorithm;

[0077] S53, Input the preliminary plan, run a multi-objective optimization model, and generate a set of Pareto optimal solutions;

[0078] S54, Dynamically adjust the weights according to user characteristics and select the optimal recommended plan.

[0079] Among them, S50 uses a multi-objective optimization model to balance time, cost, and user experience;

[0080] The objective function is:

[0081] F(x) = w1·T(x) + w2·C(x) + w3·E(x)

[0082] Wherein, T(x) is the transportation time; C(x) is the transportation cost; E(x) is the user experience; w1, w2, and w3 are weights, reflecting the user's preferences for time, cost, and experience;

[0083] The constraint conditions are:

[0084] T(x) ≤ T max

[0085] C(x) ≤ C max

[0086] As Figure 2 shown, the intelligent customization system for cultural and tourism products of the present invention includes:

[0087] A keyword database module that stores and manages a set of keywords related to cultural and tourism products.

[0088] The tourism product library management module stores and manages all cultural and tourism product information, including scenic spot lists, transportation plans, time arrangements, target audiences, etc.

[0089] The matching and splitting module filters the initially matched cultural and tourism products based on the user demand vector and the keyword database, and intelligently splits the cultural and tourism products according to the user's transportation time and demands. It includes three sub-modules:

[0090] The keyword matching sub-module uses the keywords in the user demand vector to match the keywords in the tourism product library;

[0091] The transportation time calculation sub-module calculates the transportation time based on the user's geographical location and the meeting point, and determines whether the product needs to be split;

[0092] The product splitting sub-module splits the initially matched cultural and tourism product into two sub-products if the transportation time > N hours.

[0093] The recommendation and feedback module displays a recommendation window to the user, showing products with a transportation time < N and product packages composed of two sub-products after splitting. It provides a diverse combination of text and graphics for recommendation display. Below the window, the display of keywords is included, allowing users to delete or add keywords. According to the user's feedback, it returns to the matching and splitting module to adjust the recommendation plan.

[0094] The order and itinerary management module automatically generates the user's travel itinerary and payment link, and supports the user to independently screen and confirm orders and itinerary plans.

[0095] The keyword database module and the tourism product library management module are dynamically called by the matching and splitting module to re-screen the data according to the updated demand vector;

[0096] After the user interaction module receives user input (such as text, voice, or pictures), it calls the multi-modal feature processing module to generate a user demand vector. The matching and splitting module triggers the keyword database module to query the keywords that match the user demand vector, and inputs the user demand vector, including the keyword combination generated from multi-modal feature extraction;

[0097] Find candidate keywords similar to the keywords in the user demand vector in the keyword database, and use a similarity calculation method to match the user keywords with the database keywords:

[0098]

[0099] where Q is the keyword in the user demand vector; K is the keyword in the keyword database;

[0100] Return the set of keywords with similarity greater than the threshold, and the filtered keyword combination is used as the basis for matching tourist products in the matching and splitting module.

[0101] Call the tourist product library management module with the user demand vector and the filtered keyword combination to screen out the initially matched cultural and tourism products;

[0102] Use the user's geographical location and the set location of Product A to calculate the transportation time through the transportation API;

[0103] If T ≤ N, directly recommend Product A;

[0104] If T > N, call the product splitting sub-module;

[0105] If Product A is split into Product B and Product C:

[0106] Product B: Recommend transfer scenic spots or activities close to the user's departure place.

[0107] Product C: Retain the main destination and core scenic spots.

[0108] In another embodiment, Product B includes the main destination and core scenic spots relatively close to the user's departure place, and other main destinations and core scenic spots are included in Product C.

[0109] The above are only the preferred embodiments of the present invention patent, and are not intended to limit the present invention patent. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention patent shall be included within the protection scope of the present invention patent.

Claims

1. An intelligent customization method for cultural and tourism products based on multimodal fusion, characterized in that It includes the following steps: S10. Establish multiple AI models for processing different types of data by using the APIs of publicly available AI models; S20. Establish a keyword database based on the tag data of the tourism product library; S30. Through multi-modal fusion technology, uniformly model various data features provided by users to generate a comprehensive user demand vector; S40. Based on the user demand vector and combined with the user's geographical location, screen out the tourism keyword combination that best matches the user's needs from the keyword database. According to the matching results, screen out the initially conforming tourism product A from the tourism product library, and extract the scenic spots and route information it contains; S50. For each user, combine their departure location and the gathering location of tourism product A, and calculate the transportation time by using geographical data and transportation tool information. If the transportation time is less than the set time, display it on the recommendation page. If the transportation time is greater than the set time, split tourism product A into multiple ones; S60. Further optimize the split products by using multi-modal data; S70. Recommend the split tourism products B and C to the user, and display the scenic spot list, transportation time and route plan.

2. The intelligent customization method of cultural and tourism products based on multi-modal fusion according to claim 1, wherein Configure the text model based on the BERT API, configure the image model based on the ResNet API, configure the voice model based on the Whisper API; obtain the time information of transportation modes based on the transportation data interface.

3. The intelligent customization method of cultural and tourism products based on multi-modal fusion according to claim 1, characterized in that, Step S30 includes: S31. Analyze the relationship between the user's geographical location and the transportation time to the destination through the KNN or geographical data regression model; S32. Map the features extracted from each sub-task to a unified feature space, and use a shared network layer to achieve interaction between modalities; S33. Input the fused features into the task head to generate the final user demand vector.

4. A method for intelligent customization of cultural and tourism products based on multi-modal fusion according to claim 1, characterized in that, S51. Import user features, tourism product data and actual transportation data; S52. Calculate the preliminary transportation plan through the path planning algorithm; S53. Input the preliminary plan and run the multi-objective optimization model to generate a set of Pareto optimal solutions; S54. Dynamically adjust the weights according to user features and select the optimal recommendation plan.

5. A method for intelligent customization of cultural and tourism products based on multimodal fusion according to claim 1, characterized in that, When the refrigeration mode is turned on, the dew point is greater than 0°C, and the indoor fan is turned on.

6. An intelligent customization system for cultural and tourism products based on multimodal fusion, which is used to implement any one of the methods of claims 1-5, and is characterized in that It includes: A user interaction module that receives the data input by the user (text, voice, picture, geographical location), sends a request to obtain location information, and the multi-modal feature processing module sends the user demand vector and displays the recommended cultural and tourism products and their details (including scenic spots, transportation routes, time arrangements, etc.). A multi-modal feature processing module that extracts and fuses various data input by the user, and then the multi-modal feature processing module outputs a comprehensive user demand vector; A keyword database module that stores and manages the keyword set related to cultural and tourism products; A tourism product library management module that stores and manages all cultural and tourism product information, including scenic spot lists, transportation plans, time arrangements, recommended groups, etc.; A matching and splitting module that screens out the initially matching cultural and tourism products based on the user demand vector and the keyword database, and intelligently splits the cultural and tourism products according to the user's transportation time and needs; Recommendation and Feedback Module, which shows a recommendation window to users, displays products with travel time < N and product packages composed of two sub-products after splitting, provides diverse recommendation displays combining text and pictures, includes the display of keywords below the window, supports users to delete or add keywords, and returns to the Matching and Splitting Module according to user feedback to adjust the recommendation plan. Order and Itinerary Management Module, which automatically generates users' travel itinerary and payment link, and supports users to independently screen and confirm orders and itinerary plans.

7. The intelligent customization method of cultural and tourism products based on multi-modal fusion according to claim 6, wherein, The Multimodal Feature Processing Module includes three sub-modules: Text Processing Sub-module, which uses Natural Language Processing (NLP) technology to extract travel keywords in users' requirements; Image Processing Sub-module, which uses Computer Vision technology (CNN) to analyze the pictures uploaded by users and extract visual features; Geographical Location Processing Sub-module, which calculates the travel time to the assembly location based on the user's current geographical location.

8. The intelligent customization method of cultural and tourism products based on multi-modal fusion according to claim 6, wherein The Matching and Splitting Module includes: Keyword Matching Sub-module, which uses the keywords in the user requirement vector to match the keywords in the travel product library; Travel Time Calculation Sub-module, which calculates the travel time according to the user's geographical location and the assembly location and determines whether the product needs to be split; Product Splitting Sub-module, which splits the initially matched cultural and tourism products into two sub-products if the travel time > N hours.

Images