Travel assistant system

Abstract

The invention discloses a travel assistant system. The travel assistant system comprises the steps that the format of input multi-modal data is converted into to-be-recognized data of a uniform format; decomposing to-be-identified data of the user into at least one sub-query task, and extracting task parameters of each sub-query task from the task calling mapping table; calling task parameters of the sub-query tasks through the task calling interface to execute task retrieval; executing the sub-query task based on the task parameters of the sub-query task, and obtaining multi-modal data related to the sub-query task in the spatial grid; scoring the retrieval result according to the preference information of the user; planning the output of the system at least according to the scoring result; and generating a standardized output template, performing structured assembly on the retrieval result or the reasoning result of the large language model based on the standardized output template, and outputting an answer. The comprehensive performance of the intelligent travel assistant in the aspects of personalized service, response real-time performance, result accuracy, complex travel problem processing capacity and the like is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent assistants, and in particular to a travel assistant system. BACKGROUND

[0002] The rapid development of the tourism industry has brought a series of problems to the city's tourism system. Among them, the more prominent problem is that the existing tourism system obtains data from complex sources, with huge volume and diverse modalities, which is difficult to effectively obtain and manage. The travel system is difficult to provide multi-modal travel data that travelers want, which has a great negative impact on the travel experience of tourists. Moreover, tourists will choose tourist destinations and determine travel arrangements according to personal preferences, travel time budget, etc. when they go out, which also puts higher requirements on the retrieval ability of the tourism system. In the prior art, artificial intelligence, especially large language models, have made significant progress in understanding natural language and generating text content, providing new technical means to solve the above problems. However, there is no intelligent travel assistant in the existing scheme that can provide personalized services, real-time response, high accuracy, and complex travel problem processing capabilities. SUMMARY

[0003] The present application provides a travel assistant system, which can provide a user with an intelligent travel assistant with personalized services, real-time response, high accuracy, and complex travel problem processing capabilities.

[0004] According to an aspect of the present application, a travel assistant system is provided, comprising: a multi-modal data input module, which obtains multi-modal data input by a user, pre-processes data of different modalities respectively, and converts the format of the multi-modal data into uniform format of to-be-recognized data; an intent recognition and task mapping module, which is configured to decompose the to-be-recognized data of the user into at least one sub-query task, extract task parameters of each sub-query task from a task call mapping table, and call the task parameters of the sub-query task through a task call interface to perform task retrieval; a hybrid index module, which is configured to execute the sub-query task based on the task parameters of the sub-query task, and obtain multi-modal data related to the sub-query task in a spatial grid; the spatial grid corresponds to an actual geographic area and stores multi-modal data in the corresponding geographic area; a comprehensive evaluation module, which is configured to score the retrieval results according to the preference information of the user; a planner module, which is configured to plan the output of the system according to at least the scoring results; an output analysis module, which is configured to structure process the retrieval results or the inference results of the large language model, and generate a standardized output template according to a preset mapping relationship between the task type and the output format. The structured output module is used to output answers based on the output plan and structured output assembly results.

[0005] Optionally, it also includes: The user preference extraction module is used to extract the user's behavioral habit preferences based on the user's input information, and to build a user preference table based on the user's behavioral habit preferences; extract the behavioral habit preferences in the user's real-time input information and update the user preference table.

[0006] Optionally, it also includes: The pre-answer module is used to find the ID of the target user's location based on the distance between the user's current location and the stored location cluster; obtain the user portrait corresponding to the user preference cluster with the highest similarity to the user preference data based on the similarity calculation result between the user preference data and the user preference cluster; find the query information cluster to be queried based on the user's location ID and the user portrait; calculate the similarity between the query information input by the target user and the query information cluster to be queried; and output the pre-answer content corresponding to the query information cluster to be queried with the highest similarity as the output result.

[0007] Optionally, the pre-answer module is also used to divide the region into multiple layers of location clusters using a hierarchical clustering algorithm, and store the clustered location clusters in a database; obtain the query information, current location and user preference data input by the target user; find the ID of the target user's location based on the distance between the current location and the stored location cluster; based on the similarity calculation result between the user preference data and the user preference cluster, obtain the user portrait corresponding to the user preference cluster with the highest similarity to the user preference data; find the query information cluster to be queried based on the user location ID and the user portrait; calculate the similarity between the query information input by the target user and the query information cluster to be queried; and output the pre-answer content corresponding to the query information cluster to be queried with the highest similarity as the output result.

[0008] Optionally, the multimodal data input module further includes: a multimodal input access module and a multimodal preprocessing module; The multimodal input access module is used to receive input data in different modalities; The multimodal preprocessing module is used to add preset modality fields to data of different modalities respectively, and convert the format of the multimodal data into data to be identified in a unified format.

[0009] Optionally, the intent recognition and task mapping module is further configured to match the to-be-recognized data of the user with a subquery task in a preset task invocation mapping table, obtain at least one matched subquery task, obtain a parameter template corresponding to the at least one subquery task, and perform normalized processing on a query parameter of the subquery task based on the parameter template, invoke a query interface of the at least one subquery task, and obtain the normalized query parameter of the at least one subquery task. The task invocation mapping table includes a plurality of subquery tasks, and each subquery task corresponds to a parameter template and a query interface.

[0010] Optionally, the hybrid index module is further configured to identify a main query data modality of the subquery task, execute the subquery task based on the main query data modality to obtain main query data, extract location association information in the main query data, index to a corresponding spatial grid based on the location association information, and obtain query data of other modalities related to the main query data in the spatial grid. The spatial grid corresponds to an actual geographic area and stores multi-modal data in the corresponding geographic area.

[0011] Optionally, the comprehensive evaluation module is further configured to obtain an information list of the user, the information list including at least basic information of the user, collection information of the user, and planning information of the user, score a plurality of evaluation parts of the travel location according to the information list and objective data of the travel location, the plurality of evaluation parts including at least location, user preference, cost performance, network evaluation, advantages, and disadvantages, input the preference information of the user into a trained user preference model to output first weight values of the adjusted evaluation parts, and calculate a final evaluation score of the retrieval result based at least on the first weight values of the adjusted evaluation parts and score scores of the evaluation parts, the evaluation score being used to adjust an output planning result of the large model.

[0012] Optionally, the planner module is further configured to plan the retrieval result according to the score result, the retrieval result including at least a play site and a traffic route, and constrain a planning strategy based at least on a budget limit, a time window, and traffic accessibility to form a play order and a travel arrangement plan that meet the constraint conditions.

[0013] Optionally, the output analysis module is further configured to generate a standardized output template according to a preset mapping relationship between a task type and an output format, and structure and assemble the retrieval result or the inference result of the large language model based on the standardized output template.

[0014] The technical scheme of the embodiment of the present application is that the multi-modal data input by the user is acquired, the data of different modalities is preprocessed respectively, and the format of the multi-modal data is converted into the unified format of the to-be-recognized data, so that the multi-modal data input by the user can be processed, and the convenience and usability of the user are improved; the user input is matched with the subquery task by using the task mapping table, without calling a large model for task planning, so that the calculation and time cost are reduced, and the quick query of the generated subquery task is ensured; the multi-modal data is searched in the spatial grid, so that when the information is searched, global search is no longer needed, and only the required other modal query data needs to be searched in the spatial grid corresponding to the geographical area, so that the efficiency of data search is greatly improved; in addition, the user preference data is used to score the search result, and the output of the system is planned according to the scoring result, so that the planning result wanted by the user is output for different user demands. The present application can improve the comprehensive performance of the intelligent travel assistant in the aspects of personalized service, real-time response, result accuracy, and complex travel problem processing capability.

[0015] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 is a system structure diagram of a travel assistant system provided by the embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to enable the personnel in the technical field to better understand the present application scheme, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0019] It should be noted that the terms "first", "second", and the like in the description and claims of the application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0020] Embodiment one Figure 1 A flowchart of a travel assistant system is provided for the embodiment one of the present application. As shown in the figure, the method comprises: Figure 1 A multi-modal data input module 101 acquires multi-modal data input by the user, pre-processes data of different modalities respectively, and converts the format of the multi-modal data into unified format of to-be-identified data.

[0021] It should be noted that the multi-modal data includes voice data, text data, picture data and location data. The pre-processing can include pre-processing and converting the input multi-modal data into text data respectively, for example, converting voice data into text data, extracting text data in picture data, and extracting geographic location information in location data; the pre-processing also includes adding preset modal fields, for example, for voice data, "voice input" can be added, for picture data, "image input" can be added, for text data, "text input" can be added, and for location data, "geographic location" can be added.

[0022] In this embodiment, the format of the multi-modal data can be converted into unified format of to-be-identified data. For example, the input multi-modal data can be converted into JSON format to retain information of all input channels of the user, and the efficiency and controllability of information retrieval and subsequent processing are improved through structured field setting.

[0023] An intent recognition and task mapping module 102 is configured to decompose the to-be-identified data of the user into at least one sub-query task, extract task parameters of each sub-query task from a task call mapping table, and call the task parameters of the sub-query task through a task call interface to perform task retrieval.

[0024] ​It should be noted that the to-be-identified data of the user is data in a unified format converted from multi-modal data input by the user, and the to-be-identified data can include at least one query request, that is, the intent recognition and task mapping module of the embodiment can recognize the query task contained in the user input information.

[0025] In the embodiment, a plurality of sub-query tasks can be preset in the task call mapping table, each sub-query task can include a task name, a keyword or a task feature of the sub-query task, and each sub-query task corresponds to a standardized query parameter and a preset task call interface. By performing similarity calculation or other data matching methods on the task name, the keyword or the task feature and the to-be-identified data, at least one sub-query task most matched with the to-be-identified data is found, and the task parameter of the sub-query task is obtained. Finally, the task parameter of the sub-query task can be called through the task call interface, so as to perform task retrieval and query data related to the sub-query task. The task parameter is data describing a task query range, for example, for a query task of food, the task parameter can include data such as a place to be queried, user taste, budget, etc.

[0026] The hybrid index module 103 is configured to execute the sub-query task based on the task parameter of the sub-query task, and obtain multi-modal data related to the sub-query task in a space grid. The space grid corresponds to an actual geographic area, and stores multi-modal data in the corresponding geographic area. It should be noted that the city can be divided into a plurality of geographic areas, so that each geographic area corresponds to a space grid, and each space grid can include multi-modal data stored in the geographic area corresponding to the space grid, for example, a space grid can store data such as tourist attractions, strategies, food, transportation, etc. in the corresponding geographic area. The stored data can include image data, transportation topology data, text data, etc.

[0027] The comprehensive evaluation module 104 is configured to score the retrieval result according to the preference information of the user.

[0028] The preference data of the user can include a travel location, a travel time, a travel budget, a travel transportation, a residence type, a food type, a travel activity, a travel season, shopping, and the like preferred by the user. In this embodiment, the plurality of data input or stored by the user can be used to score the retrieval result. For example, when the user prefers the cost-effective part of the travel, the proportion of the output of the cost-effective part is higher, and the proportions of other evaluation parts are correspondingly reduced. In addition, when the preference information of the user is the proportion data of each part, the user preference model can also fine-tune the proportion data of each evaluation part of the initial proportion, so that the adjusted proportion data of each evaluation part is more in line with the preference requirements of the user.

[0029] The planner module 105 is configured to plan the output of the system according to the scoring result.

[0030] In this embodiment, the output of the system can be planned according to the scoring result. For example, when the user prefers the cost-effective part of the travel, the travel path and travel arrangement with the optimal cost-effectiveness can be selected for planning.

[0031] The output analysis module 106 is configured to generate a standardized output template according to a preset mapping relationship between a task type and an output format, and structure and assemble the retrieval result or the inference result of the large language model based on the standardized output template.

[0032] It should be noted that the output analysis module 106 can output the standardized output template corresponding to the task type according to the preset mapping relationship between the task type and the output format. The task type can include a scenic spot recommendation, a food recommendation, a hotel query, a route planning, and the like. The standardized output template can be an output field parameter template preset for each task type, and the output format type corresponding to each task type can also be set. In addition, the key fields can be extracted from the retrieval result or the inference result of the large language model, and the structured output can be output according to the standardized output template.

[0033] The structured output module 107 is configured to output an answer based on the output planning and the structured output assembly result.

[0034] It should be noted that the structured output module 107 can be configured to encode the structured output assembly result of the task into a standard recognizable data format (such as a JSON or a rich text format), and output the result according to the output planning, and then push the output result to the front end for the user to view.

[0035] The technical scheme of the embodiment of the application can obtain multi-modal data input by a user, pre-process data of different modalities respectively, and convert the format of the multi-modal data into unified format to be recognized, thereby being capable of processing multi-modal data input by the user, improving the convenience and usability of the user, and achieving matching of user input and subquery tasks by using a task mapping table, without calling a large model for task planning, thereby reducing the calculation and time cost and ensuring quick query of the generated subquery task. In addition, the multi-modal data is searched in a spatial grid, so that global search is no longer needed when searching for information, and only the required other modal query data needs to be searched in the spatial grid corresponding to the geographical area, thereby greatly improving the efficiency of data search. In addition, the search result is scored by using user preference data, and the output of the system is planned according to the scoring result, thereby outputting the planning result that the user wants for different user needs. The application can improve the comprehensive performance of the intelligent travel assistant in terms of personalized service, real-time response, result accuracy, and complex travel problem processing capability.

[0036] In an embodiment, the multi-modal data input module 101 further comprises a multi-modal input access module and a multi-modal pre-processing module; the multi-modal input access module is configured to receive input data of different modalities; and the multi-modal pre-processing module is configured to add a preset modal field to data of different modalities respectively, and convert the format of the multi-modal data into unified format to be recognized.

[0037] It should be noted that the modal input access module is configured to receive multi-modal data of the user through different channels (voice, picture, text, and location); and the multi-modal pre-processing module is configured to add a preset modal field to data of different modalities respectively, and convert the format of the multi-modal data into unified format to be recognized, such as unified arrangement into structured JSON format.

[0038] Specifically, for voice input data, a speech recognition model can be used to transcribe the audio data, extract the corresponding text content, and fill it into the field "voice input". For example, the user inputs "please help me check what is good to eat near the Temple of Heaven" by voice, and the multi-modal pre-processing module can fill the recognized text "please help me check what is good to eat near the Temple of Heaven" into the voice input field.

[0039] For image input, the multi-modal pre-processing module can use a lightweight OCR recognition tool to extract text information in the picture and fill it into the "image input" field. For example, the user uploads a menu photo containing "Beijing Roast Duck, Golden Dingxuan Restaurant, Address: Dongcheng District", and the system fills the recognized text "Beijing Roast Duck, Golden Dingxuan Restaurant, Address: Dongcheng District" into the field and adds the "image input" field.

[0040] For text input, the multi-modal preprocessing module can retain the user's original input to the "text input" field, while extracting keywords and geographic location terms from the input text through a Chinese word segmentation tool. The keywords are filled into the "key word segmentation" field, and the geographic location terms are filled into the "geographic location" field. For example, for the input "Please help me find out what to eat near the Temple of Heaven", the system extracts the keywords ["check", "near", "delicious"] and the geographic term ["Temple of Heaven"] and fills them into the corresponding fields.

[0041] In addition, the multi-modal preprocessing module can also extract geographic location information from the user's device or image / text. If the user authorizes the positioning permission, the system can directly obtain the latitude and longitude of the current location; if not, it can infer the geographic location based on the recognized place names in the text or image and fill them into the "geographic location" field.

[0042] Finally, the multi-modal preprocessing module integrates various types of input information into a unified structured JSON format, as shown in the following example: { "voice input": "Please help me find out what to eat near the Temple of Heaven", "image input": "Beijing Roast Duck, Golden Dingshan Restaurant, Address: Dongcheng District", "text input": "Please help me find out what to eat near the Temple of Heaven", "key word segmentation": ["check", "near", "delicious"], "geographic location": ["Temple of Heaven", "Dongcheng District"] } Using JSON format not only retains all the information from the user's input channels, but also improves the efficiency and controllability of information retrieval and subsequent processing through structured field settings. Compared with traditional text-based interaction methods, this module has more advantages in semantic understanding and spatial perception, especially in scenarios such as tourism and travel that require the combination of text, images, voice and location for task processing.

[0043] In an embodiment, the intent recognition and task mapping module 102 is also configured to match the user's to-be-recognized data with a plurality of sub-query tasks in a preconfigured task invocation mapping table, obtain at least one matched sub-query task, obtain a parameter template corresponding to the at least one sub-query task, and perform standardized processing on the query parameters of the at least one sub-query task based on the parameter template; and call a query interface of the at least one sub-query task to obtain the standardized query parameters of the at least one sub-query task. The task invocation mapping table includes a plurality of sub-query tasks, and each sub-query task corresponds to a parameter template and a query interface.

[0044] In this embodiment, a task call mapping table can be created first, which includes a plurality of pre-set subquery task names, task feature descriptions, API interface parameter templates, and API query interface names. Among them, the API interface parameter template includes a plurality of query parameters, and the task feature description is used to describe the characteristics of the subquery task, which is not limited to task keywords, task descriptions, etc., and is intended to assist the large language model in accurately identifying the task target in the user input. In addition, the task call mapping table can be stored in a database and can be updated and modified in time according to the needs of business adjustment, so as to ensure that the content of the task call mapping table always reflects the latest business logic and interface requirements.

[0045] In an embodiment, the task call mapping table can be as shown in Table 1 below: Table 1: Intent recognition and task mapping example

[0046] When the input data to be recognized is input, the intent recognition and task mapping module 102 can extract the keywords of the data to be recognized, match the subquery tasks in the task call mapping table through the keywords, find at least one subquery task with the highest similarity, and then distribute the matched multiple subquery tasks to different threads for parallel processing. In each thread, the query parameters of the corresponding subquery task are extracted first. Specifically, the user input is matched with the parameter template defined in the task call mapping table for the subquery task, the value of each field is extracted, and it is converted into a standardized format. The standardized parameter template adopts JSON format, which clearly specifies the field name, data type, annotation and value range.

[0047] According to the task module interface defined in the mapping table, each task corresponds to a task call interface, such as the task interface of the food recommendation task getRestaurantsByBudget. In a multi-threaded environment, the system uses the HTTP request library to concurrently call each interface, uses the standardized parameters obtained in step 3.2 as input, executes the specific task, and obtains the corresponding processing result.

[0048] In an embodiment, the hybrid index module 103 is further configured to identify a main query data modality of the subquery task, execute the subquery task based on the main query data modality to obtain main query data, extract location association information in the main query data, index to a corresponding spatial grid based on the location association information, and obtain query data of other modalities related to the main query data in the spatial grid. The spatial grid corresponds to an actual geographic area and stores multi-modal data in the corresponding geographic area.

[0049] In this embodiment, the data modalities can include vector data, graph model data, and document data, and the main query data modality can be one of the vector data, the graph model data, and the document data. In this embodiment, the main query data modality corresponding to the query task can be identified. For example, the query task is: “How to get to a station”, it can be identified that the main query data of the query task is “station”, and then it can be further identified that the data modality of the “station” is the graph model data, and the station can be found by searching the graph model data.

[0050] After determining the main query data modality, the query task can be executed, and the corresponding main query data can be searched in the database corresponding to the main query data modality, that is, for the query task: “How to get to a station”, the specific station information can be found in the graph model database. The main query data can include location association information, and the location association information at least includes coordinates and places, and can also include information related to the coordinates and places, for example, it can include a station, a scenic spot, an airport, and a railway station, and then the location information can be extracted from the station, the scenic spot, the airport, and the railway station.

[0051] In this embodiment, the spatial grid can correspond to an actual geographical area, for example, a city can be divided into multiple rectangular regions in a grid form, and each rectangular region can be associated with a spatial grid. The multi-modal data in the geographical area corresponding to the spatial grid is stored in each spatial grid. When executing the query task, after the main query data is searched, the data of other modalities related to the main query data can be searched in the spatial grid where the main query data is located. For example, when the station data in the graph model data needs to be searched, the document type data in the spatial grid where the station is located can be searched, such as scenic spot data, restaurant data, and the like.

[0052] In addition, in the aspect of spatial grid division in this embodiment, GeoHash, Quadtree, and other spatial division technologies can be used for high-precision grid processing of city maps, each grid unit is taken as a spatial unit, and three types of data index pointers are respectively associated: vector data, document data, and graph model node data, to realize efficient query scheduling under spatial restriction. In the index organization mode, the vector data takes the GeoHash grid number and the semantic vector as the joint index key and is stored in the vector database such as Milvus, which is used for semantic-driven deep interest retrieval; the document data takes the entity name and the spatial grid number to construct the inverted index and is deployed in the full-text retrieval system such as Elasticsearch, which supports structured and keyword search; the graph model data constructs the traffic network based on the adjacency list structure, and each node is labeled with the grid number, to realize the path-based topological calculation and station aggregation recommendation.

[0053] In an embodiment, the comprehensive evaluation module 104 is further configured to obtain an information list of the user, the information list comprising at least basic information of the user, collection information of the user, and planning information of the user; score a plurality of evaluation parts of the travel destination according to the information list and objective data of the travel destination; the plurality of evaluation parts comprising at least location, user preference, cost performance, network evaluation, advantages, and disadvantages; input the preference information of the user into the trained user preference model to output first weight values of the adjusted evaluation parts; and calculate a final evaluation score of the search result based at least on the first weight values of the adjusted evaluation parts and the score of each evaluation part, the evaluation score being used to adjust the output planning result of the large model.

[0054] The information list of the user comprises basic information of the user, collection information of the user, and planning information of the user. The basic information can comprise name, email, age, gender, language, and the like of the user; the collection information of the user comprises the name, type, and project name of the favorite travel destination collected by the user; and the planning information of the user can comprise a travel plan of the user, the travel plan comprising a start time, an end time, a travel destination, a project in the travel, a time of each project, and a transportation tool used, and the like. The information list can further comprise preference information of the user, the preference information comprising a preferred travel destination, a preferred travel time, a preferred travel budget, a preferred travel transportation tool, a preferred accommodation type, a preferred food type, a preferred travel activity, a preferred travel season, and shopping, and the like. The objective data of the travel destination comprises traffic conditions, surrounding environment, commercial and living facilities, weather and climate, geographical location, surrounding living cost, surrounding shops, and shop conditions, and the like of the travel destination.

[0055] In the embodiment, the plurality of evaluation parts of the travel destination can be scored according to the information list and the objective data of the travel destination. The scoring method can be that the information list and the objective data of the travel destination are converted into calculable data, and the converted data is respectively input into a preset evaluation model. A plurality of evaluation models can be set in the embodiment, comprising a location evaluation function, a user preference evaluation function, a cost performance evaluation function, a network evaluation evaluation function, an advantage evaluation function, and a disadvantage evaluation function. Finally, a location evaluation score, a user preference score, a cost performance score, a network evaluation score, an advantage score, and a disadvantage score are calculated.

[0056] The preference information of the user can comprise a preferred travel destination, a preferred travel time, a preferred travel budget, a preferred travel transportation tool, a preferred accommodation type, a preferred food type, a preferred travel activity, a preferred travel season, and shopping, and the like. The preference information of the user can also comprise proportion data of each part of the location, the user preference, the cost performance, the network evaluation, the advantage, and the disadvantage set by the user. The first weight value represents the proportion data of each evaluation part.

[0057] The user preference model in the present application is a deep learning model. The user can input the user's preference information into the user preference model, and the proportion data of multiple evaluation parts can be output. For example, when the user prefers the cost-effective part of the trip, the proportion of the cost-effective part output will be higher, and the proportions of other evaluation parts will be correspondingly reduced. In addition, when the user's preference information is the proportion data of each part, the user preference model can also fine-tune the initial proportion data of each evaluation part, so that the adjusted proportion data of each evaluation part is more in line with the user's preference demand.

[0058] The first weight value of each evaluation part can be multiplied by the scoring score of each evaluation part respectively and then added, and the final evaluation score can be obtained. Of course, the user can also adjust the first weight value according to his own preference to obtain an adjusted weight value, and then multiply the adjusted weight value by the scoring score of each evaluation part respectively and add them to obtain the final evaluation score.

[0059] In an embodiment, the planner module 105 is further configured to plan the search result according to the scoring result; the search result at least includes a scenic spot and a traffic route; a planning strategy is constrained based on at least a budget limit, a time window, and traffic accessibility to form a planning of a playing order and a travel arrangement that meet the constraint conditions.

[0060] The planner module 105 can be configured to plan the search result according to the scoring result. For example, when the user prefers the cost-effective part of the trip, the trip path and travel arrangement with the optimal cost-effectiveness can be selected for planning. In addition, the user's budget limit, time window, traffic accessibility, etc. can also be reasonably modeled in the present embodiment. When an uncontrollable factor (such as the closure of a scenic spot or traffic congestion) is detected, the system automatically triggers a feedback mechanism to perform dynamic planning adjustment.

[0061] Specifically, for the planning task of the travel route, the planner module 105 can access the third-party API and the internal graph model of the system in real time to detect the route accessibility and real-time influencing factors, and form a dynamically variable constraint set. Multi-constraint modeling includes: - Budget limit constraint: ∑cost_i ≤ B (B is the total budget) - Time window constraint:

[0062] - Distance and continuity constraint: dist(i, i+1) ≤ d_threshold - Preference weight function:

[0063] wherein cost_i is the budget of a single item; T_open_i represents the start time, T_close_i represents the end time, t_arrive_i represents the arrival time; dist(i, i+1) represents the distance between adjacent items, d_threshold represents the distance threshold; w_i represents the weight of preference i, pref(i) represents the value of preference i, and Score(P) represents the total score.

[0064] When the environmental perception module detects a sudden abnormality (such as a scenic spot closure or traffic congestion), the planner module 105 triggers a hybrid RAG (Retrieval-Augmented Generation) engine to perform path re-planning. Finally, the planner module 105 outputs a daily play list, a visual map path, alternative solutions, and an exportable table, etc.

[0065] In an embodiment, the output parsing module 106 is further configured to determine a standardized output template of the task according to a preset mapping relationship table of task types and output formats, extract key fields from the inference result or the retrieval result of the large language model, and perform structured assembly according to the standardized output template.

[0066] It should be noted that the mapping relationship table of task types and output formats predefines different task types, output format templates corresponding to the task types, and output format types, as shown in Table 2 below: Table 2: Mapping relationship table of task types and output formats

[0067] Through the mapping relationship table of task types and output formats, different types of tasks can be mapped to specific output format templates to ensure consistency in content organization. Secondly, for intermediate results obtained from language model generation or multi-modal information retrieval, the output parsing module 106 can complete the corresponding field extraction, structure assembly, and semantic arrangement work according to the format requirements of the task. In addition, the output parsing module 106 is also responsible for encoding the formatted content in the form of JSON, rich text, card structure, etc. to meet the calling needs of different display components on the front end. To improve the robustness of the system, the output parsing module 106 also designs a structure anomaly recognition and fault handling mechanism. When the returned result does not conform to the expected format, automatic degradation display, supplementary prompt information, or calling of default templates can be implemented to ensure the integrity and continuity of the interaction.

[0068] In the process of processing specific tasks, the output analysis module operates according to the following process: first, the system receives intermediate results from a large language model or a retrieval system; then, according to the judgment result of the intention recognition module, the current task type is determined; the system finds the corresponding output template in the mapping relationship table of task type and output format according to the task type; then, the key fields are extracted from the original intermediate results and assembled according to the template requirements; finally, the assembled results are encoded into standard recognizable data formats (such as JSON or rich text format), and pushed to the structured output module, thereby completing the interactive closed loop of the entire travel assistant system.

[0069] In an embodiment, a user preference extraction module 107 is further included for extracting user behavior habit preferences based on user input information, and constructing a user preference table based on the user behavior habit preferences; extracting behavior habit preferences in real-time user input information, and updating the user preference table.

[0070] In this embodiment, the user preference extraction module 108 is a user preference extractor with self-learning ability. The user preference extraction module 108 can continuously learn and understand the user's behavior and selection habits in multiple rounds of dialogue with the help of a large language model, thereby continuously enriching and dynamically updating the user preference table, forming a user portrait, and further improving the accuracy and personalization level of recommended content.

[0071] The user preference table includes a global preference table (Global Preferences) and a single planning preference table (Trip Preferences). The global preference table is used to store user global preference information, and the single planning preference table is used to store user single planning preference information.

[0072] The global preference is used to represent the stable preferences formed by the user in long-term travel habits. For example, the user's regular budget level, preferred hotel type, eating habits, and transportation method. To further improve the system's understanding of user needs in a multicultural context, the system introduces two key fields: "user country of origin (nationality)" and "religion". These two fields are of great significance for cultural adaptation of content recommendation, especially in terms of dietary restrictions, social etiquette, and festival arrangements, which can significantly improve the system's recommendation rationality and user satisfaction. The global preference table is shown in Table 3: Table 3 Global Preference Table

[0073] The single-planning preference is used to model the user's temporary needs in a specific trip planning. For example, the departure time, the trip duration, the theme of this trip, and other information. Such preferences have short-term and time-limited nature, allowing users to make flexible adjustments without affecting long-term preferences, thereby improving the system's adaptability in personalized scenarios. Specifically, the single-planning preference table is shown in Table 4. Table 4 Single-planning preference table

[0074] In actual operation, the system calls the large language model to analyze the user's input in each round of user dialogue, automatically identifies and extracts preference-related elements. For example, when the user input is "I prefer natural scenic spots, but this time I want to experience urban nightlife", the system will automatically identify and update the travel theme (travel_theme) in the current single-planning preference to "urban nightlife", while keeping its global preference still "natural scenery". Through the accumulation and learning of multiple rounds of interaction, the system can gradually establish a comprehensive understanding of the user's behavior patterns and potential preferences, and constantly update the user preference table to achieve more intelligent and personalized content recommendations.

[0075] In an embodiment, it also includes a pre-answer module 109 for finding the ID of the target user's location based on the distance between the user's current location and the stored location cluster; obtaining the user portrait corresponding to the user preference cluster with the highest similarity to the user preference data based on the similarity calculation result of the user preference data and the user preference cluster; finding the query information cluster to be queried based on the user's location ID and the user portrait; calculating the similarity between the query information input by the target user and the query information cluster to be queried; and taking the pre-answer content corresponding to the query information cluster with the highest similarity as the output result.

[0076] The stored location cluster is a location cluster generated by the system in advance for clustering of geographic locations. The location cluster can be stored in a database. When a user needs to query information, the current location of the user can be compared with the stored location cluster for similarity determination, and then the location cluster corresponding to the ID of the location of the user is determined. In this embodiment, the clustering of geographic locations can be performed according to three levels, including clustering based on a city level, clustering based on an administrative region level, and clustering based on a popular point. For example, the levels of clustering are C city-D region-E location (popular location). If the range of the region is large, for example, when the number of points in the region / the area of the region (which can be estimated by a square) is less than a threshold, it can be determined that the region has the feature of “wide area and sparse points”. Therefore, more detailed density clustering can be performed to achieve better division. The DBSCAN algorithm can be used to perform clustering based on points of interest on the basis of the region, to improve the range accuracy of the classification. For example, the E location can be further clustered into E location east region, E location west region, E location north region, and the like.

[0077] It should be noted that the same query information can have different query results for different locations. For example, when a user inputs “what is good to eat nearby”, the output query result will display different restaurants and food according to the different current locations.

[0078] The stored user preference cluster is a user preference cluster generated by the system in advance for clustering of user preference data of different users. Each user preference cluster corresponds to a user portrait. The user portrait can be a label such as “student”, “office worker”, “photography enthusiast”, and the like. The clustered user preference cluster can be stored in a database. When a user needs to query information, after the current location ID is located, the user preference data of the user can be compared with the user preference cluster for similarity determination, and then the user portrait is determined.

[0079] It should be noted that the same query information can have different query results for different users. For example, when a user inputs “what is good to eat nearby”, the output query result will display different food according to the different user preferences (prefer sweet food or spicy food).

[0080] After the location ID of the user and the user portrait are determined, a plurality of query information clusters to be queried are found. Each query information cluster corresponds to a query question, such as “food query”, “location query”, “route query”, and the like.

[0081] The query information of different users can be collected, the query information can be clustered, and the clustered query information clusters correspond to different hot issues. The clustered query information clusters can be stored in a database. When a user needs to query information, the query information of the user can be compared with the query information clusters to be queried for similarity, and then the query information category of the user is determined.

[0082] After finding the query information clusters to be queried, the query problem corresponding to the query information cluster closest to the query information input by the target user is found as the query problem input by the target user according to the calculation result of the similarity between the query information input by the target user and the query information clusters to be queried.

[0083] It should be noted that for the same query information input by the user, the query results of different users in different places and with different preferences can be different. For example, when the user inputs "what is good to eat nearby", the output query result will be different food according to the current location of the user and the preference of the user.

[0084] For different query problems, a large model can be used to summarize each query problem when constructing a query index, and a pre-answer for each query problem is output, that is, each query information cluster corresponds to a pre-answer. The pre-answer is stored. When the target user inputs the query information, the query information is compared with the query information clusters to be queried for similarity, and then the query information cluster with the highest similarity is found, and the pre-answer corresponding to the query information cluster is the final output result.

[0085] The above specific embodiments do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent replacement and improvement within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A travel assistant system, characterized in that: include: The multimodal data input module obtains the multimodal data input by the user, preprocesses the data of different modalities respectively, and converts the format of the multimodal data into the data to be identified in a unified format; The intent recognition and task mapping module is used to decompose the user's to-be-recognized data into at least one sub-query task, extract the task parameters of each sub-query task from the task call mapping table; call the task parameters of the sub-query task through the task call interface to execute the task retrieval; A hybrid index module, configured to execute a subquery task based on task parameters of the subquery task and obtain multimodal data related to the subquery task in a spatial grid; the spatial grid corresponds to an actual geographical area and stores the multimodal data in the corresponding geographical area; Comprehensive evaluation module, used to score search results based on user preference information; a planner module for planning the output of the system based at least on the scoring results; The output parsing module is used to generate a standardized output template according to the preset mapping relationship between task type and output format, and to perform structured assembly of search results or inference results of large language models based on the standardized output template; The structured output module is used to output answers based on the output plan and structured output assembly results.

2. The travel assistant system according to claim 1, characterized in that Also includes: A user preference extraction module is used to extract the user's behavior habit preferences based on the user's input information, and to construct a user preference table based on the user's behavior habit preferences; Extracting the behavioral habit preferences from the user's real-time input information and updating the user preference table.

3. The travel assistant system according to claim 1, characterized in that Also includes: A pre-answer module is used to find the ID of the target user's location based on the distance between the user's current location and the stored location cluster; Based on the similarity calculation results between the user preference data and the user preference cluster, obtain the user profile corresponding to the user preference cluster with the highest similarity to the user preference data; Find the query information cluster to be queried based on the user's location ID and user profile; Calculate the similarity between the query information input by the target user and the query information cluster to be queried; and output the pre-answer content corresponding to the query information cluster to be queried with the highest similarity.

4. The travel assistant system according to claim 1, wherein: The pre-answer module is further used to divide the region into multi-layer location clusters using a hierarchical clustering algorithm, and store the clustered location clusters in a database; Obtain the query information, current location, and user preference data entered by the target user; Find the ID of the target user's location based on the distance between the current location and the stored location cluster; Based on the similarity calculation results between the user preference data and the user preference cluster, obtain the user profile corresponding to the user preference cluster with the highest similarity to the user preference data; Finding a query information cluster to be queried based on the user location ID and the user profile; Calculate the similarity between the query information input by the target user and the query information cluster to be queried; and output the pre-answer content corresponding to the query information cluster to be queried with the highest similarity.

5. The travel assistant system according to claim 1, characterized in that The multimodal data input module further includes: a multimodal input access module and a multimodal preprocessing module; The multimodal input access module is used to receive input data in different modalities; The multimodal preprocessing module is used to add preset modality fields to data of different modalities respectively, and convert the format of the multimodal data into data to be identified in a unified format.

6. The travel assistant system according to claim 1, characterized in that The intention recognition and task mapping module is also used to match the user's data to be recognized with the sub-query tasks in the preset task call mapping table to obtain at least one matched sub-query task; obtain a parameter template corresponding to at least one sub-query task, and normalize the query parameters of the sub-query task based on the parameter template; call the query interface of at least one sub-query task to obtain the normalized query parameters of at least one sub-query task; the task call mapping table includes multiple sub-query tasks, and each sub-query task corresponds to one parameter template and one query interface.

7. The travel assistant system according to claim 1, characterized in that: The hybrid index module is further configured to identify the main query data modality of the sub-query task; execute the sub-query task based on the main query data modality to obtain the main query data; and extract location association information from the main query data. Indexing to a corresponding spatial grid based on the location association information, and obtaining query data of other modalities related to the main query data in the spatial grid; The spatial grid corresponds to an actual geographical area and stores multimodal data within the corresponding geographical area.

8. The travel assistant system according to claim 1, wherein: The comprehensive evaluation module is further configured to obtain a user's information list, the information list including at least the user's basic information, the user's favorite information, and the user's planned information; and score multiple evaluation components of the travel destination based on the information list and objective data of the travel destination; the multiple evaluation components including at least location, user preference, cost-effectiveness, online reviews, advantages, and disadvantages; Input the user's preference information into the trained user preference model, and output the adjusted first weight value of each evaluation part; The final evaluation score of the retrieval result is calculated based on at least the adjusted first weight value of each evaluation part and the scoring score of each evaluation part, and the evaluation score is used to adjust the output planning result of the large model.

9. The travel assistant system according to claim 1, characterized in that: The planner module is also used to plan the search results based on the scoring results; the search results include at least tourist attractions and transportation routes; the planning strategy is constrained based on at least budget constraints, time windows, and transportation accessibility to form a plan for the order of play and travel arrangements that meets the constraints.

10. The travel assistant system according to claim 1, wherein: The output parsing module is also used to determine the standardized output template of the task based on the preset mapping relationship table between task type and output format, extract key fields from the inference results or retrieval results of the large language model, and perform structured assembly according to the standardized output template.

Images