Large language model scenic spot intelligent interaction method and system fusing multi-source information

Abstract

The invention is suitable for the technical field of artificial intelligence and computer application, and provides a large language model scenic spot intelligent interaction method and system fusing multi-source information, and the method comprises the steps: constructing a local ontology for a multi-source structured data source; taking a local body of the first accessed data source as an initial framework, and constructing a fusion body through an iterative structure alignment function; aligning attributes and relationships in the fused ontology; the fusion ontology is stored in a graph database, and aligned data is mapped to a relational database in a structured mode; performing intelligent interaction based on the fusion ontology and the large language model; and utilizing an ontology link matching method to assist the large language model in identifying problem types and solving paths. According to the method, the convenience and interaction experience of tourists in information acquisition are improved, the response efficiency of practitioners in information retrieval and operation decision making is also enhanced, the precision, individuation and intelligence of services are realized, and the method has good intelligent level and practical application value.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer application technology, and in particular to a method and system for intelligent interaction in scenic areas that integrates multi-source information and a large language model. Background Technology

[0002] In the construction of smart scenic areas, achieving higher levels of human-computer interaction and intelligent services has become a key direction for improving visitor experience and operational efficiency. Existing service systems are mostly based on static information display or rule-based queries, which are insufficient to meet visitors' demands for flexible, personalized, and context-sensitive intelligent services. Furthermore, current scenic area systems suffer from weak inter-system collaboration, information silos, rigid responses, and a lack of intelligent understanding and unified response capabilities to complex requests. Therefore, there is an urgent need to develop a method that can solve the problems of data fragmentation and semantic inconsistency among multiple systems and achieve information fusion and semantic collaboration based on a unified and shared data model, providing efficient and intelligent interactive service support for scenic areas. Summary of the Invention

[0003] To enhance the intelligent interactive capabilities of smart scenic spots in terms of visitor services and information response, this invention provides a method and system for intelligent interaction in scenic spots that integrates multi-source information and a large language model.

[0004] A method for intelligent interaction in scenic areas using a large language model that integrates multi-source information, the method comprising the following steps:

[0005] S1. Construct local ontology for multi-source structured data sources, where each local ontology includes a set of classes, a set of attributes, and a set of relations;

[0006] S2. Using the local ontology of the first accessed data source as the initial framework, a fusion ontology is constructed through an iterative structure alignment function to achieve semantic integration of multi-source data;

[0007] S3. Align attributes and relationships in the fusion ontology to eliminate semantic ambiguity;

[0008] S4. Store the merged ontology in a graph database and map the aligned data structure to a relational database.

[0009] S5. Intelligent interaction based on fusion ontology and large language model, including business judgment, task decomposition and multi-round reasoning;

[0010] S6. Utilize ontology link matching method to assist large language models in identifying problem types and solution paths.

[0011] Preferably, the steps for constructing a local ontology include:

[0012] Let there be n structured data sources, denoted as D = {D1, D2, ..., D...} nFor any data source D i ∈D, define its field structure set as F i ={f i1 ,f i2 ,…,f im}, where f im For data source D i The m-th field. Construct the local ontology O. i The process is represented as: Among them, E i For data source D i A collection of classes ij It is class e ij The set of attributes, R ij Represents class e ij A collection of relationships with other classes.

[0013] Preferably, the steps for constructing the fusion ontology include:

[0014] Let D1 be the first data source to be connected, and let O1, the local ontology of the first data source, be used as the initial fusion ontology. Right now When subsequent data source D j (j>1) When accessing, define the structure alignment function. This represents the fused ontology after k local ontologies have been integrated, O j This represents the next local ontology to be accessed, and is aligned using a structure alignment function. and O j After structural alignment and expansion, the updated fused ontology is obtained: After n-1 iterations, a unified fusion ontology is finally formed.

[0015] Preferably, the step of aligning attributes and relations in the fusion ontology to eliminate semantic ambiguity includes:

[0016] The set of attributes in different data sources is Where A i For data source D i The set of attributes, for semantically overlapping attribute pairs (a p ,a q ), a p ∈A i a q ∈A j If i < j, assume that it already exists in the fusion ontology O. fusion Attribute a in p For the data source D that was accessed first i a qFor the data source D that is being connected i A certain attribute in the code, when semantic overlap is detected, should have an alignment direction of 'a'. q →a p That is, the attribute field in the data is replaced with 'a'. p Remove 'a' from the ontology definition q The set of relationships from different data sources is For relation pairs (r) with semantic overlap p ,r q ), r p ∈R i r q ∈R j , i < j, where r p R represents i There is a semantically overlapping relation in r. q R represents j There is a semantically overlapping relationship in the text, and their alignment and attributes are the same, so it will not be elaborated further.

[0017] Preferably, the steps of storing the fused ontology in a graph database and mapping the aligned data structure to a relational database include:

[0018] The fusion ontology is stored in the graph database GraphDB. The fusion ontology has a global semantic perspective on multiple heterogeneous data sources and can support semantic query and reasoning operations based on SPARQL, thereby meeting the needs of unified retrieval and intelligent response of cross-source and semantically consistent data in complex business scenarios.

[0019] The aligned data is structured and mapped according to the classes, attributes and relational models defined by the fusion ontology, and uniformly converted into a consistent data representation, which is then written into the relational database MySQL to achieve static data storage after semantic fusion.

[0020] Preferably, the steps for intelligent interaction based on the fusion ontology and large language model include:

[0021] Business judgments are made, local ontology is constructed using data sources, and the fused ontology is built. Intelligent interaction is carried out based on the fused ontology and the large language model. The category of the request is determined by querying ontology category information, and all ontology category information is used as prompt words to assist the large language model in intent recognition and service judgment, thereby achieving preliminary screening of the request.

[0022] Preferably, the steps of using ontology link matching to assist large language models in identifying problem types and solution paths include:

[0023] The system analyzes business problems, performs reasoning and decision-making based on the AI ​​Agent, locates the required business support data, and executes corresponding processing. It provides prompts to the agent through ontology link matching methods, assisting it in identifying problem types and solution paths. With the semantic support of the fused ontology, the agent can initiate a multi-round closed-loop reasoning process of "observation-reasoning-action." This multi-round closed-loop reasoning process dynamically formulates query strategies based on context, and for complex multi-source businesses, it breaks down the overall task into multiple independently executable sub-tasks based on the information source relationships extracted from the ontology, gradually completing the query and response, and achieving cross-system collaborative processing and intelligent response.

[0024] Preferably, the step of providing prompting information to the agent through the ontology link matching method includes:

[0025] The ontology link matching method aims to identify the starting class (representing the starting semantic point of the problem) and the target class (representing the information point that the user ultimately cares about) in the business problem, and to build a semantic link connecting the two in the fusion ontology to support the reasoning and prompt generation of LLM.

[0026] Let the fusion ontology structure be a directed graph: G = (C, R), where C is the set of ontology categories of the system: C = {C1, C2, ..., C} n}, R represents the semantic relationship between classes R={(C i C j )|C i C j ∈C}; The user input question Q is natural language text: Q∈Text, Text is a set of natural language texts; The prompt word template T is as follows: T1(C,Q)=[“Please identify the starting entity category (starting class) and the desired information category (target class) expressed in the following user questions: Existing ontology category information: {C}; User question: {Q}; Return format: Starting class {class} S}, target class {class D}”];

[0027] The first step is to use the aforementioned prompt word template to enable the LLM model to identify the starting class and target class in the problem:

[0028] F LLM (T1(C,Q))= <class S ,class D If classS = classD, the matching method directly returns the class information and does not allow subsequent link matching; if classS ≠ classD, the subsequent link matching continues.

[0029] The second step is to obtain the class. Sto class D Find all paths (usually 3-5 when length is limited), and get the top K paths based on path distance: P = FindPaths(G, class S ,class D ), where each path P i Represented as an ordered sequence: P i = <class S ,class i1 ,…,class D >; Define the path scoring function as Score(P i ) = 1 / length(P i The shorter the path, the higher the score. The top K paths are selected as the candidate path set P. k =TopKf(P,K), then for each path P i ∈P k Generate its natural language description: Di = describe(Pi), Di ∈ Text;

[0030] The third step is to use LLM to select the most suitable path from K paths: Construct a language model prompt word template T(Q,D) = [“Based on question Q, determine the most suitable path among the following: {D1, ...,Dk}”]; Input the prompt word into the LLM model to obtain the optimal path: P = F LLM (T(Q,D)).

[0031] Another objective of this invention is to provide a large language model-based intelligent interactive system for scenic areas that integrates multi-source information, the system comprising:

[0032] The ontology support module, serving as the system's knowledge foundation, provides semantic basis and structural support. It consists of an ontology management component, an ontology storage component, and an ontology query component. This module is responsible for steps S1-S4 in the method, handling the construction, alignment, and storage of the fused ontology and providing access to it. The ontology management component offers both graphical and code-based interfaces, supporting full lifecycle management of ontology based on the RDF / OWL standard, including class definition, attribute declaration, and semantic relationship construction. The ontology storage component persistently stores ontology knowledge in GraphDB based on the RDF / OWL standard, modeling class hierarchy, attribute relationships, and constraint rules in triple form. The ontology query component provides ontology access interfaces to other modules via the SPARQL protocol, supporting multi-dimensional semantic retrieval.

[0033] The service awareness module, as a front-end unit for user interaction, consists of a user interaction component, a service query component, and an LLM discriminant component. It is responsible for executing the business judgment in step S5 of the method. The user interaction component provides a front-end dialogue interface for multimodal input and preprocesses user questions (cleaning characters and unifying encoding). The LLM discriminant component has a built-in large language model and determines whether the user question belongs to the system service scope based on ontology category information. The service query component queries the ontology library through SPARQL and outputs ontology category information in a structured manner according to the ontology category information description specification.

[0034] The multi-source query module, based on the RAG architecture, enables heterogeneous data interaction and consists of a task receiving component, an information location component, an SQL generation and execution component, and a multi-source information storage component. The task receiving component receives natural language query requests from the decision-making module and parses them into structured task data. The information location component uses a large language model combined with ontology category information to generate a database schema and locate the data source. The SQL generation and execution component converts the query request into SQL statements based on the schema, executes the query in a MySQL database, and supports error feedback and correction. The multi-source information storage component integrates data management and ontology annotation functions, enabling automatic mapping from the ontology model to the database structure.

[0035] The decision-making module, based on the AI ​​Agent architecture, enables intelligent decision-making and consists of an information receiving component, a prompt generation component, and a reasoning decision-making component. It is responsible for task decomposition and multi-round reasoning in method step S5. The information receiving component receives user questions through the RESTful API and triggers the multi-round reasoning process. The prompt word generation component obtains semantic links through the ontology query interface and generates LLM prompt words in combination with the question background. The reasoning decision-making component decomposes complex problems into sub-tasks based on the "observation-reasoning-action" closed loop and dynamically calls the multi-source query module until the task is completed.

[0036] Preferably, the multi-source query module has an SQL statement error feedback and reconstruction mechanism. When executing the SQL statement generated by the large language model and connecting to the structured database, if the system detects anomalies such as syntax errors, missing fields, or logical conflicts returned by the database, it immediately triggers the reconstruction process: first, it extracts the error information and constructs contextual hints based on the query intent, and then inputs them back into the large language model to generate the corrected SQL statement; at the same time, it supports a joint verification strategy of syntax hint templates and ontology schema to assist in generating query statements with semantic consistency and structural rationality, effectively improving the query success rate and robustness in complex and variable database environments.

[0037] The beneficial effects achieved by this invention are as follows:

[0038] This method not only improves the convenience and interactive experience for tourists in obtaining information, but also enhances the responsiveness of practitioners in information retrieval, facility management, and operational decision-making. Leveraging semantic fusion and natural language understanding capabilities, the method effectively overcomes the limitations of information fragmentation and traditional query methods, achieving more precise, personalized, and intelligent services. By constructing a unified ontology model, the method can dynamically adapt to the collaborative needs of multiple systems, supporting cross-source data retrieval and intelligent responses in complex business scenarios. Furthermore, the method possesses good scalability and adaptability, showing broad application prospects and promotional value in areas such as smart tourism and public services. Attached Figure Description

[0039] Figure 1 This is a schematic diagram illustrating the principle of multi-source fusion.

[0040] Figure 2 This is a block diagram of the overall system structure of the present invention.

[0041] Figure 3 This is a flowchart illustrating the system operation process of the present invention.

[0042] Figure 4 This is a schematic diagram of the system reasoning and decision-making principle of the present invention. Detailed Implementation

[0043] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0044] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0045] Unless otherwise stated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. While only preferred methods and materials have been described herein, any methods and materials similar or equivalent to those described herein may be used in the implementation or testing of this invention. All references to this specification are incorporated by way of citation to disclose and describe methods and / or materials associated with those references. In the event of any conflict with any incorporated reference, the content of this specification shall prevail.

[0046] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be apparent to those skilled in the art. This specification and embodiments are merely exemplary.

[0047] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0048] This invention provides a method for intelligent interaction in scenic areas using a large language model that integrates multi-source information. The method includes the following steps:

[0049] S1. Construct local ontology for multi-source structured data sources, where each local ontology includes a set of entities, a set of attributes, and a set of relations;

[0050] Let there be n structured data sources, denoted as D = {D1, D2, ..., D...} n For any data source D i ∈D, define its field structure set as F i ={f i1 ,f i2 ,…,f im}, where f im For data source D i The m-th field. Construct the local ontology O. i The process is represented as: Among them, E i For data source D i The set of entities in A ij It is entity e ij The set of attributes, R ij Represents entity e ij A set of relationships with other entities.

[0051] S2. Using the local ontology of the first accessed data source as the initial framework, a fusion ontology is constructed through an iterative structure alignment function to achieve semantic integration of multi-source data;

[0052] Let D1 be the first data source to be connected, and let O1, the local ontology of the first data source, be used as the initial fusion ontology. Right now When subsequent data source D j (j>1) When accessing, define the structure alignment function. This represents the fused ontology after k local ontologies have been integrated, O j This represents the next local ontology to be accessed, and is aligned using a structure alignment function. and Oj After structural alignment and expansion, the updated fused ontology is obtained: After n-1 iterations, a unified fusion ontology is finally formed.

[0053] S3. Align attributes and relationships in the fusion ontology to eliminate semantic ambiguity;

[0054] The set of attributes in different data sources is Where A i For data source D i The set of attributes, for semantically overlapping attribute pairs (a p ,a q ), a p ∈A i a q ∈A j If i < j, assume that it already exists in the fusion ontology O. fusion Attribute a in p For the data source D that was accessed first i a q For the data source D that is being connected i A certain attribute in the code, when semantic overlap is detected, should have an alignment direction of 'a'. q →a p And replace the attribute field in the data with 'a' p Remove 'a' from the ontology definition q The set of relationships from different data sources is For relation pairs (r) with semantic overlap p ,r q ), r p ∈R i r q ∈R j , i < j, where r p R represents i There is a semantically overlapping relation in r. q R represents j There is a semantically overlapping relationship in the text, and their alignment and attributes are the same, so it will not be elaborated further.

[0055] S4. Store the merged ontology in a graph database and map the aligned data structure to a relational database.

[0056] The fusion ontology is stored in the graph database GraphDB. The fusion ontology has a global semantic perspective on multiple heterogeneous data sources and can support semantic query and reasoning operations based on SPARQL, thereby meeting the needs of unified retrieval and intelligent response of cross-source and semantically consistent data in complex business scenarios.

[0057] The aligned data is structured and mapped according to the classes, attributes and relational models defined by the fusion ontology, and uniformly converted into a consistent data representation, which is then written into the relational database MySQL to achieve static data storage after semantic fusion.

[0058] S5. Intelligent interaction based on fusion ontology and large language model, including business judgment, task decomposition and multi-round reasoning;

[0059] Business judgments are made, local ontology is constructed using data sources, and the fused ontology is built. Intelligent interaction is carried out based on the fused ontology and the large language model. The category of the request is determined by querying ontology category information, and all ontology category information is used as prompt words to assist the large language model in intent recognition and service judgment, thereby achieving preliminary screening of the request.

[0060] S6. Utilize ontology link matching method to assist large language models in identifying problem types and solution paths.

[0061] The system analyzes business problems, performs reasoning and decision-making based on the AI ​​Agent, locates the required business support data, and executes corresponding processing. It provides prompts to the agent through ontology link matching methods, assisting it in identifying problem types and solution paths. With the semantic support of the fused ontology, the agent can initiate a multi-round closed-loop reasoning process of "observation-reasoning-action." This multi-round closed-loop reasoning process dynamically formulates query strategies based on context, and for complex multi-source businesses, it breaks down the overall task into multiple independently executable sub-tasks based on the information source relationships extracted from the ontology, gradually completing the query and response, and achieving cross-system collaborative processing and intelligent response.

[0062] The method further includes:

[0063] S7. Identify the information sources required by the subtask. First, based on the class and attribute descriptions in the fusion ontology, identify the specific information sources that the subtask depends on. Then, based on the structural definition of the information source in the ontology, automatically generate the corresponding database schema structure for use in constructing subsequent query statements.

[0064] S8. Query Generation and Data Retrieval: Based on the identified information source schema and the semantic content of the subtasks, a large language model is used to automatically generate matching SQL query statements. These query statements will be used to retrieve the required data from the structured database MySQL, and the results will be extracted for further use by the AI ​​Agent.

[0065] S9, the AI ​​agent, combines the newly acquired query results to continue the next round of reasoning and task planning, repeating steps eight and nine until the system determines that the information is sufficient to accurately answer the business question, thus completing a complete intelligent interaction process.

[0066] In this embodiment, the steps of providing prompt information to the agent through the ontology link matching method include:

[0067] The ontology link matching method aims to identify the starting class (representing the starting semantic point of the problem) and the target class (representing the information point that the user ultimately cares about) in the business problem, and to build a semantic link connecting the two in the fusion ontology to support the reasoning and prompt generation of LLM.

[0068] Let the ontology structure be a directed graph: G = (C, R), where C is the set of ontology categories of the system: C = {C1, C2, ..., C} n}, R represents the semantic relationship between classes R={(C i C j )|C i C j ∈C}; The user input question Q is natural language text: Q∈Text, Text is a set of natural language texts; The prompt word template T is as follows: T1(C,Q)=[“Please identify the starting entity category (starting class) and the desired information category (target class) expressed in the following user questions: Existing ontology category information: {C}; User question: {Q}; Return format: Starting class {class} S}, target class {class D}”];

[0069] The first step is to use the aforementioned prompt word template to enable the LLM model to identify the starting class and target class in the problem:

[0070] F LLM (T1(C,Q))= <class S ,class D If classS = classD, the matching method directly returns the class information and does not allow subsequent link matching; if classS ≠ classD, the subsequent link matching continues.

[0071] The second step is to obtain the class. S to class D Find all paths (usually 3-5 when length is limited), and get the top K paths based on path distance: P = FindPaths(G, class S ,class D ), where each path Pi Represented as an ordered sequence: P i = <class S ,class i1 ,…,class D >; Define the path scoring function as Score(P i ) = 1 / length(P i The shorter the path, the higher the score. The top K paths are selected as the candidate path set P. k =TopKf(P,K), then for each path P i ∈P k Generate its natural language description: Di = describe(Pi), Di ∈ Text;

[0072] The third step is to use LLM to select the most suitable path from K paths: Construct a language model prompt word template T(Q,D) = [“Based on question Q, determine the most suitable path among the following: {D1, ...,Dk}”]; Input the prompt word into the LLM model to obtain the optimal path: P = F LLM (T(Q,D)).

[0073] Another objective of this invention is to provide a large language model-based intelligent interactive system for scenic areas that integrates multi-source information. The system comprises an ontology support module, a service perception module, a multi-source query module, and a decision-making module.

[0074] The ontology support module, serving as the system's knowledge foundation, provides semantic basis and structural support. It is the core infrastructure for realizing intelligent interaction and decision-making within the system, consisting of an ontology management component, an ontology storage component, and an ontology query component. This module is responsible for steps S1-S4 in the method steps, handling the construction, alignment, storage, and providing access to the merged ontology. The ontology management component offers both graphical interface and code-based programming interfaces, with a front-end page supporting visual display and editing of the ontology. The graphical interface, based on WebVOWL, intuitively displays the ontology's class hierarchy, attribute relationships, and semantic structure; the programming interface supports SPARQL-based query and update operations, meeting developers' needs for programmatic ontology management. It supports full lifecycle management of ontology based on RDF (Resource Description Framework) / OWL (Web Ontology Language) standards, including class definition, attribute declaration, and semantic relationship construction; it allows users to create, modify, and delete class hierarchies, customize attribute features and constraints, and flexibly construct diverse semantic relationships. The component conforms to the RDF and OWL standards, supporting standardized import and export of ontology structures. The ontology storage component is responsible for the centralized management of ontology knowledge in the structured storage system. Based on the RDF / OWL standard, this component uniformly models the semantic information of the ontology, such as class hierarchy, attribute relationships, and constraint rules, and persistently stores it in GraphDB. The ontology query component provides unified ontology access and query services to other modules in the system. This component corresponds to a service interface that receives SPARQL query language from other components or modules, enabling multi-dimensional retrieval of classes, attributes, and their semantic relationships within the ontology.

[0075] The service awareness module, as a front-end unit for user interaction, is responsible for the core functions of receiving user questions and determining service scope based on ontology. This module consists of a user interaction component, a service query component, and an LLM discriminant component, and is responsible for executing the business judgment in step S5. The user interaction component corresponds to a front-end conversational interface, serving as the direct entry point for user interaction with the system and supporting multimodal input methods. This component preprocesses user questions, cleaning the original user input by removing special characters, redundant spaces, and other invalid information, while also standardizing the character encoding format. The service query component corresponds to a service interface that interfaces with the ontology managed by the system, providing the LLM discriminant component with the ontology information required to determine service scope. This component uses SPASQL statements to query the ontology stored in GraphDB, retrieves all category information, and processes the retrieved information using the ontology category information description specification. The LLM discriminant component is a built-in open-source large language model service interface and is the core unit for the service awareness module to implement service scope determination. This component receives user questions from the user interaction component and ontology knowledge base category information from the service query component. The category information covers the service scope of the system. Both are passed to the built-in large language model in the form of prompt words to determine whether the user question belongs to the service scope of the system.

[0076] The query component processes the retrieved ontology information according to the ontology category information description specification, which is as follows:

[0077] Let C represent an ontology class, expressed as: C =<class_id,class_name,desc,attrs> .

[0078] Where class_id is the unique identifier of the ontology class, class_name is the name of the ontology class, desc is a brief description of the ontology class, and attrs is the set of attributes contained in the ontology class. It is represented as attrs = {a1, a2, ..., an}, where n is the number of attributes.

[0079] Each attribute can be represented as:

[0080] ai =<attr_id,attr_name,attr_type,range,constraints> .

[0081] Where attr_id is the unique identifier of the attribute, attr_name is the name of the attribute, attr_type is the data type of the attribute, range is the value range of the attribute, and constraints are the constraints of the attribute.

[0082] The multi-source query module is a key execution component of the system, responsible for receiving task instructions from the intelligent decision-making module and interacting with multiple heterogeneous data sources at the underlying level. This module is designed based on the RAG architecture and consists of a task receiving component, an information location component, an SQL generation and execution component, and a multi-source information storage component. The task receiving component corresponds to one service interface, responsible for receiving natural language query requests from the decision-making module, which include explicit query targets and implicit information requirements. The information location component corresponds to one service interface, primarily responsible for using LLM to determine the information sources involved in the task and generating a database schema based on ontology category information. This component uses ontology category information as prompts, which also follow the aforementioned ontology category information description specifications, to guide LLM in locating the information sources and generating a complete database schema. The SQL generation and execution component corresponds to two core service interfaces: one is the SQL generation interface, which relies on the module's built-in large language model, combining the SQL generation prompts with the database schema information transmitted by the service location component to accurately convert the task request into an SQL query statement; the other is the SQL execution interface, responsible for connecting with the MySQL database used by the system, executing the generated SQL statements, and retrieving the corresponding data information. If syntax or logical errors are detected in the SQL statement during the query process, the database query interface can feed back the error information to the SQL generation interface, which will then correct and reconstruct the statement based on the feedback. The multi-source information storage component is a key link in the system's underlying data management, consisting of data management and ontology annotation, aiming to achieve unified processing and storage of multi-source heterogeneous data. The data management part is geared towards system administrators, providing a convenient operation interface and management interface, supporting centralized management of various data sources such as scenic area databases, geographic information platforms, and user review systems. The ontology annotation part corresponds to a key service interface responsible for automatically generating semantic constraints and database structure based on the platform's ontology knowledge base during the data import process. The system uses a MySQL database as the data storage backend, and the ontology knowledge base serves as the core basis for structural design, achieving automatic mapping from the ontology model to the database structure. Specifically, during database construction, this interface automatically maps classes in the ontology to tables in the database, and attributes to fields in the tables, strictly adhering to the data types and constraints defined in the ontology to set fields, ensuring that the database structure is highly consistent with the ontology model at the semantic level.

[0083] In the information location component, a corresponding database schema is generated based on the ontology information. The mapping relationship and representation structure between the ontology information and the database schema are as follows:

[0084] Suppose that each table structure in the database is mapped from ontology C, following the following specifications:

[0085] Schema(C) =<table_name,description,columns> ;

[0086] Where `table_name` corresponds to the `class_name` of the ontology class, used to identify the table name in the database; `description` is a brief description of the table, corresponding to the `description` of the ontology class; `columns = {col1, col2, ..., col...}` n} is a collection of fields, each field col i From the attribute a in the ontology class i Mapped from elsewhere. Each field col i The definition is as follows:

[0087] Col i =<column_name,data_type,range,constraints> In this context, column_name corresponds to attr_name of the attribute, serving as the field name in the database; data_type corresponds to attr_type, used to define the data type of the field; range corresponds to range of the attribute, representing the range of values ​​the field can take; and constraints corresponds to constraints of the attribute, representing the constraints of the field.

[0088] Through the above mapping relationships, the service location component can structurally convert ontology category information into a standard database schema, providing complete and accurate data structure support for the SQL generation component.

[0089] The decision-making module, based on an AI Agent architecture, implements intelligent decision-making and consists of an information receiving component, a prompt generation component, and an inference decision-making component. It is responsible for task decomposition and multi-round inference in method step S5. The information receiving component corresponds to a service interface, serving as the interaction interface between the decision-making module and external modules, and handling the transmission of natural language questions. The prompt generation component consists of two service interfaces: first, an ontology query interface, which is responsible for finding relevant links to solve the problem in the ontology based on the user's question; second, a prompt generation interface, which uses the LLM (Local Language Model) to generate corresponding prompts based on the results obtained from the ontology query interface and the problem background, providing targeted and effective prompt information to the LLM component. The inference decision-making component is the core functional unit of the decision-making module, possessing multi-round inference and task planning capabilities, focusing on semantic understanding and step-by-step solution of complex problems. This component uses prompts generated based on domain knowledge in the ontology knowledge base, enabling the large language model to understand the user's question, supporting the gradual construction of the problem-solving path in dynamic interaction, generating staged instructions, and continuously calling relevant query interfaces until the task is completed. Through collaboration with the question receiving component and the prompt generation component, the inference decision-making component achieves a closed loop of "understanding-planning-execution-feedback" for the task. In this embodiment, the information receiving component receives user questions through a RESTful API and triggers a multi-round reasoning process; the prompt word generation component obtains semantic links through an ontology query interface and generates LLM prompt words in combination with the question background; the reasoning decision component decomposes complex questions into sub-tasks based on the "observation-reasoning-action" closed loop and dynamically calls the multi-source query module until the task is completed.

[0090] In this embodiment, the multi-source query module has an SQL statement error feedback and reconstruction mechanism. When executing the SQL statement generated by the large language model and connecting to the structured database, if the system detects anomalies such as syntax errors, missing fields, or logical conflicts returned by the database, it immediately triggers the reconstruction process: first, it extracts the error information and constructs contextual hints based on the query intent, and then inputs them back into the large language model to generate the corrected SQL statement; at the same time, it supports a joint verification strategy of syntax hint templates and ontology schema to assist in generating query statements with semantic consistency and structural rationality, effectively improving the query success rate and robustness in complex and variable database environments.

[0091] To facilitate a better understanding of the technical solution of the present invention by those skilled in the art, specific embodiments of the present invention are provided below:

[0092] The schematic diagram of multi-source fusion is as follows: Figure 1 As shown, the scenic area contains three types of data sources with different structures, namely:

[0093] Tourist Management Data Source D1: Contains entities "Tourists" and "Orders." Tourists have attributes: Tourist ID, Name; orders have attributes: Order ID, Tourist ID. A "Generation" relationship exists between entities, indicating that tourists generate orders, thus generating the corresponding local ontology O1. Scenic Spot Guide Data Source D2: Contains entities "Scenic Spots" and "Facilities." Scenic Spots have attributes: Scenic Spot ID, Name, Capacity; facilities have attributes: Facility ID, Type. A "Containment" relationship exists between entities, indicating that scenic spots contain facilities, generating the corresponding local ontology O2. Real-time Monitoring Data Source D3: Contains the entity "Person Flow," with attributes: Scenic Spot ID, Real-time Number of People, used to record real-time visitor flow information for each scenic spot. This data source does not explicitly provide relationships between entities, but implicit associations can be established with scenic spot entities through the scenic spot ID, generating the corresponding local ontology O3.

[0094] The fusion ontology construction adopts an iterative alignment strategy. The construction process is as follows:

[0095] Initial fusion First, based on the visitor management data source, an initial fusion ontology is constructed. This includes the "generation" of physical tourists, orders, and their relationships. Initial expansion, access... Incorporating the local ontology O2 of the attraction guide into the fusion process, adding physical attractions, facilities, and their relationships "containment," the updated fused ontology is as follows: Secondary expansion: Access By integrating with the real-time monitoring local ontology O3, the entity "Pedestrian Flow" and its attributes are introduced. Simultaneously, based on the semantic consistency of the attraction ID field, an implicit semantic relationship "monitoring" is established between pedestrian flow and attractions, representing the attraction whose pedestrian flow is being monitored. The final merged ontology update is as follows:

[0096] The overall structural block diagram of the personalized scenic spot recommendation system based on retrieval enhancement proposed in this invention is as follows: Figure 2 As shown, the system is mainly divided into four modules: service perception module, decision-making module, multi-source query module, and ontology support module. The system first establishes connections with multiple data sources within the scenic area to acquire structured and unstructured information, and then constructs a unified ontology based on domain knowledge as the foundation for the system's semantic understanding and reasoning. During system operation, the functional modules communicate through standardized service interfaces to achieve data interaction and collaborative operation between modules. The system can complete a full intelligent service process from receiving user questions, semantic parsing, information fusion and reasoning, to finally generating accurate answers, ensuring the intelligence and efficiency of the service.

[0097] The service awareness module, as the system entry point, fully encompasses the user interaction and service scope determination processes. Its operational logic is as follows: Figure 2As shown, this module consists of a user interaction component, an LLM discriminant component, and a service query component working together. The user interaction component appears as a dialog box interface on the official scenic area platform, supporting multimodal interaction methods such as text input and voice input. After a user submits a question, this component first performs preprocessing: real-time transcription of voice input, removal of special symbols and redundant spaces from text input, and unified character encoding to UTF-8 format to ensure the standardization of input information. The service query component dynamically retrieves service category information supported by the system from the ontology database GraphDB via the SPARQL protocol, specifically including the hierarchical structure, attribute definitions, and semantic relationships of ontology categories. After structured processing, the query results are transmitted to the LLM discriminant component in a format conforming to the ontology category information description specifications. The LLM discriminant component, as the core processing unit, semantically fuses the user's preprocessed question with the ontology service category information and performs a classification task using a built-in large language model. It verifies that both categories are within the ontology's coverage, thus routing the question to the decision module for subsequent processing. If the user's question involves categories outside the ontology, this component directly returns a friendly prompt explaining why it cannot be processed and providing relevant suggestions.

[0098] The decision-making module consists of an information receiving component, a prompt word generation component, and an inference decision-making component. The information receiving component is a RESTful API interface that supports the transmission of natural language questions in JSON format. It receives user questions from the service awareness module and passes them to the prompt word generation and inference decision-making components, triggering a multi-round inference process. The prompt word generation component works in conjunction with an ontology query interface and a prompt generation interface. The ontology query interface, based on the ontology link matching method, retrieves relevant classes, attributes, and relationships from the ontology storage component (GraphDB) using the SPARQL protocol. The prompt generation interface combines the ontology query results with the question context to generate targeted prompt words. Furthermore, the information receiving and prompt word generation components pass the question and relevant prompt words to the inference decision-making component. This component, based on an "observation-reasoning-action" closed loop, decomposes complex tasks into sub-tasks and dynamically calls the multi-source query module to pass these sub-tasks.

[0099] The multi-source query module is the core of the system's data interaction. Based on the RAG architecture, it consists of a task receiving component, an information location component, an SQL generation and execution component, and a multi-source information storage component. The task receiving component receives natural language query tasks from the decision-making module via a RESTful API interface, parsing them into structured data containing task ID, query intent, target class, and contextual constraints. The information location component uses LLM to analyze the query intent, combines ontology category information to locate the data source, generates a database schema based on ontology class attributes, maps ontology classes to tables, and attributes to fields, including table names, descriptions, and field sets. The SQL generation and execution component uses a built-in LLM to generate SQL statements based on the schema and query intent, interfaces with the MySQL database to execute queries, and supports error feedback and statement reconstruction, such as automatically correcting field name spellings based on syntax errors returned by the database. The multi-source information storage component is responsible for the unified management and storage of multi-source heterogeneous data. The data management part provides a visual interface to support centralized management of data sources and data cleaning. The ontology annotation part automatically generates semantic constraints and database structure based on the ontology knowledge base when data is entered into the database, mapping ontology classes to tables and attributes to fields, ensuring that the database structure is semantically consistent with the ontology model.

[0100] The ontology support module forms the knowledge foundation of the system, consisting of an ontology management component, an ontology storage component, and an ontology query component. The ontology management component provides both a graphical interface and a code-based programming interface, supporting both visual editing and programmatic management of the ontology structure. The graphical interface, based on WebVOWL, intuitively displays the ontology's class hierarchy, attribute relationships, and semantic structure. Users can create classes, define attributes, and build relationships between classes through drag-and-drop operations. The programming interface conforms to the RDF / OWL standard, supporting ontology queries and updates via SPARQL statements. The ontology storage component, also based on the RDF / OWL standard, uniformly models semantic information such as the ontology's class hierarchy, attribute relationships, and constraint rules, and persistently stores it in GraphDB. The storage structure is represented in triples, leveraging the efficient query capabilities of the graph database to support rapid retrieval of inheritance relationships, attribute associations, and logical constraints between classes. The ontology query component provides a unified ontology access interface to other modules in the system, receiving SPARQL query requests and returning semantic retrieval results.

[0101] The principle diagram of reasoning and decision-making is as follows: Figure 4As shown, the decision module, as the core control component of the system, is mainly responsible for judging and scheduling whether to call external data sources for information completion during the user question processing process. Its working principle is based on the Large Language Model (LLM)'s preliminary understanding of the user input and the system's current information state, performing intelligent reasoning and decision control. When the system receives a user question, the LLM first combines semantic link hints related to the question in the ontology library to perform preliminary parsing of the question, attempting to directly generate the answer. During this process, the system records the current context state through the environment module (Env) and forms an information perception result called "observation". If the system's current information cannot meet the question answering requirements, the decision module will intervene, judging whether the current information is sufficient based on the observation. If insufficient, it will activate the multi-source query module to complete the required information. This module calls the query toolchain driven by the LLM, first locating the service type and data content required for the missing information, then constructing and executing the corresponding data source call to obtain the completed content. After the query is completed, the system re-enters the LLM processing stage, combining the completed information to attempt a new question solution. The decision-making module continuously performs cyclic scheduling control during this process until the system obtains enough information to complete the final answer to the user's question.

[0102] It should be noted that, in this document, the term "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0103] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for intelligent interaction in scenic areas using a large language model that integrates multi-source information, characterized in that: The method includes the following steps: S1. Construct local ontology for multi-source structured data sources, where each local ontology includes a set of classes, a set of attributes, and a set of relations; S2. Using the local ontology of the first accessed data source as the initial framework, construct the fused ontology through iterative structure alignment functions; S3. Align attributes and relationships in the fusion ontology to eliminate semantic ambiguity; S4. Store the merged ontology in a graph database and map the aligned data structure to a relational database. S5. Intelligent interaction based on fusion ontology and large language model, including business judgment, task decomposition and multi-round reasoning; S6. Utilize ontology link matching method to assist large language models in identifying problem types and solution paths.

2. The intelligent interactive method for scenic areas using a large language model that integrates multi-source information as described in claim 1, characterized in that, The steps for constructing a local ontology include: Let there be n structured data sources, denoted as D = {d1, D2, ..., Dn}. n For any data source D i ∈D, define its field structure set as F i ={f i1 ,f i2 ,…,f im }, where f im For data source D i The m-th field is used to construct the local ontology O. i The process is represented as: Among them, E i For data source D i A collection of classes ij It is class e ij The set of attributes, R ij Represents class e ij A collection of relationships with other classes.

3. The intelligent interactive method for scenic areas using a large language model that integrates multi-source information as described in claim 1, characterized in that, The steps to construct a fused ontology include: Let D1 be the first data source to be connected, and let O1, the local ontology of the first data source, be used as the initial fusion ontology. Right now When subsequent data source D j Define the structure alignment function during access. This represents the fused ontology after k local ontologies have been integrated, O j This represents the next local ontology to be accessed, and is aligned using a structure alignment function. and O j After structural alignment and expansion, the updated fused ontology is obtained: After n-1 iterations, a unified fusion ontology is finally formed.

4. The intelligent interactive method for scenic areas using a large language model that integrates multi-source information as described in claim 1, characterized in that, The steps to align attributes and relations in a fusion ontology and eliminate semantic ambiguity include: The set of attributes in different data sources is Where A i For data source D i The set of attributes, for semantically overlapping attribute pairs (a p ,a q ), a p ∈A i a q ∈A j If i < j, assume that i already exists in the fusion ontology O. fusion Attribute a in p For the data source D that was accessed first i a q For the data source D that is being connected i A certain attribute in the code, when semantic overlap is detected, should have an alignment direction of 'a'. q →a p That is, the attribute field in the data is replaced with 'a'. p Remove 'a' from the ontology definition q The set of relationships from different data sources is For relation pairs (r) with semantic overlap p ,r q ), r p ∈R i r q ∈R j , i < j, where r p R represents i There is a semantically overlapping relation in r. q R represents j There is a semantically overlapping relationship in the text, and their alignment and attributes are the same.

5. The intelligent interactive method for scenic areas using a large language model that integrates multi-source information as described in claim 1, characterized in that, The steps of storing the merged ontology in a graph database and mapping the aligned data structure to a relational database include: The fused ontology is stored in the graph database GraphDB. The fused ontology has a global semantic perspective on multiple heterogeneous data sources and can support semantic query and reasoning operations based on SPARQL. The aligned data is structured and mapped according to the classes, attributes and relational models defined by the fusion ontology, and then uniformly converted into a consistent data representation and written into the relational database MySQL.

6. The intelligent interactive method for scenic areas using a large language model that integrates multi-source information as described in claim 1, characterized in that, The steps for intelligent interaction based on fusion ontology and large language model include: Business judgments are made, local ontology is constructed using data sources, and ontology fusion is completed to construct a fused ontology. Intelligent interaction is carried out based on the fused ontology and the large language model. The category of the request is determined by querying the category information of the fused ontology, and all ontology category information is used as input prompt words to assist the large language model in performing intent recognition and service judgment, thereby achieving preliminary screening of the request.

7. The intelligent interactive method for scenic areas using a large language model that integrates multi-source information as described in claim 1, characterized in that, The steps of using ontology link matching to assist large language models in identifying problem types and solution paths include: The system analyzes business problems, performs reasoning and decision-making based on the AI ​​Agent, locates the required business support data, and executes corresponding processing. It provides prompts to the agent through ontology link matching to help it identify problem types and solution paths. With the semantic support of the fused ontology, the agent can initiate a multi-round closed-loop reasoning process of "observation-reasoning-action." This multi-round closed-loop reasoning process dynamically formulates query strategies based on context, and for complex multi-source businesses, it breaks down the overall task into multiple independently executable sub-tasks based on the information source relationships extracted from the fused ontology, gradually completing the query and response, and achieving cross-system collaborative processing and intelligent response.

8. The intelligent interactive method for scenic areas using a large language model that integrates multi-source information, as described in claim 7, is characterized in that... The steps for providing prompts to an agent using the ontology link matching method include: The ontology link matching method aims to identify the starting class and target class in the business problem and build a semantic link connecting the two in the fusion ontology to support the reasoning and prompt generation of LLM. Let the fusion ontology structure be a directed graph: G = (C, R), where C is the set of ontology categories of the system: C = {C1, C2, ..., C} n }, R represents the semantic relationship between classes R={(C i C j )|C i C j ∈C}; The user input question Q is natural language text: Q∈Text, Text is a set of natural language texts; The prompt word template T is as follows: T1(C,Q)=["Please identify the starting class and target class expressed in the following user questions: Existing ontology category information: {C}; User question: {Q}; Return format: Starting class {class S }, target class {class D }”]; The first step is to use the aforementioned prompt word template to enable the LLM model to identify the starting class and target class in the problem: F LLM (T1(C,Q))= <class S ,class D If classS = classD, then the category information is returned directly, and subsequent link matching is not allowed; if classS ≠ classD, then subsequent link matching continues. The second step is to obtain the class. S to class D Find all paths and get the top K paths based on path distance: P = FindPaths(G, class S ,class D ), where each path P i Represented as an ordered sequence: P i = <class S ,class i1 ,…,class D >; Define the path scoring function as Score(P i ) = 1 / length(P i The shorter the path, the higher the score. The top K paths are selected as the candidate path set P. k =TopKf(P,K), then for each path P i ∈P k Generate its natural language description: Di = describe(Pi), Di ∈ Text; The third step is to use LLM to select the most suitable path from K paths: Construct a language model prompt template T(Q,D) = ["Based on question Q, determine the most suitable path among the following: {D1, ...,Dk}"]; Input the prompt into the LLM model to obtain the optimal path: P = F LLM (T(Q,D)).

9. A large-scale language model-based intelligent interactive system for scenic areas that integrates multi-source information, characterized in that: The system includes: The ontology support module, serving as the system's knowledge foundation, provides semantic basis and structural support. It consists of an ontology management component, an ontology storage component, and an ontology query component. This module is responsible for steps S1-S4 in the method, including building, aligning, storing, and providing access to the fused ontology. The ontology management component provides both graphical and code-based interfaces, supporting full lifecycle management of ontology based on the RDF / OWL standard, including class definition, attribute declaration, and semantic relationship construction. The ontology storage component persistently stores ontology knowledge in GraphDB based on the RDF / OWL standard, modeling class hierarchy, attribute relationships, and constraint rules in triple form. The ontology query component provides ontology access interfaces to other modules via the SPARQL protocol, supporting multidimensional semantic retrieval. The service awareness module, as a front-end unit for user interaction, consists of a user interaction component, a service query component, and an LLM discriminant component. It is responsible for executing the business judgment in step S5 of the method. The user interaction component provides a front-end dialogue interface with multimodal input and preprocesses user questions. The LLM discriminant component has a built-in large language model and determines whether the user question belongs to the system service scope based on ontology category information. The service query component queries the ontology library through SPARQL and outputs ontology category information in a structured manner according to the ontology category information description specification. The multi-source query module, based on the RAG architecture, enables heterogeneous data interaction and consists of a task receiving component, an information location component, an SQL generation and execution component, and a multi-source information storage component. The task receiving component receives natural language query requests from the decision-making module and parses them into structured task data. The information location component uses a large language model combined with ontology category information to generate a database schema and locate the data source. The SQL generation and execution component converts the query request into SQL statements based on the schema, executes the query in a MySQL database, and supports error feedback and correction. The multi-source information storage component integrates data management and ontology annotation functions, enabling automatic mapping from the ontology model to the database structure. The decision-making module, based on the AI ​​Agent architecture, enables intelligent decision-making and consists of an information receiving component, a prompt generation component, and an inference decision-making component. It is responsible for task decomposition and multi-round inference in method step S5. The information receiving component receives user questions through a RESTful API and triggers the multi-round inference process. The prompt word generation component obtains semantic links through the ontology query interface using the ontology link matching method and generates LLM prompt words in combination with the question background. The inference decision-making component, based on the "observation-reasoning-action" closed loop, decomposes complex problems into sub-tasks and dynamically calls the multi-source query module until the task is completed.

10. A large language model intelligent interactive system for scenic areas that integrates multi-source information according to claim 9, characterized in that, The multi-source query module has an SQL statement error feedback and reconstruction mechanism. When executing an SQL statement generated by the large language model and connecting to a structured database, if the system detects an anomaly, it immediately triggers the reconstruction process: first, it extracts the error information and constructs contextual hints based on the query intent, and then inputs them back into the large language model to generate a corrected SQL statement; at the same time, it supports a joint verification strategy of syntax hint templates and ontology schema to help generate query statements with semantic consistency and structural rationality.

Images