Multilingual knowledge graph-based school history culture intelligent guide system and method

Abstract

The invention discloses a school history culture intelligent guide system and method based on a multilingual knowledge graph, and relates to the technical field of knowledge graphs and intelligent guide, and the system comprises a multilingual school history knowledge graph construction module which constructs a multilingual ontology model and generates the multilingual knowledge graph through cross-lingual semantic alignment; the edge collaborative perception interaction module is used for collecting interaction data and realizing low-delay response based on an edge-cloud collaborative strategy; the cross-language semantic retrieval and recommendation module is used for calculating semantic similarity and generating personalized recommendation; the knowledge-enhanced multilingual intelligent question and answer module constructs an enhancement prompt and generates a question and answer result; according to the school history culture international communication system and method, the problems of language barriers and single forms in school history culture international communication are solved, and the tour guide service quality is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of knowledge graph and intelligent guide technology, and particularly relates to a school history and culture intelligent guide system and method based on a multilingual knowledge graph. BACKGROUND

[0002] With the acceleration of the internationalization process of higher education, the demand for international dissemination of school history and culture is increasingly urgent. As an important carrier of cultural heritage in colleges and universities, the school history museum undertakes the important responsibility of showing the history of the school, outstanding alumni, major events and precious cultural relics to domestic and foreign visitors. However, the existing school history museum guide system faces significant challenges in international dissemination, mainly in two aspects: language barrier and single content presentation form.

[0003] In the prior art, researchers have proposed an application scheme of knowledge graph technology in the field of museum guide. For example, a Chinese patent application with publication number CN109657068A discloses a method for generating and visualizing cultural relic knowledge graph for smart museums. The method includes the following steps: designing a cultural relic knowledge graph ontology model according to the application requirements of cultural relic guide; obtaining cultural relic data; extracting entity, relationship and attribute information based on the ontology model and cultural relic data using a knowledge extraction tool; representing the extracted information using RDF and other knowledge graph description languages; completing cultural relic knowledge storage; generating a visualized knowledge graph and an interactive interface. This technical solution realizes the structured organization and visualized display of cultural relic information through knowledge graph technology, which to some extent improves the intelligent level of museum guide.

[0004] However, the above-mentioned prior art solution has the following significant shortcomings. First, in terms of language support, the existing solution only supports knowledge representation and query in a single language, and cannot meet the multilingual guide needs of international visitors and international student groups. For visitors from different countries and regions, the existing system cannot provide guide services in their native language environment, which seriously affects the coverage and depth of cultural dissemination. Secondly, in terms of content presentation, the existing solution uses a simple two-dimensional visualization interface based on Python Tkinter, which can only display the node association graph of the knowledge graph, and lacks immersive scene experience capability. Visitors can only passively receive static text information and cannot obtain a sense of being in the historical scene. Thirdly, in terms of interaction mode, the existing solution only supports structured retrieval based on query languages such as SPARQL, and ordinary visitors cannot interact and explore knowledge through natural language. In addition, the existing solution uses a completely cloud-based architecture, which can cause high response delay during periods of high visitor density, affecting user experience.

[0005] In summary, the existing knowledge graph guide technology has obvious deficiencies in multilingual support, immersive experience, intelligent interaction and real-time response, and it is difficult to meet the actual needs of international dissemination of school history and culture. Therefore, an intelligent guide technology scheme for school history and culture is urgently needed, which can break through the language barrier, provide an immersive experience, support intelligent questioning and answer, and has a low delay response capability. SUMMARY

[0006] In view of the above technical problems existing in the prior art, the present application provides a school history and culture intelligent guide system and method based on a multilingual knowledge graph, aiming to solve the technical problems of the existing guide system, such as prominent language barrier, single content presentation form, limited interaction mode and high response delay.

[0007] To solve the above technical problems, the present application adopts the following technical solutions:

[0008] The school history and culture intelligent guide system based on a multilingual knowledge graph comprises a multilingual school history knowledge graph construction module, an edge collaborative perception interaction module, a cross-lingual semantic retrieval and recommendation module, a knowledge-enhanced multilingual intelligent question-answering module and an AR virtual scene rendering and display module.

[0009] The multilingual school history knowledge graph construction module constructs a multilingual ontology model for the field of school history and culture, which includes five types of entities, i.e. school history figure entities, school history event entities, school history artifact entities, school history building entities and historical period entities, and the associated relationships between the entities. Multilingual knowledge extraction is performed on the collected school history text data to obtain multilingual triple data. The entities and relationships in the multilingual triple data are respectively vector-encoded to construct a multilingual semantic embedding space. The semantic space of synonymous entities in different languages is aligned through a cross-lingual semantic alignment loss function, and finally a multilingual school history knowledge graph is generated.

[0010] The edge collaborative perception interaction module sets near-field communication tags and micro voice trigger units at the display location of exhibits, and collects interaction behavior data when detecting that a visitor has entered a preset perception range. The interaction behavior data is preprocessed at the edge, and the language preference features of the visitor are extracted. Based on the edge-cloud collaborative strategy, the frequently accessed knowledge sub-graphs are cached to the edge node, and the query requests with a response delay lower than a preset delay threshold are directly processed by the edge node.

[0011] The cross-lingual semantic retrieval and recommendation module receives the query request of the visitor, encodes the query text into the multilingual semantic embedding space, calculates the semantic similarity between the query vector and the entity vector in the multilingual school history knowledge graph. According to the semantic similarity and the historical interaction data of the visitor, a recommended content list is generated through a personalized recommendation algorithm. The recommended content is automatically adapted to the target language according to the language preference features of the visitor.

[0012] The knowledge-enhanced multilingual intelligent question-answering module receives natural language questions from visitors and retrieves relevant knowledge triples from a multilingual school history knowledge graph. The retrieved knowledge triples are converted into prompt text and combined with the natural language question to construct enhanced prompts. These enhanced prompts are then input into a large language model to generate question-answering results that match the visitor's language preferences.

[0013] The AR virtual scene rendering and display module loads the corresponding 3D scene model from a pre-stored library of virtual school history scenes based on the visitor's current location and viewpoint. The 3D scene model is then overlaid with images of physical exhibits using augmented reality rendering to generate an immersive school history scene. Multilingual text annotations and audio narration are then overlaid onto the rendered virtual scene.

[0014] This invention also provides a method for intelligent navigation of school history and culture based on multilingual knowledge graphs, wherein each step of the method corresponds one-to-one with each module of the above system.

[0015] The beneficial effects of this invention are as follows:

[0016] First, by constructing a multilingual knowledge graph of the university's history and employing cross-lingual semantic alignment technology, a unified representation and cross-lingual retrieval of knowledge in different languages ​​were achieved, effectively solving the language barrier problem in the international dissemination of the university's history and culture. Tests show that the accuracy rate of cross-lingual retrieval can reach over 85%, significantly improving the guided tour experience for international visitors.

[0017] Secondly, through the edge-cloud collaborative architecture design, the knowledge subgraph accessed frequently is cached to the edge nodes, and the response latency of hot queries can be reduced to less than 500 milliseconds, which is 60%-70% lower than that of a pure cloud architecture, effectively solving the response latency problem during peak visitor periods.

[0018] Third, by integrating knowledge graphs with large language models, knowledge-enhanced question-answering technology enables visitors to explore in-depth knowledge using natural language. The accuracy of question-answering is 15%-25% higher than that of pure large model solutions, while effectively reducing the occurrence of large model illusion problems.

[0019] Fourth, through AR virtual scene rendering technology, an immersive reproduction of historical scenes was achieved, allowing visitors to intuitively experience the historical background and atmosphere of the school's history and culture, resulting in a user satisfaction rating increase of over 40%. Attached Figure Description

[0020] Fig. 1 This is an architecture diagram of the intelligent campus history and culture guide system based on multilingual knowledge graphs provided in this embodiment of the invention;

[0021] Fig. 2This is a flowchart of the intelligent navigation method for school history and culture based on multilingual knowledge graphs provided in an embodiment of the present invention. Detailed Implementation

[0022] Please refer to the attached document. Figs. 1-2 The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See Fig. 1 This invention provides an intelligent campus history and culture navigation system based on a multilingual knowledge graph. The system comprises five core modules: a multilingual campus history knowledge graph construction module, an edge-collaborative perception and interaction module, a cross-lingual semantic retrieval and recommendation module, a knowledge-enhanced multilingual intelligent question-answering module, and an AR virtual scene rendering and display module. These five modules form a deeply coupled collaborative relationship; the output of one module serves as the key input to the next, and the feedback from subsequent modules can inversely influence the parameter adjustments of preceding modules, forming a closed-loop optimization mechanism.

[0024] The multilingual school history knowledge graph construction module is the knowledge foundation layer of the entire system, responsible for building a school history and cultural knowledge graph that supports multilingual queries and retrieval. The technical implementation of this module includes four core sub-processes: multilingual ontology model design, multilingual knowledge extraction, vector encoding, and cross-lingual semantic alignment.

[0025] In terms of multilingual ontology model design, this invention, tailored to the characteristics of the school history and culture field, designs an ontology model comprising five core entities. The first category is historical figures, encompassing past presidents, renowned professors, distinguished alumni, and important historical figures, with attributes including name, birth and death dates, positions, major contributions, and related events. The second category is historical events, covering historical events such as the school's founding process, major reforms, academic achievements, and social contributions, with attributes including event name, date of occurrence, participants, impact, and related documents. The third category is historical artifacts, covering exhibits such as historical archives, valuable documents, commemorative items, and teaching equipment, with attributes including artifact name, era, material, size, origin, and cultural value. The fourth category is historical buildings, covering historical buildings, landmark sites, and demolished building ruins, with attributes including building name, construction date, architectural style, designer, and historical significance. The fifth category is historical periods, used to identify different historical stages, such as the founding period, development period, war period, and revival period, with attributes including period name, start and end years, and period characteristics.

[0026] Regarding the design of relationships between entities, this invention defines the following core relationship types: Person-Event relationships include participation, initiation, and witnessing. Person-Person relationships include teacher-student, colleague, and successor relationships. Event-Cultural Relic relationships include creation, recording, and commemoration. Event-Period relationships include belonging and symbolism. Architecture-Period relationships include construction date and continued existence date. Through the above definitions of entities and relationships, an ontology framework capable of comprehensively expressing the school's historical and cultural knowledge is constructed.

[0027] Preferably, each type of entity and relation has multilingual attribute fields. Taking Chinese, English, Japanese, Korean, French, German, Spanish, and Russian as examples, the name and description attributes of each entity include versions in all eight languages. Relationship tags also support multilingual representations. This design enables the knowledge graph to natively support multilingual knowledge storage and retrieval.

[0028] In terms of multilingual knowledge extraction, this invention employs a combination of word segmentation tools and regular expressions to extract structured knowledge from historical text data. For Chinese text, a deep learning-based Chinese named entity recognition model is used to extract personal names, place names, organization names, and time expressions. For English and other Western language text, a pre-trained multilingual BERT model is used for entity recognition and relation extraction. Coreference resolution is performed simultaneously during the extraction process, unifying different representations of the same entity into a standard entity identifier.

[0029] In terms of vector encoding, this invention employs a multilingual pre-trained language model to encode entities and relations. Specifically, for each entity, its name and description text are concatenated and input into a multilingual encoder, which outputs a fixed-dimensional semantic embedding vector. In this embodiment, the semantic embedding vector is set to 768 dimensions. For each relation, its label text is also encoded into a semantic embedding vector. Through vector encoding, discrete symbolic knowledge is transformed into a continuous vector representation, laying the foundation for subsequent semantic similarity calculation.

[0030] Regarding cross-linguistic semantic alignment, this invention designs a cross-linguistic semantic alignment loss function to map synonymous entities from different languages ​​to similar positions in the semantic space. The loss function is calculated as follows:

[0031] ,

[0032] in, This represents the cross-language semantic alignment loss value. This is a predefined set of cross-language synonymous entity pairs, obtained through manual annotation or cross-language entity alignment algorithms; For synonymous entity pairs, For the source language entity, For the corresponding entity in the target language; , They are respectively , The corresponding semantic embedding vector; This is a vector of negative instance entities randomly sampled from non-synonymous entities; This is the Euclidean distance function, used to measure the distance between vectors; The interval hyperparameter is used to control the distance difference between positive and negative pairs. Its value ranges from 1.0 to 2.0, with a preferred value of 1.5.

[0033] By minimizing the aforementioned loss function, the semantic embedding vectors of entities are optimized, ensuring that the vector representations of synonymous entities are close in semantic space, while the vector representations of non-synonymous entities are far apart. After multiple rounds of iterative training, knowledge from different languages ​​can be aligned in a unified semantic space, thereby supporting cross-language semantic retrieval. In this embodiment, the number of training iterations is set to 100 rounds, the learning rate is set to 0.001, and the Adam optimizer is used for parameter updates.

[0034] The edge-collaborative perception and interaction module is the system's perception layer, responsible for collecting visitor interaction behavior data and extracting language preference features. It also achieves low-latency query responses through an edge-cloud collaborative strategy. The module's technical implementation includes four core sub-processes: perception device deployment, interaction data acquisition, language preference extraction, and edge-cloud collaborative strategy.

[0035] Regarding the deployment of sensing devices, this invention places near-field communication (NFC) tags and miniature voice trigger units at the deployment location of each exhibit. The NFC tags utilize NFC or Bluetooth Low Energy technology; when a visitor's smart device enters the preset sensing range, a communication connection is automatically established, triggering the push of guided tour content. The preset sensing range is differentiated according to the type of exhibit and the layout of the exhibition hall; the sensing range for small exhibits is set to 0.5-1.0 meters, while the sensing range for large exhibits or architectural models is set to 1.5-3.0 meters. The miniature voice trigger unit integrates a microphone array and a local voice activity detection module; when a visitor's voice input is detected, it triggers voice acquisition and subsequent voice recognition processing.

[0036] In terms of interactive data collection, the edge collaborative perception interaction module collects multimodal interaction data from visitors in real time. The collected data types include: exhibit proximity event data, recording the timestamps and duration of visitor approach to each exhibit; touch interaction data, recording visitors' click, swipe, and zoom operations on the guide terminal; voice input data, recording visitors' voice queries and Q&A content; and text input data, recording visitors' text queries entered via the virtual keyboard. This data is first preprocessed and feature extracted at the edge nodes, and then uploaded to the cloud server for aggregation and analysis as needed.

[0037] In terms of language preference extraction, this invention employs a multimodal fusion method to determine visitors' language preferences. Specifically, for voice input data, a lightweight language recognition model deployed on edge nodes performs real-time language detection. This model determines the language category of the voice based on acoustic features. For text input data, a text language classifier based on character-level statistical features determines the language. If the language categories of the voice and text data match, that language category is directly used as the visitor's language preference. If they do not match, the language category of the most recent interaction is used as the current language preference, and the history of language preference changes is recorded for subsequent personalized services.

[0038] Regarding the edge-cloud collaboration strategy, this invention designs a dynamic caching mechanism based on access frequency, caching frequently accessed knowledge subgraphs to edge nodes to reduce query response latency. The implementation process of this strategy is as follows: First, the system continuously counts the access frequency of each knowledge subgraph for an hourly period. Knowledge subgraphs with access frequencies exceeding a preset threshold (more than 50 accesses per hour) are marked as hotspot subgraphs. Second, the marked hotspot knowledge subgraphs are synchronized from the cloud to the local cache on the edge nodes. The cache update cycle is set to 5-30 minutes, with the specific value dynamically adjusted based on the knowledge update frequency and the storage capacity of the edge nodes. Preferably, the cache update cycle is set to 5 minutes during peak access periods and 30 minutes during off-peak periods.

[0039] When an edge node receives a query request, it first calculates the expected response latency. If the expected response latency is less than a preset latency threshold of 500 milliseconds, and the knowledge subgraph involved in the query is already cached in the edge node's local cache, the edge node directly executes the query and returns the result without accessing the cloud server. If the expected response latency is greater than the preset latency threshold, or the knowledge subgraph involved in the query is not in the local cache, the query request is forwarded to the cloud server for processing, and the query result is returned to the visitor while updating the edge node's access statistics.

[0040] Through the aforementioned edge-cloud collaboration strategy, this invention enables localized processing of hot-topic queries, effectively reducing visitor waiting time. Test data shows that during peak visitor periods, approximately 65%-75% of query requests can be processed directly by edge nodes, reducing the average response latency from 1500-2000 milliseconds in a pure cloud architecture to 300-500 milliseconds, an improvement of 60%-75%.

[0041] The cross-language semantic retrieval and recommendation module is the core service layer of the system, responsible for responding to visitors' query requests and providing personalized content recommendations. The technical implementation of this module includes four core sub-processes: query understanding, semantic similarity calculation, personalized recommendation, and multilingual adaptation.

[0042] In terms of query understanding, upon receiving a visitor's query request, the query text is first preprocessed, including word segmentation, stop word removal, and part-of-speech tagging. Then, the preprocessed query text is input into a multilingual encoder to generate a semantic embedding vector for the query text. This vector resides in the same semantic space as the entity vectors in the multilingual school history knowledge graph, supporting cross-language semantic matching.

[0043] In terms of semantic similarity calculation, this invention designs a weighted similarity calculation method that integrates language matching and entity type. The specific calculation formula is as follows:

[0044] ,

[0045] in, For query With entity semantic similarity; The semantic embedding vector for the query text is generated by a multilingual encoder; The semantic embedding vector of the entity is pre-calculated and stored during the knowledge graph construction stage; the cosine similarity is the product of the vector dot product and the vector magnitude. The language matching weight is 1.0 when the query language matches the main language of the entity, and 0.7-0.9 when there is a cross-language alignment relationship between the query language and the entity. The specific value is dynamically adjusted according to the alignment quality. This is the entity type weight, ranging from 0.5 to 1.0, depending on the degree of match between the query intent and the entity type. For example, when the query intent is to find people and the candidate entity is of type "people", The value is 1.0; when the candidate entity is an event type, The value is 0.7.

[0046] In terms of personalized recommendations, this invention employs a recommendation algorithm that fuses collaborative filtering with knowledge graph embedding. This algorithm comprehensively considers user historical behavior, knowledge graph structure information, and the current context to generate personalized recommendations. The formula for calculating the recommendation score is as follows:

[0047] ,

[0048] in, For users For entities Recommended score; The similarity score is calculated by analyzing the preferences of other users with similar interaction history to the current user, based on collaborative filtering. The relevance score based on knowledge graph paths is obtained by calculating the path association strength between the entities visited by the user and the candidate entities in the knowledge graph. To determine the matching score based on the current context, contextual factors such as the user's current location, viewed content, and dwell time are taken into account. , , Let be the weighting coefficient, satisfying The default values ​​are 0.4, 0.35, and 0.25. The weighting coefficients can be customized based on user type; for example, for first-time users, a higher weighting coefficient can be applied. The weights are used to enhance knowledge structure-based recommendations.

[0049] Regarding multilingual adaptation, each item in the recommendation list is automatically adapted to the target language based on the visitor's language preferences. Specifically, entity names, descriptions, and related information for the corresponding language version are extracted from the knowledge graph and assembled into recommended content for the target language. If an item lacks corresponding content in the target language, a machine translation service is invoked for real-time translation, and the translation results are cached for later use.

[0050] The knowledge-enhanced multilingual intelligent question-answering module is the system's intelligent interaction layer, responsible for responding to visitors' natural language questions and generating accurate question-and-answer results. The module's technical implementation includes four core sub-processes: question understanding, knowledge retrieval, prompt construction, and answer generation.

[0051] Regarding question understanding, upon receiving a visitor's natural language question, the system first identifies the question type: is it a factual, explanatory, or inference question? Factual questions typically inquire about specific objective information such as time, place, and people. Explanatory questions usually ask about the cause, background, or significance of an event or phenomenon. Inference questions typically require integrating multiple knowledge points to make inferences. The result of question type identification will influence the subsequent knowledge retrieval strategy and the style of answer generation.

[0052] In terms of knowledge retrieval, this invention retrieves knowledge triples related to the question from a multilingual school history knowledge graph. The specific retrieval process is as follows: First, the question text is encoded into a semantic embedding vector. Then, the semantic similarity between the question vector and the entity vectors in the knowledge graph is calculated. Next, entities with a semantic similarity greater than a preset similarity threshold of 0.7 are selected as seed entities. Finally, a breadth-first traversal is performed in the knowledge graph centered on the seed entities to extract triples directly associated with the seed entities, with a traversal depth of 2 hops. Based on the relevance of the triples to the question, the top K triples are selected as the retrieval results, where K ranges from 3 to 10, with a default value of 5.

[0053] In terms of prompt construction, this invention converts the retrieved knowledge triples into natural language description text and combines them with user questions to construct enhanced prompts. The specific steps of prompt construction are as follows: First, each triple is converted into a declarative sentence according to the subject-verb-object format. For example, the triple (Li Dazhao, served as, Director of Peking University Library) is converted into Li Dazhao once served as Director of Peking University Library. Then, multiple declarative sentences are organized into background knowledge paragraphs according to chronological order or logical relationship. Finally, the background knowledge paragraph, user question paragraph, and output format constraint paragraph are combined into a complete enhanced prompt according to a preset prompt template. The structure of the preset prompt template is as follows: The background knowledge paragraph begins with "Based on the following school history materials:" and lists the retrieved knowledge descriptions; the question paragraph begins with "Please answer the following question:" and gives the user's original question; the output format constraint paragraph requires the model to output in a specified language and keep the answer concise and accurate.

[0054] In terms of answer generation, the constructed enhanced prompts are input into a large language model to generate question-and-answer results. The large language model used in this invention supports multilingual output and can generate answers in the corresponding language based on the language constraints in the prompts. Compared to directly using a large language model to answer questions, the knowledge augmentation method has the following advantages: First, by injecting verified school history knowledge, the illusion problem of the large model is effectively reduced, improving the accuracy of the answers. Second, by limiting the knowledge source, the answer content is ensured to be consistent with the official information of the school history museum, avoiding inaccurate or misleading content. Third, by explicitly retrieving the knowledge graph, the knowledge source of the answer can be traced, enhancing the interpretability of the answer. Test data shows that after adopting knowledge augmentation, the question-and-answer accuracy rate increased from 70%-75% with the pure large model to 85%-90%, an increase of 15%-25%.

[0055] The AR virtual scene rendering and display module is the system's display layer, responsible for presenting the school's history and culture to visitors in an immersive way. The module's technical implementation includes four core sub-processes: pose estimation, scene loading, augmented reality rendering, and multilingual annotation.

[0056] In terms of pose estimation, this invention determines the visitor's six-degree-of-freedom pose information through a visual positioning algorithm. The six-degree-of-freedom pose includes three-dimensional position coordinates and three-dimensional rotation angles, representing the visitor's position and orientation within the exhibition space, respectively. Specifically, it employs visual-inertial odometry (VIO), fusing camera image sequences and inertial measurement unit (IMU) data to estimate the visitor's pose in real time. To improve positioning accuracy, visual markers are pre-deployed in the exhibition hall. When a marker enters the camera's field of view, pose correction is performed through marker recognition, achieving centimeter-level positioning accuracy.

[0057] In terms of scene loading, the corresponding 3D scene model is loaded from a pre-stored virtual scene library of school history based on the visitor's pose information. This virtual scene library contains reconstructed campus scenes from multiple historical periods, such as the campus appearance in its early days, the westward relocation during wartime, and the development after the reform and opening up. The scene models employ a block-based loading strategy, prioritizing the loading of scene blocks within the visitor's field of view based on their current position and field of vision. Scene blocks outside the field of vision are loaded using low-precision models or with delayed loading to ensure smooth rendering.

[0058] In augmented reality rendering, this invention employs a deep learning-based image segmentation algorithm to identify the boundary regions of physical exhibits, and then overlays a 3D scene model onto the background region outside the boundary regions. The specific rendering process is as follows: First, the exhibit regions in the camera image are identified using a semantic segmentation network, generating an exhibit mask. Then, the projection position and transformation matrix of the virtual scene model in the image are calculated based on the visitor's pose. Next, the virtual scene is rendered in the region outside the exhibit mask, while maintaining the original image display of the exhibit region. Finally, a fusion process is performed at the boundary between the virtual scene and the real image, using a feathering algorithm to achieve a smooth transition and avoid obvious seams.

[0059] For multilingual annotation, multilingual text annotations and audio narration are overlaid on the rendered virtual scene. Text annotations include descriptions of key elements in the scene, historical background information, and identification of relevant figures. The annotation content is obtained from a knowledge graph, and the languages ​​are consistent with the visitor's language preferences. Simultaneously, the system provides synchronized audio narration that is generated by a multilingual speech synthesis module. Visitors can control the playback, pause, and skip to the next section of the narration using voice commands or gestures.

[0060] To ensure real-time AR rendering, this invention offloads some rendering computation tasks to edge nodes. Specifically, image segmentation and scene model projection calculations are performed on the GPU of the edge nodes, and the rendering results are transmitted as a video stream to the visitor's mobile terminal or AR glasses for display. Through edge rendering, even visitors using mobile devices with limited computing power can enjoy a smooth AR experience. Test data shows that the AR rendering frame rate can be stably maintained above 30 frames per second, and the end-to-end latency is controlled within 100 milliseconds, meeting the real-time requirements of an immersive experience.

[0061] The five modules described above form a tightly coupled and collaborative relationship. The knowledge graph generated by the multilingual school history knowledge graph construction module serves as the data foundation for all subsequent modules. The interaction data and extracted language preference features collected by the edge collaborative perception interaction module serve as input parameters for the cross-lingual semantic retrieval and recommendation module and the knowledge-enhanced multilingual intelligent question-answering module. The recommendation results from the cross-lingual semantic retrieval and recommendation module and the question-answering results from the knowledge-enhanced multilingual intelligent question-answering module are both passed to the AR virtual scene rendering and display module for visualization. Simultaneously, the content presented by the AR module and the interactive feedback from visitors are fed back to the edge collaborative perception interaction module for recording and analysis, used to update the access popularity statistics of the knowledge graph and optimize the parameters of the recommendation algorithm.

[0062] Through the aforementioned closed-loop collaborative mechanism, the system can continuously learn visitors' behavioral patterns and preferences, thereby constantly optimizing the quality and personalization of the guided tour service.

[0063] See Fig. 2 This invention also provides an intelligent navigation method for school history and culture based on a multilingual knowledge graph, wherein each step of the method corresponds one-to-one with each module in the above system embodiment. The method includes the following steps:

[0064] Step S1: Construct a multilingual ontology model for the field of school history and culture, perform multilingual knowledge extraction on the collected school history text data to obtain multilingual triple data, perform vector encoding on the entities and relations in the multilingual triple data, and perform semantic space alignment on synonymous entities in different languages ​​through cross-language semantic alignment loss function to generate a multilingual school history knowledge graph.

[0065] The design method of the multilingual ontology model, the implementation method of multilingual knowledge extraction, the specific process of vector encoding, and the calculation method of the cross-lingual semantic alignment loss function are all consistent with the description of the multilingual school history knowledge graph construction module in the above system embodiment. Through step S1, a multilingual school history knowledge graph supporting eight languages ​​is generated, providing a knowledge foundation for subsequent intelligent navigation services.

[0066] Step S2: Set up near-field communication tags and miniature voice triggering units at the exhibit deployment locations, collect visitor interaction behavior data, extract visitor language preference features, and cache the frequently accessed knowledge subgraph to edge nodes based on the edge-cloud collaborative strategy.

[0067] The deployment method of the sensing devices, the content of the collected interactive data, the method of extracting language preferences, and the implementation details of the edge-cloud collaboration strategy are all consistent with the description of the edge collaborative sensing interaction module in the above system embodiment. Through step S2, a perceptual connection is established between the visitor and the exhibits, and an edge computing infrastructure supporting low-latency response is constructed.

[0068] Step S3: Receive the visitor's query request, encode the query text into a multilingual semantic embedding space, calculate the semantic similarity between the query vector and the entity vector in the knowledge graph, and generate recommended content that is automatically adapted according to the visitor's language preferences through a personalized recommendation algorithm.

[0069] The formulas for calculating semantic similarity and the scoring formulas for the personalized recommendation algorithm are consistent with the description of the cross-language semantic retrieval and recommendation module in the above system embodiment. Through step S3, cross-language semantic retrieval and personalized recommendations are achieved, providing visitors with content tailored to their language preferences and interests.

[0070] Step S4: Receive the visitor's natural language question, retrieve relevant knowledge triples from the multilingual school history knowledge graph, construct knowledge-enhanced prompts and input them into the large language model to generate question-and-answer results that match the visitor's language preferences.

[0071] The knowledge retrieval method, the construction method of enhanced prompts, and the technical details of answer generation are all consistent with the description of the knowledge-enhanced multilingual intelligent question-answering module in the above system embodiment. Through step S4, knowledge graph-based enhanced intelligent question answering is realized, enabling visitors to conduct in-depth knowledge exploration using natural language.

[0072] Step S5: Load the 3D scene model according to the visitor's position and perspective, perform augmented reality overlay rendering with the physical exhibits, generate an immersive school history scene, and overlay multilingual annotation content.

[0073] The pose estimation method, scene loading strategy, augmented reality rendering process, and multilingual annotation implementation are all consistent with the description of the AR virtual scene rendering and display module in the above system embodiment. Step S5 achieves an immersive reproduction of historical scenes, allowing visitors to experience the school's history and culture firsthand.

[0074] The steps S1 to S5 described above form a complete intelligent tour guide service process. In practical applications, step S1 is typically executed once during the system deployment phase or periodically to update the knowledge graph. Steps S2 to S5 are continuously executed during a visitor's tour of the school history museum, responding to various interactive requests from visitors. The collaborative relationships between the steps are consistent with the collaborative relationships between the modules in the above system embodiment.

[0075] To verify the technical effectiveness of this invention, a three-month deployment and test were conducted at a university history museum. The test environment included: approximately 200 exhibits covering four categories: historical figures, events, artifacts, and architecture; a knowledge graph containing approximately 15,000 entities and 50,000 triples, supporting four languages: Chinese, English, Japanese, and Korean; edge nodes deployed at six key locations in the exhibition hall; and approximately 3,000 visitors, including domestic tourists, international students, and international visitors, participating in the test.

[0076] Test results show that: First, in terms of cross-language retrieval accuracy, the accuracy rate for Chinese queries retrieving Chinese content is 92%, for English queries retrieving English content is 89%, and for cross-language queries (such as English queries retrieving Chinese content and then translating it) is 85%, all meeting the expected targets. Second, in terms of response latency, the average response latency for popular queries is 380 milliseconds, and for less popular queries it is 1200 milliseconds, with the overall average response latency reduced by approximately 62% compared to a pure cloud architecture. Third, in terms of question-answering accuracy, the accuracy rate after knowledge augmentation is 88%, an improvement of approximately 18% compared to a pure large model solution. Fourth, in terms of user satisfaction, the average user satisfaction rating collected through questionnaires is 4.3 out of 5, an improvement of approximately 45% compared to traditional navigation systems.

[0077] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A school history and culture intelligent guide system based on a multilingual knowledge graph, characterized in that, The method comprises the following steps: A multilingual school history knowledge graph construction module is used to construct a multilingual ontology model for the school history culture field, and the multilingual ontology model comprises five types of entities, i.e., school history figure entities, school history event entities, school history cultural relic entities, school history building entities and historical period entities, and the correlation between the entities; multilingual knowledge extraction is performed on collected school history text data to obtain multilingual triple data; The entities and relationships in the multilingual triple data are respectively vector-encoded to construct a multilingual semantic embedding space, the semantic spaces of synonymous entities in different languages are aligned through a cross-language semantic alignment loss function, and a multilingual school history knowledge graph is generated; An edge collaborative perception interaction module is used to set a near-field communication tag and a miniature voice trigger unit at a display location, and when it is detected that a visitor enters a preset perception range, the interaction behavior data of the visitor is collected; The interaction behavior data is edge-pretreated to extract language preference features of the visitor; Based on an edge-cloud collaborative strategy, knowledge subgraphs with high-frequency access are cached to edge nodes, and query requests with a response delay lower than a preset delay threshold are directly processed by the edge nodes; A cross-language semantic retrieval and recommendation module is used to receive a query request of a visitor, encode the query text to the multilingual semantic embedding space, calculate the semantic similarity between the query vector and the entity vectors in the multilingual school history knowledge graph, generate a recommended content list through an individualized recommendation algorithm according to the semantic similarity and the historical interaction data of the visitor, and the recommended content is automatically adapted to a target language according to the language preference features of the visitor; A knowledge-enhanced multilingual intelligent question-answering module is used to receive a natural language question of a visitor, retrieve knowledge triples related to the question from the multilingual school history knowledge graph, convert the retrieved knowledge triples into prompt texts, combine the prompt texts with the natural language question to construct enhanced prompts, input the enhanced prompts into a large language model, and generate a question-answering result conforming to the language preference features of the visitor. In the multilingual school history knowledge graph construction module, the cross-language semantic alignment loss function is calculated in the following manner: In the edge collaborative perception interaction module, the edge-cloud collaborative strategy is as follows:

2. The system of claim 1, wherein, According to the access frequency statistical value of the knowledge subgraph, a knowledge subgraph with an access frequency greater than a preset frequency threshold is marked as a hot subgraph, the preset frequency threshold is an access frequency greater than 50 times per hour, the hot subgraph is cached to an edge node, and the cache update period is 5-30 minutes; , wherein, is a cross-lingual semantic alignment loss value; is a predefined set of cross-lingual synonym entity pairs; is a synonym entity pair, is a source language entity, is a target language entity; , are respectively , corresponding semantic embedding vectors; is a randomly sampled negative example entity vector; is an Euclidean distance function; is a margin hyper-parameter, with a value range of 1.0-2.

0.

3. The system of claim 1, wherein, When a query request is received, the expected response delay of the query request is calculated; if the expected response delay is less than a preset delay threshold of 500 milliseconds and the knowledge subgraph involved in the query has been cached to the edge node, the edge node directly returns the query result; otherwise, the query request is forwarded to a cloud server for processing. In the cross-language semantic retrieval and recommendation module, the semantic similarity is calculated in the following manner: In the cross-language semantic retrieval and recommendation module, the individualized recommendation algorithm adopts a collaborative filtering and knowledge graph embedding fusion manner, and the recommendation score is calculated in the following manner:

4. The system of claim 1, wherein, ​ ， wherein, is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to the entity; is a semantic similarity of the query to 5. The system of claim 1, wherein, ​ ， wherein, is a user recommendation score for an entity ; is a similarity score based on collaborative filtering; is a relevance score based on knowledge graph path; is a matching score based on current context; , , is a weight coefficient, satisfying , and the default values are 0.4, 0.35, and 0.25, respectively.

6. The system of claim 1, wherein, The knowledge-enhanced multilingual intelligent question-answering module, the construction mode of the enhanced prompt is: Retrieving knowledge triples with a semantic similarity greater than a preset similarity threshold 0.7 from the multilingual school history knowledge graph, selecting the top K triples in terms of similarity, and K is in the range of 3-10; Converting the selected knowledge triples into natural language description texts in the subject-predicate-object format; Combining the natural language description texts as background knowledge with user questions according to a preset prompt template, the preset prompt template including a background knowledge section, a question section, and an output format constraint section.

7. The system of claim 1, wherein, It also includes an AR virtual scene rendering and display module for loading a corresponding three-dimensional scene model from a pre-stored school history virtual scene library according to the current location and viewing angle information of the visitor; performing augmented reality superimposition rendering on the three-dimensional scene model and the entity exhibit image to generate an immersive school history scene; and superimposing multilingual text annotations and voice commentary content in the rendered virtual scene; In the AR virtual scene rendering and display module, the processing flow of the augmented reality superimposition rendering is: Determining the six-degree-of-freedom pose information of the visitor through a visual positioning algorithm, the six-degree-of-freedom pose information including three-dimensional position coordinates and three-dimensional rotation angles; Loading a three-dimensional scene model corresponding to the viewing angle from the school history virtual scene library according to the six-degree-of-freedom pose information; Using a deep learning-based image segmentation algorithm to identify the boundary area of the entity exhibit; Superimposing the three-dimensional scene model in the background area outside the boundary area to maintain the original visual effect of the entity exhibit.

8. The system of claim 1, wherein, The system also includes: A multilingual speech synthesis module for converting the question-answering results and the recommended content into voice signals, the voice signals being in a target language consistent with the language preference characteristics of the visitor; and supporting languages including Chinese, English, Japanese, Korean, French, German, Spanish, and Russian.

9. The system of claim 1, wherein, In the edge collaborative perception interaction module, the extraction method of the visitor language preference characteristics is: Collecting voice input data of the visitor and determining the language category of the voice data through a language recognition model; Collecting text input data of the visitor and determining the language category of the text data through a text language classifier; If the language categories of the voice data and the text data are consistent, the language category is taken as the language preference of the visitor; If not, the language category of the last interaction is taken as the language preference of the visitor.

10. A school history and culture intelligent guide method based on a multilingual knowledge graph, adopting the system of any one of claims 1-9, characterized in that, It includes: Step S1: constructing a multilingual ontology model for the school history and culture field, performing multilingual knowledge extraction on the collected school history text data to obtain multilingual triple data, vector encoding entities and relationships in the multilingual triple data, aligning the semantic spaces of synonymous entities in different languages through a cross-language semantic alignment loss function, and generating a multilingual school history knowledge graph; Step S2: setting near-field communication tags and miniature voice trigger units at exhibit deployment locations, collecting visitor interaction behavior data, extracting visitor language preference characteristics, and caching frequently accessed knowledge subgraphs to edge nodes based on an edge-cloud collaborative strategy; Step S3: Receive the query request of the visitor, encode the query text into a multilingual semantic embedding space, calculate the semantic similarity between the query vector and the entity vector in the knowledge graph, and generate recommended content automatically adapted to the visitor's language preference through a personalized recommendation algorithm; Step S4: Receive the visitor's natural language question, search for related knowledge triples in the multilingual school history knowledge graph, construct a knowledge-enhanced prompt and input it into a large language model, and generate a question and answer result that meets the visitor's language preference; Step S5: Load the three-dimensional scene model according to the visitor's location and perspective, and perform augmented reality superposition rendering with the entity exhibits to generate an immersive school history scene and superimpose multilingual labeled content.

Images