Augmented reality dynamic narrative guide method and system

Abstract

The invention provides an augmented reality dynamic narrative guide method and system. The method comprises the following steps: constructing a story knowledge graph containing entities, relationships and attributes from cultural data through a natural language processing technology; collecting multi-dimensional behavior data such as eye movement, position and voice of a user in real time, and generating a user state vector; executing heuristic path search in the atlas based on a user state, and planning a personalized narrative path; calling a large language model to generate style-adaptive explanation content; the multimedia content is presented to the user through the augmented reality terminal device, closed-loop feedback is formed, specifically, the problems that an existing digital navigation technology is static in content, lack of individuation and poor in interactivity are solved, real intelligent navigation of thousands of people and thousands of faces is achieved, the depth and the satisfaction degree of the visit experience of the travel are greatly improved, and the user experience is improved. The method is suitable for various travel scenes such as museums, historic building groups, archaeological sites and non-abandoned inheritance bases.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of artificial intelligence and extended reality, specifically relating to an extended reality dynamic narrative navigation method and system. Background Technology

[0002] With the digital transformation of the cultural and tourism industry, tourists increasingly expect more personalized, immersive, and interactive experiences. Existing digital tour guide technologies, such as mobile apps, QR code scanning for explanations, or rudimentary AR information overlays, have significant limitations: 1. Homogeneous content: All tourists receive the same explanations, which cannot be adjusted according to their individual interests and knowledge backgrounds, resulting in a superficial experience. 2. One-way interaction: The guided tour content is pre-set and recorded, and tourists can only passively receive it, unable to have meaningful interaction with the content or ask follow-up questions. 3. Context separation: The system cannot perceive the real-time behavior and focus of attention of visitors during the visit. For example, when a visitor stands and stares at an exhibit for a long time, the system cannot provide a more in-depth explanation. These shortcomings make it difficult for existing technological solutions to meet the high-level cultural tourism needs of in-depth tours and study tours, and they cannot truly achieve intelligent and personalized guided tours tailored to each individual. Therefore, there is an urgent need for a new technological solution that can sense the user's status in real time and dynamically generate narrative content. Summary of the Invention

[0003] Based on this, the present invention provides an extended reality dynamic narrative tour guide method and system, which aims to solve the technical problems of static content, lack of personalization, and poor interactivity in existing digital tour guide technologies.

[0004] A first aspect of this invention provides an extended reality dynamic narrative navigation method, applied in a scenario with an extended reality terminal device, the method comprising: By using natural language processing technology, entities, narrative relationships between entities, and emotional style tags are extracted from unstructured cultural data to construct a cultural story knowledge graph containing a set of entity nodes, a set of relationship edges, and a set of attribute tags. Real-time collection of multi-dimensional user behavior data, including eye-tracking data, movement data, voice data, and gesture data, to generate a multi-dimensional state vector describing the user's current state; Based on the multidimensional state vector, a heuristic path search algorithm is executed in the cultural story knowledge graph to plan the narrative path that best matches the user's current interests and behaviors; Based on the narrative path and user profile, a large language model is invoked to generate personalized explanatory text, and style adaptive adjustments are made. The generated explanations are presented to users via voice and vision using augmented reality terminal devices.

[0005] Furthermore, the step of extracting entities, narrative relationships between entities, and sentiment style tags from unstructured cultural data using natural language processing technology, and constructing a cultural story knowledge graph containing a set of entity nodes, a set of relationship edges, and a set of attribute tags, includes: A document set of unstructured cultural materials is obtained, the document set consisting of several text documents. Each text document in the document set is traversed, and an entity extraction operation is performed on each text document to obtain the entity set in the text document. Based on each text document and its extracted entity set, perform a relation extraction operation to obtain the relation set; Iterate through each entity in the entity set, and if an entity is not in the entity node set, add the entity to the entity node set. The attributes of the entity are assigned values ​​to obtain the attribute tag set. The assignment result is composed of the sentiment analysis result of the entity context and the style tag extraction result of the entity. Iterate through the relationships in the set of relationships, and if the confidence of a relationship is greater than a threshold, add the relationship to the set of relationship edges.

[0006] Furthermore, in the step of obtaining a document set of unstructured cultural data, which consists of several text documents, and traversing each text document in the document set to perform entity extraction operations on each text document to obtain the entity set in the text document, a BERT-based named entity recognition model is used to identify four core entities: people, places, cultural relics, and events. In addition, a rule matching combined with machine learning method is used to identify temporal entities.

[0007] Furthermore, the step of collecting multi-dimensional user behavior data in real time, including eye-tracking data, movement data, voice data, and gesture data, and generating a multi-dimensional state vector describing the user's current state includes: Obtain the location coordinates of the extended reality terminal device, calculate the movement speed vector based on the location coordinates, and calculate the dwell time at the current point of interest; A spatial state vector is constructed using the location coordinates, the movement speed vector, and the dwell time. The eye-tracking data is processed to obtain a set of fixation points, and an attention heatmap is generated based on each fixation point in the set of fixation points. Based on the attention heatmap, identify the target object being gazed upon, and calculate the intensity of interest based on the gaze duration and gaze frequency of the target object being gazed upon; A visual attention vector is constructed using the gaze target object, the interest intensity, and the attention heatmap; Based on the movement data, determine the user's visit path sequence and analyze the user's visit rhythm pattern; Construct a temporal behavior vector based on the visitor path sequence and the visitor rhythm pattern; The speech data is subjected to speech intent recognition to obtain the speech recognition result; The gesture data is subjected to gesture recognition to obtain the gesture recognition result; Obtain user interaction data and update it; An interaction state vector is constructed using the speech recognition results, the gesture recognition results, and the interaction data. Obtain demographic information, which includes at least age group, education level, and cultural background; Inferring interests and preferences based on historical behavior, while simultaneously assessing users' level of professional knowledge; User profile vectors are constructed using the demographic information, interest preferences, and professional knowledge level. A multi-dimensional state vector describing the user's current state is generated based on the spatial state vector, the visual attention vector, the temporal behavior vector, the interaction state vector, and the user profile vector.

[0008] Furthermore, the step of executing a heuristic path search algorithm in the cultural story knowledge graph based on the multidimensional state vector to plan the narrative path that best matches the user's current interests and behaviors includes: Obtain a candidate path set, which contains paths formed by the current interest entities, and determine whether the length of the candidate path set is less than a first preset length. If so, obtain the endpoint of each path in the candidate path set, and determine the adjacent entities of the endpoint in the cultural story knowledge graph; Generate new paths based on each adjacent entity to expand the candidate paths; Determine if the length of the new path is less than or equal to the second preset length; If so, the new path is added to the candidate path set, and the relevance score, diversity score, and coherence score of each path in the candidate path set are calculated. Based on the relevance score, the diversity score, and the coherence score, the total score of each path is calculated. The path with the highest total score is identified as the narrative path and output.

[0009] Furthermore, in the steps of adding new paths to the candidate path set and calculating the relevance score, diversity score, and coherence score of each path in the candidate path set, the relevance score is determined based on the gaze target object and the interest preference; the diversity score is determined based on the type diversity of entities in the path and the semantic distance between entities; and the coherence score is determined based on the structure of the cultural story knowledge graph.

[0010] Furthermore, the step of generating personalized explanatory text by invoking a large language model based on the narrative path and user profile, and then adaptively adjusting the style, includes: Select the corresponding narrative template based on the age group in the user profile. Specifically, use a story-based template for children, an academic template for experts, and a conversational template for other groups. Traverse each entity in the narrative path and construct a content fragment for each entity. The core information of the entity is extracted based on the cultural story knowledge graph, the entity information is expanded based on the interest preferences, and transition sentences between entities are designed. Finally, the core information, the expanded entity information, and the transition sentences are assembled into a content fragment and added to the content segment set. Based on the narrative template, the content segmentation set, and the context, generate prompt words for use by the large language model; Based on the prompt words, invoke LLM to obtain the original text; Based on the level of professional knowledge, the vocabulary difficulty of the original text is adjusted to obtain the first adjusted text; Based on the user profile, the language style of the first adjusted text is adjusted to obtain the second adjusted text; Based on the interaction data, interactive guidance is added to the second adjustment text to generate the final personalized explanation text.

[0011] A second aspect of this invention provides an extended reality dynamic narrative navigation system for implementing the extended reality dynamic narrative navigation method provided in the first aspect, the system comprising: The knowledge graph construction module is used to extract entities, narrative relationships between entities, and emotional style tags from unstructured cultural data through natural language processing technology, and to construct a cultural story knowledge graph containing a set of entity nodes, a set of relationship edges, and a set of attribute tags. The data acquisition module is used to collect multi-dimensional user behavior data in real time, including eye-tracking data, movement data, voice data, and gesture data, and generate a multi-dimensional state vector describing the user's current state. The path search module is used to execute a heuristic path search algorithm in the cultural story knowledge graph based on the multidimensional state vector to plan the narrative path that best matches the user's current interests and behaviors. The explanatory text generation module is used to generate personalized explanatory text by calling a large language model based on the narrative path and user profile, and to make adaptive style adjustments. The presentation module is used to present the generated explanatory content to the user in a voice and visual manner through extended reality terminal devices.

[0012] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the extended reality dynamic narrative navigation method provided in the first aspect.

[0013] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the extended reality dynamic narrative navigation method provided in the first aspect.

[0014] The extended reality dynamic narrative tour guide method and system provided in this invention constructs a story knowledge graph containing entities, relationships, and attributes from cultural materials using natural language processing technology; it collects multi-dimensional behavioral data such as user eye movement, location, and voice in real time to generate user state vectors; it performs heuristic path search in the graph based on user state to plan personalized narrative paths; it calls a large language model to generate style-adaptive explanation content; and it presents multimedia content to users through extended reality terminal devices, forming a closed-loop feedback. Specifically, this invention solves the problems of static content, lack of personalization, and poor interactivity in existing digital tour guide technologies, achieving truly personalized intelligent tour guides, significantly improving the depth and satisfaction of cultural tourism visits, and is applicable to various cultural tourism scenarios such as museums, ancient building complexes, archaeological sites, and intangible cultural heritage transmission bases. Attached Figure Description

[0015] Figure 1 The flowchart illustrates the implementation of an extended reality dynamic narrative navigation method provided in Embodiment 1 of the present invention. Figure 2 This is a structural block diagram of an extended reality dynamic narrative navigation system provided in Embodiment 2 of the present invention; Figure 3 This is a structural block diagram of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0016] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0017] The extended reality dynamic narrative navigation method provided in this embodiment can be executed in smart terminals, computer terminals, network devices, chips, chip modules, or similar computing devices. The reference to "embodiment" in this application means that a specific feature, structure, or characteristic described in connection with an embodiment can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. Those skilled in the art will explicitly and implicitly understand that the embodiments described in this application can be combined with other embodiments.

[0018] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0019] As indicated in this specification, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0020] Example 1 According to an embodiment of the present invention, an extended reality dynamic narrative navigation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0021] This first embodiment provides an extended reality dynamic narrative navigation method, applied to scenarios with extended reality terminal devices. Specifically, extended reality (XR) technology is a general term for human-computer interaction technologies that integrate real and virtual environments, mainly including three branches: augmented reality (AR), virtual reality (VR), and mixed reality (MR). Please refer to... Figure 1 ,Figure 1 The flowchart of an extended reality dynamic narrative navigation method provided in Embodiment 1 of the present invention is shown, specifically including steps S01 to S05.

[0022] Step S01: Extract entities, narrative relationships between entities, and emotional style tags from unstructured cultural data using natural language processing technology, and construct a cultural story knowledge graph containing a set of entity nodes, a set of relationship edges, and a set of attribute tags.

[0023] Specifically, a document set D of unstructured cultural materials is acquired, wherein the document set D consists of several text documents d i Composition, that is Iterate through each text document d in the document set D. i For each text document d i The entity extraction operation is performed to obtain the entity set in the text document. Among them, the BERT-based named entity recognition model is used to identify four core entities: people, places, cultural relics, and events. In addition, the rule matching combined with machine learning method is used to identify time-series entities such as time and dynasty. Finally, the entities are deduplicated and normalized to build an entity index table. Based on each text document d i The entity set extracted from it is subjected to relation extraction operation to obtain the relation set. It should be noted that a relation extraction algorithm based on dependency parsing is applied to identify semantic relations between entities. In the embodiments of the present invention, the defined relations include cause, influence, contrast, and inheritance. In addition, a relation classification model is trained using remote supervised learning, and the relation confidence threshold is set to 0.85. Iterate through each entity in the entity set, and if an entity is not in the entity node set V, add the entity to the entity node set V. Assign values ​​to the attributes of the entity to obtain the attribute tag set A. The assignment result is composed of the sentiment analysis result of the entity context and the style tag extraction result of the entity. Iterate through the relationships in the set of relationships. If the confidence of a relationship is greater than a threshold, add the relationship to the set of relationship edges E. This is understandable, as the cultural story knowledge graph... .

[0024] Step S02: Collect multi-dimensional user behavior data in real time, including eye-tracking data, movement data, voice data, and gesture data, and generate a multi-dimensional state vector describing the user's current state.

[0025] Specifically, the location coordinates of the extended reality terminal device are obtained, and the movement speed vector is calculated based on the location coordinates. At the same time, the dwell time at the current point of interest is also calculated. A spatial state vector is constructed using the location coordinates, the movement speed vector, and the dwell time. The eye-tracking data is processed to obtain a set of fixation points, and an attention heatmap is generated based on each fixation point in the set of fixation points. Based on the attention heatmap, the gaze target object is identified, and the interest intensity is calculated based on the gaze duration and gaze frequency of the gaze target object, wherein the interest intensity is calculated using a weighted summation method. A visual attention vector is constructed using the gaze target object, the interest intensity, and the attention heatmap; Based on the mobile data, the user's visit path sequence is determined, and the user's visit rhythm pattern is analyzed. It can be understood that the user's visit path sequence, duration distribution and other multi-dimensional features are clustered (such as K-means, hierarchical clustering, etc.), and users with similar rhythm patterns are grouped into one category, and typical visit rhythm types are summarized (such as "efficient check-in type", "deep exploration type", "random roaming type" etc.). Construct a temporal behavior vector based on the visitor path sequence and the visitor rhythm pattern; The speech data is subjected to speech intent recognition to obtain the speech recognition result; The gesture data is subjected to gesture recognition to obtain the gesture recognition result; Obtain user interaction data and update it; An interaction state vector is constructed using the speech recognition results, the gesture recognition results, and the interaction data. Obtain demographic information, which includes at least age group, education level, and cultural background; Inferring interests and preferences based on historical behavior, while simultaneously assessing users' level of professional knowledge; User profile vectors are constructed using the demographic information, interest preferences, and professional knowledge level. A multi-dimensional state vector describing the user's current state is generated based on the spatial state vector, the visual attention vector, the temporal behavior vector, the interaction state vector, and the user profile vector.

[0026] Step S03: Based on the multidimensional state vector, execute a heuristic path search algorithm in the cultural story knowledge graph to plan the narrative path that best matches the user's current interests and behaviors. It should be noted that the purpose of step S03 is to search the cultural story knowledge graph for the narrative path that best matches the user's state, i.e., the multi-dimensional state vector. Specifically, a candidate path set is obtained, which includes paths formed by the current interest entities. It is determined whether the length of the candidate path set is less than a first preset length. The interest entities can be captured by eye-tracking devices to capture the user's gaze coordinates and area of ​​interest (AOI). For example, if the user continuously gazes at an exhibit for more than a threshold (e.g., 200ms), the entity corresponding to that exhibit is marked as the current interest entity; or, the entity with the highest interest intensity score is marked as the current interest entity. If so, obtain the endpoint p of each path in the candidate path set, and determine the adjacent entities of the endpoint in the cultural story knowledge graph; Generate a new path based on each adjacent entity n. To expand candidate paths; Determine if the length of the new path is less than or equal to the second preset length; If so, the new path is added to the candidate path set, and the relevance score, diversity score, and coherence score of each path in the candidate path set are calculated. It should be noted that the relevance score is determined based on the target object and the user's interest preferences. Specifically, firstly, the semantic association between the entity and the user's attention object is analyzed (e.g., through word vector similarity, topic matching, etc.). Secondly, the user's interest preferences inferred from historical behavior (e.g., preferences for historical, science fiction, etc.) are combined to quantify the fit between the entity and the preference. Finally, the two are weighted and summed to obtain the relevance score. The relevance score is also determined based on the diversity of entity types in the path (e.g., the distribution of categories of people, events, and locations) and the semantic distance between entities (e.g., the distribution of categories of people, events, and locations). By using the cosine distance of embedded vectors to avoid excessive semantic repetition of entities in the path, the diversity score is determined, with higher scores indicating more diverse paths. Based on the structure of the cultural story knowledge graph, the coherence score is determined. Specifically, first, the strength of relationships between entities is analyzed (e.g., the weight of edges in the story graph, representing the tightness of narrative connections between entities). Then, the temporal logic of the path is checked (e.g., whether the order of events of entities conforms to narrative common sense). Finally, the overall coherence of the path is calculated by combining the adjacency relationships of entities in the graph structure; higher scores indicate smoother narratives. Based on the relevance score, the diversity score, and the coherence score, the total score for each path is calculated, expressed as: ; Where α, β, and γ are weighting parameters that control the effects of relevance, diversity, and coherence, respectively; relevance is relevance; diversity is diversity; coherence is coherence; and total_score is the total score. The path with the highest total score is identified as the narrative path and output.

[0027] Step S04: Based on the narrative path and user profile, a large language model is invoked to generate personalized explanatory text, and style adaptive adjustments are made. Specifically, the corresponding narrative template is selected based on the age group in the user profile. The age group can be determined by demographic information. Among them, the children group uses a story-based template with a basic vocabulary level, the experts group uses an academic template with a professional vocabulary level, and other groups use a conversational template with a general vocabulary level. Traverse each entity in the narrative path and construct a content fragment for each entity. The core information of the entity is extracted based on the cultural story knowledge graph, the entity information is expanded based on the interest preferences, and transition sentences between entities are designed. Finally, the core information, the expanded entity information, and the transition sentences are assembled into a content fragment and added to the content segment set. Based on the narrative template, the content segmentation set, and the context, generate prompt words for use by the large language model; Based on the prompt words, invoke LLM to obtain the original text; Based on the level of professional knowledge, the vocabulary difficulty of the original text is adjusted to obtain the first adjusted text; Based on the user profile, the language style (such as formality, friendliness, etc.) of the first adjustment text is adjusted to obtain the second adjustment text; Based on the interaction data, interactive guidance is added to the second adjustment text to generate the final personalized explanation text.

[0028] Step S05: Present the generated explanation content to the user via voice and vision using an augmented reality terminal device.

[0029] It should be noted that the AR glasses display virtual explanation content, including text annotations, highlighted indicators, and 3D animations; provide directional voice explanations through spatial audio technology; enhance the immersive experience through haptic feedback devices; and monitor changes in user attention in real time to dynamically adjust the intensity and manner of the presented content.

[0030] In summary, the extended reality dynamic narrative tour guide method in the above embodiments of the present invention constructs a story knowledge graph containing entities, relationships, and attributes from cultural materials using natural language processing technology; collects multi-dimensional behavioral data such as user eye movements, location, and voice in real time to generate user state vectors; performs heuristic path search in the graph based on user state to plan personalized narrative paths; calls a large language model to generate style-adaptive explanation content; and presents multimedia content to users through extended reality terminal devices, forming a closed-loop feedback. Specifically, the present invention solves the problems of static content, lack of personalization, and poor interactivity in existing digital tour guide technologies, achieving truly personalized intelligent tour guides, significantly improving the depth and satisfaction of cultural tourism visits, and is applicable to various cultural tourism scenarios such as museums, ancient building complexes, archaeological sites, and intangible cultural heritage transmission bases.

[0031] Example 2 Please see Figure 2 , Figure 2 This is a structural block diagram of an extended reality dynamic narrative guide system 200 provided in Embodiment 2 of the present invention. The extended reality dynamic narrative guide system 200 includes: a knowledge graph construction module 21, a data acquisition module 22, a path search module 23, an explanatory text generation module 24, and a presentation module 25, wherein: The knowledge graph construction module 21 is used to extract entities, narrative relationships between entities, and emotional style tags from unstructured cultural data through natural language processing technology, and to construct a cultural story knowledge graph containing a set of entity nodes, a set of relationship edges, and a set of attribute tags. The acquisition module 22 is used to collect multi-dimensional user behavior data in real time, including eye-tracking data, movement data, voice data and gesture data, and generate a multi-dimensional state vector describing the user's current state. The path search module 23 is used to execute a heuristic path search algorithm in the cultural story knowledge graph based on the multidimensional state vector to plan the narrative path that best matches the user's current interests and behaviors. The explanatory text generation module 24 is used to generate personalized explanatory text by calling a large language model based on the narrative path and user profile, and to make adaptive style adjustments. Presentation module 25 is used to present the generated explanatory content to the user in a voice and visual manner through an extended reality terminal device.

[0032] Furthermore, in some optional embodiments of the present invention, the knowledge graph construction module 21 includes: The acquisition unit is used to acquire a document set of unstructured cultural materials. The document set consists of several text documents. It traverses each text document in the document set and performs entity extraction operation on each text document to obtain the entity set in the text document. It uses a BERT-based named entity recognition model to identify four core entities: people, places, cultural relics, and events. In addition, it uses a rule matching combined with machine learning method to identify time-series entities. An extraction unit is used to perform a relation extraction operation based on each text document and its extracted entity set to obtain the relation set; The first traversal unit is used to traverse each entity in the entity set, wherein if an entity is not in the entity node set, the entity is added to the entity node set; The assignment unit is used to assign values ​​to the attributes of an entity to obtain the attribute tag set. The assignment result is composed of the sentiment analysis result of the entity context and the style tag extraction result of the entity. The second traversal unit is used to traverse the relations in the relation set, wherein if the confidence of a relation is greater than a threshold, the relation is added to the relation edge set.

[0033] Furthermore, in some optional embodiments of the present invention, the acquisition module 22 includes: The first calculation unit is used to obtain the location coordinates of the extended reality terminal device, calculate the movement speed vector based on the location coordinates, and calculate the dwell time at the current point of interest. The first construction unit is used to construct a spatial state vector using the position coordinates, the movement speed vector, and the dwell time. The first generation unit is used to process eye-tracking data to obtain a set of fixation points, and generate an attention heatmap based on each fixation point in the set of fixation points. The second calculation unit is used to identify the gaze target object based on the attention heatmap, and to calculate the interest intensity based on the gaze duration and gaze frequency of the gaze target object; The second construction unit is used to construct a visual attention vector using the gaze target object, the interest intensity, and the attention heatmap; The analysis unit is used to determine the user's visit path sequence and analyze the user's visit rhythm pattern based on the movement data. The third construction unit is used to construct a temporal behavior vector based on the visitor path sequence and the visitor rhythm pattern; The first recognition unit is used to perform voice intent recognition on the voice data to obtain a voice recognition result; The second recognition unit is used to perform gesture recognition on the gesture data to obtain a gesture recognition result; The first acquisition unit is used to acquire and update user interaction data; The fourth construction unit is used to construct an interaction state vector using the speech recognition result, the gesture recognition result, and the interaction data; The second acquisition unit is used to acquire demographic information, which includes at least age group, education level, and cultural background. The inference unit is used to infer user interests and preferences based on historical behavior, while also assessing the user's level of expertise. The fifth construction unit is used to construct a user profile vector using the demographic information, interest preferences, and professional knowledge level. The second generation unit is used to generate a multi-dimensional state vector describing the user's current state based on the spatial state vector, the visual attention vector, the temporal behavior vector, the interaction state vector, and the user profile vector.

[0034] Furthermore, in some optional embodiments of the present invention, the path search module 23 includes: The first judgment unit is used to obtain a candidate path set, which contains paths formed by the current interest entity, and to determine whether the length of the candidate path set is less than a first preset length. The third acquisition unit is used to acquire the endpoint of each path in the candidate path set when it is determined that the length of the candidate path set is less than the first preset length, and to determine the adjacent entity of the endpoint in the cultural story knowledge graph. The third generation unit is used to generate new paths based on each adjacent entity to expand the candidate paths; The second judgment unit is used to determine whether the length of the new path is less than or equal to the second preset length; The third calculation unit is used to add a new path to the candidate path set when the length of the new path is less than or equal to a second preset length, calculate the relevance score, diversity score, and coherence score of each path in the candidate path set, and calculate the total score of each path based on the relevance score, the diversity score, and the coherence score. The relevance score is determined based on the gaze target object and the interest preference; the diversity score is determined based on the type diversity of entities in the path and the semantic distance between entities; and the coherence score is determined based on the structure of the cultural story knowledge graph. The output unit is used to determine the path with the highest total score as the narrative path and output it.

[0035] Furthermore, in some optional embodiments of the present invention, the explanatory text generation module 24 includes: The selection unit is used to select the corresponding narrative template based on the age group in the user profile. The children group uses a story-based template, the experts group uses an academic template, and other groups use a conversational template. The third traversal unit is used to traverse each entity in the narrative path and construct a content fragment for each entity. The core information of the entity is extracted according to the cultural story knowledge graph, the entity information is expanded according to the interest preferences, and transition sentences between entities are designed. Finally, the core information, the expanded entity information and the transition sentences are assembled into a content fragment and added to the content segment set. The fourth generation unit is used to generate prompt words for use by the large language model based on the narrative template, the content segmentation set, and the context. The calling unit is used to call the LLM based on the prompt word to obtain the original text; The first adjustment unit is used to adjust the vocabulary difficulty of the original text according to the professional knowledge level to obtain the first adjusted text; The second adjustment unit is used to adjust the language style of the first adjustment text according to the user profile to obtain the second adjustment text; The fifth generation unit is used to add interactive guidance to the second adjustment text based on the interaction data, and generate the final personalized explanation text.

[0036] Example 3 In another aspect, the present invention also proposes an electronic device, please refer to [link to relevant documentation]. Figure 3 The image shows an electronic device according to Embodiment 3 of the present invention, including a memory 20, a processor 10, and a computer program 30 stored in the memory and executable on the processor. When the processor 10 executes the computer program 30, it implements the extended reality dynamic narrative navigation method as described above.

[0037] In some embodiments, the processor 10 may be a central processing unit (CPU), controller, microcontroller, microprocessor or other data processing chip, used to run program code stored in memory 20 or process data, such as executing access restriction programs.

[0038] The memory 20 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 20 can be an internal storage unit of an electronic device, such as the hard disk of the electronic device. In other embodiments, the memory 20 can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 20 can include both internal and external storage units of the electronic device. The memory 20 can be used not only to store application software and various types of data of the electronic device, but also to temporarily store data that has been output or will be output.

[0039] It should be pointed out that, Figure 3 The structure shown does not constitute a limitation on the electronic device. In other embodiments, the electronic device may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0040] This invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the extended reality dynamic narrative navigation method described above.

[0041] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0042] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0043] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0044] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0045] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. An extended reality dynamic narrative guided tour method, characterized in that, When applied to scenarios with extended reality terminal devices, the method includes: By using natural language processing technology, entities, narrative relationships between entities, and emotional style tags are extracted from unstructured cultural data to construct a cultural story knowledge graph containing a set of entity nodes, a set of relationship edges, and a set of attribute tags. Real-time collection of multi-dimensional user behavior data, including eye-tracking data, movement data, voice data, and gesture data, to generate a multi-dimensional state vector describing the user's current state; Based on the multidimensional state vector, a heuristic path search algorithm is executed in the cultural story knowledge graph to plan the narrative path that best matches the user's current interests and behaviors; Based on the narrative path and user profile, a large language model is invoked to generate personalized explanatory text, and style adaptive adjustments are made. The generated explanations are presented to users via voice and vision using augmented reality terminal devices.

2. The extended reality dynamic narrative navigation method according to claim 1, characterized in that, The steps of extracting entities, narrative relationships between entities, and sentiment style tags from unstructured cultural data using natural language processing technology, and constructing a cultural story knowledge graph containing a set of entity nodes, a set of relationship edges, and a set of attribute tags, include: A document set of unstructured cultural materials is obtained, the document set consisting of several text documents. Each text document in the document set is traversed, and an entity extraction operation is performed on each text document to obtain the entity set in the text document. Based on each text document and its extracted entity set, perform a relation extraction operation to obtain the relation set; Iterate through each entity in the entity set, and if an entity is not in the entity node set, add the entity to the entity node set. The attributes of the entity are assigned values ​​to obtain the attribute tag set. The assignment result is composed of the sentiment analysis result of the entity context and the style tag extraction result of the entity. Iterate through the relationships in the set of relationships, and if the confidence of a relationship is greater than a threshold, add the relationship to the set of relationship edges.

3. The extended reality dynamic narrative navigation method according to claim 2, characterized in that, The step of obtaining a document set of unstructured cultural data, which consists of several text documents, involves traversing each text document in the document set and performing entity extraction operations on each text document to obtain the entity set in that text document. In this step, a BERT-based named entity recognition model is used to identify four core entities: people, places, cultural relics, and events. In addition, a rule matching combined with machine learning method is used to identify time-series entities.

4. The extended reality dynamic narrative navigation method according to claim 3, characterized in that, The steps of collecting multi-dimensional user behavior data in real time, including eye-tracking data, movement data, voice data, and gesture data, and generating a multi-dimensional state vector describing the user's current state include: Obtain the location coordinates of the extended reality terminal device, calculate the movement speed vector based on the location coordinates, and calculate the dwell time at the current point of interest; A spatial state vector is constructed using the location coordinates, the movement speed vector, and the dwell time. The eye-tracking data is processed to obtain a set of fixation points, and an attention heatmap is generated based on each fixation point in the set of fixation points. Based on the attention heatmap, identify the target object being gazed upon, and calculate the intensity of interest based on the gaze duration and gaze frequency of the target object being gazed upon; A visual attention vector is constructed using the gaze target object, the interest intensity, and the attention heatmap; Based on the movement data, determine the user's visit path sequence and analyze the user's visit rhythm pattern; Construct a temporal behavior vector based on the visitor path sequence and the visitor rhythm pattern; The speech data is subjected to speech intent recognition to obtain the speech recognition result; The gesture data is subjected to gesture recognition to obtain the gesture recognition result; Obtain user interaction data and update it; An interaction state vector is constructed using the speech recognition results, the gesture recognition results, and the interaction data. Obtain demographic information, which includes at least age group, education level, and cultural background; Inferring interests and preferences based on historical behavior, while simultaneously assessing users' level of professional knowledge; User profile vectors are constructed using the demographic information, interest preferences, and professional knowledge level. A multi-dimensional state vector describing the user's current state is generated based on the spatial state vector, the visual attention vector, the temporal behavior vector, the interaction state vector, and the user profile vector.

5. The extended reality dynamic narrative navigation method according to claim 4, characterized in that, The step of executing a heuristic path search algorithm in the cultural story knowledge graph based on the multidimensional state vector to plan the narrative path that best matches the user's current interests and behaviors includes: Obtain a candidate path set, which contains paths formed by the current interest entities, and determine whether the length of the candidate path set is less than a first preset length. If so, obtain the endpoint of each path in the candidate path set, and determine the adjacent entities of the endpoint in the cultural story knowledge graph; Generate new paths based on each adjacent entity to expand the candidate paths; Determine if the length of the new path is less than or equal to the second preset length; If so, the new path is added to the candidate path set, and the relevance score, diversity score, and coherence score of each path in the candidate path set are calculated. Based on the relevance score, the diversity score, and the coherence score, the total score of each path is calculated. The path with the highest total score is identified as the narrative path and output.

6. The extended reality dynamic narrative navigation method according to claim 5, characterized in that, In the steps of adding a new path to the candidate path set and calculating the relevance score, diversity score, and coherence score of each path in the candidate path set, the relevance score is determined based on the gaze target object and the interest preference; the diversity score is determined based on the type diversity of entities in the path and the semantic distance between entities; and the coherence score is determined based on the structure of the cultural story knowledge graph.

7. The extended reality dynamic narrative navigation method according to claim 6, characterized in that, The steps of generating personalized explanatory text by calling a large language model based on the narrative path and user profile, and then adaptively adjusting the style, include: Select the corresponding narrative template based on the age group in the user profile. Specifically, use a story-based template for children, an academic template for experts, and a conversational template for other groups. Traverse each entity in the narrative path and construct a content fragment for each entity. The core information of the entity is extracted based on the cultural story knowledge graph, the entity information is expanded based on the interest preferences, and transition sentences between entities are designed. Finally, the core information, the expanded entity information, and the transition sentences are assembled into a content fragment and added to the content segment set. Based on the narrative template, the content segmentation set, and the context, generate prompt words for use by the large language model; Based on the prompt words, invoke LLM to obtain the original text; Based on the level of professional knowledge, the vocabulary difficulty of the original text is adjusted to obtain the first adjusted text; Based on the user profile, the language style of the first adjusted text is adjusted to obtain the second adjusted text; Based on the interaction data, interactive guidance is added to the second adjustment text to generate the final personalized explanation text.

8. An extended reality dynamic narrative navigation system, characterized in that, For implementing the extended reality dynamic narrative navigation method as described in any one of claims 1-7, the system comprises: The knowledge graph construction module is used to extract entities, narrative relationships between entities, and emotional style tags from unstructured cultural data through natural language processing technology, and to construct a cultural story knowledge graph containing a set of entity nodes, a set of relationship edges, and a set of attribute tags. The data acquisition module is used to collect multi-dimensional user behavior data in real time, including eye-tracking data, movement data, voice data, and gesture data, and generate a multi-dimensional state vector describing the user's current state. The path search module is used to execute a heuristic path search algorithm in the cultural story knowledge graph based on the multidimensional state vector to plan the narrative path that best matches the user's current interests and behaviors. The explanatory text generation module is used to generate personalized explanatory text by calling a large language model based on the narrative path and user profile, and to make adaptive style adjustments. The presentation module is used to present the generated explanatory content to the user in a voice and visual manner through extended reality terminal devices.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the extended reality dynamic narrative navigation method as described in any one of claims 1-7.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the extended reality dynamic narrative navigation method as described in any one of claims 1-7.

Images