Guiding method, system, electronic device, storage medium and product

Abstract

This disclosure proposes a tour guide method, system, electronic device, storage medium, and product. Applied to a tour guide system, the system includes a positioning sensing node, a user terminal, an inference server, and an augmented reality terminal. The method includes: responding to a user entering an exhibition venue and activating tour guide mode; generating a spatial perception model based on user spatial data and user behavior data; collecting user-initiated voice interaction information; generating user interest features based on the voice interaction information; determining the semantic similarity between the user interest features and the exhibits corresponding to the current tour guide path; if the semantic similarity is greater than a preset similarity threshold, generating answer information; if the semantic similarity is less than or equal to the preset similarity threshold, generating a semantically related tour guide path based on the spatial perception model; highlighting the tour guide path and preloading the explanation content corresponding to the exhibits on the tour guide path, which can improve the content association and continuous narrative logic between exhibit explanations, thereby enhancing the user's visiting experience.

Description

Technical Field

[0001] This disclosure relates to the field of augmented reality technology, and more particularly to a navigation method, system, electronic device, storage medium, and product. Background Technology

[0002] In related guided tour programs, fixed guided tour routes, paper guide maps, or stand-alone audio guide devices are usually used to provide guided tour services. Relying on fixed narration content makes it difficult to respond to users' personalized questions, and the narration of exhibits lacks content connection and continuous narrative logic, resulting in a fragmented user visit experience. Summary of the Invention

[0003] This disclosure provides a navigation method, system, electronic device, storage medium, and product to address problems in the related art.

[0004] A first aspect of this disclosure provides a navigation method applied to a navigation system, the system including a positioning sensing node, a user terminal, an inference server, and an augmented reality terminal, the method comprising:

[0005] In response to a user entering the exhibition venue and activating the guided tour mode, the positioning sensing node acquires user spatial data and user behavior data, generates a spatial perception model based on the user spatial data and user behavior data, and sends the spatial perception model to the user terminal. The user terminal collects voice interaction information initiated by the user through the augmented reality terminal and sends the voice interaction information to the inference server. The reasoning server generates user interest features based on the voice interaction information and determines the semantic similarity between the user interest features and the exhibits corresponding to the current tour path. If the semantic similarity is greater than a preset similarity threshold, it generates answer information and sends it to the augmented reality terminal. If the semantic similarity is less than or equal to the preset similarity threshold, it generates a semantically associated tour path based on the spatial perception model and sends the semantically associated tour path to the augmented reality terminal. The augmented reality terminal highlights the tour path and preloads the explanatory content corresponding to the exhibits on the tour path.

[0006] In one embodiment, generating a spatial awareness model based on the user space data and the user behavior data includes: The positioning sensing node determines the user's planar coordinates within the exhibition area based on the user space data, and the exhibition area includes multiple exhibition grids; The location sensing node determines the activity index of each exhibition grid based on the dwell time and frequency of occurrence in each exhibition grid in the user behavior data; The positioning sensing node generates the user's movement vector based on the planar coordinates and the activity index of each exhibition grid; The positioning and sensing node generates a spatial sensing model based on the movement vector.

[0007] In one embodiment, the method provided in this disclosure further includes: The user terminal determines the location of the exhibit based on the spatial perception model. The augmented reality terminal displays navigation information for the exhibit corresponding to the user's current location.

[0008] In one embodiment, sending the voice interaction information to the inference server includes: The user terminal parses the voice interaction information into first text interaction information, and generates the user's target interaction information based on the first text interaction information and the user's second text interaction information; The user terminal sends the target interaction information to the inference server.

[0009] In one embodiment, based on the spatial awareness model, a semantically related navigation path is generated, including: The reasoning server starts from the exhibit node corresponding to the user's current exhibit location, performs multi-hop semantic queries in the exhibit knowledge graph, and extracts a set of exhibit nodes associated with the user's interest features. The inference server determines the semantic association strength of each exhibit node in the exhibit node set; The reasoning server generates a semantically associated navigation path based on the spatial awareness model and the semantic association strength.

[0010] In one embodiment, based on the spatial awareness model, a semantically related navigation path is generated, including: When the voice interaction information involves exhibits from a different exhibition area than the current exhibition area, the positioning perception node sends the spatial perception model and the exhibit knowledge graph corresponding to each exhibition area to the inference server. The reasoning server determines the cross-exhibition-area spatial perception model based on the spatial perception model corresponding to each exhibition area, and determines the cross-exhibition-area knowledge graph based on the exhibit knowledge graph corresponding to each exhibition area. The reasoning server takes the exhibit node corresponding to the user's current exhibit location as the starting point and generates a semantically related navigation path based on the cross-exhibition area spatial perception model and the cross-exhibition area knowledge graph.

[0011] A second aspect of this disclosure provides a navigation system, which includes a location sensing node, a user terminal, an inference server, and an augmented reality terminal. The positioning and sensing node is used to respond to a user entering the exhibition venue and activating the guided tour mode, acquire user spatial data and user behavior data, generate a spatial perception model based on the user spatial data and user behavior data, and send the spatial perception model to the user terminal. The user terminal is used to collect voice interaction information initiated by the user through the augmented reality terminal and send the voice interaction information to the inference server. The reasoning server is used to generate user interest features based on the voice interaction information, determine the semantic similarity between the user interest features and the exhibits corresponding to the current tour path; if the semantic similarity is greater than a preset similarity threshold, generate answer information and send it to the augmented reality terminal; if the semantic similarity is less than or equal to the preset similarity threshold, generate a semantically associated tour path based on the spatial perception model, and send the semantically associated tour path to the augmented reality terminal. The augmented reality terminal is used to highlight the tour path and preload the explanatory content corresponding to the exhibits on the tour path.

[0012] A third aspect of this disclosure provides an electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods described in the first aspect of this disclosure.

[0013] A fourth aspect of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described in the first aspect of this disclosure.

[0014] A fifth aspect of this disclosure provides a computer program product including a computer program that, when executed by a processor, implements the methods described in the first aspect of this disclosure.

[0015] In summary, this disclosure proposes a tour guide method applied to a tour guide system. The system includes a positioning and sensing node, a user terminal, an inference server, and an augmented reality terminal. The method includes: in response to a user entering an exhibition venue and activating a tour guide mode, the positioning and sensing node acquires user spatial data and user behavior data, generates a spatial perception model based on the user spatial data and user behavior data, and sends the spatial perception model to the user terminal; the user terminal collects voice interaction information initiated by the user through the augmented reality terminal and sends the voice interaction information to the inference server; the inference server generates user interest features based on the voice interaction information and determines the semantic similarity between the user interest features and the exhibits corresponding to the current tour guide path; if the semantic similarity is greater than a preset similarity threshold, generates answer information and sends it to the augmented reality terminal; if the semantic similarity is less than or equal to the preset similarity threshold, generates a semantically related tour guide path based on the spatial perception model and sends the semantically related tour guide path to the augmented reality terminal; the augmented reality terminal highlights and renders the tour guide path and preloads the explanatory content corresponding to the exhibits on the tour guide path.

[0016] According to the solution provided in this disclosure, a spatial perception model is generated by acquiring user spatial data and user behavior data. User interest characteristics are obtained by analyzing the collected voice interaction information. Based on the semantic similarity between user interest characteristics and exhibits corresponding to the current tour route, either answer information is generated directly or a semantically related tour route is generated. Then, the augmented reality terminal completes the highlight rendering of the route and preloading of the explanation content. This can simultaneously meet the user's needs for immediate Q&A and personalized visits, solve the problem of fixed traditional tour routes and inability to respond to personalized user questions, improve the content association and continuous narrative logic between exhibit explanations, and thus enhance the user's visit experience.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0019] Figure 1 A flowchart illustrating a tour guide method provided in an embodiment of this disclosure; Figure 2 A flowchart illustrating a method for acquiring user space data provided in an embodiment of this disclosure; Figure 3 A schematic diagram of a spatial perception model provided in an embodiment of this disclosure; Figure 4 A flowchart illustrating a method for generating a navigation path provided in an embodiment of this disclosure; Figure 5 A flowchart illustrating yet another tour guide method provided in this embodiment of the disclosure; Figure 6 This is a schematic diagram of the structure of a tour guide system provided in an embodiment of the present disclosure; Figure 7 This is a schematic diagram of the hardware composition structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0020] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0021] To facilitate a better understanding of the technical solutions described in the embodiments of this disclosure by those skilled in the art, the technical terms in the embodiments of this disclosure are explained as follows before introducing the embodiments of this disclosure.

[0022] In related guided tour programs, fixed guided tour routes, paper guide maps, or stand-alone audio guide devices are usually used to provide guided tour services. Relying on fixed narration content makes it difficult to respond to users' personalized questions, and the narration of exhibits lacks content connection and continuous narrative logic, resulting in a fragmented user visit experience.

[0023] Currently, common tour guide technologies in museums and exhibition halls include location-based audio guides and mobile app-based tour guides. These technologies typically use RFID, Bluetooth, or GPS to locate the user's position and trigger pre-recorded audio or text explanations.

[0024] However, these solutions mainly rely on static data and have fixed content. They cannot dynamically adjust the tour route and provide professional explanations based on users' real-time questions or spatial movements. The handling of these user questions can only be done in isolated human responses, lacking a continuous narrative, resulting in a fragmented tour experience.

[0025] To address the shortcomings of related technologies, this disclosure generates a spatial perception model by acquiring user spatial data and user behavior data. It analyzes collected voice interaction information to obtain user interest characteristics. Based on the semantic similarity between user interest characteristics and exhibits corresponding to the current tour route, it selects to directly generate answer information or generate a semantically related tour route. Then, the augmented reality terminal completes the path highlighting rendering and preloading of explanation content. This can simultaneously meet users' needs for immediate Q&A and personalized visits, solve the problems of fixed traditional tour routes and inability to respond to personalized user questions, improve the content association and continuous narrative logic between exhibit explanations, and thus enhance the user's visit experience.

[0026] The present disclosure will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0027] The present disclosure provides a tour guide method that can be applied to tour guide systems, such as museum tour guide systems, art gallery tour guide systems, and exhibition hall tour guide systems.

[0028] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a tour guide method provided in an embodiment of this disclosure. The tour guide method provided in this embodiment is applied to a tour guide system, which includes a positioning sensing node, a user terminal, an inference server, and an augmented reality terminal, and includes the following steps: Step 101: In response to a user entering the exhibition venue and activating the guided tour mode, the positioning sensing node acquires user spatial data and user behavior data, generates a spatial perception model based on the user spatial data and user behavior data, and sends the spatial perception model to the user terminal. In one embodiment, the exhibition venue is a space for setting up various exhibits and providing guided tours to users. The exhibition venue may include, but is not limited to, museums, art galleries, exhibition halls, themed exhibition halls, cultural and tourism exhibition halls, etc. This application does not specifically limit the scale, type, or category of exhibits of the exhibition venue.

[0029] In one embodiment, the guided tour mode refers to the intelligent explanation and path guidance working mode activated by the terminal device. Users can activate the guided tour mode through touch, voice, buttons, etc.

[0030] In one embodiment, the positioning and sensing node is a sensing device deployed within the exhibition venue that has data collection capabilities. The positioning and sensing node may include a 5G base station in the exhibition hall, a lidar in the display case, etc.; the user terminal is an electronic device carried by the user, such as a mobile phone, a handheld smart terminal, etc.; the inference server is a service device deployed in the venue's backend or in the cloud; the augmented reality terminal is an augmented reality (AR) device worn by the user, such as AR glasses or an AR head-mounted terminal, used to overlay real-scene visualizations, display answer information, render guided tour paths, and play exhibit explanations.

[0031] In one embodiment, user space data is used to indicate the user's real-time location information within the exhibition venue, and may include the user's coordinate data, orientation data, etc.; user behavior data is used to indicate the user's behavioral status during the visit, and may include data such as the user's dwell time, activity trajectory, and frequency of visits to areas.

[0032] In one embodiment, the spatial perception model is a model that can reflect the relationship between user distribution, activity preferences, and venue spatial layout.

[0033] In one embodiment, user space data and user behavior data can be collected by display case lidar deployed in the venue; alternatively, user images can be captured by in-venue surveillance cameras to determine user space data and user behavior data in the images.

[0034] In one embodiment, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a method for acquiring user space data according to an embodiment of the present disclosure. The method can obtain parameter information such as the user's location in front of the display case, user identification ID, display case ID, and exhibit number by scanning with a LiDAR node.

[0035] In one embodiment, after generating the spatial awareness model, it can be sent to the user terminal via wireless communication.

[0036] Step 102: The user terminal collects voice interaction information initiated by the user through the augmented reality terminal and sends the voice interaction information to the inference server. In one embodiment, the voice interaction information is data of voice-based questions posed by the user regarding exhibits, tour routes, and other related content.

[0037] In one embodiment, the user asks a question via voice through an augmented reality terminal. The augmented reality terminal acquires the voice question data and transmits it to the user terminal, which then forwards the voice interaction information to the inference server via a network.

[0038] In one embodiment, the user terminal can first parse the voice question data into first text interaction information, and then upload the text data to the inference server to reduce the amount of data transmitted.

[0039] Step 103: The reasoning server generates user interest features based on the voice interaction information and determines the semantic similarity between the user interest features and the exhibits corresponding to the current tour path; if the semantic similarity is greater than a preset similarity threshold, it generates answer information and sends it to the augmented reality terminal; if the semantic similarity is less than or equal to the preset similarity threshold, it generates a semantically associated tour path based on the spatial perception model and sends the semantically associated tour path to the augmented reality terminal. In one embodiment, user interest features are user preference tags obtained from voice interaction information analysis, which are used to characterize the types of exhibits the user is currently interested in, their visit requests, etc.

[0040] In one embodiment, semantic similarity is used to measure the degree of association between voice interaction information and the current exhibit content. The higher the value, the higher the degree of association between the voice interaction information and the current exhibit content.

[0041] In one embodiment, the preset similarity threshold is a pre-set critical value used to determine the degree of association between voice interaction information and the current exhibit content. The preset similarity threshold can be adjusted according to actual usage needs.

[0042] In one embodiment, the answer information is text or voice explanation content generated in response to the user's voice interaction information, used to directly answer the user's questions.

[0043] In one embodiment, the semantically associated tour route is a redesigned visitor route based on user interest characteristics and the spatial layout of the exhibition venue, with exhibits arranged along the route that are relevant to the user's interests.

[0044] In one embodiment, the inference server can use a natural language processing algorithm to parse and extract keywords from the voice interaction information, thereby generating user interest features. Then, it can use a semantic matching algorithm to calculate the semantic similarity between the user interest features and the content of the current exhibit, and compare the calculation result with a preset similarity threshold. When the semantic similarity is greater than the preset similarity threshold, it is determined that the user's question revolves around the current exhibit, and the inference server generates answer information based on the corresponding explanatory materials and sends it to the augmented reality terminal. When the semantic similarity is less than or equal to the preset similarity threshold, it is determined that the user's request has a low relevance to the current exhibit, and the inference server needs to re-plan the path based on the spatial perception model, generate a semantically related guide path, and send it to the augmented reality terminal.

[0045] Step 104: The augmented reality terminal highlights the tour path and preloads the explanatory content corresponding to the exhibits on the tour path.

[0046] In one embodiment, highlight rendering is a visualization display processing method for augmented reality terminals. It can highlight the tour route in the user's field of vision through virtual markers such as lines, arrows, and halos to guide the user's movement.

[0047] In one embodiment, the explanation content includes introductory texts, audio materials, and graphic materials corresponding to each exhibit along the tour route.

[0048] In one embodiment, preloading the explanatory content corresponding to the exhibits on the guided tour path refers to caching the explanatory content to the local augmented reality terminal in advance, which can avoid problems such as content loading delays and playback stuttering during the user's visit.

[0049] In one embodiment, after receiving the guided tour path data, the augmented reality terminal overlays virtual visual elements onto the real-world scene to complete the path highlighting rendering, such as overlaying blue arrows connecting exhibits; at the same time, it obtains the explanatory content of the exhibits along the guided tour path, stores it in the local cache area, and calls it to play when the user arrives at the corresponding exhibit location.

[0050] In one embodiment, the path of the highlighted rendering can be adjusted based on the user's gaze; if the user is looking at an exhibit, the navigation system will enhance the highlight rendering of that point.

[0051] This embodiment constructs a spatial perception model by collecting user spatial and behavioral data within the venue. It analyzes user interests based on user voice question data and completes semantic matching. Based on the matching results, it selects to generate answer information or replan the tour route. Finally, the augmented reality terminal completes the visualization display of the route and pre-loads the explanation content. It can respond to users' real-time visit requests, distinguish different interaction scenarios and provide corresponding services. It can quickly answer users' on-site questions and adjust the tour route according to users' interests. At the same time, it enhances the user's visit experience through pre-loading and AR visualization, realizing intelligent and personalized tour guidance in exhibition venues.

[0052] In one embodiment, generating a spatial awareness model based on the user space data and the user behavior data includes: The positioning sensing node determines the user's planar coordinates within the exhibition area based on the user space data, and the exhibition area includes multiple exhibition grids; In one embodiment, the exhibition grid is the smallest statistical unit obtained after dividing the overall space of the exhibition venue into regions. The grid division can be completed according to rules such as fixed area, exhibit distribution, and visitor route. Each grid is independent of each other and completely covers the entire exhibition venue.

[0053] In one embodiment, the planar coordinates refer to the two-dimensional position coordinates of the user calculated with the exhibition venue as the reference coordinate system, which are used to identify the user's real-time location.

[0054] In one embodiment, user spatial data can be collected through methods such as in-venue positioning base stations, Bluetooth positioning, and visual positioning. The user spatial data can be analyzed and coordinates converted to determine the user's planar coordinates in the two-dimensional plane of the exhibition venue.

[0055] For example, the 5G base station in the exhibition hall receives response signals from user terminals and LiDAR nodes in each booth. After receiving the signals via the 5G NR uplink, the SA and DID variables are extracted. SA is the round-trip time of one radio frequency signal between the base station and the user terminal, and its value is proportional to the distance between the user and the base station. The 5G base station in the exhibition hall estimates the distance between the user and the base station using the following formula: Where d represents the distance between the user terminal and the base station, and c represents the speed of electromagnetic wave propagation in air (approximately 3 × 10⁸ m / s). The DID variable represents the coordinate distance between a certain display case and each user. The 5G base station in the exhibition hall retrieves the absolute physical coordinates of the display case from the spatial database, and, combined with the distance estimated by SA and DID, further calculates the dynamic planar position coordinates (x, y) of all users on the current exhibition hall plane.

[0056]

[0057] in, For the first The planar coordinates of each display case; For users and the The distance between base stations; Let represent the coordinate distance between the user and the i-th display case.

[0058] The 5G base station in the exhibition hall constructs a triangulation coordinate system by receiving the user's dynamic planar position coordinates from at least three different base stations. Assume the user's three base station coordinates are as follows: , , Then, the user's final dynamic planar position coordinates (x, y) can be calculated as follows:

[0059]

[0060]

[0061] The 5G base station in the exhibition hall uses a multi-source least squares positioning algorithm to iteratively solve the above equations, ultimately obtaining the user's precise coordinates on the exhibition hall's plane. .

[0062] The location sensing node determines the activity index of each exhibition grid based on the dwell time and frequency of occurrence in each exhibition grid in the user behavior data; In one embodiment, the activity index is a numerical value representing the degree of attention a single exhibition grid receives from users. A higher activity index indicates that more people visit the area and that users are more willing to stay.

[0063] In one embodiment, dwell time refers to the length of time a user stays continuously within a single exhibition grid, which can be used to reflect the user's level of attention; frequency of occurrence refers to the cumulative number of times a user enters the same exhibition grid during a single visit, which can be used to reflect the frequency of the user's visits to the area.

[0064] In one embodiment, quantitative scores and weighting coefficients can be configured for dwell time and frequency of occurrence, respectively. After weighted fusion and normalization of dwell time and frequency of occurrence, the activity index corresponding to each exhibition grid can be calculated. Alternatively, a graded rating method can be adopted, which divides dwell time and frequency of occurrence into grades according to preset intervals, and combines the grade results to comprehensively determine the activity index of each exhibition grid.

[0065] For example, the display case's LiDAR obtains the user's planar position coordinates from the 5G base station. And combined with the real-time user movement heat map dataset captured by LiDAR Calculate the user's mobile hotspot vector Quantify the probability of user movement direction and divide the exhibition hall floor plan into The grid, each grid cell Record all users' dynamic planar position coordinates Duration of stay in the area Frequency of occurrence ; Calculate the thermal activity index ,in Configurable weights; This indicates the intensity of the user hotspot in the grid, i.e., the aforementioned activity index.

[0066] The positioning sensing node generates the user's movement vector based on the planar coordinates and the activity index of each exhibition grid; In one embodiment, the movement vector is vector data determined based on the user's real-time location and activity index, used to characterize the user's current movement direction, movement trend, and potential travel preferences.

[0067] In one embodiment, the user's current grid position is determined based on the user's planar coordinates. The activity index of each exhibition grid around the user's current grid position is then used to determine the user's next movement tendency and generate the corresponding user movement vector.

[0068] For example, the exhibition hall floor plan is divided into several grids, with each display case located in one or more grids. The activity index is based on the aforementioned exhibition grids. The system retrieves the user's current grid location. For display case grids the user hasn't visited, a weighted calculation of the user's movement vector, or movement tendency value, is performed based on distance and activity index. These movement tendency values ​​constitute the user's movement heatmap vector. The Each element represents the user's interaction with the display case. The user's potential movement interests are then analyzed. Ultimately, the user's movement vector is used to recommend the most likely next showroom path for the user. The mathematical expression for the user's movement vector is as follows:

[0069] in, : Indicates user's opinion on the display case The intensity of the tendency to move; Display case The activity index of the hot zone in the grid; User's current location and display case The Euclidean distance; (The maximum possible distance within the exhibition hall, such as the diagonal length of the exhibition hall). , , These are configurable weight parameters.

[0070] The positioning and sensing node generates a spatial sensing model based on the movement vector.

[0071] In one embodiment, a movement vector containing directional and preference components is generated by taking the user's current planar coordinates as a reference point and combining the activity index of the exhibition grids that the user has not yet visited in the adjacent exhibition grids and the spatial connectivity between the grids.

[0072] For example, such as Figure 3 As shown, Figure 3 This is a schematic diagram of a spatial perception model provided in an embodiment of this disclosure. Figure 3 The system uses all users and display cases as nodes, and connects the nodes by establishing edges between them through the user's movement vector and the spatial distance between the nodes.

[0073] In one embodiment, after generating the spatial awareness model, the 5G base station in the exhibition hall encapsulates the spatial awareness model into a data packet in JavaScript Object Notation (JSON) format through the 5G slice network, and then sends the data packet to the user terminal, such as the user's smartphone.

[0074] This embodiment first determines the user's planar coordinates based on spatial data, then determines the activity index of each exhibition grid based on user behavior data, and finally generates user movement vectors based on the user's planar coordinates and the activity index of each exhibition grid. Based on the movement vectors, a spatial perception model is generated. By generating a spatial perception model from multiple dimensions such as location, regional popularity, and movement trend, the deviation of the spatial perception model caused by single data modeling is avoided.

[0075] In one embodiment, the tour guiding method further includes: The user terminal determines the location of the exhibit based on the spatial perception model. In one embodiment, the exhibit location of the user refers to the exhibit location corresponding to the area where the user is currently located within the exhibition venue. The exhibit location can be determined by the coordinate range of the exhibit, the grid to which the exhibit belongs, and the unique identifier of the exhibit.

[0076] In one embodiment, the coordinate information of the exhibition venue, the distribution mapping relationship of exhibits, and the real-time planar coordinates of the user can be extracted from the spatial perception model. The user's coordinates are compared and matched with the coordinate range of each exhibit to determine the current location of the exhibit where the user is located.

[0077] In one embodiment, the grid area to which the user belongs can be determined first, based on the exhibition grid division rules in the spatial perception model, and then the location of the exhibit where the user is located can be determined based on the binding relationship between the grid and the exhibit.

[0078] For example, the spatial perception model pre-stores the coordinate ranges and grid numbers of all exhibits in the exhibition area. After the user terminal obtains the model data, it matches the calculated real-time user coordinates with the coordinate ranges of the exhibits one by one. When the user's coordinates fall within the coordinate range of a certain exhibit, it is determined that the user is at the position of that exhibit, and the unique identifier number of that exhibit is output.

[0079] The augmented reality terminal displays navigation information for the exhibit corresponding to the user's current location.

[0080] In one embodiment, the guide information for an exhibit refers to the explanatory materials for the corresponding exhibit. The guide information may include, but is not limited to, text descriptions, audio explanations, picture displays, video explanations, historical background information, and other information.

[0081] In one embodiment, based on the determined location of the user at the exhibit, the guide information corresponding to the exhibit is obtained, and the guide information is sent to the augmented reality terminal for presentation.

[0082] In one embodiment, the augmented reality terminal combines real-world images to overlay and display tour information. For example, text and images can be displayed floating around the exhibits, and audio explanations can be played simultaneously.

[0083] In one embodiment, the display format of the guide information can be adaptively switched according to the device's capabilities. For example, when the device has excellent display effects, it will prioritize displaying content combining text and images, and when the device emphasizes audio interaction, it will prioritize playing audio explanations.

[0084] For example, the exhibition space is divided into Each exhibition grid can use LiDAR to obtain the real-time crowd density of all display cases within the current exhibition area. ,in Indicates the first Individual display cases, Indicates the first How many people are in each display case?

[0085] like If so, the display case is marked as a high-density area, and its priority is reduced in path planning; if If so, the display case will be marked as a recommendable area, and its exhibits will be considered based on semantic tags. semantic tags of exhibits that users have already viewed The similarity scores are ranked, and the similarity calculation uses a semantic distance function, the mathematical expression of which is as follows:

[0086] in, The preset number of participants threshold is used; the final recommended path is... Sort by size (largest to smallest) and exclude display cases in high-density areas to obtain... ,in To recommend the number of display cases, This refers to the showcase node in the recommended path.

[0087] This embodiment determines the user's location on the exhibit based on a spatial perception model, and the augmented reality terminal displays the corresponding exhibit's guide information. The spatial perception model enables automatic identification of the user's location on the exhibit, eliminating the need for manual operation by the user to trigger the explanation. It can automatically match and push exhibits and guide content, improving the automation level of the guide service.

[0088] In one embodiment, as the user moves around, the positioning and sensing node continuously receives user spatial data and user behavior data, updates the spatial perception model, and re-evaluates the priority of the guided tour path and the explanatory content.

[0089] In one embodiment, sending the voice interaction information to the inference server includes: The user terminal parses the voice interaction information into first text interaction information, and generates the user's target interaction information based on the first text interaction information and the user's second text interaction information; The user terminal sends the target interaction information to the inference server.

[0090] In one embodiment, the voice interaction information is used to carry the user's questioning intent, and the first text interaction information is the text content obtained after the voice interaction information has been processed into text, which is used to present the user's question, keywords and expressive requests more intuitively.

[0091] In one embodiment, the second text interaction information is the content, frequency, and topic of the questions asked by the user in the exhibition venue before this question was asked. The second text interaction information can reflect the user's long-term focus and visiting preferences.

[0092] In one embodiment, the target interaction information is a continuous semantic data sequence formed by the first text interaction information and the second text interaction information. The target interaction information can relatively completely restore the context and expression logic of the user's multi-turn interaction.

[0093] In one embodiment, the voice interaction information can be converted using a speech recognition algorithm to extract the text content from the voice interaction information and obtain the corresponding first text interaction information; alternatively, the speech can be parsed locally based on a locally deployed speech recognition model to obtain the corresponding first text interaction information.

[0094] In one embodiment, the user's second text interaction information can be obtained by organizing past user questions in chronological order from the local storage area; alternatively, the second text interaction information can be categorized according to topic tags to filter out the second text interaction information related to the currently viewed content.

[0095] In one embodiment, the first text interaction information and the second text interaction information can be sequentially concatenated and fused with keywords according to semantic association rules to generate the user's target interaction information; alternatively, the first text interaction information and the second text interaction information can be fused using a pre-trained semantic fusion model to obtain the user's target interaction information.

[0096] For example, the user terminal will ask the user questions via real-time voice. Convert to standard text Extract core keywords and question intent from the text; read the user's historical question records from the local cache. This process involves identifying the thematic thread and key focus of past questions; linking real-time text content with past questions in chronological order of interaction; and supplementing this with semantic association markers to form the user's target interaction information, which serves as the contextual semantic flow. Contextual semantic flow usually exists in the form of structured data, which contains both the independent semantics of a single question and the contextual associations of multiple rounds of interaction.

[0097] In one embodiment, the user terminal can send target interaction information to the inference server via communication methods such as wireless local area network or mobile communication network.

[0098] In one embodiment, the user terminal may first compress and encapsulate the target interactive information, and then complete the data transmission according to a preset communication protocol to improve transmission efficiency and data security.

[0099] This embodiment first parses the voice interaction information into first text interaction information, then combines it with the user's second text interaction information to generate the user's target interaction information, and finally sends the target interaction information to the inference server. The inference server performs data preprocessing based on the first and second text interaction information to avoid the problem of one-sided semantic understanding caused by relying on a single question text, and enables the inference server to accurately identify the user's intent by combining the complete context.

[0100] For example, the inference server... Each question in the query is encoded using word embeddings to generate a semantic vector. The semantic weights between the current question and historical questions are calculated using an attention mechanism. Identify contextual dependencies; use the softmax function to generate exhibition interest prediction vectors. This indicates the strength of a user's preference for a specific topic (such as historical events, figures, or cultural background). If... If the semantic similarity between the semantic vectors of the current navigation path and the semantic vectors of the next three exhibits is below a set threshold, it is considered an isolated problem, and a new navigation route map is generated; otherwise, the basic explanation content is returned directly. The mathematical expression for semantic similarity is as follows:

[0101] in, This represents the user's current interest prediction vector; represents the semantic label vector of the i-th recommended showcase on the current navigation path; k represents the number of recommended exhibits on the navigation path; This represents the cosine similarity function, used to measure the semantic consistency between user interest vectors and exhibit semantic tags; This represents the semantic connectivity threshold, which can be set to 0.5.

[0102] If the above expression is true, it means that the semantic correlation between the user's interests and the exhibits on the current navigation path is weak, and the navigation route needs to be replanned.

[0103] In one embodiment, based on the spatial awareness model, a semantically related navigation path is generated, including: The reasoning server starts from the exhibit node corresponding to the user's current exhibit location, performs multi-hop semantic queries in the exhibit knowledge graph, and extracts a set of exhibit nodes associated with the user's interest features. In one embodiment, an exhibit node is a basic data unit within the exhibit knowledge graph used to represent a single exhibit. Each exhibit node is bound to data such as the name, category, theme, description information, and exhibition area of ​​the corresponding exhibit.

[0104] In one embodiment, the exhibit knowledge graph is a structured network built based on the inherent relationships between exhibits. The exhibit knowledge graph may include, but is not limited to, semantic relationships such as the category association of exhibits, historical context association, technological association, and thematic content association.

[0105] In one embodiment, multi-hop semantic query refers to a query method that starts from the initial exhibit node (i.e., the exhibit corresponding to the current exhibit location of the user) and extends the retrieval layer by layer outward along the semantic relationships in the knowledge graph, which can discover various exhibits that are directly or indirectly related.

[0106] In one embodiment, the exhibit node set is a set of all exhibit nodes that are semantically related to the user's interest features, selected after multi-hop semantic querying.

[0107] In one embodiment, a graph retrieval algorithm can be used to traverse the exhibit knowledge graph. Starting from the current exhibit node, adjacent nodes are traversed according to a preset hop limit (such as 3 hops). Exhibit nodes that match the user's interest characteristics are selected according to similarity rules to form an exhibit node set.

[0108] For example, the node corresponding to the exhibit currently viewed by the user is used as the starting point for the query. The core keywords and topic tags in the user's interest features are extracted. According to the semantic association links of the exhibit knowledge graph, related exhibits are retrieved in sequence to complete multi-hop queries. All retrieved exhibit nodes are summarized and deduplicated to obtain a set of exhibit nodes. All exhibits in the set of exhibit nodes are semantically matched to the user's visiting interests.

[0109] The inference server determines the semantic association strength of each exhibit node in the exhibit node set; In one embodiment, the semantic association strength is a numerical value representing the degree of correlation between the semantic tags of exhibit nodes and user interest features. The higher the value of the semantic association strength, the higher the degree of correlation between the corresponding exhibit and the user's interest.

[0110] In one embodiment, keyword matching can be performed based on the attribute tags, thematic keywords, and user interest characteristics of the exhibit nodes to determine the semantic association strength of each exhibit node.

[0111] In one embodiment, a semantic association level standard can be preset, which is divided into three levels according to the degree of association: strong association, general association, and weak association. Each level is assigned a corresponding score to determine the semantic association strength of each exhibit node.

[0112] The reasoning server generates a semantically associated navigation path based on the spatial awareness model and the semantic association strength.

[0113] In one embodiment, the semantic association strength can be used as the main sorting criterion, and the path can be spliced ​​by combining the exhibit location and regional connectivity of the exhibition venue in the spatial perception model, and the exhibit nodes can be connected in sequence to generate a semantically associated tour path.

[0114] In one embodiment, the semantic association strength and the distance between exhibits in the exhibition area can be weighted separately, and a comprehensive path score can be calculated by weighting. The layout scheme with the higher comprehensive score is selected to generate a semantically associated guide path.

[0115] This embodiment uses multi-hop semantic queries based on the exhibit knowledge graph to filter out a set of exhibit nodes that match the user's interests. Then, it determines the semantic association strength between the semantic tags of each exhibit node and the user's interest features. Finally, it generates a tour guide path based on the spatial perception model and the semantic association strength. This ensures that the tour guide content matches the user's interests while also taking into account the actual spatial layout and access logic of the exhibition venue. It avoids the problems of route detours and unreasonable visitor routes caused by relying solely on semantically generated paths, thus improving the rationality of the tour guide path.

[0116] For example, the semantic correlation between user interests and exhibits on the current navigation path is weak. When exhibits that the user is interested in and exhibits corresponding to the user's current location are in the same exhibition area, such as... Figure 4 As shown, Figure 4 A flowchart illustrating a method for generating a navigation path provided in this embodiment of the disclosure includes: an inference server accessing a pre-built knowledge graph. The map contains historical event nodes. Character nodes Cultural background nodes Edges represent the semantic association strength. .

[0117] Continuous semantic stream Importing knowledge graphs Generate a reasoning path, the steps are as follows: Based on the current exhibit node ,from Extract the set of nodes with direct semantic association. ,in Indicates historical events related to the current exhibits. Indicates the relevant figures, Representing cultural background, this set constitutes the first hop semantic association path. Based on continuous semantic flow... Transform it into a set of semantic tags and from Extracting and Related semantic node set: ,right For each node in the sequence, its semantically related nodes are extracted again to form a set of multi-hop semantic extension paths. The mathematical expression is as follows:

[0118] in, Indicates the hop count limit (default is 3). The function returns the number of hops starting from the current node. The set of all semantically related exhibit nodes that can be reached after layer diffusion.

[0119] right Each node in Calculate the sum of its semantic strengths. The mathematical expression is as follows:

[0120] in, Represents a node The association strength in the semantic hop at the h-th layer, where h represents an influencing factor constant that can be adjusted according to business needs.

[0121] Combined with exhibit spatial perception model Calculate the priority score of the recommended path. This score integrates three dimensions: semantic strength, spatial accessibility, and user hotspot behavior. Its mathematical expression is as follows: ; in, This represents an adjustable weight parameter that satisfies... ; Indicates the semantic intensity attenuation coefficient; Indicates the user's current location and the exhibits Spatial distance between them; This represents the maximum recommended distance threshold for the exhibition hall, used to normalize spatial distances. This indicates the activity index of users walking around the recommended exhibits. This represents the maximum value of the heat zone during movement within the exhibition hall, used to normalize the heat zone intensity.

[0122] Output The highest-ranking path is used as the recommended path, i.e., the aforementioned semantically related navigation path. If multiple recommended paths exist and Approaching (i.e.) Then iterative adjustment Until the recommended path is unique.

[0123] The semantic correlation between user interests and exhibits on the current navigation path is weak. When the exhibits that the user is interested in and the exhibits corresponding to the user's current location are in different exhibition areas, the following navigation method can be used to generate a navigation path with semantic correlation across exhibition areas.

[0124] In one embodiment, based on the spatial awareness model, a semantically related navigation path is generated, including: When the voice interaction information involves exhibits from a different exhibition area than the current exhibition area, the positioning perception node sends the spatial perception model and the exhibit knowledge graph corresponding to each exhibition area to the inference server. In one embodiment, the exhibition area is an independent visitor area divided by the exhibition venue according to the theme of the exhibits, functional areas, and spatial location. Different exhibition areas display different types and themes of exhibits.

[0125] In one embodiment, the exhibit knowledge graph corresponding to each exhibition area is a semantic network constructed from the exhibits within the corresponding exhibition area, recording semantic relationships such as theme association, category association, and content association between exhibits within the exhibition area.

[0126] In one embodiment, a semantic recognition algorithm can be used to parse the voice interaction information, extract exhibit keywords and topic tags from the voice interaction information, compare the extracted content with the exhibit information of the current exhibition area, and determine whether the voice interaction information involves exhibits from different exhibition areas than the current exhibition area.

[0127] In one embodiment, the exhibit's exhibition area identifier can also be matched based on the voice interaction information. If the matching result is inconsistent with the identifier of the exhibition area where the user is currently located, it is determined that the voice interaction information involves exhibits in a different exhibition area than the current exhibition area.

[0128] For example, the inference server parses the user's target interaction information, extracts the exhibit theme and name that the user is interested in, and retrieves the exhibition area number to which the exhibit belongs; it compares the exhibition area number with the exhibition area number where the user is currently located, and if the two do not match, it means that the voice interaction information involves exhibits in a different exhibition area than the current exhibition area.

[0129] The reasoning server determines the cross-exhibition-area spatial perception model based on the spatial perception model corresponding to each exhibition area, and determines the cross-exhibition-area knowledge graph based on the exhibit knowledge graph corresponding to each exhibition area. In one embodiment, the cross-exhibition area spatial perception model can reflect global spatial information such as the positional relationship, travel routes, and regional connectivity between different exhibition areas.

[0130] In one embodiment, the cross-exhibition area knowledge graph not only includes the associations of exhibits within each exhibition area, but also establishes cross-domain semantic relationships between exhibits in different exhibition areas.

[0131] In one embodiment, when determining a cross-exhibition-area spatial perception model based on the spatial perception model corresponding to each exhibition area, coordinate alignment and data splicing can be performed on the spatial coordinates, area boundaries, and passage paths of multiple exhibition areas to supplement spatial data such as channels and connecting road segments between exhibition areas, thereby forming a cross-exhibition-area spatial perception model.

[0132] In one embodiment, when determining the cross-exhibition area knowledge graph based on the exhibit knowledge graph corresponding to each exhibition area, the original semantic relationships within each exhibition area can be retained first, and then the cross-exhibition area semantic links between exhibits in different exhibition areas can be determined according to the exhibit theme, historical context, and process characteristics, so as to realize the connection and fusion of exhibit knowledge graphs corresponding to multiple exhibition areas.

[0133] For example, after receiving spatial perception models corresponding to multiple exhibition areas, the inference server unifies the coordinate reference, splices the data of the exhibition area connection channels, and generates a cross-exhibition area spatial perception model that can reflect the location of multiple exhibition areas in the entire venue; at the same time, it merges the received multiple exhibit knowledge graphs, adds semantic associations between exhibits of the same theme and exhibits of the same origin in different exhibition areas, and obtains a cross-exhibition area knowledge graph.

[0134] The reasoning server takes the exhibit node corresponding to the user's current exhibit location as the starting point and generates a semantically related navigation path based on the cross-exhibition area spatial perception model and the cross-exhibition area knowledge graph.

[0135] In one embodiment, the exhibit node corresponding to the user's current exhibit location is used as the starting point for path planning. Multi-hop semantic retrieval is performed based on the cross-exhibition area knowledge graph to filter out cross-exhibition area exhibit nodes that match the user's interest characteristics. At the same time, the global location, connectivity routes, and regional distribution information in the cross-exhibition area spatial perception model are combined to connect the filtered exhibit nodes into a route.

[0136] In one embodiment, the path can be sorted and combined by taking into account factors such as the semantic association strength between exhibits, the travel distance between exhibition areas, and the main visitor routes of the venue, and the target exhibits in different exhibition areas can be connected in sequence to generate a cross-exhibition area semantic association guide path.

[0137] This embodiment generates a cross-exhibition-area spatial perception model and a cross-exhibition-area knowledge graph based on the spatial perception model and exhibit knowledge graph of each exhibition area when user interaction information involves exhibits from other exhibition areas. Then, it completes cross-exhibition-area navigation path planning based on these models. This approach breaks through the data and semantic limitations of a single exhibition area. It ensures semantic relevance between exhibits in different exhibition areas based on the cross-exhibition-area knowledge graph, while relying on the cross-exhibition-area spatial perception model to guarantee smooth route flow across multiple exhibition areas. This solves the problems of unreasonable path planning and incomplete semantic matching in cross-exhibition-area scenarios.

[0138] For example, the inference server receives exhibit spatial awareness models from each exhibition area. The model will incorporate the knowledge graphs of each exhibition area. By merging the data, a cross-exhibition area semantic graph can be generated. This forms a unified semantic space; and integrates the spatial perception models of exhibits in each exhibition area. Integrate into a cross-exhibition area spatial perception model It is used to support path recommendations for users in different exhibition areas.

[0139] The inference server starts from the current exhibit node and will... and As input, following the steps described above—that exhibits of interest to the user and exhibits corresponding to the user's current location are located in the same exhibition area—a semantically related navigation path is generated. Specifically, the inference server takes the exhibit node corresponding to the user's current exhibit location as the starting point, performs multi-hop semantic queries in the cross-exhibition area knowledge graph, and extracts a set of target exhibit nodes associated with the user's interest features; the inference server determines the target semantic association strength of each exhibit node in the target exhibit node set; and the inference server generates a semantically related navigation path based on the cross-exhibition area spatial perception model and the target semantic association strength.

[0140] For example, such as Figure 5 As shown, Figure 5 A flowchart illustrating another tour guide method provided in this disclosure includes: S1: When a user enters the exhibition hall wearing AR glasses and starts the guided tour mode, the 5G base station in the exhibition hall sends a fixed radio frequency command signal to the user terminal; at the same time, the lidar node in the display case continuously scans the user terminal, generates the exhibit perception point cloud map signal and broadcasts it to the 5G base station.

[0141] S2: The 5G base station in the exhibition hall analyzes the response information and the SA and DID variables in the point cloud map signal to calculate the spatial distance between the user and the display case. The dynamic spatial coordinates are determined by indoor triangulation. The lidar combines the coordinates and the user's walking hot zone data to divide the grid into an N×N grid, calculates the grid hot zone activity index, determines the user's movement hot zone vector, and finally generates a dynamically updated exhibit spatial perception model and uploads it to the user's AR glasses.

[0142] S3: The user's smartphone determines the ID of the display case based on the exhibition space perception model, and plays the corresponding guide content in real time through AR glasses; at the same time, a navigation route map is generated based on the real-time crowd density in front of the display case, the user's movement hot zone vector, and the correlation of the display case content, and is fed back to the AR glasses to guide the user to the next display case.

[0143] S4: During navigation, the user initiates a question using AR glasses. The phone's semantic layer collects the voice content and parses it into text. This text is then input into a cloud-based knowledge reasoning server for contextual question-and-answer behavior sequence analysis, generating an exhibition interest prediction vector. The system then determines whether the question is a non-isolated question (answerable within the current navigation route). If it is not an isolated problem: the server directly generates a solution using a built-in large model and feeds it back to the AR glasses; If not (isolated problem): The server combines the exhibit spatial perception model to perform semantic correlation evaluation, regenerates the recommended spatial navigation route, and feeds it back to the AR glasses.

[0144] S5: If it is an isolated problem, the cloud-based knowledge reasoning server combines the exhibit space perception model to perform semantic correlation evaluation and processing. Starting from the current exhibit node, it performs multi-hop queries to generate a semantic path containing a continuous semantic flow (historical event timeline, interpersonal relationship network, cultural background), and combines the exhibit space perception model to generate a recommended spatial navigation route.

[0145] S6: If the question involves cross-exhibition area exhibit association issues, notify the exhibition hall's 5G base station to send the exhibit spatial perception models of multiple exhibition areas to the cloud knowledge reasoning server. In the reasoning layer, integrate the cross-exhibition area spatial coordinate models, perform cross-area level semantic association degree evaluation processing, generate new spatial navigation line recommended paths, and dynamically update them to the AR glasses.

[0146] S7: After the cloud-based knowledge reasoning server generates a recommended path for spatial navigation, it sends the result to the user's AR glasses, triggering path highlighting rendering and preloading of explanation content in the AR glasses.

[0147] S8: The 5G base station in the exhibition hall continuously receives spatial data from the lidar of the display cases and user terminals, dynamically updates the spatial perception model of the exhibits, and re-evaluates the priority of path recommendations and explanation content as users move around.

[0148] In summary, the solution provided in this public disclosure is as follows: By acquiring user spatial data and user behavior data to generate a spatial perception model, and analyzing the collected voice interaction information to obtain user interest characteristics, the system selects to directly generate answer information or generate a semantically related guide path based on the semantic similarity between user interest characteristics and the exhibits corresponding to the current guide path. The augmented reality terminal then completes the path highlighting rendering and preloading of explanation content. This can simultaneously meet users' needs for immediate Q&A and personalized visits, solving the problems of fixed traditional guide routes and inability to respond to personalized user questions. It also improves the content association and continuous narrative logic between exhibit explanations, thereby enhancing the user's visitor experience.

[0149] To implement the tour guide method provided in this disclosure, this disclosure also provides a tour guide system, such as... Figure 6 As shown. Figure 6 This is a schematic diagram of the structure of a tour guide system provided in an embodiment of the present disclosure. The tour guide system 600 includes a positioning sensing node 601, a user terminal 602, an inference server 603, and an augmented reality terminal 604. The positioning sensing node 601 is used to respond to a user entering the exhibition venue and activating the guided tour mode, acquire user space data and user behavior data, generate a spatial perception model based on the user space data and user behavior data, and send the spatial perception model to the user terminal 602. The user terminal 602 is used to collect voice interaction information initiated by the user through the augmented reality terminal 604 and send the voice interaction information to the inference server 603. The reasoning server 603 is used to generate user interest features based on the voice interaction information, determine the semantic similarity between the user interest features and the exhibits corresponding to the current tour path; if the semantic similarity is greater than a preset similarity threshold, generate answer information and send it to the augmented reality terminal 604; if the semantic similarity is less than or equal to the preset similarity threshold, generate a semantically associated tour path based on the spatial perception model, and send the semantically associated tour path to the augmented reality terminal 604. The augmented reality terminal 604 is used to highlight the tour path and preload the explanatory content corresponding to the exhibits on the tour path.

[0150] In one embodiment, the positioning sensing node 601 is used to determine the planar coordinates of a user within the exhibition space based on the user space data, the exhibition space including multiple exhibition grids; The location sensing node 601 is used to determine the activity index of each exhibition grid based on the dwell time and frequency of occurrence in each exhibition grid in the user behavior data. The positioning sensing node 601 is used to generate the user's movement vector based on the planar coordinates and the activity index of each exhibition grid; The positioning and sensing node 601 is used to generate a spatial sensing model based on the movement vector.

[0151] In one embodiment, the user terminal 602 is used to determine the location of the exhibit where the user is located based on the spatial perception model; The augmented reality terminal 604 is used to display guide information for the exhibits corresponding to the user's current location.

[0152] In one embodiment, the user terminal 602 is configured to parse the voice interaction information into first text interaction information, and generate the user's target interaction information based on the first text interaction information and the user's second text interaction information; The user terminal 602 is used to send the target interaction information to the inference server 603.

[0153] In one embodiment, the inference server 603 is used to perform multi-hop semantic queries in the exhibit knowledge graph, starting from the exhibit node corresponding to the user's current exhibit location, and extract a set of exhibit nodes associated with the user's interest features. The inference server 603 is used to determine the semantic association strength of each exhibit node in the exhibit node set; The reasoning server 603 is used to generate a semantically associated navigation path based on the spatial awareness model and the semantic association strength.

[0154] In one embodiment, in response to the voice interaction information involving exhibits in a different exhibition area from the current exhibition area, the positioning perception node 601 is used to send the spatial perception model corresponding to each exhibition area and the exhibit knowledge graph corresponding to each exhibition area to the inference server 603. The reasoning server 603 is used to determine a cross-exhibition-area spatial perception model based on the spatial perception model corresponding to each exhibition area, and to determine a cross-exhibition-area knowledge graph based on the exhibit knowledge graph corresponding to each exhibition area. The reasoning server 603 is used to generate a semantically related navigation path based on the cross-exhibition area spatial perception model and the cross-exhibition area knowledge graph, starting from the exhibit node corresponding to the user's current exhibit location.

[0155] It should be noted that the tour guide system provided in the above embodiments is only illustrated by the division of the above program modules. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the tour guide system can be divided into different program modules to complete all or part of the processing described above. In addition, the tour guide system provided in the above embodiments and the tour guide method embodiments provided in this disclosure belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0156] Figure 7 This is a schematic diagram of the hardware composition structure of the electronic device provided in the embodiments of this disclosure, such as... Figure 7 As shown, the electronic device 700 includes at least one processor 702; and a memory 701 communicatively connected to the at least one processor 702; wherein the memory 701 stores instructions executable by the at least one processor 702, the instructions being executed by the at least one processor 702 to implement the steps of the navigation method of the present disclosure embodiments.

[0157] Optionally, the electronic device may specifically be a tour guide system according to the embodiments of this application, and the electronic device may implement the corresponding processes implemented by the tour guide system in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0158] It is understood that the electronic device also includes a communication interface 703. Various components in the electronic device are coupled together via a bus system 704. It is understood that the bus system 704 is used to implement communication between these components. In addition to a data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general designated all buses as Bus System 704.

[0159] It is understood that memory 701 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 701 described in this embodiment of the invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0160] The methods disclosed in the above embodiments can be applied to or implemented by processor 702. Processor 702 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the hardware of processor 702 or by instructions in software form. Processor 702 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 702 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, specifically memory 701. Processor 702 reads information from memory 701 and, in conjunction with its hardware, completes the steps of the aforementioned methods.

[0161] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0162] This disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, which are used to cause a computer to execute the steps of the navigation method of the present invention.

[0163] Optionally, the computer-readable storage medium can be applied to the tour guide system in the embodiments of this application, and the computer instructions cause the computer to execute the corresponding processes implemented by the tour guide system in the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0164] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the navigation method steps provided in this embodiment of the invention.

[0165] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0166] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0167] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0168] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0169] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0170] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A tour guide method, characterized in that, Applied to a navigation system, the system including a positioning sensing node, a user terminal, an inference server, and an augmented reality terminal, the method includes: In response to a user entering the exhibition venue and activating the guided tour mode, the positioning sensing node acquires user spatial data and user behavior data, generates a spatial perception model based on the user spatial data and user behavior data, and sends the spatial perception model to the user terminal. The user terminal collects voice interaction information initiated by the user through the augmented reality terminal and sends the voice interaction information to the inference server. The reasoning server generates user interest features based on the voice interaction information and determines the semantic similarity between the user interest features and the exhibits corresponding to the current tour path. If the semantic similarity is greater than a preset similarity threshold, it generates answer information and sends it to the augmented reality terminal. If the semantic similarity is less than or equal to the preset similarity threshold, it generates a semantically associated tour path based on the spatial perception model and sends the semantically associated tour path to the augmented reality terminal. The augmented reality terminal highlights the tour path and preloads the explanatory content corresponding to the exhibits on the tour path.

2. The method according to claim 1, characterized in that, The generation of a spatial awareness model based on the user space data and the user behavior data includes: The positioning sensing node determines the user's planar coordinates within the exhibition area based on the user space data, and the exhibition area includes multiple exhibition grids; The location sensing node determines the activity index of each exhibition grid based on the dwell time and frequency of occurrence in each exhibition grid in the user behavior data; The positioning sensing node generates the user's movement vector based on the planar coordinates and the activity index of each exhibition grid; The positioning and sensing node generates a spatial sensing model based on the movement vector.

3. The method according to claim 1, characterized in that, The method further includes: The user terminal determines the location of the exhibit where the user is located based on the spatial perception model. The augmented reality terminal displays navigation information for the exhibit corresponding to the user's current location.

4. The method according to claim 1, characterized in that, Sending the voice interaction information to the inference server includes: The user terminal parses the voice interaction information into first text interaction information, and generates the user's target interaction information based on the first text interaction information and the user's second text interaction information; The user terminal sends the target interaction information to the inference server.

5. The method according to claim 1, characterized in that, The generation of semantically related navigation paths based on the spatial awareness model includes: The inference server starts from the exhibit node corresponding to the user's current exhibit location, performs multi-hop semantic queries in the exhibit knowledge graph, and extracts a set of exhibit nodes associated with the user's interest features. The inference server determines the semantic association strength of each exhibit node in the exhibit node set; The reasoning server generates a semantically associated navigation path based on the spatial awareness model and the semantic association strength.

6. The method according to claim 1, characterized in that, The generation of semantically related navigation paths based on the spatial awareness model includes: When the voice interaction information involves exhibits from a different exhibition area than the current exhibition area, the positioning perception node sends the spatial perception model and the exhibit knowledge graph corresponding to each exhibition area to the inference server. The reasoning server determines the cross-exhibition-area spatial perception model based on the spatial perception model corresponding to each exhibition area, and determines the cross-exhibition-area knowledge graph based on the exhibit knowledge graph corresponding to each exhibition area. The reasoning server takes the exhibit node corresponding to the user's current exhibit location as the starting point and generates a semantically related navigation path based on the cross-exhibition area spatial perception model and the cross-exhibition area knowledge graph.

7. A tour guide system, characterized in that, The system includes a positioning and sensing node, a user terminal, an inference server, and an augmented reality terminal. The positioning and sensing node is used to respond to a user entering the exhibition venue and activating the guided tour mode, acquire user spatial data and user behavior data, generate a spatial perception model based on the user spatial data and user behavior data, and send the spatial perception model to the user terminal. The user terminal is used to collect voice interaction information initiated by the user through the augmented reality terminal and send the voice interaction information to the inference server. The reasoning server is used to generate user interest features based on the voice interaction information, determine the semantic similarity between the user interest features and the exhibits corresponding to the current tour path; if the semantic similarity is greater than a preset similarity threshold, generate answer information and send it to the augmented reality terminal. If the semantic similarity is less than or equal to a preset similarity threshold, a semantically related navigation path is generated based on the spatial perception model, and the semantically related navigation path is sent to the augmented reality terminal. The augmented reality terminal is used to highlight the tour path and preload the explanatory content corresponding to the exhibits on the tour path.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.

Images