Interactive tourism guide system based on AR (Augmented Reality) technology

Abstract

The invention discloses an interactive travel guide system based on an AR technology, and relates to the field of travel guide. Through the three-dimensional virtual tour guide generation module and the historical scene miniature reduction module, a unique virtual tour guide image is constructed in combination with local cultural characteristics, and a historical scene is presented in a miniature model manner, so that a user can intuitively perceive the original appearance and change of a relics or cultural place, and the manner greatly enriches tour guide contents and improves the tour guide efficiency. According to the method, the cultural cognition depth of tourists is improved, the immersion and interestingness of sightseeing are enhanced, and compared with traditional character labeling or static information superposition, a more vivid and visual historical culture display means is provided. Besides, the data processing and interaction control module is used for dynamically updating the presented virtual tour guide information and the historical scene restoration content, and the personalized interaction mode not only improves the flexibility of tour guide, but also can adjust the display content according to the real-time feedback of the user, thereby meeting the diversified requirements of tourists.

Description

Technical Field

[0001] This application relates to the technical field of tourist guides, and particularly to an interactive tourist guide system based on AR technology. Background Art

[0002] In the current trend of digital upgrading in the tourism industry, using augmented reality (AR) technology to provide interactive guide services for tourists has become an important means to enhance the tourism experience. By superimposing virtual scenes and guide information on the real environment where users are located, AR technology can help tourists more intuitively and vividly understand the scenic area background and cultural connotations.

[0003] However, most of the existing technologies focus on simple text annotations or static information superimposition, lacking in-depth perception of user interest and dynamic adaptation to environmental changes. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide an interactive tourist guide system based on AR technology.

[0005] This application provides an interactive tourist guide system based on AR technology, including:

[0006] A scene collection and positioning module, configured to perform spatial positioning and scene capture on the tourist environment where the user is located, and obtain real-scene images;

[0007] A three-dimensional virtual tour guide generation module, configured to construct a virtual tour guide image according to local cultural characteristics;

[0008] An AR display and fusion module, configured to superimpose the virtual tour guide image on the real-scene image in the AR interface, and perform illumination estimation, perspective projection transformation, and dynamic rendering optimization on the layer where the virtual tour guide image is located;

[0009] A historical scene miniature restoration module, configured to construct a three-dimensional historical scene model corresponding to the real-scene image and present it in the form of a miniature model in the AR interface;

[0010] A data processing and interaction control module, configured to obtain the interaction information of the user, and dynamically update the virtual tour guide image and the three-dimensional historical scene model based on the interaction information;

[0011] Among them, the data processing and interaction control module works in cooperation with the scene collection and positioning module, the three-dimensional virtual tour guide generation module, the AR display and fusion module, and the historical scene miniature restoration module to form a multi-threaded data processing pipeline.

[0012] In one embodiment, the three-dimensional virtual tour guide generation module includes:

[0013] A hierarchical rendering unit for fusing several layers including a clothing texture layer, a facial expression layer, and a lip-sync layer on a GPU-accelerated rendering pipeline;

[0014] An animation driving unit for realizing the animation switching of the virtual tour guide image by adopting a state machine-based management method.

[0015] In one embodiment, the historical scene micro-reduction module includes:

[0016] A historical data storage unit for storing historical information resources related to real-scene images;

[0017] A 3D reconstruction engine for converting historical information resources into a 3D historical scene model;

[0018] A scale scaling and transparency control unit for micro-reducing the 3D historical scene model according to a preset ratio and superimposing it on the real-scene image, and at the same time realizing a gradual contrast between the 3D historical scene model and the real-scene image by using transparency control means.

[0019] In one embodiment, the data processing and interaction control module includes:

[0020] A gaze tracking unit for identifying the gaze direction and dwell time of the user in the AR interface, so that the data processing and interaction control module updates the virtual tour guide image and / or the 3D historical scene model according to the recognition result of the gaze tracking unit as interaction information;

[0021] A gesture recognition unit for real-time detecting and pose estimating the key points of the user's hand, so that the data processing and interaction control module updates the virtual tour guide image and / or the 3D historical scene model according to the recognition result of the gesture recognition unit as interaction information.

[0022] In one embodiment, the data processing and interaction control module further includes a language understanding unit and a voice interaction unit;

[0023] The language understanding unit is used to convert the user's voice question into a text or semantic command through speech recognition and natural language processing algorithms;

[0024] The voice interaction unit is used to give a voice response to the text or semantic command through the virtual tour guide image.

[0025] In one embodiment, the system further includes an environment perception module for real-time monitoring environmental information;

[0026] The AR display and fusion module adjusts the brightness and contrast of the virtual tour guide image according to the environmental information by using dynamic exposure compensation and shadow mapping algorithms;

[0027] Based on the environmental information, the data processing and interaction control module makes a rendering decision on the three-dimensional historical scene model by increasing / decreasing the number of rendered polygons or enabling / disabling the intelligent occlusion culling algorithm.

[0028] In one embodiment, the data processing and interaction control module is also used to generate a rendering complexity coefficient based on the user interest factor and the environmental factor, and perform adaptive rendering optimization on the AR interface based on the rendering complexity coefficient;

[0029] Among them, the user interest factor is related to the interaction information, and the environmental factor is related to the lighting and the number of tourists in the tourist environment.

[0030] In one embodiment, the data processing and interaction control module is also used to adaptively adjust the level of detail of the three-dimensional historical scene model based on the user distance and the rendering complexity coefficient.

[0031] In one embodiment, the system further includes a geofence trigger unit, which is used to automatically trigger the virtual tour guide image to start the explanation mode and the three-dimensional historical scene model to be miniaturized and restored when it detects that the user enters the predefined geographical area, and release the resources occupied by the virtual tour guide image and the three-dimensional historical scene model when it detects that the user leaves the predefined geographical area.

[0032] In one embodiment, the system combines GPS / IMU, visual SLAM and geofence technology to detect in real time whether the user enters or leaves the predefined geographical area.

[0033] The above interactive tourism guide system based on AR technology, through the three-dimensional virtual tour guide generation module and the historical scene miniaturization and restoration module, constructs a unique virtual tour guide image in combination with local cultural characteristics, and presents the historical scene in the form of a miniaturized model, enabling users to intuitively perceive the original appearance and changes of the relics or cultural sites. This method greatly enriches the content of the tour guide, improves the depth of tourists' cultural cognition, enhances the immersion and interest of the tour. Compared with the traditional text annotation or static information overlay, the present invention provides a more vivid and intuitive means of historical and cultural display. In addition, the data processing and interaction control module is used to dynamically update the presented virtual tour guide information and historical scene restoration content. This personalized interaction method not only improves the flexibility of the tour guide, but also can adjust the display content according to the user's real-time feedback, so as to meet the diverse needs of tourists. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is the overall architecture diagram of the interactive tourism guide system based on AR technology in one embodiment;

[0035] Figure 2 It is the scene acquisition and positioning flow chart in one embodiment;

[0036] Figure 3 Flow chart of three - dimensional virtual tour guide generation in an embodiment

[0037] Figure 4 Flow chart of historical scene miniature restoration in an embodiment

[0038] Figure 5 Structural diagram of user multi - modal interaction in an embodiment

[0039] Figure 6 Structural diagram of multi - parameter rendering optimization formula in an embodiment Detailed implementation manners

[0040] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] As Figure 1 shown, the interactive tourism navigation system based on AR technology provided by the present invention mainly consists of a scene acquisition and positioning module, an AR display and fusion module, a three - dimensional virtual tour guide generation module, a historical scene miniature restoration module, a data processing and interaction control module, an environment perception module, and a geofence trigger unit, etc. Each module interacts and collaborates through a multi - thread data processing pipeline, and can provide tourists with in - depth cultural background explanations, immersive historical scene restoration, personalized content push, and automated environment - adaptive rendering effects in a real tourism environment.

[0042] Among them, the scene acquisition and positioning module relies on a camera device (usually a high - resolution RGB camera or a TOF depth camera) and GPS / IMU sensors to obtain the spatial information of the user's environment. For large outdoor scenic spots, GPS can provide geographical coordinates at the meter level; IMU samples the user's movement and rotation postures in real time through an accelerometer, a gyroscope, and a magnetometer. In some areas with weak GPS signals, this module can also cooperate with Bluetooth Beacon or other indoor positioning technologies to ensure the continuity and accuracy of positioning.

[0043] In terms of high - precision spatial positioning, in order to achieve centimeter - level or better accuracy, the scene acquisition and positioning module incorporates a visual SLAM algorithm. Its basic principle is to detect, match, and track feature points in consecutive video frames, thereby estimating the user's pose in three - dimensional space and simultaneously constructing a local map. The system can use GPU parallel acceleration technology on a mobile terminal to calculate the external parameters (rotation and translation matrices) of the camera in real time, and send the result to the AR display and fusion module for subsequent virtual overlay rendering.

[0044] In one embodiment, after the scene acquisition and positioning module obtains the location information, it will match it with the geo-fence database to determine whether the user has crossed the predefined geographical area. The scene acquisition and positioning module transmits the acquired location information, orientation information, and area information matched with the geo-fence database to the data processing and interactive control module. Once it is determined that the user has crossed the predefined geographical area, the geo-fence trigger unit can trigger the corresponding three-dimensional historical scene model display or virtual tour guide explanation.

[0045] The scene acquisition and positioning module can ensure continuous and accurate tracking of the user's position and posture, providing a reliable spatial reference for the subsequent AR display and fusion module and the historical scene miniature restoration module.

[0046] The 3D virtual tour guide generation module stores 3D resources such as local traditional clothing textures, regional cultural elements, and images of historical celebrities in the database in advance, which are used to combine and generate unique virtual tour guide images. For example, in a historical city with a long tradition, a tour guide wearing ancient official uniforms or traditional ethnic costumes can be designed; in a modern theme scenic area, a technological or cartoon-like appearance can also be adopted. Through layered rendering technology, the clothing texture layer is synthesized frame by frame with the facial expression layer, lip synchronization layer, etc. on the GPU to form a 3D virtual image.

[0047] To enhance the vividness of the tour guide, the 3D virtual tour guide generation module manages animation clips based on the finite state machine (FSM) and logically arranges different explanation scenes. For example, when a user enters a certain area of ​​the scenic spot for the first time, the tour guide image is in the "welcoming" state and actively conducts a brief guidance; if the user looks at the tour guide image or makes a gesture request, its state switches to "explaining" and lip-syncs the output of speech; if the user no longer pays attention to or leaves the fenced area, the tour guide can switch to the "sleep" state to save computing resources.

[0048] In a more preferred embodiment, the virtual tour guide image can support multiple languages, and the user can specify the language in the system interface or voice command. With the help of text-to-speech (TTS) and lip shape automatic matching algorithm, the guide's lip shape and pronunciation are roughly consistent, giving tourists a more natural sense of human-computer communication.

[0049] Based on the virtual tour guide image generated by the 3D virtual tour guide generation module, the AR display and fusion module superimposes it on the real scene image. The specific steps are as follows: read the real scene image and the virtual tour guide image, and ensure that the virtual tour guide image contains a transparency channel; adjust the size of the virtual tour guide image and determine the superimposed position; separate the RGB channel and transparency channel of the virtual tour guide image; create a mask of the same size as the virtual tour guide image; extract the area to be superimposed in the real scene image and perform image fusion; put the fused result back into the real scene image, display and save the result.

[0050] The historical scene miniature restoration module includes a historical data storage unit, which stores a wealth of pictures, archaeological documents, old photos, and academic research materials. Through multi-perspective reconstruction algorithms or laser point cloud reconstruction, a high-precision three-dimensional model of the ruins can be generated. For some missing details, the system can use manual verification and reference information to perform restorative interpolation to ensure that the historical scene has a high textual research value.

[0051] When the user stays in a certain site area for more than a preset time, or asks about its "past use" through voice, the data processing and interactive control module will call this module to superimpose the past scenes on the current real scene in the form of a miniature model. The system can present it in a preset ratio such as 1:100 or 1:50, and through a gradual change of transparency, the site is presented to the user in a "contrast between the past and the present", helping the user to intuitively understand historical changes.

[0052] The data processing and interactive control module includes an eye tracking unit and a gesture recognition unit. The eye tracking unit captures the user's pupil position through a camera or a dedicated eye movement sensor, and calculates the gaze direction in combination with a facial key point detection algorithm. When a user stares at a building or virtual element for more than 3 seconds, the system determines that the user may be interested in the target, and then actively pops up a brief information card or a tour guide explanation.

[0053] The gesture recognition unit uses a convolutional neural network to extract the key points of the user's hand and classify its posture. When a specific gesture (such as pinching, double-clicking, swiping) is detected and the gaze falls on the same target, a deeper interaction is triggered, such as switching the tour guide's explanation mode or zooming in on the local details of the historical model.

[0054] The data processing and interactive control module also includes a language understanding unit and a voice interaction unit. The two work together. After the language understanding unit obtains the user's spoken language, it is first transcribed into text through ASR, and semantic understanding is performed on the server side or local engine to find the historical data or explanation script that best matches the inquiry. Then the voice interaction unit drives the virtual tour guide image to give a verbal answer. If deep historical scene restoration is required, the system automatically calls the historical scene miniature restoration module.

[0055] The data processing and interactive control module will record the user's gaze trajectory, gesture frequency, voice query keywords and other interactive information in the background to build a user portrait and behavior model. Combined with the geo-fence information, it can analyze where tourists stay the longest and which cultural elements they are more interested in. When visiting or going to related attractions in the future, the system can give priority to pushing corresponding in-depth content. Combined with multimodal interaction methods, it can more accurately capture tourists' focus and achieve a deeper interactive experience.

[0056] The environment perception module collects the current ambient light intensity, color temperature and weather conditions (such as cloudy, sunny, day and night, and rainfall) through external light sensors and weather API interfaces, and outputs them to the AR display and fusion module and the data processing and interactive control module in real time. The system adjusts the brightness, contrast and shadow of virtual models such as the virtual tour guide image and the three-dimensional historical scene model accordingly, so that the virtual elements always maintain visual consistency with the real world.

[0057] The environment perception module can also use machine learning to identify and count human heads in real-life images and detect the number of tourists within the visible range. When a large number of tourists are detected within the visible range, the system can reduce the number of rendered polygons or enable the intelligent occlusion culling algorithm to ensure smoothness, so that only the most necessary parts of the model are rendered, which can significantly reduce the amount of data that needs to be processed, speed up the rendering speed, and greatly increase the frame rate of real-time image updates, thereby bringing a smoother visual experience.

[0058] The present invention pre-sets several geographic fences in a scenic area or museum, for example, the entire park is divided into four areas A, B, C, and D, and each fence corresponds to an independent historical database and tour guide script. When the user enters area B from area A, the scene acquisition and positioning module detects the position change and sends the event to the data processing module. The data processing and interactive control module combines the user's existing interest preferences to determine whether to automatically open a specific historical scene or demonstration animation in area B. For example, if a user asks a lot of questions about building structures in area A, then after entering area B, the present invention can give priority to presenting the miniature restoration and explanation of the relevant buildings to enhance coherence and depth. When the user leaves the fenced area, the system can release the three-dimensional model resources associated with the area or place them in a low-priority cache, thereby saving memory and computing load; if the user returns to the area again, it can be quickly restored to ensure smooth use.

[0059] Through the linkage with geofencing technology, this embodiment realizes point-to-point customized cultural tour guide services. When a user enters a specific geographical area, the system can automatically trigger the corresponding virtual tour guide commentary mode and miniaturized restoration content of historical scenes, and load the corresponding historical database according to the regional cultural characteristics. This method effectively avoids the problems of tourists getting lost or missing important scenic spots in large scenic areas, improves the coherence and depth of the tour guide. At the same time, after leaving the geofence, the associated models and resource occupations are automatically released, improving the system operation efficiency and the battery life performance of mobile terminals.

[0060] In one embodiment, the system anonymously records information such as the tourist's line-of-sight trajectory, gesture operations, voice keywords, and stay duration during the entire tour, and stores it in the background server or local terminal. Such big data can reflect the tourist's preference for certain scenic spots, and also help the scenic area management to understand which cultural elements are most concerned.

[0061] When the same tourist enters the scenic area again at different times or with different terminals, the system can directly push the historical scenes or related stories that have not been deeply browsed before according to their past interaction data, making the tour experience continuously enriched and personalized. In addition, at the system level, the interaction data of a large number of tourists can be summarized, and the timing of the tour guide explanation and the path recommendation strategy can be optimized through statistical analysis.

[0062] After the tour guide ends, the system can automatically generate an AR review segment according to the user's high-interest scenes, photos taken, and viewing duration, allowing users to share "customized sightseeing notes" or tour guide experiences on social platforms, forming good cultural dissemination and secondary interaction.

[0063] This embodiment not only improves the tourist's tour experience, but also provides an effective means for the digital upgrade and intelligent management of the scenic area. The system can record and analyze user behavior data, providing a basis for personalized push in subsequent visits. In addition, by summarizing the interaction data of a large number of tourists, the scenic area can optimize the timing of the tour guide explanation and the path recommendation strategy, further improving the management efficiency and tourist satisfaction.

[0064] In one embodiment, the AR display and fusion module uses methods such as light estimation and compensation, perspective projection transformation and alignment, and frame rate and latency control to optimize the fusion effect.

[0065] Specifically, when the real-time image captured by the camera shows a strong change in light (such as walking from the shadow to the sun), if there is no corresponding light estimation algorithm, the virtual model will often show sudden brightness changes or incoordination. To this end, the present invention extracts brightness, contrast and color temperature information from each frame of the image, combines the data of the environment perception module, and dynamically calculates the ambient light model; then, the virtual object is subjected to corresponding exposure compensation and shadow mapping, so that the virtual and the real are optically integrated as much as possible.

[0066] According to the camera pose and focal length parameters output by the scene acquisition and positioning module, the virtual tour guide and historical scene model are transformed into perspective projection to ensure that the position, size, and orientation in the user's field of view are consistent with the real environment. In addition, this implementation can perform corrections based on the camera's intrinsic parameters and distortion coefficients during rendering to ensure that the virtual scene does not appear to "drift" or "layer" when the user moves or turns his head, thus achieving perspective projection transformation and alignment.

[0067] To achieve a smooth interactive experience, the AR display and fusion module can automatically select different levels of model details based on the adaptive rendering algorithm of multi-parameter fusion, so as to better achieve high-quality AR navigation under limited hardware resources. For example, when the user is less interested in a certain relic or there are too many tourists around, the system can temporarily enable simplified models and low-resolution maps to ensure that the key frame rate is not less than 30FPS; when the user shows high interest (long-term staring, frequent gestures or voice inquiries) and the hardware performance allows, the system will enable a more sophisticated historical restoration model for rendering.

[0068] The above multi-parameter fusion adaptive rendering algorithm quantitatively evaluates the user interest factor and environmental factor, and finally generates the rendering complexity coefficient R comp . R comp When the value is high, the system tends to use high-precision models, superimpose more details and historical animations; otherwise, it will automatically simplify or turn off some special effects.

[0069] Among them, the user interest factor is defined as:

[0070]

[0071] In the formula, T is the duration of the user's gaze on a specific target; G is the total number of gesture interactions; Q is the number of voice inquiries; α, β, γ, δ are weights trained based on historical data. By performing nonlinear processing (square root, logarithm, exponential) on gaze, gesture, and voice, the user's real interest changes can be captured more sensitively. T, G, and Q are all interactive information, which are collected and obtained by the gaze tracking unit, gesture recognition unit, language understanding unit, and voice interaction unit in the data processing and interaction control module.

[0072] Environmental factors are defined as:

[0073]

[0074] Among them, L is the current light intensity, L0 is the reference light; P is the number of tourists within the current visible range, P0 is the threshold; ρ, κ1, and κ2 are adjustment parameters. This formula dynamically measures the influence of light and the number of people through a logical function form. The light intensity and the number of tourists are collected through the environmental perception module.

[0075] After fusing the user interest degree and environmental factors, we get;

[0076]

[0077] Among them, ω, μ, ν, and λ are used to control the strength of the influence of each part. When R comp exceeds the preset threshold, the system automatically enables high-precision models, layer rendering, and more historical animations; if it is lower than the threshold, some special effects are simplified or turned off to ensure an available frame rate even in poor lighting or when there are many tourists.

[0078] The historical scene miniature restoration module also uses a rendering complexity coefficient for optimization. After loading the three-dimensional historical scene model, through a multi-level detail strategy, it flexibly switches according to the user's distance and the rendering complexity coefficient. If the user is far away, only low-resolution textures are displayed; if the user gestures to zoom in or the line of sight strongly focuses, it automatically switches to a higher-precision model and texture to present details such as building structures and character costumes.

[0079] This embodiment proposes a rendering complexity control formula with multi-parameter fusion for complex environmental scenarios such as insufficient light or dense tourists, dynamically adjusting the fineness, light, and transparency of the displayed content. This adaptive rendering optimization strategy not only ensures the visual effect under performance-limited conditions but also ensures the smoothness and clarity under different lighting and tourist densities. Compared with the prior art, this embodiment performs more stably in complex environments and can provide users with a continuous high-quality AR navigation experience.

[0080] In summary, the present invention can realize the multimodal interaction of three-dimensional virtual tour guides, the miniature restoration of historical scenes, and the comprehensive adaptive rendering of environmental perception and user interest in real scenes. Specifically, high-precision map information, historical databases and three-dimensional modeling resources can be configured in advance in tourist attractions or cultural places. After entering the scenic area, tourists wear or use matching AR terminal devices (such as AR glasses, smart phones or tablets, etc.) to start the system. The system uses the visual information captured by the camera and the position and posture data provided by GPS / IMU, combined with visual SLAM technology to locate the user in real time, and superimpose the three-dimensional virtual tour guide image, scene annotations and historical scene miniature models on the real scene. When tourists are interested in a relic or make a voice inquiry, the system calls the multi-parameter fusion rendering complexity control formula proposed by the present invention according to their sight stay time, gesture operation, language instructions, and current lighting, tourist density and other environmental factors, and dynamically adjusts the fineness, lighting and transparency of the displayed content, so as to achieve a reasonable balance between performance and visual effects.

[0081] The technical effects of the above scheme are as follows: through the three-dimensional reconstruction of relics or cultural sites and the superimposition of miniature models on the user side, tourists can intuitively compare the past and present styles, and deeply perceive the cultural background and changes of the times behind them. Through the three interactive methods of sight tracking, gesture recognition, and voice input, the focus of tourists can be flexibly combined and captured in real time; at the same time, combined with geographic fences, partition triggering and point-to-point interpretation in large-scale scenes can be realized. In complex environments such as lighting and multi-user concurrency, the use of multi-parameter fusion rendering complexity control formula can ensure the balance between model accuracy and frame rate, so as to still present a high level of visual effects in limited hardware resources. The system will record and analyze user behavior, actively push historical content that is more in line with tourists' needs in subsequent visits, and can also generate personalized review scenes to promote cultural dissemination and word-of-mouth fermentation.

[0082] It can be seen that the technical solution of the present invention not only meets the tourists' needs for cultural depth, interactive fun and immersive experience, but also provides an effective means for the digital upgrade, intelligent management and subsequent big data analysis of scenic spots. Therefore, the specific implementation methods of the present invention can play a significant role and have good application prospects in the field of tourism.

[0083] Each module in the above system can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0084] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0085] Those of ordinary skill in the art can understand that to implement all or part of the functions of the above-mentioned embodiments, it can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the functions of the above-mentioned modules. Among them, any reference to a memory, database, or other medium used in the various embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the various embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the various embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., and are not limited thereto.

[0086] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0087] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An interactive tourist guide system based on AR technology, characterized in that: The system comprises: The scene acquisition and positioning module is used to spatially locate and capture the scene of the user's tourism environment to obtain real-scene images; A 3D virtual tour guide generation module is used to build a virtual tour guide image based on local cultural characteristics; An AR display and fusion module, used to superimpose the virtual tour guide image onto the real scene image in an AR interface, and perform lighting estimation, perspective projection transformation, and dynamic rendering optimization on the layer where the virtual tour guide image is located; A historical scene miniature restoration module, used to construct a three-dimensional historical scene model corresponding to the real scene image and present it in the form of a miniature model in the AR interface; A data processing and interactive control module, used to obtain user interaction information and dynamically update the virtual tour guide image and the three-dimensional historical scene model based on the interaction information; Among them, the data processing and interactive control module works together with the scene acquisition and positioning module, the three-dimensional virtual tour guide generation module, the AR display and fusion module, and the historical scene miniature restoration module to form a multi-threaded data processing pipeline.

2. The system according to claim 1, characterized in that The three-dimensional virtual tour guide generation module includes: A layered rendering unit that fuses multiple layers, including clothing texture layers, facial expression layers, and lip sync layers, on a GPU-accelerated rendering pipeline; The animation driving unit is used to realize the animation switching of the virtual tour guide image by adopting a state machine-based management method.

3. The system according to claim 1, characterized in that The historical scene miniature restoration module includes: A historical data storage unit, used to store historical information resources related to the real scene image; A three-dimensional reconstruction engine, used for converting the historical information resource into the three-dimensional historical scene model; The scaling and transparency control unit is used to miniaturize the three-dimensional historical scene model according to a preset ratio and superimpose it on the real scene image, and at the same time use transparency control means to achieve gradual contrast between the three-dimensional historical scene model and the real scene image.

4. The system according to claim 1, characterized in that The data processing and interactive control module includes: A gaze tracking unit, used to identify the gaze direction and duration of the user in the AR interface, so that the data processing and interaction control module updates the virtual tour guide image and / or the three-dimensional historical scene model according to the recognition result of the gaze tracking unit as the interaction information; The gesture recognition unit is used to detect the key points of the user's hand and estimate the posture in real time, so that the data processing and interaction control module updates the virtual tour guide image and / or the three-dimensional historical scene model according to the recognition result of the gesture recognition unit as the interaction information.

5. The system according to claim 1, characterized in that The data processing and interactive control module also includes a language understanding unit and a voice interaction unit; The language understanding unit is used to convert the user's voice question into text or semantic commands through speech recognition and natural language processing algorithms; The voice interaction unit is used to give a voice response to the text or semantic command through the virtual tour guide image.

6. The system according to claim 1, characterized in that The system also includes an environmental perception module for real-time monitoring of environmental information; The AR display and fusion module adjusts the brightness and contrast of the virtual tour guide image according to the environmental information using dynamic exposure compensation and shadow mapping algorithms; The data processing and interactive control module makes rendering decisions such as increasing / decreasing the number of rendered polygons or turning on / off the intelligent occlusion culling algorithm for the three-dimensional historical scene model according to the environmental information.

7. The system according to claim 1, characterized in that The data processing and interaction control module is further used to generate a rendering complexity coefficient according to a user interest factor and an environmental factor, and to perform adaptive rendering optimization on the AR interface based on the rendering complexity coefficient; The user interest factor is related to the interaction information, and the environmental factor is related to the lighting of the tourism environment and the number of tourists.

8. The system according to claim 7, characterized in that The data processing and interactive control module is also used to adaptively adjust the detail level of the three-dimensional historical scene model based on the user distance and the rendering complexity coefficient.

9. The system according to claim 1, characterized in that The system also includes a geo-fence trigger unit, which is used to automatically trigger the virtual tour guide image to start the explanation mode and the miniature restoration of the three-dimensional historical scene model when it is detected that the user enters the predefined geographical area, and release the resources occupied by the virtual tour guide image and the three-dimensional historical scene model when it is detected that the user leaves the predefined geographical area.

10. The system according to claim 9, characterized in that The system combines GPS / IMU, visual SLAM and geo-fencing technology to detect in real time whether a user enters or leaves the predefined geographic area.

Images