Interaction system and method for cultural and tourism scenarios

Abstract

The application discloses a kind of interactive system and method for cultural and tourism scene, the system includes digital twin platform, wearable smart terminal, mobile terminal auxiliary APP, game engine and command management center.Digital twin platform is used to build scenic spot three-dimensional model and configure level data to generate game script;Wearable smart terminal is used to report position by fusing multi-source positioning data, and interactive data is collected through NFC, with fall detection and SOS alarm function;Mobile terminal auxiliary APP is used to collect AR interactive data based on spatial anchoring;Game engine is used to trigger level according to position coordinates, and verify level completion condition by integrating NFC interactive data, AR interactive data and position coordinates;Command management center is used to receive SOS alarm and dispatch unmanned aerial vehicle and monitoring resources.The application realizes the integration of virtual and real space interaction and safety control in scenic spot.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, and more specifically to an interactive system and method for cultural and tourism scenarios. Background Technology

[0002] With the deepening of digital transformation in the cultural tourism industry, augmented reality technology and role-playing game mechanics are gradually being applied to scenic area guides and visitor experience systems. Chinese patent application CN105469330A discloses a task-based intelligent scenic area guide system. This system navigates visitors along a predetermined route using a handheld terminal based on their chosen role. When a visitor reaches a task point on the route, they perform the task specified for that point. The system determines whether the visitor has completed the task; if so, navigation continues to the next task point; otherwise, it switches to normal tour mode. Chinese patent application CN107705224A discloses a guide method based on AR glasses. This method obtains the visitor's current and destination locations, acquires all paths from the current location to the destination, and identifies scene experience points along the way. It then presents scenes or information to the visitor, reducing manpower costs in scenic areas and providing scene-based services.

[0003] While the aforementioned existing technologies have improved visitor participation and the overall experience to some extent, they still have significant drawbacks. In the solution disclosed in CN105469330A, the system's determination of whether a visitor has reached a task point relies entirely on the positioning signal of the handheld terminal. However, factors commonly found in scenic environments, such as indoor-outdoor space transitions, mountain obstructions, and satellite signal drift, can lead to decreased positioning accuracy, making the system prone to misjudgments or omissions when determining whether a visitor has reached a task point. Furthermore, this solution uses a single method to determine task completion, failing to comprehensively utilize the visitor's multi-dimensional behaviors in the real physical space, such as touching, observing, and moving, to determine the true completion status of the task. In the solution disclosed in CN107705224A, there is a lack of dynamic spatial mapping between the scene or information presented by the AR glasses and the visitor's actual spatial location. The visitor passively receives pre-set content associated with their location; the system cannot perceive whether the visitor has actually engaged in substantial spatial contact or interaction with the presented scene, resulting in a disconnect between the virtual content and the visitor's spatial behavior.

[0004] Therefore, how to determine the spatial relationship between tourists' spatial location and virtual scene in real time in the real environment of a scenic spot and perceive tourists' interactive behavior based on multi-source spatial perception data has become an urgent technical problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide an interactive system and method for cultural and tourism scenarios, which aims to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides an interactive system for cultural and tourism scenarios, the system comprising:

[0007] A digital twin platform is used to construct a digital twin model of a scenic area. The location, triggering conditions, completion conditions, and virtual content anchoring coordinates of the level nodes are configured on the digital twin model of the scenic area, and a game script is generated and stored in the game engine.

[0008] Wearable smart terminal is used to integrate multi-source positioning data to generate continuous location data and report it to the game engine, generate NFC interactive data by reading real-scene tag encoding through NFC and report it to the game engine, detect tourist fall events based on motion sensor data or trigger SOS events in response to physical buttons, and send SOS alarms containing the continuous location data to the command and management center.

[0009] A mobile auxiliary APP is used to bind to the wearable smart terminal and collect AR interaction data based on real-time screen spatial anchoring and report it to the game engine.

[0010] The game engine is used to read the game script content of the current level, map the continuous position data to a digital twin coordinate system, trigger the level according to the mapped coordinates, verify the level completion conditions based on the NFC interaction data, the AR interaction data and the position coordinates, mark the level as completed after verification, and determine the next level according to the node order of the game script.

[0011] The command and management center is used to receive SOS alarms, dispatch drones to the tourist's location based on the continuous location data, retrieve video surveillance footage of the surrounding area, and establish voice communication through the wearable smart terminal.

[0012] Secondly, the present invention provides an interactive method for cultural and tourism scenarios, applied to the system described in the first aspect, the method comprising:

[0013] Construct a digital twin model of the scenic area, configure the location, triggering conditions, completion conditions and virtual content anchoring coordinates of the level nodes on the digital twin model, generate a game script and store it in the game engine;

[0014] The wearable smart terminal integrates multi-source positioning data to generate continuous location data and reports it to the game engine; it also generates NFC interactive data by reading real-world tag encoding via NFC and reports it to the game engine.

[0015] The mobile auxiliary APP is bound to the wearable smart terminal, and AR interaction data is collected based on real-time screen spatial anchoring and reported to the game engine;

[0016] The game engine reads the game script content of the current level, maps the continuous position data to a digital twin coordinate system, triggers the level according to the mapped coordinates, verifies the level completion conditions based on the NFC interaction data, the AR interaction data and the position coordinates, marks the level as completed after verification, and determines the next level according to the node order of the game script.

[0017] During the interaction, the wearable smart terminal detects a fall event based on motion sensor data or triggers an SOS event by responding to a physical button, and sends an SOS alarm containing the continuous location data to the command and management center; the command and management center receives the SOS alarm, dispatches a drone to the tourist's location based on the continuous location data, retrieves video surveillance footage of the surrounding area, and establishes voice communication through the wearable smart terminal.

[0018] In summary, the present invention has the following advantages:

[0019] By constructing a 3D model of the scenic area through a digital twin platform and configuring the location, triggering conditions, completion conditions, and virtual content anchoring coordinates of checkpoint nodes, the multi-source fused continuous location data reported by wearable smart terminals can be mapped to a unified digital twin coordinate system. The game engine then triggers levels based on the matching results between the mapped coordinates and the geofences of the checkpoint nodes, achieving precise spatial alignment between virtual interactive content and the real physical space of the scenic area. This avoids misjudgments caused by insufficient positioning accuracy and inconsistent spatial benchmarks. Furthermore, the game engine collaboratively verifies checkpoint completion conditions using NFC interaction data, AR interaction data, and location coordinates. This allows the system to comprehensively determine the interactive behavior status of visitors from three dimensions: physical touch, augmented reality operation, and spatial arrival. Compared to solutions relying on only a single judgment condition, this significantly improves the accuracy and reliability of interaction verification. Meanwhile, after detecting a fall or triggering an SOS, the wearable smart terminal sends an alarm containing continuous location data to the command and management center. The command and management center directly dispatches the drone based on this continuous location data and simultaneously retrieves surrounding monitoring footage. The location data carried by the alarm and the location data used by the game engine to trigger levels are from the multi-source fusion positioning results under the same spatial reference. They can be directly used for drone route planning and monitoring retrieval without coordinate transformation, thus effectively shortening the time path from location awareness to resource dispatch in emergency response and improving the overall response efficiency of the system. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the modules of an embodiment of the interactive system of the present invention for cultural and tourism scenarios;

[0021] Figure 2 This is a flowchart illustrating an embodiment of the interactive method for cultural and tourism scenarios according to the present invention.

[0022] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0024] It should be noted that the descriptions involving "first," "second," etc., in this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0025] Reference Figure 1 The diagram shown is a modular schematic of an embodiment of the interactive system for cultural and tourism scenarios according to the present invention. The interactive system for cultural and tourism scenarios is characterized in that the system includes:

[0026] A digital twin platform is used to construct a digital twin model of a scenic area. The location, triggering conditions, completion conditions, and virtual content anchoring coordinates of the level nodes are configured on the digital twin model of the scenic area, and a game script is generated and stored in the game engine.

[0027] Wearable smart terminal is used to integrate multi-source positioning data to generate continuous location data and report it to the game engine, generate NFC interactive data by reading real-scene tag encoding through NFC and report it to the game engine, detect tourist fall events based on motion sensor data or trigger SOS events in response to physical buttons, and send SOS alarms containing the continuous location data to the command and management center.

[0028] A mobile auxiliary APP is used to bind to the wearable smart terminal and collect AR interaction data based on real-time screen spatial anchoring and report it to the game engine.

[0029] The game engine is used to read the game script content of the current level, map the continuous position data to a digital twin coordinate system, trigger the level according to the mapped coordinates, verify the level completion conditions based on the NFC interaction data, the AR interaction data and the position coordinates, mark the level as completed after verification, and determine the next level according to the node order of the game script.

[0030] The command and management center is used to receive SOS alarms, dispatch drones to the tourist's location based on the continuous location data, retrieve video surveillance footage of the surrounding area, and establish voice communication through the wearable smart terminal.

[0031] In this embodiment, the system includes a digital twin platform, a game engine, and a command and management center on the scenic area side, and wearable smart terminals and mobile auxiliary apps on the tourist side. Because traditional scenic area experiences lack story-driven interactive guidance, there is no correlation between changes in tourists' positions in the real physical space and the progression of the virtual storyline. Furthermore, scenic area management cannot accurately locate tourists scattered throughout the area or respond quickly to emergencies. To address this issue, this embodiment uses a digital twin model as a unified spatiotemporal reference layer, anchoring the virtual game script to precise coordinates in the real scenic area. Wearable smart terminals and mobile auxiliary apps respectively handle the data collection tasks for physical interaction and augmented reality interaction. Finally, the game engine completes the closed loop of spatial triggering and level verification, while the command and management center uses the same location data to implement emergency dispatch.

[0032] Specifically, the digital twin platform is responsible for constructing a digital twin model of the scenic area and configuring game data on that model. Specifically, the platform receives multi-source spatial data, including UAV aerial survey imagery and ground laser point clouds, unifies this data to a common spatial reference, and then processes it through denoising, registration, and mesh reconstruction to generate a digital twin model of the scenic area that includes geometric information about the terrain surface and building facades. The construction of this digital twin model ensures that every real-world location within the scenic area has corresponding three-dimensional virtual coordinates, providing a unified spatial reference framework for subsequent level design and the overlay of virtual content.

[0033] After the digital twin model of the scenic area is constructed, the digital twin platform marks multiple candidate checkpoint locations on the model. A geofence with a preset radius is generated centered on each candidate checkpoint location, and the boundary coordinates of each geofence are stored in the game engine. The game engine assigns trigger and completion conditions to each candidate checkpoint location based on its spatial region attributes within the digital twin model. These conditions define what conditions must be met for a visitor to trigger the checkpoint and what interactive actions are required to complete it.

[0034] The game engine then binds anchor coordinates of virtual content to each level in the spatial coordinate system of the scenic area's digital twin model, arranging the configuration information of all levels into a game script according to the predetermined tour order and storing it in the game engine. After the above configuration, the digital twin platform provides spatial reference data and game logic data for the entire system, and all subsequent level triggering and verification operations are based on the data produced by the digital twin platform.

[0035] Wearable smart terminals, worn by tourists, perform three basic functions: location awareness, near-field interaction, and safety monitoring. The terminals integrate a satellite positioning chip, Bluetooth positioning unit, inertial measurement unit, and barometric pressure sensor. Outdoors, satellite positioning is the primary method, and the altitude data calculated by the barometric pressure sensor is fused with the satellite positioning to correct vertical accuracy. When tourists enter indoor areas or areas with satellite signal obstruction, the terminals automatically switch to Bluetooth positioning. The inertial measurement unit and barometric pressure sensor continuously provide motion posture and altitude change data. The fusion algorithm within the terminals integrates all positioning data to generate continuous and smooth 3D position data, which is reported to the game engine at a preset frequency. For near-field interaction, the terminals use an NFC reader / writer module to read the code of tags placed in the scenic area. When a tourist brings the terminal close to an NFC tag at a checkpoint, the terminal reads the tag's unique code and packages it with the current 3D position data, reporting it to the game engine as verification of whether the tourist has reached a specific physical location and completed a specific action.

[0036] The wearable smart terminal's voice intercom and team broadcast unit also supports a team voice broadcast mode during team tasks. Specifically, when tourists are in team mode, any team member can press and hold the PTT button on the side of the wearable smart terminal to start recording. The voice message is encoded by Opus and uploaded to the game engine server via the 4G cellular communication module. The server then forwards it in real time to the wearable smart terminals of all other team members via the 4G network. After receiving the team voice message, other team members' terminals automatically play it through their speakers, thus achieving a real-time broadcast intercom effect where one person speaks and the whole team hears.

[0037] Both team voice broadcasts and management center broadcast notifications are implemented through the same 4G cellular communication module, but operate in different communication modes: team broadcasts use a two-way forwarding mode from team members to the server to the entire team, while broadcast notifications use a one-way push mode from the server to the terminal. The terminal has a built-in circular voice buffer that stores the most recent twenty team voice messages. Each voice message includes the sender's role name, timestamp, and message sequence number as metadata tags. Users press the voice playback button on the side of the casing to enter the playback interface. The display shows a list of voice messages; users can short-press to turn pages and long-press to play selected voice messages, enabling them to review previously played messages at any time. This fundamentally differs from the single-communication mode of traditional walkie-talkies where messages disappear after being spoken. The voice intercom and team broadcast units are equipped with an audio playback priority management mechanism. Priorities, from highest to lowest, are: SOS emergency calls, management center emergency broadcasts, management center general notifications, and team voice broadcasts. Higher-priority events automatically interrupt lower-priority playback.

[0038] In terms of safety monitoring, the wearable smart terminal uses an inertial measurement unit to detect tourists' falls in real time. When a fall is detected or a tourist presses and holds a physical button on the wearable smart terminal, the wearable smart terminal immediately generates an SOS alarm message containing three-dimensional location data, tourist identity and device status and sends it to the command and management center through the built-in communication module.

[0039] The wearable smart terminal is also equipped with a vibration-encoded silent reminder module. This module can solve the problem of silent transmission of team collaboration information in immersive gaming experiences, avoiding interference from voice or screen brightness. The vibration-encoded silent reminder module includes a linear resonant motor and a vibration encoding and parsing unit. The linear resonant motor achieves multi-mode vibration output through PWM drive. The vibration encoding and parsing unit stores a predefined vibration pattern lookup table, encoding and distinguishing different event types through different combinations of vibration duration, interval, and intensity. The predefined vibration patterns include at least one short vibration corresponding to successful NFC payment confirmation or successful check-in; two short vibrations corresponding to teammate completing a task or triggering a level; three short vibrations corresponding to game plot progression notifications or reminders to obtain key items; one long vibration corresponding to reminding the wearer to pay attention to team dispersal when team members leave beyond a preset distance threshold; two long vibrations corresponding to the arrival of a broadcast notification from the management center; and continuously increasing vibrations corresponding to level triggering or the imminent start of an important plot event.

[0040] When the main control module of the wearable smart terminal receives a collaborative event notification from the game engine or detects a teammate's distance deviation, it prioritizes outputting the corresponding vibration pattern through the vibration-encoded silent alert module to convey the event information to the wearer. This allows the wearer to perceive the event type through touch without needing to look at the screen or listen to voice prompts. The vibration encoding lookup table supports OTA updates and expansions to adapt to the alerting needs of different scenarios and scenarios. The wearable smart terminal obtains the relative position data of other terminals in the same team in real time through the Bluetooth indoor positioning module. When it detects that the distance between any teammate and this terminal exceeds a preset threshold, the main control module triggers a long, single vibration mode, achieving silent tactile alerts for team dispersion.

[0041] The wearable smart terminal is also equipped with an SOS alarm light module, which solves the problem of search and rescue personnel quickly locating the person in distress at night or in low-visibility environments after a tourist triggers an SOS. The SOS alarm light module includes six high-brightness red LED beads and an SOS encoding driver unit. The six LED beads are respectively located on the top, bottom, and left and right sides of the terminal casing, forming a 360-degree omnidirectional light signal coverage. The SOS encoding driver unit drives each LED bead through six independent GPIOs. After an SOS event is triggered, the LED beads automatically flash synchronously according to the international standard Morse code SOS. The flashing rhythm of the Morse code SOS is three short flashes, each lasting 200 milliseconds on and 200 milliseconds off; three long flashes, each lasting 600 milliseconds on and 200 milliseconds off; and three short flashes, each lasting 200 milliseconds on and 200 milliseconds off, with a complete cycle of approximately six seconds. The SOS alarm light module also supports alternating flashing mode and constant illumination mode. The alternating flashing mode uses three groups of LED beads to alternately create a rotating chasing motion effect, while the constant illumination mode is remotely controlled by the command and management center. When the terminal battery level falls below a preset threshold, the SOS encoding driver unit automatically switches to a power-saving flashing mode, extending the alarm light's operating duration. The SOS alarm light module operates independently of other terminal functions after SOS is triggered. Even if the terminal's main control module fails in extreme circumstances, the SOS encoding driver unit can still maintain Morse code flashing output for a preset duration.

[0042] The wearable smart terminal is also equipped with a camera module, which enables the terminal to take independent photos and perform AI image recognition, allowing tourists to complete image-capture games independently without their phones. The camera module includes a 5-megapixel CMOS image sensor and lens. A physical shutter button is located on the side of the terminal's casing. A short press of the physical shutter button triggers a shot, a long press triggers continuous shooting, and a double press triggers playback. The captured photos are automatically uploaded to a backend AI analysis server via a 4G communication module. The AI ​​analysis server is equipped with a pre-trained object detection model that supports the recognition of at least fifty categories of common animals, plants, and scenic area landmarks. After completing target recognition, the AI ​​analysis server sends the target category and confidence level back to the terminal. When the confidence level is greater than or equal to 85%, the main control module of the wearable smart terminal determines that the task target has been captured, displays the recognition result on the screen, and sends a confirmation vibration via a vibration-coded silent reminder module. When the 4G communication signal is unavailable, the photos are automatically temporarily stored in the terminal's built-in memory and automatically uploaded in batches and processed by AI recognition once the signal is restored.

[0043] The main control module of the wearable smart terminal maintains the current task's collection list, recording the target objects to be photographed and their completed status. Once all target objects in the collection list have been successfully captured, the main control module displays a task completion notification on the screen and emits a corresponding vibration pattern via the vibration module. Simultaneously, it reports the task completion event to the game engine to advance the game's storyline. Photos captured by the camera module are transmitted via TLS encryption and are not persistently stored locally. The cache is automatically cleared after the task is completed to protect visitor privacy.

[0044] The mobile auxiliary app runs on the visitor's smartphone and is paired with a wearable smart terminal via Bluetooth for collecting augmented reality interaction data. Specifically, the mobile auxiliary app uses the phone's camera to capture real-time footage, extracts visual feature points from the footage, and matches these points with a pre-deployed digital twin model feature point map from the game engine. This calculates the camera's position and orientation within the digital twin space, thus completing spatial anchoring. After initializing spatial anchoring, the mobile auxiliary app continuously tracks the displacement changes of feature points in the real-time footage to update the camera's real-time pose within the digital twin space. Based on the anchor coordinates of the virtual content corresponding to the current level in the game script and the camera's real-time pose, the mobile auxiliary app determines whether each virtual element is within the camera's current field of view. Virtual elements within the field of view are then rendered and overlaid onto the real-time footage with the correct spatial position and perspective.

[0045] When tourists click or swipe on overlaid virtual content on the mobile auxiliary app interface, the app generates AR interaction data containing the identifier of the manipulated virtual content, the type of operation, and the current spatial pose, and reports this data to the game engine via wireless network. The mobile auxiliary app enables tourists to see virtual elements precisely aligned with the physical space within a real scenic environment, and through spatial anchoring, ensures that different tourists see the same virtual content in the same location, expanding the interactive dimensions of level tasks.

[0046] As the logical hub of the entire system, the game engine is responsible for triggering levels based on the visitor's location, receiving and processing various interactive data, and verifying level completion conditions. During the visitor's movement, the game engine continuously receives 3D location data reported by the wearable smart terminal, mapping the coordinates in the 3D location data to the internal coordinate system of the digital twin model to obtain the visitor's real-time coordinates in the virtual space. The game engine reads currently incomplete level nodes and their geofence boundaries from the stored game script, determines the spatial inclusion relationship between the visitor's real-time coordinates and the geofence boundaries of each level, and triggers the level when the real-time coordinates fall within the geofence of a certain level.

[0047] After triggering a level, the game engine extracts the verification requirements for the current level from the game script. These requirements specify the types of interaction methods needed to prove the visitor has completed the level, which can include one or more combinations of location verification, NFC verification, and AR verification, as well as the logical relationships between these verification items. The verification parameters include the target location coordinates, the target NFC tag code, and the target virtual content identifier. The game engine determines the verification order based on the requirements, comparing the visitor's real-time location with the target location, matching the tag code in the NFC interaction data with the target tag code, and comparing the virtual content identifier in the AR interaction data with the target virtual content identifier. The game engine determines whether the level completion conditions are met based on the logical relationships specified in the verification requirements. The level completion conditions are considered met when all required verification items pass or when any one verification item passes. After determining that the level completion conditions are met, the game engine marks the current level as completed, reads the exit node identifier of the current level from the game script, and determines the next level pointed to by the exit node as the level to be triggered. This cycle of location monitoring and triggering continues until all levels are marked as completed.

[0048] The command and management center is responsible for emergency response and overall monitoring. After receiving the SOS alarm reported by the wearable smart terminal, the command and management center extracts the current location of the tourist from the SOS alarm data, uses the current location of the tourist as the target point, obtains the no-fly zone boundary around the current location of the tourist from the digital twin model of the scenic area, uses the current location of the idle drone as the starting point, and uses the no-fly zone boundary as the obstacle constraint to generate a flight path from the drone's nest location to the tourist's location without crossing the no-fly zone.

[0049] The command and control center converts the flight path into flight commands and sends them to the drone's pod, controlling the drone to take off and fly along the flight path to the tourist's location. Simultaneously, the command and control center retrieves real-time footage from fixed video surveillance cameras deployed around the tourist's location based on the location data from the SOS alarm, displaying both the drone's transmitted footage and the fixed surveillance footage on a large command screen. Command personnel establish two-way communication with the tourist through the voice communication module built into a wearable smart terminal to understand the situation on-site and coordinate rescue resources. This implementation method of the command and control center directly uses the location data from the SOS alarm for drone flight path planning and video surveillance retrieval, establishing a data pathway between alarm, positioning, dispatch, and communication, thus shortening the response time from alarm receipt to resource dispatch.

[0050] In addition, the scenic area's infrastructure layer also deploys AI-powered intelligent NPCs. These NPCs can be deployed as independent intelligent interactive terminals indoors or outdoors within the scenic area, providing game participants with information such as Q&A, clue provision, and voice communication. The hardware components of the AI-powered intelligent NPC include a display screen, camera, microphone, speaker, and NFC module. The display screen uses an outdoor high-brightness LCD screen or e-ink screen to display NPC character images, game task information, menu-style question clues, and player character information. It has high brightness display capabilities in strong outdoor light environments and supports automatic backlight adjustment at night. The camera is integrated into the upper part of the NPC's casing to recognize special badges displayed on the player's handheld terminal or mobile phone screen as one of the input conditions for answering questions and providing clues. It can also be used to collect facial or posture information of the player to enhance the interactive experience.

[0051] The microphone employs an array microphone design and supports noise reduction processing to capture players' voice questions. The built-in speaker is used for NPC voice broadcasts, providing players with answers, clues, and game guidance information. The NFC module supports the ISO / IEC 14443 protocol for near-field identification and triggering with the smart devices worn by game players.

[0052] The AI-powered Smart NPC supports multiple power supply modes to adapt to different indoor and outdoor deployment environments, including convenient battery swapping, solar power, and cable power. In convenient battery swapping mode, the NPC has a built-in, quickly removable rechargeable battery compartment. The battery uses a standard lithium battery pack, and operators can quickly replace the battery using a dedicated key or clip, achieving uninterrupted battery swapping. In solar power mode, the NPC integrates a solar photovoltaic panel on top, charging the built-in battery in sunny conditions. The accompanying power management unit automatically switches between solar charging, battery discharging, and AC power, prioritizing solar power. In cable power mode, the NPC supports standard AC power or low-voltage DC cable power, suitable for indoor locations or outdoor locations with AC power access. The AI-powered Smart NPC has an IP65 protection rating, providing dust and water resistance, and can be safely deployed in outdoor environments. The shell is made of fiberglass or weather-resistant engineering plastic, and can be customized into any cultural tourism design based on different scenic area IPs, deeply integrating with the scenic area theme and combining decorative and interactive features.

[0053] The AI-powered intelligent NPC is equipped with a dedicated triggering mechanism to ensure access is only available to game participants, preventing unauthorized personnel from monopolizing resources. The NPC integrates a human infrared sensor. When the infrared sensor detects a person approaching, the NPC's display switches from standby to a ready-to-trigger state, showing the NPC's image or prompts, but not entering full working mode. Only when a game player touches the NPC's NFC module with their wearable smart device does the NPC read the player's identity via NFC, which is then verified by the backend game engine. Once confirmed as a legitimate game participant, the NPC enters working mode. Unauthorized personnel can only see the standby screen and cannot access the NPC's core functions such as question-and-answer and clue-giving. If the NPC does not detect a valid NFC touch or game interaction within a preset time, it automatically exits working mode and returns to standby, freeing up resources for other players.

[0054] Once the AI-powered NPC enters working mode, it reads the player's identity via NFC and uploads it to the game engine. The game engine then returns the player's current character identity, a list of pending tasks, and task progress based on the player's profile. The NPC uses this information to present players with menu-style question clues. These clues are pre-configured by operations staff in the script editor according to character and plot progression. Players can select clues on the screen to obtain guidance information closely related to their own storyline. The AI-powered NPC also integrates a large-scale language model for reasoning, supporting both menu-style questioning and the collection of the player's natural speech via microphone. Speech recognition is then used by the language model to generate answers relevant to the game's plot, characters, and clues. The answers are displayed in text on the NPC's screen and broadcast via a speaker, achieving a dual-mode presentation. Players can choose according to their preferences or environmental conditions: reading text in noisy environments and listening to audio in quiet environments.

[0055] The text and voice content output by the NPC undergoes backend content review, and the large-scale model's responses are subject to manual or rule-based verification at key points to ensure safe and compliant output. The AI-powered NPC's camera module is configured to capture images of special badges displayed on the player's wearable device or mobile phone screen. Image recognition identifies the badge category, and the NPC uses the identified badge as input for answering questions and providing clues. Only after the player completes a specific task and obtains the corresponding badge can the corresponding question answer or plot clue be unlocked, creating an interactive closed loop using screen badges for game unlocking and progression. The NPC maintains a real-time connection with the game engine server and command and management center via wired Ethernet or 4G wireless modules, synchronizing player status, question and answer logs, and NPC operational status information. It also regularly reports its own battery level, online status, and abnormal alarms to the command and management center for unified maintenance and management by operations personnel.

[0056] Taking a nature exploration-themed scenario in a large botanical garden as an example, this illustrates the entire interactive process for visitors through the collaborative efforts of various modules within the system. The overall layout of the scenic area comprises five main areas: the entrance plaza, the tropical greenhouse, the desert plant area, the medicinal plant garden, and the wetland viewing area. The scenario is designed around plant classification knowledge, divided into four levels: spring, summer, autumn, and winter. After arriving at the visitor center, visitors select the nature exploration scenario on a mobile auxiliary app and complete the payment. Staff then distribute a wearable smart terminal to the visitors, who scan the terminal's QR code to complete the binding process via the app. The app displays the scenario's background for visitors to learn beforehand, and the terminal screen displays that the first level has been unlocked and prompts visitors to proceed to the tropical greenhouse area.

[0057] Visitors wear devices as they walk along the trails. The device's internal positioning fusion module reports their 3D location data to the game engine once per second. When a visitor approaches within 20 meters of the tropical greenhouse entrance, the game engine matches their real-time location with the pre-defined geofence of the tropical greenhouse in the digital twin model, triggering the "Spring Sprouts" level. The device emits a series of progressively increasing vibrations as a reminder, the screen displays a task prompt, and the app simultaneously pushes detailed level information, requiring the visitor to check in at the NFC checkpoint at the greenhouse entrance. When the visitor brings the device close to the checkpoint, the device's NFC reader / writer module reads the tag code and packages it along with the current location coordinates into NFC interactive data, which is then reported to the game engine. The engine compares the tag code with the target code stored in the script. If a match is found, the level is considered complete, the device emits a short vibration to confirm, and the app plays a completion animation and updates the storyline.

[0058] When visitors enter the desert plant area, the game engine triggers the "Summer's Abundance" level. The app prompts visitors to use their phone's camera to find and record four desert plants. Visitors open the app's camera and point it at surrounding plants such as cacti. The app uses synchronous positioning and mapping technology to anchor the camera's view with the spatial coordinates of a digital twin model, overlaying virtual plant name tags onto the target plants. Visitors click on the virtual name tags one by one to complete the recording. The app reports the AR interaction data generated by each operation to the game engine. After verifying all four AR interaction data, the engine determines that the level is complete, the terminal emits three short vibrations as a notification, and the app displays "Level Completed" and unlocks the next level. After visitors enter the medicinal plant garden, satellite signals are blocked by tree canopies. The terminal automatically increases the weight of Bluetooth positioning and maintains continuous location reporting by interacting with the Bluetooth beacon network deployed in the garden. The game engine triggers the "Autumn Harvest" level based on the location coordinates. This level requires visitors to find three scattered NFC tags and touch them in the marked order. Visitors find the three tags using the map clues in the app and read them in sequence with the terminal. The engine verifies that the touch order is correct and determines that the level is complete.

[0059] Upon arrival at the wetland viewing area, the game engine triggers the final level, "Winter's Hidden Treasures," which requires successful location verification and AR verification to complete. The engine confirms the visitor's location is within the wetland viewing area's geographical fence. The app overlays a virtual animation of migratory bird routes onto the camera feed. After viewing, the visitor clicks the confirmation button, and the app reports this confirmation to the engine. Once confirmed, the engine determines the level is complete. The terminal displays completion information, the app shows a leaderboard and awards prizes to the visitor's NFC payment account. The visitor generates a full replay via the app, shares it on social media, and then returns the terminal. Throughout the process, the terminal's inertial measurement unit and barometric pressure sensor continuously monitor the visitor's movement. If a fall occurs, an SOS alarm is automatically triggered, sending the precise location to the command and control center. The center then dispatches a drone to the location and retrieves surrounding surveillance footage, establishing voice communication with the visitor through the terminal.

[0060] In this embodiment, the digital twin platform pre-models the scenic area and configures the script. Wearable smart terminals continuously report tourist locations, driving the game engine to trigger levels step by step. Wearable smart terminals and mobile auxiliary apps collect tourist interaction evidence through NFC and augmented reality, respectively. The game engine integrates location data, NFC interaction data, and AR interaction data to verify level conditions and advance the plot. In emergencies, the command and management center dispatches drones and monitoring resources based on location data. The system uses the real-time spatial location of tourists as the core clue throughout all stages. From script configuration, location triggering, interaction verification to emergency dispatch, everything revolves around the same set of spatial coordinate data, achieving integrated virtual-real experience and safety management.

[0061] In one embodiment, the construction of a digital twin model of a scenic area, configuring the location, triggering conditions, completion conditions, and virtual content anchor coordinates of checkpoint nodes on the digital twin model, generating a game script, and storing it in the game engine includes:

[0062] The digital twin platform acquires multi-source spatial data of the scenic area, performs coordinate registration and gridding processing on the multi-source spatial data, and generates a digital twin model of the scenic area containing geometric information of terrain surface and building facade. The multi-source spatial data includes UAV aerial survey images and ground laser point clouds.

[0063] Candidate checkpoint locations are marked on the digital twin model of the scenic area. A geofence with a preset radius is generated with each candidate checkpoint location as the center. The set of geofence coordinates corresponding to each candidate checkpoint location is stored in the checkpoint location database of the game engine.

[0064] The set of geofence coordinates is read from the checkpoint location database. Based on the spatial region attributes of each candidate checkpoint location in the scenic area digital twin model, trigger conditions and completion conditions are assigned to each candidate checkpoint location, and checkpoint node definition data containing location data and logical rules is generated.

[0065] In the spatial coordinate system of the digital twin model of the scenic area, anchor coordinates of virtual content are bound to each checkpoint node. The checkpoint node definition data and the anchor coordinates are associated and stored as checkpoint node configuration records. All checkpoint node configuration records are arranged in the order of tour to generate a game script and stored in the game engine.

[0066] The digital twin platform first acquires multi-source spatial data of the scenic area, including at least UAV aerial survey imagery and ground-based laser point clouds. Specifically, UAV aerial survey imagery is obtained by capturing the scenic area from multiple angles using a tilting camera mounted on a UAV. This provides texture information of the scenic area's surface, building roofs and facades, and spatial structure from a bird's-eye view. However, it has blind spots in areas such as the ground-floor walls and under eaves of buildings. Ground-based laser point clouds are obtained by scanning the scenic area using a laser scanner mounted on a ground tripod. This accurately collects the geometric coordinates of building walls, column bases, and ground facilities. However, its complete spatial structure information from a bird's-eye view is not as rich as that of UAV aerial survey imagery. The two types of data complement each other in terms of spatial coverage and information dimensions. UAV aerial survey imagery provides a wide-ranging bird's-eye view of spatial structure, while ground-based laser point clouds provide detailed geometric structures of building sides. Only by fusing these two types of data can a digital twin model of the scenic area be constructed, containing complete terrain surface and building facade geometric information.

[0067] Because UAV aerial imagery and terrestrial laser point clouds use different spatial reference bases during acquisition—UAV aerial imagery records the geographic coordinates of each pixel using a global geocentric coordinate system (GCS), while terrestrial laser point clouds record their spatial coordinates using the local coordinate system of the scanner's location—the two types of data cannot be directly overlaid without registration. The digital twin platform uses the GCS coordinates of the UAV aerial imagery as a reference and spatially transforms and registers the local coordinate system of the terrestrial laser point clouds to the GCS of the UAV aerial imagery. The digital twin platform extracts ground control points with distinct geometric features from the UAV aerial imagery, such as road intersection corners, building corners, or the center of artificial targets. Simultaneously, it identifies the 3D point cloud coordinates corresponding to the same physical location in the terrestrial laser point cloud, calculates the rotation matrix and translation vector required to transform the laser point cloud coordinates to the image coordinate system, and precisely aligns the two types of data spatially, generating a fused point cloud dataset with a unified spatial reference.

[0068] After coordinate registration, the digital twin platform divides the fused point cloud dataset into a 3D voxel grid according to a preset voxel size. A voxel grid can be understood as dividing 3D space into several identical cubic cells, each containing several laser points. By statistically analyzing the spatial distribution characteristics of all laser points within each cell, the surface information for that location can be extracted. The digital twin platform calculates the average elevation value of all laser points within each voxel grid, using this average elevation value as the representative surface elevation value for that voxel grid location. After processing each voxel grid individually, a regularized grid elevation model is generated. The regularized grid elevation model records the surface height corresponding to each planar location within the scenic area in the form of a grid matrix, representing a digital representation of the scenic area's terrain surface. The process of transforming discrete laser point clouds into a regularized grid elevation model transforms the originally disordered point cloud data into an ordered raster data structure.

[0069] While generating a regularized grid elevation model, the digital twin platform also needs to extract geometric information of building facades from the fused point cloud dataset to compensate for the inability of a pure elevation model to represent the vertical morphology of building facades. Specifically, the digital twin platform separates ground point clouds and non-ground point clouds from the fused point cloud dataset. Ground point clouds refer to laser points that fall on the ground or road surface, while non-ground point clouds include laser points that fall on objects higher than the ground, such as building walls, tree branches, and artificial structures.

[0070] The digital twin platform calculates the normal vector direction for each laser point in the non-ground point cloud. The normal vector can be understood as a direction vector perpendicular to the local surface where the point is located. For laser points falling on a flat wall, the normal vector direction should approximately point horizontally, meaning the angle with the vertical direction is close to 90 degrees. For laser points falling on horizontal ground or tree canopies, the normal vector direction is approximately pointing vertically. Based on this geometric characteristic, the digital twin platform marks point cloud segments in the non-ground point cloud where the angle between the normal vector direction and the vertical direction is less than a preset angle threshold as building facade point clouds, because building walls are the most typical non-ground structures in scenic areas with near-horizontal normal vectors. The digital twin platform projects the geometric boundary of the building facade point cloud, i.e., the outermost contour line of the building wall, onto the corresponding horizontal grid position in the regularized grid elevation model. The projection operation maps the building boundary line in three-dimensional space onto a two-dimensional planar grid, so that each grid position, in addition to recording the ground elevation value, can also record whether a building facade exists at that location and the height range of that facade. The digital twin platform organizes the projected point cloud information of building facades into a geometric information layer of the building facade. This layer has the same spatial range and grid division method as the regularized grid elevation model, and the two can be regarded as different information layers of the same map.

[0071] The digital twin platform overlays a regularized grid elevation model with a layer of building facade geometry information. The regularized grid elevation model provides the ground elevation at each grid location, while the building facade geometry information layer provides the spatial boundaries and heights of the building facades at the same grid location, generating a digital twin model of the scenic area that simultaneously includes the topographical undulations and the spatial structure of the building facades. This scenic area digital twin model provides a unified three-dimensional spatial reference for subsequent level design and virtual content overlay. The game engine can directly perform coordinate queries and geometric calculations within the coordinate system of this scenic area digital twin model when performing position mapping and spatial judgment.

[0072] Further, the step of performing coordinate registration and gridding processing on the multi-source spatial data to generate a digital twin model of the scenic area containing geometric information of terrain surfaces and building facades includes:

[0073] Using the global geocentric coordinate system coordinates of the UAV aerial survey image as a reference, the local coordinate system of the ground laser point cloud is registered to the global geocentric coordinate system of the UAV aerial survey image to generate a fused point cloud dataset with unified spatial reference.

[0074] The fused point cloud dataset is divided into a three-dimensional voxel grid according to a preset voxel size. The average elevation value of all laser points in each voxel grid is calculated. The average elevation value is used as the representative value of the ground elevation to generate a regular grid elevation model.

[0075] Non-ground point clouds are separated from the fused point cloud dataset. Point cloud segments in the non-ground point clouds with a normal vector and a vertical angle smaller than a preset angle are extracted as building facade point clouds. The boundary of the building facade point cloud is projected onto the regularized grid elevation model to form a geometric information layer of the building facade.

[0076] The regularized grid elevation model is overlaid with the building facade geometry information layer to generate a digital twin model of the scenic area that includes terrain surface and building facade geometry information.

[0077] During the acquisition of UAV aerial survey imagery, each pixel records its precise latitude and longitude coordinates on Earth. This coordinate system serves as a unified spatial reference benchmark within the scenic area. In contrast, when acquiring point clouds, ground-based laser scanners establish a local coordinate system with the scanner's installation location as the origin. Each time the scanner moves its installation position, a new local coordinate system origin is generated. Therefore, point cloud data obtained from different stations reside in their respective independent local coordinate systems and cannot be directly overlaid with aerial survey imagery containing global geocentric coordinates. The digital twin platform identifies multiple ground control points with distinct geometric features from the UAV aerial survey imagery, such as intersections of road markings, the planar projection of building corners, or the center of artificially placed crosshairs. Simultaneously, it locates the corresponding three-dimensional coordinates of the same physical location in the ground-based laser point cloud. The digital twin platform uses these paired control point coordinates to calculate the rotation matrix and translation vector required to transform the laser point cloud from the local coordinate system to the global geocentric coordinate system through the iterative nearest point algorithm. It transforms the point clouds obtained from different stations in the local coordinate system to the global geocentric coordinate system one by one, so that all point cloud data and aerial survey images are accurately aligned in spatial position, generating a fused point cloud dataset with a unified spatial reference.

[0078] The distribution of laser points in the fused point cloud dataset is non-uniform and discrete in space, with horizontal distances between adjacent laser points ranging from a few centimeters to tens of centimeters. This discrete and disordered data structure is not conducive to rapid spatial querying and calculation by computer programs. The digital twin platform divides the scenic area into regular rectangular grids in the horizontal direction according to a preset grid spacing. Each grid cell forms a columnar voxel space in the vertical direction. All laser points falling within the same voxel grid collectively reflect the surface morphology at that location. The digital twin platform statistically analyzes the elevation values ​​of all laser points within each voxel grid and calculates their arithmetic mean. This average value is used as the representative surface elevation value for that grid location. After traversing all voxel grids, a regularized grid elevation model is formed.

[0079] The fused point cloud dataset contains surface point clouds and a large number of non-surface point clouds. Surface point clouds record the elevation information of the ground and road surfaces, while non-surface point clouds record information about various features above the ground. The digital twin platform distinguishes between surface and non-surface point clouds by identifying the rate of elevation change between each laser point and its neighbors. Laser points falling on the ground have gradual elevation changes in their neighbors, while laser points falling on building walls may have drastic horizontal elevation changes in their neighbors. For each laser point in the non-surface point cloud, the digital twin platform calculates its normal vector direction. If the angle between the normal vector direction and the vertical direction is less than a preset angle threshold, it indicates that the local surface where the laser point is located is approximately perpendicular to the horizontal plane, conforming to the geometric characteristics of the building facade. The digital twin platform separates the qualified point cloud segments from the non-surface point cloud, extracts their outermost boundary coordinates along the horizontal direction, and projects the boundary coordinates onto the corresponding grid positions of the regularized grid elevation model, forming a geometric information layer of the building facade.

[0080] The regularized grid elevation model records the surface elevation of each grid location, which can express the undulations of the scenic area's terrain. However, surface elevation alone cannot express the vertical shape and spatial occupancy of building walls. The building facade geometry information layer records whether a building facade exists at the same grid location and the elevation range of its bottom and top. The digital twin platform overlays the two layers, so that each grid location simultaneously obtains two sets of data: surface elevation and building facade elevation. When there is no building at the grid location, the surface elevation is used as the surface height; when there is a building at the grid location, the top elevation of the building facade is used as the surface height, and the bottom elevation is used as the building's base position. Thus, the same model can simultaneously express the terrain undulations and the three-dimensional structure of buildings.

[0081] In one embodiment, the step of verifying the level completion conditions based on the NFC interaction data, the AR interaction data, and the location coordinates, marking the level as complete after successful verification, and determining the next level according to the node order of the game script includes:

[0082] Read the game script content of the current level, and extract the verification type field and verification parameter list from the game script content;

[0083] The verification order is determined according to the verification type field, the location coordinates are compared with the target location in the verification parameter list to generate a location verification result, the NFC interaction data is matched with the target tag code in the verification parameter list to generate an NFC verification result, and the AR interaction data is compared with the target virtual content identifier in the verification parameter list to generate an AR verification result.

[0084] The verification logic relationship is determined based on the verification type field. When the verification type field indicates that all verification items must pass, the game engine determines that the level completion condition is met if all verification results are true.

[0085] When the verification type field indicates that any verification item can pass, the checkpoint completion condition is met if any verification result is true.

[0086] After determining that the level completion conditions are met, the current level is marked as completed. The exit node identifier of the current level is read from the game script, and the level node pointed to by the exit node identifier is determined as the next level node.

[0087] During game execution, the game engine continuously receives 3D location data reported by wearable smart terminals, NFC interaction data, and AR interaction data reported by mobile auxiliary apps. The 3D location data arrives at a frequency of once per second or higher; NFC interaction data is triggered when the player touches a real-world tag; and AR interaction data is triggered when the player interacts with virtual content on their phone screen. Upon receiving this data, the game engine first needs to identify the currently active level node. Then, it reads the configuration record corresponding to the current level from the game script stored locally in the game engine. This configuration record contains two key data structures: a verification type field and a verification parameter list.

[0088] The Verification Type field is an identifier indicating the verification combination used in the current level. Different types of level tasks have different verification requirements. For example, a check-in level that only requires visitors to reach a certain location only needs location verification; a punch-in level that requires visitors to touch a specific NFC tag needs NFC verification; an exploration level that requires visitors to identify virtual content using their mobile phones needs AR verification; and a comprehensive level may require a combination of two or three of the above verification methods. The Verification Type field is used to express the identification information of this combination requirement, and its value determines which interactive data needs to be processed in the subsequent verification process and how it is processed. The Verification Parameter List is a specific set of comparison benchmark data used in conjunction with the Verification Type field, which includes specific values ​​such as target location coordinates, target tag code, and target virtual content identifier.

[0089] Since location data, NFC interaction data, and AR interaction data may arrive at the game engine at different times, and the processing computational load varies for each type of data, setting the verification order helps the game engine rationally arrange the execution sequence of verification resources. Therefore, the game engine calculates the spatial distance between the coordinate values ​​in the 3D location data and the target location coordinates in the verification parameter list. If the calculated distance is less than a preset distance threshold, the location verification result is considered true; otherwise, it is considered false. The game engine compares the tag code carried in the NFC interaction data with the target tag code in the verification parameter list bit by bit. If they match exactly, the NFC verification result is considered true; otherwise, it is considered false. The game engine compares the virtual content identifier carried in the AR interaction data with the target virtual content identifier in the verification parameter list. If they match, the AR verification result is considered true; otherwise, it is considered false.

[0090] After calculating the verification results, the game engine determines whether the level completion conditions are met based on the logical relationship indicated by the verification type field. Different level tasks have different requirements for the combination of verification results. Some simple check-in or attendance tasks may only require a single verification method to pass, while some deeply interactive tasks that require full player participation may require multiple verification methods to prove that the player has fully completed all the requirements of the level. When the verification type field indicates that all verification items need to pass, the game engine checks the location verification result, NFC verification result, and AR verification result one by one. Only when all verification results are true is the level completion condition determined to be met; any false verification result results in failure. When the verification type field indicates that any verification item only needs to pass, the game engine checks whether at least one of the three types of verification results is true. As long as any true value exists, the level completion condition is determined to be met.

[0091] After determining that the level completion conditions are met, the game engine marks the current level as completed and writes a completion timestamp and the verification method used during completion into the level node data structure of the game script. The game engine then reads the exit node identifier of the current level from the game script. The exit node identifier is a pointer to the next level node, recording which subsequent level should be jumped to from the current level in the directed acyclic graph of the game script. The game engine identifies the level node pointed to by the exit node identifier as the next level node and sets it as the currently active level to be triggered. It continues to receive and process the location and interaction data reported by the player, repeating the above verification process until all level nodes in the game script are marked as completed.

[0092] In one embodiment, the step of collecting AR interaction data based on real-time screen spatial anchoring and reporting it to the game engine includes:

[0093] The camera is used to capture real-time images, visual feature points are extracted from the real-time images, and the visual feature points are matched with the feature point map in the digital twin model issued by the game engine. Based on the matching results, the initial pose of the camera in the digital twin space is calculated to complete the spatial anchoring initialization.

[0094] After the spatial anchoring initialization is completed, the real-time pose of the camera in the digital twin space is continuously updated by tracking the displacement of visual feature points in the real-time image between consecutive image frames.

[0095] Based on the anchor coordinates of the virtual content corresponding to the current level in the game script and the real-time pose of the camera, it is determined whether the anchor coordinates of the virtual content are within the field of view of the camera. The virtual content within the field of view is rendered and superimposed onto the real-time screen. AR interactive data is generated in response to the touch operation of the guest on the superimposed virtual content. The AR interactive data includes the identifier of the virtual content being operated, the operation type, and the current spatial pose.

[0096] The AR interaction data is reported to the game engine wirelessly.

[0097] Real-time video streams are dynamic data composed of a sequence of image frames continuously captured by a camera, generating approximately thirty frames per second. Each frame contains color and texture information of the current scene within the scenic area. Upon first acquiring a clear real-time frame, the mobile auxiliary app extracts visual feature points from that frame. Visual feature points are local pixel regions in an image that exhibit significant texture changes and can be repeatedly identified from different viewpoints, such as the corners of building windows, the textured edges of rock surfaces, or the intersections of road markings. These feature points can serve as reference anchors for subsequent spatial matching. The mobile auxiliary app then matches the extracted visual feature points with a feature point map in a digital twin model pre-deployed by the game engine.

[0098] The feature point map is generated synchronously by the digital twin platform when constructing the digital twin model of the scenic area. It records the visual feature description vectors corresponding to each salient spatial location in the spatial coordinate system of the scenic area's digital twin model, which can be understood as the feature index of the digital twin model at the visual level. The mobile auxiliary APP compares the similarity between the description vectors of feature points in the current image and the pre-stored description vectors in the feature point map to find the spatial coordinates of the digital twin model corresponding to each feature point in the image. Based on the geometric projection relationship between multiple matching point pairs, the PnP algorithm is used to calculate the position and orientation of the camera in the digital twin space. This position and orientation is the initial pose of the camera, and the spatial anchoring initialization is completed at this point.

[0099] After spatial anchoring initialization, the mobile auxiliary app enters continuous pose tracking mode. The app receives each newly arrived real-time frame, compares the position of visual feature points in the current frame with the position of the same feature point in the previous frame, and calculates the pixel displacement of that feature point between the two frames. This pixel displacement reflects the spatial position and orientation changes of the camera during the brief time interval between acquiring two images. The mobile auxiliary app synthesizes the displacements of multiple feature points in the image and, by solving the essential matrix or homography matrix, infers the camera's motion parameters in three-dimensional space, including rotation angles and translation distances, thereby continuously updating the camera's real-time pose in the digital twin space. This pose update method based on visual feature point tracking does not rely on external positioning signals such as GPS or Bluetooth; it only relies on the images captured by the camera itself, thus maintaining continuous spatial positioning capabilities in various indoor and outdoor environments.

[0100] The mobile auxiliary app determines whether the anchor coordinates of each virtual content element in the current level are within the camera's field of view (FOV) based on the game script's context and the camera's real-time pose. The FAV refers to a pyramidal spatial region extending from the camera's optical center, with the camera's current orientation as the central axis and the camera's field of view as the subtended angle. Only 3D spatial points falling within this region will appear on the camera's image. The mobile auxiliary app transforms the anchor coordinates of each virtual content element from the digital twin model's spatial coordinate system to the camera's local coordinate system, then projects them onto the image pixel coordinate system. It checks whether the projected pixel coordinates fall within the effective display area and whether the distance between the anchor point and the camera is within the preset rendering distance. When the virtual content's anchor coordinates meet these conditions, the mobile auxiliary app determines that the virtual content is within the camera's FAV. The mobile auxiliary app then performs perspective projection transformation on the virtual content within the FAV according to the relative spatial relationship between its anchor coordinates and the camera, rendering and superimposing it onto the real-time image with the correct size, angle, and occlusion relationship, making the virtual content appear fixed in a specific location in real physical space.

[0101] AR interaction data includes at least the identifier of the manipulated virtual content, the type of operation, and the real-time pose of the camera in the digital twin space. The identifier of the manipulated virtual content uniquely identifies which virtual item the visitor interacted with. The type of operation records whether the visitor performed a click, long press, or swipe, etc. The current spatial pose records the precise position and orientation of the camera at the moment the interaction occurred, serving as spatial coordinate evidence to verify whether the interaction was performed in the correct spatial location. The mobile auxiliary app reports the generated AR interaction data to the game engine wirelessly via Wi-Fi or cellular networks for use in verifying level completion conditions.

[0102] In one embodiment, scheduling the drone to fly to the tourist's location based on the continuous location data includes:

[0103] Extract the continuous location data of tourists from the SOS alarm, and obtain the no-fly zone boundary data around the location indicated by the continuous location data from the scenic area digital twin model;

[0104] Using the location indicated by the continuous location data as the target point, the current location of the idle UAV's nest as the starting point, and the no-fly zone boundary data as obstacle constraints, a flight path is generated from the starting point to the target point without crossing the no-fly zone.

[0105] The flight path is converted into drone flight commands, which are then sent to the drone nest via a wireless communication link. Upon receiving the drone flight commands, the drone nest controls the drone to take off and fly to the tourist's location along the flight path.

[0106] When the wearable smart terminal detects a tourist fall or triggers an SOS event by pressing and holding a physical button, it immediately sends an SOS alarm containing continuous location data to the command and management center via a 4G communication module. This continuous location data includes the tourist's current latitude, longitude, and altitude coordinates. This data is collected and packaged into the alarm data by the wearable smart terminal's positioning fusion module at the moment the SOS is triggered. Upon receiving the SOS alarm, the command and management center first extracts the tourist's continuous location data from the SOS alarm data to obtain the target coordinates of the tourist's current location.

[0107] The command and management center uses the tourist's current location as the target point, the location of the currently idle drone's nest as the starting point, and the no-fly zone boundary data stored in the scenic area's digital twin model as obstacle constraints to generate a flight path from the drone's nest location to the tourist's location without crossing the no-fly zone. The scenic area's digital twin model pre-stores no-fly zone boundary data within the scenic area. No-fly zones are areas within the scenic area where drones are prohibited from flying, typically including densely populated tourist areas, airspace above important cultural relics protection units, wildlife habitats, and areas surrounding communication base stations. The boundaries of these areas are represented in the digital twin model by a sequence of closed polygon coordinates. When planning the flight path, the command and management center treats the no-fly zone as an insurmountable obstacle, ensuring that the generated flight path completely bypasses the no-fly zone boundaries.

[0108] Specifically, the command and management center projects the scenic area's digital twin model onto a horizontal plane as a two-dimensional planar map. This map is then divided into multiple grid units according to a preset grid size, each corresponding to a fixed-size square area within the scenic area. Based on no-fly zone boundary data, the command and management center marks grid units falling within the no-fly zone as impassable and those falling within the flyable zone as passable, generating a gridded access map. The command and management center maps the drone's nest location to the corresponding starting grid unit in the gridded access map and the visitor's location to the corresponding target grid unit. Using the starting grid unit as the path's origin and the target grid unit as the path's destination, the command and management center searches the gridded access map for a sequence of path nodes composed of passable grid units.

[0109] During the search process, the command and management center uses the spatial distance between adjacent traversable grid cells as the path cost, and the sum of the actual cumulative cost from the path start point to the current grid cell and the estimated cost from the current grid cell to the path end point as the priority evaluation index. In each round, the grid cell with the lowest priority evaluation index is selected from the grid set to be expanded for expansion. The adjacent traversable grid cells of the expanded grid are added to the grid set to be expanded. The search ends when the target grid cell is expanded, thus obtaining the grid path from the starting grid cell to the target grid cell with the minimum path cost. The command and management center connects the center point coordinates of each grid cell in the grid path sequentially according to the search order to generate a navigation path composed of a sequence of geographic coordinates.

[0110] The command and control center converts the flight path into UAV flight commands. A UAV flight command is a structured control command data set, containing parameters such as takeoff instructions, climb altitude, horizontal speed along waypoints, hovering altitude above the target point, and landing or return-to-home instructions. The command and control center transmits the UAV flight commands to the UAV pod via a wireless communication link. Upon receiving the flight commands, the UAV pod controls the UAV to take off and autonomously fly to the tourist's location according to the flight path, transmitting its position status and surrounding environment video back to the command and control center in real time.

[0111] Simultaneously, the command and management center retrieves real-time footage from fixed video surveillance cameras deployed around the tourist's location based on the continuous location data from the SOS alarm. Fixed video surveillance cameras are pre-deployed at key road sections and major nodes within the scenic area, and each camera's installation coordinates and shooting angle are recorded in the scenic area's digital twin model. The command and management center calculates the spatial distance and angle between the tourist's location and the installation locations of each camera, selecting the camera closest to the tourist's location and displaying its real-time footage on the command center's large screen. Command personnel establish two-way voice communication with the tourist via the voice communication module of a wearable smart terminal to verify the situation and coordinate subsequent rescue efforts. The entire dispatch process, from receiving the SOS alarm to the drone's takeoff, remains within a preset response time threshold, ensuring rapid resource allocation and response deployment in emergencies.

[0112] Further, the step of using the location indicated by the continuous location data as the target point, the current location of the idle UAV's nest as the starting point, and the no-fly zone boundary data as obstacle constraints to generate a flight path from the starting point to the target point without crossing the no-fly zone includes:

[0113] The digital twin model of the scenic area is projected into a two-dimensional planar map and divided into multiple grid units. Based on the no-fly zone boundary data, the grid units that fall within the no-fly zone are marked as impassable grids, and a gridded access map is generated.

[0114] The drone nest location is mapped to the starting grid cell in the gridded access map, and the tourist's location is mapped to the target grid cell in the gridded access map. The starting grid cell is used as the path start point and the target grid cell is used as the path end point. The path node sequence composed of accessible grid cells is searched on the gridded access map.

[0115] The spatial distance between adjacent passable grid cells is used as the path cost, and the sum of the actual cumulative cost from the path start point to the current grid cell and the estimated cost from the current grid cell to the path end point is used as the priority evaluation index. In each round, the grid cell with the smallest priority evaluation index is selected from the grid cell set to be expanded and expanded. The adjacent passable grid cells of the expanded grid cell set are added to the grid cell set to be expanded. The search ends when the target grid cell is expanded, and the grid path with the smallest path cost from the starting grid cell to the target grid cell is obtained.

[0116] The center point coordinates of each grid cell in the grid path are connected in the search order to generate a navigation path composed of a sequence of geographic coordinates.

[0117] After receiving an SOS alarm and extracting the continuous location data of the tourists, the command and management center needs to plan a safe flight path from the current location of the idle drone's nest to the tourist's location. The no-fly zone has clearly defined polygonal boundaries, covering areas with high tourist density, important cultural relics sites, wildlife habitats, and areas surrounding communication base stations. Since drones typically maintain a relatively fixed altitude during flight, and the projections of tall buildings and mountains within the scenic area on the horizontal plane constitute the main no-fly obstacles, flight path planning can be carried out on the horizontal projection plane.

[0118] The command and management center projects the scenic area's digital twin model onto a horizontal plane as a two-dimensional planar map, dividing the map into multiple grid cells according to a preset grid size. Specifically, the projection process involves extracting the latitude and longitude components of each 3D coordinate point in the scenic area's digital twin model as the horizontal and vertical coordinates, discarding the height component, to obtain a planar layout of the scenic area on the horizontal plane. The selection of the preset grid size needs to balance the accuracy of path planning with the calculation speed; too large a size may prevent paths from passing through narrow passable passages, while too small a size will increase the number of grid cells and the computational load. Based on the no-fly zone boundary data, the command and management center marks grid cells falling within the no-fly zone as impassable grid cells and grid cells outside the no-fly zone as passable grid cells, generating a gridded access map. Each grid cell in this map has one of two states: impassable or passable. An impassable grid indicates that drones are prohibited from flying over that location, while a passable grid indicates that drones are permitted to fly over that location.

[0119] The command and management center maps the location of the drone's nest to the corresponding starting grid cell in the rasterized access map, and maps the location of the tourist to the corresponding target grid cell in the same rasterized access map. This mapping process involves converting latitude and longitude coordinates into row and column indices in the raster map by calculating the grid cell position of the coordinate point on the two-dimensional map. After obtaining the starting and target grid cells, the command and management center uses the starting grid cell as the path's starting point and the target grid cell as the path's ending point, searching the rasterized access map for a sequence of path nodes consisting of accessible grid cells. This sequence of path nodes is the set of all grid cells traversed from the starting point to the ending point, where any two adjacent grid cells are edge-adjacent or corner-adjacent on the map.

[0120] During the search process, the command and management center uses the spatial distance between adjacent walkable grid cells as the path cost. Spatial distance refers to the Euclidean distance between the center points of two adjacent grid cells. For two horizontally or vertically adjacent grid cells, the spatial distance is equal to the grid size. For two diagonally adjacent grid cells, the spatial distance is equal to the grid size multiplied by the square root of two. The command and management center uses the sum of the actual cumulative cost from the path start point to the current grid cell and the estimated cost from the current grid cell to the path end point as a priority evaluation index. The actual cumulative cost is the sum of the spatial distances between all adjacent grid pairs traversed from the path start point along the searched path to the current grid cell; this value reflects the length of the path already traversed. The estimated cost is the straight-line distance from the current grid cell to the path end point; this value is an estimate of the remaining path length. The smaller the sum of the two, the more likely the potentially complete path passing through the current grid cell is to become the shortest path.

[0121] In each round, the command and control center selects the grid cell with the lowest priority evaluation index from the grid set to be expanded for expansion. Initially, the grid set contains only the starting grid cell. Each round, the grid cell with the lowest priority evaluation index is selected and designated as the expanded grid. Then, the eight adjacent grid cells surrounding this grid cell are checked, and those that are passable and have not yet been expanded are added to the grid set to be expanded. The parent node of these newly added grid cells is recorded as the currently expanded grid cell. When the target grid cell is added to the grid set to be expanded and then removed for expansion, the command and control center terminates the search process. Starting from the target grid cell, it traces back along the recorded parent node pointers to the starting grid cell, arranging the traversed grid cells in order from start to finish, thus obtaining the grid path from the starting grid cell to the target grid cell. Since the grid cell with the lowest priority evaluation index is selected for expansion each time during the search process, this grid path is the path with the lowest path cost among all possible paths.

[0122] The command and management center connects the center point coordinates of each grid cell in the grid path sequentially according to the search order, generating a flight path composed of a sequence of geographic coordinates. The center point coordinates of each grid cell are the geographic coordinates of the geometric center of the rectangular area covered by that grid cell. Connecting the center points of all grid cells in the path in order from the start to the end results in a polyline path composed of sequentially connected straight line segments. This flight path does not cross any no-fly zones and has the shortest total length among all paths that satisfy the constraint of not crossing no-fly zones. The command and management center converts the flight path into UAV flight commands and sends them to the UAV nest, controlling the UAV to fly along the path to the tourist's location. This embodiment uses a gridded map for path search, enabling the calculation of no-fly zone constraints and the shortest path objective within a unified grid space. An executable UAV flight path can be generated on the two-dimensional projection plane of the digital twin model without complex geometric calculations.

[0123] In one embodiment, the system further includes a data analysis module, which generates a leaderboard based on play data reports and extracts bonuses from the game revenue pool to distribute to the corresponding tourists' NFC payment accounts.

[0124] The data analysis module receives a play data report sent by the game engine. This play data report records the actual values ​​of various key indicators during the visitor's play, including the total time spent completing all levels, the ratio of the number of levels completed to the total number of levels (i.e., the completion rate), the total number of virtual items successfully collected by the visitor through AR interaction during the entire play, and the total number of times the visitor triggered and completed the hidden tasks preset in the game script.

[0125] The data analysis module extracts the aforementioned indicators from the play data reports of all visitors who have finished playing that day, and sorts all visitors according to preset ranking dimensions. The preset ranking dimensions include at least one of the following: completion speed, completion rate, number of collected items, and number of times hidden quests are triggered. The data analysis module ranks visitors from highest to lowest according to the primary ranking dimension pre-set by the system operator. When multiple visitors have the same value in the primary dimension, they are ranked according to the value in the secondary dimension, and so on, generating a daily game leaderboard containing the ranking sequence. The data analysis module pushes the generated leaderboard data to a mobile auxiliary APP, allowing visitors to view their ranking position and the score difference between adjacent visitors in the APP interface.

[0126] While generating the leaderboard, the data analysis module reads the total daily game revenue pool from the game engine's fund management submodule. The daily game revenue pool refers to a unified pool of funds collected from all revenue generated by tourists using the NFC payment function of wearable smart terminals for ticket purchases, value-added services, and other in-park consumption, after deducting payment channel fees. The data analysis module calculates the total prize money extracted from the pool according to the reward extraction ratio preset by the system operator, and distributes this prize money to the top-ranked tourists according to their leaderboard rankings. The first-ranked tourist receives the preset maximum share of the prize money, the second-ranked tourist receives the second-largest share, and subsequent ranked tourists receive shares in a preset decreasing tier until all preset prize money slots are allocated. The data analysis module generates a prize money distribution record for each winning tourist, which includes at least the winning tourist's identity identifier, ranking, and the prize money amount due.

[0127] The data analysis module sends the prize money distribution record to the payment management submodule, which then executes the prize money distribution operation to the corresponding winning tourist's NFC payment account through the scenic area's unified payment system. The NFC payment account is an electronic payment account bound to the NFC reader / writer module in the wearable smart terminal. This account is part of the same account system as the payment account used by the tourist when purchasing tickets before the game started. The payment management submodule transfers the prize money to the winning tourist's NFC payment account. The tourist can then use the wearable smart terminal to directly spend the remaining prize money in the account via NFC payment during subsequent scenic area visits, or withdraw the account balance to their personal bank card after the visit. After completing the prize money distribution, the data analysis module simultaneously pushes the leaderboard data and prize money distribution results to the mobile auxiliary APP. The APP sends notification messages containing the winning tourist's ranking and prize money amount, and displays the complete leaderboard list and corresponding ranking scores to all participating tourists.

[0128] The above implementation directly links leaderboard rankings with prize distribution. The data analysis module uniformly processes the aggregation, sorting, and prize calculation and distribution of play data, creating a direct correlation between visitor performance and tangible rewards. The leaderboard's competitive mechanism incentivizes visitors to strive for faster completion times, higher completion rates, and more complete collections during gameplay. The prize distribution mechanism enhances visitor motivation and repeat visits by returning a portion of the daily revenue to high-performing visitors. The entire data analysis module operates without human intervention; the entire process, from receiving play data reports to leaderboard generation and prize distribution, is automated, ensuring the objectivity and timeliness of ranking calculations and prize distribution.

[0129] In one embodiment, the system further includes a visitor abnormal behavior recognition module, which is configured in the game engine or the command and management center or set up independently.

[0130] The abnormal tourist behavior recognition module is used to receive continuous location data reported by the wearable smart terminal and AR interaction data reported by the mobile auxiliary APP.

[0131] The abnormal tourist behavior identification module calculates the real-time movement trajectory of tourists in the scenic area based on the continuous location data, compares the real-time movement trajectory with the predefined tour route in the digital twin model, and generates a deviation alarm when the real-time movement trajectory deviates from the predefined tour route by more than a preset distance threshold and sends it to the command and management center.

[0132] And / or, the abnormal tourist behavior recognition module calculates the duration of the tourist's stay within the checkpoint geofence based on the continuous location data. When the interaction frequency of the AR interactive data is lower than a preset threshold and the stay duration exceeds a preset time threshold, an abnormal stay alarm is generated and sent to the command and management center.

[0133] The abnormal visitor behavior recognition module can be configured within the game engine, running as a functional submodule. This module receives continuous location data reported by wearable smart terminals and AR interaction data reported by mobile auxiliary apps in real time. Continuous location data arrives at a frequency of once per second or higher, containing the visitor's three-dimensional spatial coordinates at every moment within the scenic area. AR interaction data is reported when a visitor clicks or swipes on overlaid virtual content using the mobile auxiliary app; this data includes the identifier of the manipulated virtual content, the type of operation, and the spatial pose at the time of the operation.

[0134] The abnormal visitor behavior detection module calculates the real-time movement trajectory of visitors within the scenic area based on continuous location data. This trajectory is a spatial path formed by connecting the location coordinates of visitors at consecutive time points in chronological order. The module extracts all continuous location data points received within a recent period using a fixed time window, arranges them from earliest to latest according to their timestamps, forming a sequence of location points within that time window, and then connects adjacent points in this sequence with straight line segments to generate the visitor's real-time movement trajectory within the scenic area. The module also reads predefined tour routes from the scenic area's digital twin model. These predefined routes are recommended walking paths marked on the digital twin model by the scenic area operators when designing the game script; they consist of a series of ordered spatial coordinate points, representing the walking direction visitors should follow under normal tour conditions.

[0135] The abnormal tourist behavior detection module calculates the distance between each location point in the real-time movement trajectory and the predefined tour route. Specifically, it calculates the vertical distance from each location point to the nearest coordinate point on the tour route. When the vertical distances from multiple consecutive location points to the tour route exceed a preset distance threshold, it indicates that the tourist has continuously deviated from the predefined tour route, and the abnormal tourist behavior detection module generates a deviation alarm. The deviation alarm includes at least the tourist's identification, the current coordinates of the deviation location, the time the deviation started, and the coordinate sequence of the nearest location in the real-time movement trajectory. The abnormal tourist behavior detection module sends the deviation alarm to the command and management center, which then highlights the tourist's location on the situational awareness screen and notifies nearby staff to pay attention.

[0136] Meanwhile, the abnormal visitor behavior recognition module also calculates the visitor's dwell time within the level's geofence based on continuous location data. The level's geofence is a circular spatial area centered on the level node location, and its boundary coordinates are stored in the game engine during the game script configuration stage. The abnormal visitor behavior recognition module continuously determines whether the continuous location data received at each moment falls within the geofence range of the current level. When the location coordinates first enter the geofence, the entry time is recorded; when the location coordinates leave the geofence, the departure time is recorded. The time length between the entry and departure times is the visitor's dwell time within the level's geofence. The abnormal visitor behavior recognition module obtains the total amount of AR interaction data reported by the mobile auxiliary APP within the current time window, divides the total amount of AR interaction data by the length of the time window, and calculates the frequency of the visitor's AR interactions within the current time period.

[0137] When a visitor's dwell time within the geofence of a checkpoint exceeds a preset time threshold, and the frequency of AR interactions within the same time period is lower than a preset threshold, it indicates that the visitor has been staying in the area for an extended period without engaging in the required augmented reality interactions. This may indicate that the visitor is disoriented, experiencing physical discomfort, or is unable to complete the checkpoint task for other reasons. The visitor abnormal behavior recognition module generates an abnormal dwelling alarm and sends it to the command and management center. The abnormal dwelling alarm includes at least the visitor's identification, current location coordinates, dwell time value, and geofence boundary information for the checkpoint, allowing the command and management center to determine whether to dispatch personnel to investigate.

[0138] The visitor abnormal behavior recognition module analyzes real-time trajectory deviation and abnormal loitering to automatically identify abnormal visitor behavior using existing continuous location data and AR interaction data without requiring additional hardware. Route deviation detection utilizes geometric comparison between location data and predefined routes, while abnormal loitering detection combines spatial dwelling information from location data with operational activity information from AR interaction data. These two types of detection cover various abnormal situations that visitors may encounter during their visit, providing data support for proactive safety intervention by the command and management center.

[0139] Reference Figure 2 The diagram shown is a flowchart illustrating an embodiment of the interactive method for cultural and tourism scenarios according to the present invention. This method is executed by the aforementioned system, which is implemented in software and / or hardware. The interactive method for cultural and tourism scenarios includes:

[0140] Construct a digital twin model of the scenic area, configure the location, triggering conditions, completion conditions and virtual content anchoring coordinates of the level nodes on the digital twin model, generate a game script and store it in the game engine;

[0141] The wearable smart terminal integrates multi-source positioning data to generate continuous location data and reports it to the game engine; it also generates NFC interactive data by reading real-world tag encoding via NFC and reports it to the game engine.

[0142] The mobile auxiliary APP is bound to the wearable smart terminal, and AR interaction data is collected based on real-time screen spatial anchoring and reported to the game engine;

[0143] The game engine reads the game script content of the current level, maps the continuous position data to a digital twin coordinate system, triggers the level according to the mapped coordinates, verifies the level completion conditions based on the NFC interaction data, the AR interaction data and the position coordinates, marks the level as completed after verification, and determines the next level according to the node order of the game script.

[0144] During the interaction, the wearable smart terminal detects a fall event based on motion sensor data or triggers an SOS event by responding to a physical button, and sends an SOS alarm containing the continuous location data to the command and management center; the command and management center receives the SOS alarm, dispatches a drone to the tourist's location based on the continuous location data, retrieves video surveillance footage of the surrounding area, and establishes voice communication through the wearable smart terminal.

[0145] The specific implementation method of the interactive method for cultural and tourism scenarios of the present invention is largely the same as the specific implementation method of the interactive system for cultural and tourism scenarios described above, and will not be repeated here.

[0146] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware simulation platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of the present invention.

[0148] Although specific embodiments of the present invention have been described in detail with reference to the accompanying drawings, this should not be construed as limiting the scope of protection of this patent. Various modifications and variations that can be made by those skilled in the art without inventive effort within the scope described in the claims still fall within the scope of protection of this patent.

Claims

1. An interactive system for cultural and tourism scenarios, characterized in that, The system includes: A digital twin platform is used to construct a digital twin model of a scenic area. The location, triggering conditions, completion conditions, and virtual content anchoring coordinates of the level nodes are configured on the digital twin model of the scenic area, and a game script is generated and stored in the game engine. Wearable smart terminal is used to integrate multi-source positioning data to generate continuous location data and report it to the game engine, generate NFC interactive data by reading real-scene tag encoding through NFC and report it to the game engine, detect tourist fall events based on motion sensor data or trigger SOS events in response to physical buttons, and send SOS alarms containing the continuous location data to the command and management center. A mobile auxiliary APP is used to bind to the wearable smart terminal and collect AR interaction data based on real-time screen spatial anchoring and report it to the game engine. The game engine is used to read the game script content of the current level, map the continuous position data to a digital twin coordinate system, trigger the level according to the mapped coordinates, verify the level completion conditions based on the NFC interaction data, the AR interaction data and the position coordinates, mark the level as completed after verification, and determine the next level according to the node order of the game script. The command and management center is used to receive SOS alarms, dispatch drones to the tourist's location based on the continuous location data, retrieve video surveillance footage of the surrounding area, and establish voice communication through the wearable smart terminal.

2. The system as described in claim 1, characterized in that, The process of constructing a digital twin model of the scenic area, configuring the location, triggering conditions, completion conditions, and virtual content anchor coordinates of the level nodes on the digital twin model, generating a game script, and storing it in the game engine includes: The digital twin platform acquires multi-source spatial data of the scenic area, performs coordinate registration and gridding processing on the multi-source spatial data, and generates a digital twin model of the scenic area containing geometric information of terrain surface and building facade. The multi-source spatial data includes UAV aerial survey images and ground laser point clouds. Candidate checkpoint locations are marked on the digital twin model of the scenic area. A geofence with a preset radius is generated with each candidate checkpoint location as the center. The set of geofence coordinates corresponding to each candidate checkpoint location is stored in the checkpoint location database of the game engine. The set of geofence coordinates is read from the checkpoint location database. Based on the spatial region attributes of each candidate checkpoint location in the scenic area digital twin model, trigger conditions and completion conditions are assigned to each candidate checkpoint location, and checkpoint node definition data containing location data and logical rules is generated. In the spatial coordinate system of the digital twin model of the scenic area, anchor coordinates of virtual content are bound to each checkpoint node. The checkpoint node definition data and the anchor coordinates are associated and stored as checkpoint node configuration records. All checkpoint node configuration records are arranged in the order of tour to generate a game script and stored in the game engine.

3. The system as described in claim 2, characterized in that, The step of performing coordinate registration and gridding processing on the multi-source spatial data to generate a digital twin model of the scenic area containing geometric information of terrain surfaces and building facades includes: Using the global geocentric coordinate system coordinates of the UAV aerial survey image as a reference, the local coordinate system of the ground laser point cloud is registered to the global geocentric coordinate system of the UAV aerial survey image to generate a fused point cloud dataset with unified spatial reference. The fused point cloud dataset is divided into a three-dimensional voxel grid according to a preset voxel size. The average elevation value of all laser points in each voxel grid is calculated. The average elevation value is used as the representative value of the ground elevation to generate a regular grid elevation model. Non-ground point clouds are separated from the fused point cloud dataset. Point cloud segments in the non-ground point clouds with a normal vector and a vertical angle smaller than a preset angle are extracted as building facade point clouds. The boundary of the building facade point cloud is projected onto the regularized grid elevation model to form a geometric information layer of the building facade. The regularized grid elevation model is overlaid with the building facade geometry information layer to generate a digital twin model of the scenic area that includes terrain surface and building facade geometry information.

4. The system as described in claim 1, characterized in that, The process of verifying the level completion conditions based on the NFC interaction data, the AR interaction data, and the location coordinates, marking the level as complete upon successful verification, and determining the next level according to the node order of the game script includes: Read the game script content of the current level, and extract the verification type field and verification parameter list from the game script content; The verification order is determined according to the verification type field, the location coordinates are compared with the target location in the verification parameter list to generate a location verification result, the NFC interaction data is matched with the target tag code in the verification parameter list to generate an NFC verification result, and the AR interaction data is compared with the target virtual content identifier in the verification parameter list to generate an AR verification result. The verification logic relationship is determined based on the verification type field. When the verification type field indicates that all verification items must pass, the game engine determines that the level completion condition is met if all verification results are true. When the verification type field indicates that any verification item can pass, the checkpoint completion condition is met if any verification result is true. After determining that the level completion conditions are met, the current level is marked as completed. The exit node identifier of the current level is read from the game script, and the level node pointed to by the exit node identifier is determined as the next level node.

5. The system as described in claim 1, characterized in that, The reporting of AR interaction data collected based on real-time screen spatial anchoring to the game engine includes: The camera is used to capture real-time images, visual feature points are extracted from the real-time images, and the visual feature points are matched with the feature point map in the digital twin model issued by the game engine. Based on the matching results, the initial pose of the camera in the digital twin space is calculated to complete the spatial anchoring initialization. After the spatial anchoring initialization is completed, the real-time pose of the camera in the digital twin space is continuously updated by tracking the displacement of visual feature points in the real-time image between consecutive image frames. Based on the anchor coordinates of the virtual content corresponding to the current level in the game script and the real-time pose of the camera, it is determined whether the anchor coordinates of the virtual content are within the field of view of the camera. The virtual content within the field of view is rendered and superimposed onto the real-time screen. AR interactive data is generated in response to the touch operation of the guest on the superimposed virtual content. The AR interactive data includes the identifier of the virtual content being operated, the operation type, and the current spatial pose. The AR interaction data is wirelessly reported to the game engine.

6. The system as described in claim 1, characterized in that, The step of scheduling the drone to fly to the tourist's location based on the continuous location data includes: Extract the continuous location data of tourists from the SOS alarm, and obtain the no-fly zone boundary data around the location indicated by the continuous location data from the scenic area digital twin model; Using the location indicated by the continuous location data as the target point, the current location of the idle UAV's nest as the starting point, and the no-fly zone boundary data as obstacle constraints, a flight path is generated from the starting point to the target point without crossing the no-fly zone. The flight path is converted into drone flight commands, which are then sent to the drone nest via a wireless communication link. Upon receiving the drone flight commands, the drone nest controls the drone to take off and fly to the tourist's location along the flight path.

7. The system as described in claim 6, characterized in that, The step of using the location indicated by the continuous location data as the target point, the current location of the idle UAV's nest as the starting point, and the no-fly zone boundary data as obstacle constraints to generate a flight path from the starting point to the target point without crossing the no-fly zone includes: The digital twin model of the scenic area is projected into a two-dimensional planar map and divided into multiple grid units. Based on the no-fly zone boundary data, the grid units that fall within the no-fly zone are marked as impassable grids, and a gridded access map is generated. The drone nest location is mapped to the starting grid cell in the gridded access map, and the tourist's location is mapped to the target grid cell in the gridded access map. The starting grid cell is used as the path start point and the target grid cell is used as the path end point. The path node sequence composed of accessible grid cells is searched on the gridded access map. The spatial distance between adjacent passable grid cells is used as the path cost, and the sum of the actual cumulative cost from the path start point to the current grid cell and the estimated cost from the current grid cell to the path end point is used as the priority evaluation index. In each round, the grid cell with the smallest priority evaluation index is selected from the grid cell set to be expanded and expanded. The adjacent passable grid cells of the expanded grid cell set are added to the grid cell set to be expanded. The search ends when the target grid cell is expanded, and the grid path with the smallest path cost from the starting grid cell to the target grid cell is obtained. The center point coordinates of each grid cell in the grid path are connected in the search order to generate a navigation path composed of a sequence of geographic coordinates.

8. The system as described in claim 1, characterized in that, The system also includes a data analysis module, which generates a leaderboard based on play data reports and extracts bonuses from the game revenue pool to distribute to the corresponding tourists' NFC payment accounts.

9. The system according to any one of claims 1 to 8, characterized in that, The system also includes a visitor abnormal behavior recognition module, which is configured in the game engine or the command and management center, or set up independently. The abnormal tourist behavior recognition module is used to receive continuous location data reported by the wearable smart terminal and AR interaction data reported by the mobile auxiliary APP. The abnormal tourist behavior identification module calculates the real-time movement trajectory of tourists in the scenic area based on the continuous location data, compares the real-time movement trajectory with the predefined tour route in the digital twin model, and generates a deviation alarm when the real-time movement trajectory deviates from the predefined tour route by more than a preset distance threshold and sends it to the command and management center. And / or, the abnormal tourist behavior recognition module calculates the duration of the tourist's stay within the checkpoint geofence based on the continuous location data. When the interaction frequency of the AR interactive data is lower than a preset threshold and the stay duration exceeds a preset time threshold, an abnormal stay alarm is generated and sent to the command and management center.

10. An interactive method for cultural and tourism scenarios, applied to the system as described in claim 1, characterized in that, The method includes: Construct a digital twin model of the scenic area, configure the location, triggering conditions, completion conditions and virtual content anchoring coordinates of the level nodes on the digital twin model, generate a game script and store it in the game engine; The wearable smart terminal integrates multi-source positioning data to generate continuous location data and reports it to the game engine; it also generates NFC interactive data by reading real-world tag encoding via NFC and reports it to the game engine. The mobile auxiliary APP is bound to the wearable smart terminal, and AR interaction data is collected based on real-time screen spatial anchoring and reported to the game engine; The game engine reads the game script content of the current level, maps the continuous position data to a digital twin coordinate system, triggers the level according to the mapped coordinates, verifies the level completion conditions based on the NFC interaction data, the AR interaction data and the position coordinates, marks the level as completed after verification, and determines the next level according to the node order of the game script. During the interaction, the wearable smart terminal detects a fall event based on motion sensor data or triggers an SOS event by responding to a physical button, and sends an SOS alarm containing the continuous location data to the command and management center; the command and management center receives the SOS alarm, dispatches a drone to the tourist's location based on the continuous location data, retrieves video surveillance footage of the surrounding area, and establishes voice communication through the wearable smart terminal.

Images