Virtual character memory extraction and application system and method based on geographic position triggering

Abstract

The invention discloses a virtual role memory extraction and application system and method based on geographic position triggering, and relates to the technical field of artificial intelligence and geographic information processing. The system comprises a mobile client and a cloud server, the client collects multi-modal data and triggers retrieval through an H3 index, and a cloud constructs a space-time memory bank to achieve hybrid storage and retrieval. According to the method, a smooth trajectory is obtained through multi-sensor fusion positioning, geo-fence pre-filtering is carried out based on an H3 index, an S-T-S model is adopted for sorting memory, and memory is adaptively updated after a context-related reply is generated through LLM. According to the method, time-space-semantic integrated memory management is realized, the problems of context dislocation, low retrieval efficiency, insufficient virtual-real fusion precision and the like of an existing system are solved, a virtual character has anthropomorphic geographical environment perception and memory wake-up capability, and interaction immersion and real-time performance are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and geographic information fusion technology, specifically to a virtual character memory extraction and application system and method based on geographic location triggering. Background Technology

[0002] With the rapid development of artificial intelligence technology, virtual character systems based on Large Language Models (LLM) have demonstrated strong potential in applications such as games, education, and navigation. These systems can understand natural language and dynamically generate content, providing users with a more intelligent and immersive interactive experience. To maintain consistency in long-term interactions, Retrieval-Enhanced Generation (RAG) technology has been introduced into virtual character systems, using external memory mechanisms to store and retrieve historical interaction information.

[0003] In existing technologies, pure semantic vector retrieval systems based on LangChain or LlamaIndex are common solutions. These systems retrieve relevant memories from vector databases by calculating the semantic similarity of text (such as cosine similarity), which can maintain the coherence of the conversation to some extent. However, these systems have a fundamental flaw: they completely lack the ability to perceive physical spatial location. When applied to location-based services (LBS) scenarios, the inability to distinguish the user's actual geographical location can lead to memory mismatches across geographical locations. For example, when a user asks a question in Nanjing, the system may incorrectly retrieve a similar conversation that occurred in Beijing, severely damaging the contextual relevance and immersion of the interaction.

[0004] Another representative approach is the "generative agent" sandbox architecture proposed by Stanford University. This approach introduces a more complex memory model, retrieving memories by comprehensively considering temporal relevance, importance, and semantic relevance, and operates within a virtual 2D sandbox map. While this approach performs well in virtual environments, its spatial model is virtual and simplified, unable to handle the challenges of complex geographic projections, GPS positioning noise, and large-scale geographic data indexing in the real world, thus making it unsuitable for direct application in real-world outdoor LBS scenarios.

[0005] In addition, while traditional point-of-interest (POI)-based tour guide systems can achieve location-based triggering, these systems typically use pre-recorded audio or video content, and the interaction is static and rigid. They lack the ability to learn, remember, and evolve dynamically, and therefore cannot meet users' needs for intelligent and natural dialogue.

[0006] In summary, existing technical solutions have significant shortcomings in deeply coupling LLM's intelligent dialogue capabilities with real geospatial reality: they either lack spatial dimension, cannot operate effectively in the real physical world, or have insufficient interactive capabilities. Therefore, there is an urgent need for a system that can deeply integrate LLM's semantic understanding capabilities, memory retrieval mechanisms, and real geospatial perception to support a coherent, realistic, and highly contextualized interactive experience in scenarios such as outdoor augmented reality (AR) navigation and immersive gaming. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention discloses a virtual character memory extraction and application system and method based on geolocation triggering, in order to solve the problems mentioned in the background technology.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a virtual character memory extraction and application system based on geographical location triggering, comprising a mobile client and a cloud server connected by communication;

[0009] The mobile client includes a multimodal perception module, an LBS trigger engine, and an AR rendering engine. The multimodal perception module is used to collect GNSS data, IMU data, environmental image streams, and user voice commands. The LBS trigger engine has a built-in H3 index calculation library, which is used to calculate the H3 index of the current location in real time and compare it with the locally cached plot trigger list to trigger a memory retrieval request. The AR rendering engine is used to load virtual character models, play animations, render speech bubbles, and handle the occlusion relationship between virtual objects and the real environment.

[0010] The cloud server includes an API gateway, a spatiotemporal memory bank, a memory retrieval engine, and a generative reasoning service. The API gateway receives client requests and performs authentication and distribution. The spatiotemporal memory bank includes a vector storage area, a spatial index area, and a metadata area. The vector storage area stores text embedding vectors. The spatial index area stores H3 indexes and PostGIS geometric objects. The metadata area stores structured data such as timestamps, sentiment tags, and associated NPCIDs. The memory retrieval engine executes a hybrid retrieval algorithm to calculate the comprehensive score of memory entries. The generative reasoning service deploys an LLM (Limited Language Management) to receive system prompts containing spatiotemporal memories and generate responses that match the character's persona.

[0011] Preferably, the multimodal perception module of the mobile client includes a GNSS receiver, an IMU sensor, a vision sensor, and an audio module. The GNSS receiver is used to collect latitude, longitude, altitude, and accuracy factors. The IMU sensor is used to collect triaxial acceleration, angular velocity, and magnetometer data. The vision sensor is used to collect environmental image streams through an RGB camera. The audio module is used to collect user voice commands through a microphone array.

[0012] Preferably, the multimodal perception module uses the extended Kalman filter (EKF) algorithm to fuse position data and motion sensing data, and dynamically adjusts the observation noise covariance matrix when the position signal accuracy factor exceeds the threshold; when the position signal accuracy is insufficient, the mobile client automatically switches to visual positioning mode and obtains the device's six-degree-of-freedom attitude as a spatial anchor point through image feature matching.

[0013] Preferably, the spatiotemporal memory bank uses a PostgreSQL database, with the pgvector plugin to process vector data and the PostGIS plugin to process geographic data, enabling hybrid queries within a single database; each memory entry stored in the spatiotemporal memory bank is a spatiotemporal-semantic memory unit, which is a JSON object containing multidimensional attributes, and is forcibly bound to an H3 hierarchical geospatial index, high-precision GPS coordinates, and timestamp.

[0014] This invention also provides a method for extracting and applying virtual character memories based on geographic location triggering, applied to the aforementioned system, comprising the following steps:

[0015] S1: Multi-sensor fusion positioning and triggering. The mobile client collects GPS location data and IMU acceleration data, uses the extended Kalman filter algorithm to fuse the data to obtain a smooth trajectory, converts the smoothed coordinates into H3 indexes, checks whether the H3 index is in the locally cached plot trigger list, and if it matches, initiates a retrieval request to the cloud.

[0016] S2: Geofencing-based hybrid retrieval. After receiving the request in the cloud, the k_ring algorithm of H3 index is used to obtain the ID of the adjacent hexagonal grid centered on the user. The memory entries in the corresponding grid are queried, the semantic similarity between the memory entry vector and the query vector is calculated, and then the candidate memories are reordered through a dual decay hybrid scoring model.

[0017] S3: Context injection and reasoning. Select the N memories with the highest comprehensive scores, convert them into natural language descriptions, inject them into system prompts (SystemPrompt), and LLM generates responses and pushes them to the client in a streaming manner.

[0018] S4: Adaptive memory update. Summarize the interactive content and calculate semantic similarity with historical memories. If the similarity exceeds the threshold, update the timestamp and importance weight of the old memory. Otherwise, create a new memory entry and bind it to the current high-precision geographic location coordinates to upload it to the spatiotemporal memory bank.

[0019] Preferably, in step S1, the state vector of the extended Kalman filter algorithm includes position and velocity, and the state transition equation and observation equation are as follows:

[0020]

[0021] Where F is the state transition matrix, and B is the control input matrix / input mapping matrix. The acceleration input is measured by the IMU. This is process noise; For GPS observation location, For observation matrix / measurement matrix, To observe noise;

[0022] The observation noise variance is dynamically adjusted based on the HDOP returned by the GPS receiver.

[0023]

[0024] When HDOP increases (signal quality deteriorates), As the frequency of memory triggering increases exponentially, the filter automatically reduces its reliance on GPS and instead places greater trust in the integral calculation results of the IMU, thereby ensuring the continuity of memory triggering.

[0025] Preferably, the scoring formula for the dual-attenuation hybrid scoring model in step S2 is:

[0026]

[0027] Each item is defined as follows:

[0028] This represents semantic similarity, with a range of values.

[0029] It is a spatial distance decay function;

[0030] It is a time decay function;

[0031] The importance of memory;

[0032] The spatial distance attenuation function is:

[0033] d is the Euclidean distance between the location where the memory occurred and the current location;

[0034] The spatial half-decay distance, d= At that time, the attenuation coefficient is 0.5;

[0035] α is the decay rate factor, which controls how quickly the weight decreases with increasing distance;

[0036] The time decay function is:

[0037]

[0038] in, For the time difference, Forgetting rate parameter This is the minimum memory residual value.

[0039] Preferably, the response generated in step S3 is streamed to the mobile client, and the AR rendering engine synchronously loads the virtual character model and blends it with the real environment for rendering.

[0040] Preferably, in step S2, the inverse ranking fusion algorithm is used to fuse the pure semantic ranking and the pure spatiotemporal ranking. The fusion formula is as follows:

[0041]

[0042] in, It is a smoothing constant. This represents the rank of entry m in the purely semantic search results. This represents the rank of entry m in the pure spatiotemporal search results.

[0043] Preferably, the system prompt in step S3 includes the role name, current time, user's current location information, and retrieved memories.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] 1. This invention constructs a spatiotemporal-semantic integrated memory storage structure. In the storage unit of the vector database, in addition to storing the embedded representation vector of the text, it forcibly binds the H3 hierarchical geospatial index, high-precision GPS coordinates and timestamps to construct multi-dimensional index key values, thereby supporting efficient hybrid queries and realizing the "landing" of memory. This solves the problems of virtual characters lacking "sense of place" and responses lacking geographical context in the existing technology.

[0046] 2. This invention proposes a dual-attenuation hybrid scoring algorithm based on physical distance and time span. It quantifies the impact of spatial distance and time passage on memory weight through a mathematical model, and simulates the forgetting curve and scene triggering mechanism of biological memory in a mathematical form, effectively solving the problems of memory fragmentation and context loss in long-term cross-regional interactions.

[0047] 3. This invention employs multi-sensor fusion positioning technology, combining extended Kalman filter algorithm to process GPS and IMU data, and can switch to visual positioning service (VPS) mode when GPS signal accuracy is insufficient, ensuring accurate alignment between memory trigger and physical landmark, and solving the problems of false triggering and virtual-real fusion occlusion caused by GPS drift in complex outdoor scenarios.

[0048] 4. This invention utilizes the hierarchical nesting characteristics of the H3 hierarchical geospatial index. When a user enters a parent large grid area, the memory vectors of all child small grids in that area are automatically preloaded from the cloud to edge nodes or local cache, enabling low-latency retrieval on mobile devices. It implements a hierarchical preloading and efficient filtering mechanism based on geographic grids, instantly reducing the retrieval space from the entire database to a local area, greatly reducing system latency and computing power costs, resolving the contradiction between retrieval efficiency and real-time performance under large-scale spatiotemporal data, and meeting the needs of large-scale commercial applications.

[0049] 5. This invention uses a "location-aware" prompt word dynamic construction technology to structurally transform the retrieved spatiotemporal memory into a natural language description (including meta-information such as relative location, distance, and time difference), and injects it into the large language model in real time as an explicit context for system prompt words. This drives the virtual character to generate dialogue content with environmental awareness. At the same time, combined with a memory adaptive update mechanism, it realizes the accumulation and dynamic evolution of memory, improving the coherence and personalization of interaction. Attached Figure Description

[0050] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0051] In the attached diagram:

[0052] Figure 1 This is a schematic diagram of the structure of a virtual character memory extraction and application system based on geographical location triggering according to the present invention;

[0053] Figure 2 This is a flowchart illustrating a method for extracting and applying virtual character memories based on geographic location triggering according to the present invention. Detailed Implementation

[0054] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0055] Example: Figure 1As shown, this invention provides a system for virtual character memory extraction and application based on geographic location triggering, integrating multimodal perception, hybrid data storage, adaptive retrieval algorithms, and a generative reasoning engine, specifically including the following:

[0056] 1. System Overall Architecture

[0057] The system consists of a three-tier architecture: a mobile client, an edge computing node (optional), and a cloud server.

[0058] 1.1 Mobile Client

[0059] Running on smartphones or AR glasses, it is responsible for collecting multimodal data and rendering the interactive interface.

[0060] Multimodal sensing module:

[0061] GNSS receiver: Collects latitude, longitude, altitude, and accuracy factor (HDOP).

[0062] IMU sensor: Collects triaxial acceleration, angular velocity and magnetometer data to assist in positioning and attitude calculation.

[0063] Visual sensor: An RGB camera captures environmental image streams for VPS positioning and AR background rendering.

[0064] Audio module: Microphone array to capture user voice commands.

[0065] LBS trigger engine: Built-in lightweight H3 index calculation library, calculates the H3Index of the current location in real time, compares it with the "hot zone list" in the local cache, and triggers memory retrieval request.

[0066] AR rendering engine: Based on UnityARFoundation or UnrealEngine, it is responsible for loading virtual character models, playing animations, rendering speech bubbles, and handling the occlusion relationship between virtual objects and the real environment.

[0067] 1.2 Cloud Server

[0068] The core brain of the system is responsible for complex logical operations and data storage.

[0069] API Gateway: Receives HTTP / WebSocket requests from clients, performs authentication, and distributes them.

[0070] Spatio-Temporal Memory Bank: The core storage unit.

[0071] Vector storage area: Stores text embedding vectors (e.g., 768-dimensional or 1536-dimensional).

[0072] Spatial Index Area: Stores H3 indexes and precise PostGIS geometric objects (Point / Polygon).

[0073] Metadata area: Stores structured data such as timestamps, sentiment tags, and associated NPCIDs.

[0074] Technology Selection: We recommend using PostgreSQL, with the pgvector plugin for vector processing and the PostGIS plugin for geographic data processing, enabling mixed queries within a single database.

[0075] Retrieval Engine: Executes a hybrid retrieval algorithm to calculate the hybrid score of memory entries.

[0076] Generative Inference Service (InferenceService): Deploys an LLM (such as GPT-4, Claude 3.5, or a finely tuned DeepSeek model), receives a SystemPrompt containing spatiotemporal memory, and generates a response that matches the character's persona.

[0077] Urban governance analysis module: Aggregates spatiotemporal interaction data from all users to generate an emotional heat map reflecting the cultural intensity of the region.

[0078] 2. Core Data Structure: Spatio-Temporal Semantic Memory Unit

[0079] To support efficient location-based retrieval, this invention defines a standardized memory storage schema. Each interaction record (MemoryEntry) is structured as a JSON object containing multi-dimensional attributes and stored in the database.

[0080] Table 1: Definition of Memory Unit Data Structure

[0081]

[0082] 3. Key processing steps, such as Figure 2 As shown

[0083] 3.1 Step S1: Multi-Sensor Fusion Positioning & Triggering

[0084] To address the issue of false triggering caused by outdoor GPS drift, this invention designs a multi-level positioning process based on Kalman filtering:

[0085] Data Acquisition: The client synchronously acquires GPS location data P at a frequency of 1Hz-10Hz. gps and IMU acceleration data A imu .

[0086] Filtering and smoothing: The above data is fused using the Extended Kalman Filter (EKF) algorithm.

[0087] State prediction: Based on the position and IMU acceleration at the previous moment, calculate the predicted position P at the current moment. pred .

[0088] Status Update: Predicted position is corrected using GPS observations. When the GPS signal accuracy factor (HDOP) is poor (e.g., >2.0), the weight of GPS observations is automatically reduced (by increasing the measurement noise covariance matrix R), relying more on IMU calculations to obtain a smooth trajectory P. est .

[0089] Grid mapping: smoothing the coordinates P est Convert to H3 index (Resolution10, side length approximately 66 meters).

[0090] Hotspot detection: Check if the current H3 index is in the locally cached "Story Trigger List". If it is, send a retrieval request to the cloud; if it is not, remain silent to save bandwidth and battery.

[0091] 3.2 Step S2: Geo-Fenced Hybrid Retrieval

[0092] When the cloud receives a request containing user coordinates (Lat, Lon) and query text Q, the following retrieval logic is executed:

[0093] Spatial pre-filtering: Using the k-ring algorithm of H3 index, quickly obtain the IDs of all adjacent hexagonal grids centered on the user with a radius of K layers (e.g., K=1, covering a radius of about 200-300 meters). Only the entries in the h3_indexINGrid_{set} memory are retrieved from the database. This step instantly narrows the search space from the entire database (millions) to a local one (thousands), greatly improving search speed.

[0094] Semantic similarity calculation: For the pre-filtered memory set, calculate its vector V. mem With query vector V query Cosine similarity:

[0095] Spatiotemporal Scoring: The candidate memories are reordered using the decay scoring model at the core of this invention (see Algorithm Section 4.2 for details).

[0096] 3.3 Step S3: Context Injection and Reasoning

[0097] Top-N truncation: Select the N items with the highest overall scores (usually 3-5 items) for memorization.

[0098] Prompt dynamic assembly: The selected memories are converted into natural language descriptions and injected into SystemPrompt. The template is as follows:

[0099] SystemInstruction: You are [Character Name]. Current time is [Time]. CurrentLocationContext: The user is currently near [POIName] (distance in meters). RetrievedMemories:

[0100] [Time: 2 days ago | Location: Bookstore | Emotion: Happy] The user once said that he likes to read historical novels.

[0101] [Time: 10 minutes ago | Location: Cafe | Emotion: Calm] The user ordered a latte.

[0102] Task: Based on the current location and the aforementioned memories, engage in a conversation with the user. If the current location is highly relevant to a particular memory (e.g., next to a bookstore), proactively bring up related topics.

[0103] Streaming generation: LLM generates responses and streams them to the client via WebSocket for playback.

[0104] 3.4 Step S4: Adaptive Update of Memory

[0105] Summary and Storage: After the conversation ends, the background uses a lightweight LLM to summarize the conversation.

[0106] Deduplication mechanism: Calculate the semantic similarity between the new summary and the 5 most recent memories. If the similarity is >0.85, instead of creating a new entry, update the last_accessed_time and importance weights of the old entry to strengthen that memory.

[0107] Location binding: Bind the current high-precision coordinates to the new memory to complete the closed loop.

[0108] 4. Core Algorithm Implementation

[0109] 4.1 Localization Smoothing Algorithm Based on Extended Kalman Filter (EKF)

[0110] To address the abrupt changes in GPS data within urban environments, the following state equations and observation equations are established:

[0111] State vector: (Position and velocity)

[0112] State transition equation:

[0113] Where F is the state transition matrix, and B is the control input matrix / input mapping matrix. The acceleration input is measured by the IMU. This is process noise.

[0114] Observation equation: in For GPS observation location, For observation matrix / measurement matrix, To observe noise.

[0115] Adaptive noise covariance: Dynamically adjusts the observation noise variance based on the HDOP (Horizontal Dilution of Precision) returned by the GPS receiver. :

[0116] When HDOP increases (signal quality deteriorates), As the frequency of memory triggering increases exponentially, the filter automatically reduces its reliance on GPS and instead places greater trust in the integral calculation results of the IMU, thereby ensuring the continuity of memory triggering.

[0117] 4.2 Dual-Decay Hybrid Scoring

[0118] This invention innovatively proposes a semantic-temporal-spatial dual-decay hybrid comprehensive scoring model, the STS model, to quantify the relevance of memories. The scoring formula is defined as follows:

[0119]

[0120] Each item is defined as follows:

[0121] Semantic similarity, range of values.

[0122] 2) Spatial distance decay function. An improved Inverse Distance Weighting (IDW) model is adopted:

[0123] d: Euclidean distance (meters) between the location where the memory occurred and the current location.

[0124] : Spatial half-life distance. This is a configurable parameter (e.g., 50 meters). When d= At that time, the attenuation coefficient is 0.5.

[0125] α: Decay rate factor (usually α=2), controls how quickly the weight decreases with increasing distance. This function ensures that the closer the distance, the higher the weight; and that the function is smooth at d=0, without any singular values.

[0126] 3) Time decay function. Ebbinghaus forgetting curve simulating human memory:

[0127] Time difference (hours or days).

[0128] Forgetting rate parameter.

[0129] Minimum memory retention value (e.g., 0.1) ensures that even memories from long ago will not be completely forgotten if they are semantically extremely similar.

[0130] 4) I m The importance of a memory is scored by the LLM when it generates a memory (0-1), representing the degree of importance of the information (e.g., the importance of a task cue > the importance of small talk).

[0131] 4.3 Reciprocal Rank Fusion (RRF) Algorithm

[0132] In some cases, a simple product formula might miss crucial information because a single term is too low (e.g., due to a large distance). To enhance robustness, this system employs the RRF algorithm to fuse "pure semantic ranking" and "pure spatiotemporal ranking":

[0133]

[0134] The system ultimately returns the optimal memory set selected based on the above hybrid scoring strategy.

[0135] The definitions of each item are as follows:

[0136] : A candidate memory / document entry (memoryentry);

[0137] : Total score after RRF fusion (higher score indicates a higher ranking);

[0138] Smoothing constant (to prevent high-ranking numbers from dominating and to avoid denominators being zero; typically 60).

[0139] :entry The ranking in the "pure semantic search / sorting" results (generally starting from 1, where 1 indicates the most relevant).

[0140] :entry The ranking in the "pure spatiotemporal search / sort" (referred to as "pure spatiotemporal ranking" in the document) results (also usually starting from 1).

[0141] Specifically, in order to help understand the actual operating logic of the system, the following scenarios are described:

[0142] Scenario: A user is strolling through the "Laomendong" historical district in Nanjing, with "Li Bai" as their virtual companion.

[0143] Phase 1: Roaming

[0144] The user is strolling through the neighborhood. The client reports GPS data at 1Hz.

[0145] The system calculates the H3 index and finds that the user has entered within 50 meters of "Junhui Bookstore" (H3Res=10).

[0146] Phase 2: Triggering

[0147] The system detected a hotspot and sent a request to the cloud.

[0148] Hybrid Search:

[0149] Space: Search for memories within a 100-meter radius of "Junhui Bookstore".

[0150] Semantics: No explicit query (passively triggered), uses "current environment description" as the query.

[0151] Result: The search revealed that the user had mentioned "wanting to buy a book of poetry" 3 days ago. Due to its extremely close proximity to the bookstore, the space score was very high, and despite the passage of 3 days, it still ranked first in the overall score.

[0152] Phase 3: Interaction

[0153] LLM received a prompt: "The user is next to a bookstore. Memory indicates he wants to buy a poetry collection."

[0154] The NPC initiates a conversation (via headphones / AR glasses): "Hey, isn't that the bookstore up ahead? That poetry collection you mentioned last time, shall we go in and look for it?"

[0155] Phase 4: Augmentation

[0156] The user raises their phone to activate AR mode.

[0157] VPS relocation: Identify the bookstore's signboard.

[0158] AR rendering: Li Bai's virtual image "sits" on the steps at the entrance of the bookstore, with a stable position that does not drift.

[0159] 2. Hardware Deployment Recommendations

[0160] Cloud computing resources:

[0161] Vector database: Alibaba Cloud RDSPostgreSQL (v16) or AWS Aurora, with pgvector extension enabled.

[0162] Inference nodes: NVIDIA A10 or A100 GPU clusters, with quantized LLM deployed to reduce latency.

[0163] Edge / Terminal Requirements:

[0164] iOS devices: iPhone 12 Pro and above (using LiDAR to assist in VPS scanning).

[0165] Android devices: Models that support the ARCore depth API.

[0166] 3. Table of Mathematical Symbols

[0167]

[0168] 4. This technical solution references the following general standards and cutting-edge research in the field as citations of existing technology:

[0169] OGCGeoAPI: The Geospatial Union's standard interface for geospatial information processing.

[0170] H3CoreLibrary: Documentation for Uber's open-source hexagonal hierarchical indexing algorithm.

[0171] RFC7946: GeoJSON format standard for data exchange.

[0172] Ebbinghaus Forgetting Curve: A classic mathematical model in psychology concerning memory forgetting.

[0173] Furthermore, alternative methods to the technical solution in this embodiment include:

[0174] 1. Alternatives to spatial indexing technology

[0175] Original solution: Use H3 (Uber hexagonal hierarchical spatial index).

[0176] Advantages: The hexagonal topology makes the neighborhood distance more uniform and the hierarchical scaling smooth, which is suitable for trajectory calculation of continuous movement.

[0177] Alternative Option A: Use Google S2 indexes.

[0178] Description: Based on Hilbert curves, the sphere is projected onto the surface of a cube and then subdivided into a quadtree mesh.

[0179] Applicability: S2 performs exceptionally well when handling rectangular region queries and offers better compatibility with the Google Maps ecosystem. It can be used as an alternative if the system relies more on rectangular region determination than circular radius search.

[0180] Alternative Option B: Use R-Tree (Region Tree) or Quad-Tree (Quad Tree).

[0181] Description: Traditional spatial database indexing structures organize data using the minimum bounding rectangle.

[0182] Applicability: Suitable for querying static data. However, in scenarios with high-frequency writes (real-time updates of user trajectories), the rebalancing overhead of the tree is significant. It can be used as an alternative for low-frequency update scenarios.

[0183] 2. Alternatives to positioning and triggering technologies

[0184] Original solution: GPS + IMU (EKF) + VPS.

[0185] Alternative Option A: Bluetooth Beacons / UWB (Ultra-Wideband).

[0186] Description: Deploy hardware base stations in specific locations (such as inside museums or underground spaces) to perform triangulation positioning via RSSI or ToF ranging.

[0187] Applicability: Suitable for indoor depth experience areas where GPS signals are completely blocked. UWB provides centimeter-level accuracy, outperforming VPS in low-light environments.

[0188] Alternative solution B: Wi-Fi fingerprint recognition.

[0189] Description: Location is determined using a database of MAC addresses and signal strength characteristics of surrounding Wi-Fi hotspots.

[0190] Applicability: No additional hardware required, low cost, but poor accuracy (5-10 meters), only suitable for coarse area triggering.

[0191] 3. Alternatives to the scoring algorithm

[0192] Original solution: Product decay model .

[0193] Alternative Solution A: Weighted Summation Model .

[0194] Description: Linear weighting after normalizing each indicator.

[0195] Features: The computation is simpler, but it is difficult to simulate a "veto" (i.e., if the distance is too far, it should not be triggered regardless of how semantically related it is). The product model has the characteristics of an AND gate, making it more suitable for strict triggering logic.

[0196] Alternative Option B: A machine learning-based ranking model.

[0197] Description: Train a lightweight GBDT or neural network model (such as LambdaMART) with input features such as semantics, distance, and time to directly predict click-through rate (CTR) or relevance score.

[0198] Features: As user feedback data accumulates, the effect will be better than manually designed formulas, but the effect is poor in the cold start phase and the system complexity is high.

[0199] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A virtual character memory retrieval and application system based on geolocation triggering, characterized in that: This includes mobile clients and cloud servers for communication connectivity; The mobile client includes a multimodal perception module, an LBS trigger engine, and an AR rendering engine. The multimodal perception module is used to collect GNSS data, IMU data, environmental image streams, and user voice commands. The LBS trigger engine has a built-in H3 index calculation library, which is used to calculate the H3 index of the current location in real time and compare it with the locally cached plot trigger list to trigger a memory retrieval request. The AR rendering engine is used to load virtual character models, play animations, render speech bubbles, and handle the occlusion relationship between virtual objects and the real environment. The cloud server includes an API gateway, a spatiotemporal memory bank, a memory retrieval engine, and a generative reasoning service. The API gateway receives client requests and performs authentication and distribution. The spatiotemporal memory bank includes a vector storage area, a spatial index area, and a metadata area. The vector storage area stores text embedding representation vectors. The spatial index area stores H3 indexes and PostGIS geometric objects. The metadata area stores structured data such as timestamps, sentiment tags, and associated NPCIDs. The memory retrieval engine executes a hybrid retrieval algorithm to calculate the comprehensive score of memory entries. The generative reasoning service deploys an LLM (Local Management Model) to receive system prompts containing spatiotemporal memories and generate responses that match the character's persona.

2. The virtual character memory extraction and application system based on geographical location triggering according to claim 1, characterized in that: The multimodal perception module of the mobile client includes a GNSS receiver, an IMU sensor, a vision sensor, and an audio module. The GNSS receiver is used to collect latitude, longitude, altitude, and accuracy factors. The IMU sensor is used to collect triaxial acceleration, angular velocity, and magnetometer data. The vision sensor is used to collect environmental image streams through an RGB camera. The audio module is used to collect user voice commands through a microphone array.

3. The virtual character memory extraction and application system based on geographical location triggering according to claim 1, characterized in that: The multimodal perception module uses an extended Kalman filter algorithm to fuse position data and motion sensing data. When the position signal accuracy factor exceeds a threshold, the observation noise covariance matrix is ​​dynamically adjusted. When the position signal accuracy is insufficient, the mobile client automatically switches to visual positioning mode and obtains the device's six-degree-of-freedom attitude as a spatial anchor point through image feature matching.

4. The virtual character memory extraction and application system based on geographical location triggering according to claim 1, characterized in that: The spatiotemporal memory bank uses a PostgreSQL database, with the pgvector plugin to process vector data and the PostGIS plugin to process geographic data, enabling hybrid queries within a single database. Each memory entry stored in the spatiotemporal memory bank is a spatiotemporal-semantic memory unit, which is a JSON object containing multidimensional attributes and is forcibly bound to an H3 hierarchical geospatial index, high-precision GPS coordinates, and a timestamp.

5. A method for extracting and applying virtual character memories based on geolocation triggering, applied to the system described in claim 1, characterized in that, Includes the following steps: S1: Multi-sensor fusion positioning and triggering. The mobile client collects GPS location data and IMU acceleration data, uses the extended Kalman filter algorithm to fuse the data to obtain a smooth trajectory, converts the smoothed coordinates into H3 indexes, checks whether the H3 index is in the locally cached plot trigger list, and if it matches, initiates a retrieval request to the cloud. S2: Geofencing-based hybrid retrieval. After receiving the request in the cloud, the k_ring algorithm of H3 index is used to obtain the ID of the adjacent hexagonal grid centered on the user. The memory entries in the corresponding grid are queried, the semantic similarity between the memory entry vector and the query vector is calculated, and then the candidate memories are reordered through a dual decay hybrid scoring model. S3: Context injection and reasoning. Select the N memories with the highest comprehensive scores, convert them into natural language descriptions and inject them into system prompts. LLM generates responses and pushes them to the client in a streaming manner. S4: Adaptive memory update. Summarize the interactive content and calculate semantic similarity with historical memories. If the similarity exceeds the threshold, update the timestamp and importance weight of the old memory. Otherwise, create a new memory entry and bind it to the current high-precision geographic location coordinates to upload it to the spatiotemporal memory bank.

6. The method for extracting and applying virtual character memories based on geographical location triggering according to claim 5, characterized in that: In step S1, the state vector of the extended Kalman filter algorithm includes position and velocity. The state transition equation and observation equation are as follows: ； Where F is the state transition matrix, and B is the control input matrix / input mapping matrix. The acceleration input is measured by the IMU. This is process noise; For GPS observation location, For observation matrix / measurement matrix, To observe noise; The observation noise variance is dynamically adjusted based on the HDOP returned by the GPS receiver. ； When HDOP increases As the number of GPS devices increases exponentially, the filter automatically reduces its trust in GPS.

7. The method for extracting and applying virtual character memories based on geographical location triggering according to claim 5, characterized in that: The scoring formula for the dual-attenuation hybrid scoring model in step S2 is: ； Each item is defined as follows: This represents semantic similarity, with a range of values. It is a spatial distance decay function; It is a time decay function; The importance of memory; The spatial distance attenuation function is: ； d is the Euclidean distance between the location where the memory occurred and the current location; The spatial half-life distance; α is the decay rate factor; The time decay function is: ； in, For the time difference, Forgetting rate parameter This is the minimum memory residual value.

8. The method for extracting and applying virtual character memories based on geographical location triggering according to claim 5, characterized in that: The response generated in step S3 is streamed to the mobile client, and the AR rendering engine simultaneously loads the virtual character model and blends it with the real environment for rendering.

9. A method for extracting and applying virtual character memories based on geographical location triggering according to claim 5, characterized in that: In step S2, the inverse ranking fusion algorithm is used to fuse the pure semantic ranking and the pure spatiotemporal ranking. The fusion formula is as follows: ； in, It is a smoothing constant. This represents the rank of entry m in the purely semantic search results. This represents the rank of entry m in the pure spatiotemporal search results.

10. A method for extracting and applying virtual character memories based on geographical location triggering according to claim 5, characterized in that: In step S3, the system prompt includes the character name, current time, user's current location information, and retrieved memories.

Images