Intelligent gamification knowledge base architecture based on multi-modal semantic understanding and dynamic value feedback

Abstract

The invention discloses an intelligent gamification knowledge base architecture based on multi-modal semantic understanding and dynamic value feedback. The intelligent gamification knowledge base architecture comprises an AI enhancement processing layer, an intelligent data storage layer, an AI model and service layer, a data application layer and an echo feedback loop. The AI enhancement processing layer is used for realizing semantic level analysis and priority processing of the unstructured data through dynamic value evaluation and a Transform-OCR (Optical Character Recognition) model; the intelligent data storage layer adopts a heterogeneous database cluster to store diversified structured data, and an NPC memory verification module is arranged in the intelligent data storage layer; the AI model and the service layer realize query routing and fusion sorting through a full-layer optimization engine, and a project master AI dynamically distributes knowledge base subsets; and the data application layer generates echo signals to drive dynamic optimization of the whole system. The problems that an existing game knowledge base is static, NPC dialogue is prone to generating illusion, and response efficiency is low are solved, the technical effects of knowledge dynamic evolution, accurate and credible interaction and system lightweight deployment are achieved, and game interaction immersion and intelligent experience are greatly improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and game technology, specifically to an intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback. Background Technology

[0002] With the explosion of generative artificial intelligence (AIGC) and large language models (LLM), the digital entertainment industry is transforming from "script-driven" to "data-driven." Especially in fields such as open-world games, digital museums, and immersive script-based murder mystery games, developers hope to use massive amounts of historical archives and literary works to build intelligent NPCs with deep memory and personalized interactive capabilities.

[0003] Existing technologies still have significant shortcomings: the conversion of traditional unstructured data (historical documents, PDF scans, etc.) into NPC knowledge relies on manual input or simple OCR, which leads to fragmented knowledge due to a lack of contextual understanding, easily causing logical gaps or illusions in NPC interactions; multi-source heterogeneous data (vectors, graphs, time series, etc.) are stored in a scattered manner, forming data silos that are difficult to collaboratively support complex scene interactions; traditional databases are static storage and lack dynamic data value assessment mechanisms based on user feedback; in game AI architectures, the model layer and application layer are tightly coupled, resulting in poor scalability and an inability to achieve flexible iteration of "Model as a Service (MaaS)". The two existing mainstream solutions (RAG-based general knowledge bases and traditional game database architectures) also have inherent defects such as context loss, lack of entity relationships, data rigidity, and lack of multimodality, making it difficult to meet the needs of intelligent interaction. Summary of the Invention

[0004] The purpose of this invention is to overcome or at least partially solve the above problems by proposing an intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: an intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback, comprising:

[0006] The AI-enhanced processing layer receives unstructured raw data, predicts the potential value of documents through a dynamic data value assessment module, completes contextual optical character understanding by combining the Transformer-OCR model, and then outputs multi-type structured data through multimodal structured processing.

[0007] The intelligent data storage layer adopts a heterogeneous database cluster composed of vector database, graph database, relational / document database, time-series database and object storage to uniformly and relatedly store the structured data, and the relational / document database has a built-in NPC memory and verification module;

[0008] The AI ​​model and service layer includes a full-layer optimization engine, a project master AI, and a model service gateway. The full-layer optimization engine receives echo signals from the data application layer and dynamically adjusts data weights and retrieval strategies. The project master AI enables on-demand knowledge distribution.

[0009] The data application layer is used to realize end-user interaction and team data asset management, generate echo signals containing user dialogue and LBS data and feed them back to the AI ​​model and service layer; through the echo signals, the layers are coordinated to achieve dynamic evolution of system data and models.

[0010] In a preferred embodiment, the dynamic data value assessment module uses the following algorithm to calculate the document value:

[0011]

[0012] in, Rate the overall value of the document;

[0013] , , These are weighting coefficients;

[0014] To ensure the relevance of the documents to the system requirements;

[0015] For document scarcity;

[0016] The echo signal reflects the current hot-button demands;

[0017] The dynamic data value assessment module constructs a priority queue based on the comprehensive value score of the documents. High-scoring documents enter the GPU-accelerated processing queue, while low-scoring documents enter cold storage to wait for processing during idle time.

[0018] In a preferred embodiment, the processing steps of the Transformer-OCR model include: first, performing layout analysis on the document image, segmenting it into Header, Footer, Body, Sidebar, Table, and Caption regions, and constructing a document DOM tree; then, achieving multimodal alignment of text with surrounding images; and finally, outputting a compressed token stream rich in semantic information.

[0019] In a preferred embodiment, the multimodal structuring process specifically involves performing entity extraction, semantic vectorization, fact tuple extraction, and time-series event parsing on the compressed token stream, and correspondingly distributing the data to a graph database, a vector database, a relational / document database, and a time-series database.

[0020] In a preferred embodiment, the NPC memory and verification module structures the NPC memory into tuples:

[0021] {EventID,Timestamp,Location,Participants,Emotion}

[0022] When generating NPC dialogue, the system automatically verifies the consistency between key facts in the dialogue and the tuples. If there is a discrepancy, the full-layer optimization engine is triggered to intercept and correct it.

[0023] In a preferred embodiment, the project master AI dynamically mounts or unmounts corresponding subsets of the knowledge base based on the user's real-time LBS data and task progress, thereby achieving lightweight deployment and real-time response of a large-scale knowledge base.

[0024] In a preferred embodiment, the echo signal includes positive echoes and negative echoes: positive echoes are positive interactive behavior data of users appreciating an NPC's answer or staying for a long time, triggering the weight enhancement of the corresponding knowledge node; negative echoes are negative interactive behavior data of users leaving quickly, triggering the reduction of the data weight of the relevant plot line.

[0025] In a preferred embodiment, the model service gateway provides a unified API interface to support hot switching between different LLM models.

[0026] In a preferred embodiment, the full-layer optimization engine also has query routing and fusion sorting functions, selects the target database according to the query intent of the data application layer, and feeds back the weighted fusion of the search results of multiple databases.

[0027] In a preferred embodiment, the data application layer is further used to collect high-quality dialogue pairs and corrected OCR data, and use them as training data to fine-tune the Transformer-OCR model and LLM model, so as to achieve continuous iterative optimization of the model.

[0028] Compared with existing technologies, the intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback provided by this invention has the following beneficial effects:

[0029] Through the dynamic data value assessment module and echo feedback mechanism, the system can capture players' hot needs in real time, prioritize GPU acceleration processing for high-value documents, and cold store low-value documents for processing during idle time. At the same time, it can dynamically adjust knowledge weights and supplement scarce content based on feedback signals, breaking the limitations of the static nature of traditional knowledge bases.

[0030] By using the NPC memory and verification module built into the relational / document database, the character memory is structured into the tuple format defined in this invention. When generating dialogue, the consistency of key facts is automatically verified. When there is a conflict, the full-layer optimization engine intercepts and corrects it, providing "unique source of facts" support for NPC dialogue and ensuring the accuracy of interaction.

[0031] The Transformer-OCR model is used to achieve multimodal semantic-level parsing and output a compressed token stream containing visual semantics. Combined with structured processing such as entity extraction and semantic vectorization, different types of data are distributed to heterogeneous clusters such as vector databases and graph databases, which not only meets the diverse data storage needs, but also optimizes retrieval and calling efficiency.

[0032] The project's main AI dynamically mounts subsets of the knowledge base based on player LBS data and task progress, avoiding resource consumption caused by loading the full data. Combined with the LLM hot-switching function of the model service gateway, it ensures real-time interactive response and system stability, adapting to the deployment needs of game servers of different sizes.

[0033] By collecting positive / negative echo signals and high-quality interactive data through the data application layer, the system drives the full-layer optimization engine to adjust retrieval strategies and knowledge weights. On the other hand, it provides fine-tuning data for the Transformer-OCR model and LLM, enabling continuous iterative optimization of the model and continuously improving the system's intelligence level. Attached Figure Description

[0034] Figure 1 This is a diagram showing the various levels of this invention, the core functional modules within each level, and the information interaction relationships between the modules. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the accompanying drawings. This embodiment takes an intelligent game NPC interaction scenario as an example to illustrate the technical implementation details of each level, but it is not intended to limit the present invention. All equivalent substitutions or improvements based on the technical solution of the present invention fall within the protection scope of the present invention.

[0036] Combination Figure 1 The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback of this invention is divided into four main layers from top to bottom, as well as a data access and feedback loop running through the right side. The specific implementation details of each layer are as follows:

[0037] I. Implementation of the AI ​​Enhancement Processing Layer

[0038] The AI-enhanced processing layer, acting as the system's "sensory organs," is the first barrier for data entering the system. Its core innovation lies in surpassing traditional OCR character recognition to achieve "semantic-level" document parsing, transforming chaotic raw data into ordered, structured information. On the right, in the "Intelligent Data Access Layer," the system receives unstructured raw data such as "historical document PDFs / copies." Before the data is actually processed, it first passes through the "Dynamic Data Value Assessment" module. The technical implementation process is as follows: the system first performs metadata sniffing, extracting the document's metadata (creation time, author, source); then, based on the current system's "data gap index" (calculated from lower-layer feedback), it scores the document using a value scoring algorithm. The algorithm formula is:

[0039]

[0040] in, A comprehensive value score is given to a document, indicating its overall priority under the "current stage / current task";

[0041] , , These are weighting coefficients used to control the relative contributions of the three types of factors to the final score;

[0042] The relevance of a document to system requirements is used to measure the degree to which the document matches the current system requirements / hot topics / plot tasks / search intent.

[0043] To assess document scarcity, we measure "the uniqueness / irreplaceability / ability to fill knowledge gaps within the library";

[0044] The echo signal reflects current hot demands, such as players currently exploring "Ming Dynasty firearms," ​​which leads to a surge in related document ratings;

[0045] Finally, a priority queue is constructed based on the overall value score of the documents. High-scoring documents enter the GPU-accelerated processing queue, while low-scoring documents enter cold storage to wait for processing during idle time.

[0046] After value assessment, the document enters the Transformer-OCR contextual optical character understanding module. Unlike traditional OCR engines based on CNN+RNN, this module uses a Transformer-based LargeVisionModel. Its technical principle is as follows: The model first performs layout analysis on the document image, segmenting it into regions such as Header, Footer, Body, Sidebar, Table, and Caption, constructing a "document DOM tree." While recognizing text, it achieves multimodal alignment between the text and surrounding images; for example, it recognizes the text next to a map as "battle route description," rather than just isolated characters. Finally, it completes compression and tokenization processing, outputting not plain text but "compressed tokens" that retain semantically rich information such as font size (indicating importance) and color (indicating emotion or category).

[0047] Afterwards, the token stream after OCR enters the "Multimodal Understanding" module for split processing: entity extraction identifies names, places, times, and events in the text, generating structured data streams to the "Graph Database"; semantic vectorization converts paragraph text into high-dimensional vectors (Embedding), generating structured data streams to the "Vector Database"; fact tuples extract atomic facts of "who-when-what" to generate structured data streams to the "Relational / Document Database"; and time-series event parsing identifies behavior descriptions with timestamps, generating structured data streams to the "Time-Series Database".

[0048] II. Implementation of the Intelligent Data Storage Layer

[0049] The intelligent data storage layer, acting as the system's "memory cortex," abandons the limitations of a single database and adopts a heterogeneous database cluster to store information of different dimensions. The specific heterogeneous design is shown in the table below.

[0050]

[0051] The above-described technical selections are merely illustrative and can be replaced by other database products with equivalent functionality, and do not constitute a limitation on the present invention. This heterogeneous database cluster is interconnected through a unified ID system, with its core innovation being the NPC memory and verification mechanism: an NPC memory and verification module is specifically designed within the "relational / document database," where each NPC's memory is no longer chaotic text but is structured into tuples.

[0052] {EventID,Timestamp,Location,Participants,Emotion}

[0053] EventID is a unique identifier for an event (used for indexing, deduplication, and associating evidence).

[0054] Timestamp is the time or time range in which the event occurred (used for chronological reasoning and plot sequencing).

[0055] Location is the location of the event (which can be a place name, coordinates, or geographic entity ID).

[0056] Participants is a collection of participants (a list of entities or IDs such as people, NPCs, or organizations).

[0057] Emotion is an emotion / attitude label (used to drive changes in NPC tone, stance, and favorability).

[0058] When the generative AI generates a line of dialogue, the system will automatically query the relational database to verify whether the key facts in the dialogue conflict with the configuration table of the NPC. If there is a conflict, the full-layer optimization engine will intercept and correct it.

[0059] III. Implementation of AI Models and Service Layers

[0060] The AI ​​model and service layer, acting as the system's "brain," is responsible for scheduling, inference, and optimization. It comprises three core modules: a full-layer optimization engine, a project-controlled AI, and a model service gateway. The overall data AI full-layer optimization engine is a distributed scheduling system. It doesn't directly store data but maintains an "index" and "heatmap" of the data. Its working mechanism is as follows: when the application layer initiates a request, the engine determines whether to query the vector library (casual chat), the graph library (inference), or the time-series library (behavioral analysis) based on the intent, thus implementing query routing. Then, the search results from different databases are weighted and fused, i.e., re-ranking. Simultaneously, the engine sends "optimization instructions" to the "project-controlled AI." For example, if it discovers a large number of users inquiring about a lesser-known poet, the engine will generate an instruction: "Increase the retrieval weight of vectors related to this poet and trigger a background crawler to supplement relevant information."

[0061] The project's master AI is a dedicated AI gateway designed for specific game projects or application scenarios. Its knowledge distribution strategy implements "knowledge streaming," meaning it doesn't feed the entire database to NPCs. Instead, it dynamically loads relevant knowledge base subsets (KnowledgeShards) based on the player's current LBS location and task progress through the "NPC configuration / knowledge retrieval" interface. The model service gateway provides a unified API interface, shielding the differences in underlying models and supporting hot switching between different LLMs (such as switching from GPT-4 to the self-developed Transformer-Chat), ensuring the stability of upper-layer applications.

[0062] IV. Implementation of Data Application Layer and Echo Loop

[0063] The data application layer, acting as the "limb" of the system, enables interaction with end users and the development team, and is the source of the "echoes." In the game's NPC interaction and dialogue recording phase, players interact with NPCs on the front end. Each dialogue generates a record containing {Query, Response, Latency, Satisfaction_Score}, where Query is the user's question / input, Response is the system / NPC's response, Latency is the time taken from receiving the Query to generating the Response (used for performance and SLA optimization), and Satisfaction_Score is the user's satisfaction rating (which can come from likes, ratings, dwell time mapping, etc., used to form "echo" feedback signals). These "dialogue records" are sent back in real-time to the "user dialogue and LBS data" module, which then feeds back to the full-layer optimization engine, forming an echo path: if a player appreciates an NPC's answer (likes or prolonged dwell time), it's a positive echo, and the engine strengthens the knowledge nodes referenced by that answer; if a player quickly leaves (shows disinterest) through LBS data, it's a negative echo, and the engine reduces the data weight of the relevant storyline.

[0064] In the team's data asset management and analysis phase, operations personnel can view the "Data Value Score" report through the corresponding module. The system will automatically suggest relevant data supplementation, such as "A surge in access to data related to 'Song Dynasty Tea Culture' has been detected in the last 24 hours, but the current knowledge base coverage is only 30%. It is recommended to supplement relevant PDF documents." At the same time, the data application layer is also used to collect "high-quality dialogue pairs" and "corrected OCR data," and use them as training data to fine-tune the underlying Transformer-OCR and LLM, enabling the model to self-evolve.

[0065] Through the coordinated operation of the above four-layer structure and the feedback drive of the echo signal, each layer achieves dynamic adjustment, ultimately realizing the dynamic evolution of system data and model, and fully reproducing the technical solution of the present invention.

[0066] The above description of the embodiments is provided to facilitate understanding and use of the present invention by those skilled in the art. It is obvious to those skilled in the art that various modifications can be made to the embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments. Improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the protection scope of the present invention.

Claims

1. An intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback, characterized in that, Including: The AI-enhanced processing layer receives unstructured raw data, predicts the potential value of documents through a dynamic data value assessment module, completes contextual optical character understanding by combining the Transformer-OCR model, and then outputs multi-type structured data through multimodal structured processing. The intelligent data storage layer adopts a heterogeneous database cluster composed of vector database, graph database, relational / document database, time-series database and object storage to uniformly and relatedly store the structured data, and the relational / document database has a built-in NPC memory and verification module; The AI ​​model and service layer includes a full-layer optimization engine, a project master AI, and a model service gateway. The full-layer optimization engine receives echo signals from the data application layer and dynamically adjusts data weights and retrieval strategies. The project master AI enables on-demand knowledge distribution. The data application layer is used to realize end-user interaction and team data asset management, generate echo signals containing user dialogue and LBS data and feed them back to the AI ​​model and service layer; through the echo signals, the layers are coordinated to achieve dynamic evolution of system data and models.

2. The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback as described in claim 1, characterized in that: The dynamic data value assessment module uses the following algorithm to calculate document value: ； in, Rate the overall value of the document; , , These are weighting coefficients; To ensure the relevance of the documents to the system requirements; For document scarcity; The echo signal reflects the current hot-button demands; The dynamic data value assessment module constructs a priority queue based on the comprehensive value score of the documents. High-scoring documents enter the GPU-accelerated processing queue, while low-scoring documents enter cold storage to wait for processing during idle time.

3. The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback as described in claim 1, characterized in that: The processing steps of the Transformer-OCR model include: first, performing layout analysis on the document image, segmenting it into Header, Footer, Body, Sidebar, Table, and Caption regions, and constructing the document DOM tree; then, achieving multimodal alignment of text with surrounding images; and finally, outputting a compressed token stream rich in semantic information.

4. The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback as described in claim 1, characterized in that: The multimodal structuring process specifically involves performing entity extraction, semantic vectorization, fact tuple extraction, and time-series event parsing on the compressed token stream, and correspondingly distributing the data to a graph database, vector database, relational / document database, and time-series database.

5. The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback according to claim 1, characterized in that: The NPC memory and verification module structures NPC memories into tuples: {EventID,Timestamp,Location,Participants,Emotion}; When generating NPC dialogue, the system automatically verifies the consistency between key facts in the dialogue and the tuples. If there is a discrepancy, the full-layer optimization engine is triggered to intercept and correct it.

6. The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback according to claim 1, characterized in that: The project's master AI dynamically mounts or unmounts corresponding subsets of the knowledge base based on real-time user LBS data and task progress, enabling lightweight deployment and real-time response of large-scale knowledge bases.

7. The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback according to claim 1, characterized in that: The echo signals include positive echoes and negative echoes: positive echoes are positive interactive behavior data of users appreciating an NPC's answer or staying for a long time, which triggers the weight enhancement of the corresponding knowledge node; negative echoes are negative interactive behavior data of users leaving quickly, which triggers the reduction of the data weight of the relevant plot line.

8. The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback according to claim 1, characterized in that: The model service gateway provides a unified API interface, supporting hot switching between different LLM models.

9. The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback according to claim 1, characterized in that: The full-layer optimization engine also has query routing and fusion sorting functions, selects the target database according to the query intent of the data application layer, and performs weighted fusion of the search results of multiple databases before feeding back the results.

10. The intelligent gamified knowledge base architecture based on multimodal semantic understanding and dynamic value feedback according to claim 1, characterized in that: The data application layer is also used to collect high-quality dialogue pairs and corrected OCR data, which are then used as training data to fine-tune the Transformer-OCR model and the LLM model, thereby achieving continuous iterative optimization of the model.

Images