A game AI middleware system and hardware architecture supporting heterogeneous multi-GPU collaboration and companion learning

Abstract

The application discloses a game AI middleware system and hardware architecture supporting heterogeneous multi-GPU cooperation and accompanying learning. The system contains a physically isolated main rendering GPU and at least one coprocessor AI computing power GPU, and the two are directly connected through a PCIe bus. The application provides a unified AI middleware service interface, and a separate circular buffer area is opened in the main rendering GPU memory, so that the coprocessor AI computing power GPU can directly obtain the game real-time context in a zero-copy mode. At the same time, the coprocessor AI computing power GPU is configured with a local lightweight vector database, and the long-term memory of the player's behavior habits is realized through retrieval augmented generation (RAG) technology; and during the low-load period of system rendering, the coprocessor AI computing power GPU is scheduled to fine-tune and train the local large language model in a low-rank adaptive (LoRA) mode. The application solves the problem of power occupation of complex NPC logic on graphics rendering, and realizes the physical-level personalized behavior evolution of local AI characters.

Description

Technical Field

[0001] This invention relates to the fields of computer architecture, artificial intelligence, and video game development technology. Specifically, it relates to a local game AI scheduling and accompanying evolution system based on multi-GPU physical isolation and zero-copy communication. Background Technology

[0002] With the increasing application of local artificial intelligence (especially large language models and complex multi-agent systems) in video games, the interaction logic of game NPCs is shifting from pre-set scripts to real-time generation. However, both high-quality AI inference and high-fidelity 3D game rendering require massive amounts of GPU computing power. Existing single-GPU architectures struggle to meet both demands simultaneously, leading to resource contention and system power consumption throttling under high-load game scenarios. Furthermore, existing cloud-based AI solutions suffer from extremely high network latency and the risk of player privacy breaches. In addition, traditional local game AI lacks long-term memory and continuous learning capabilities, making it unable to evolve physically based on individual player gaming habits. Summary of the Invention

[0003] To address the aforementioned technical challenges, this invention proposes a game AI middleware system and hardware architecture that supports heterogeneous multi-GPU collaboration and adjoint learning. The core innovation of this invention lies in breaking down the system barriers between multiple independent GPUs on the motherboard. The system includes a main rendering GPU and at least one coprocessor AI computing GPU, connected directly via a PCIe bus in a point-to-point (P2P) manner. This invention provides a unified AI middleware service interface (API), creating an isolated circular buffer within the main rendering GPU's memory, enabling the coprocessor AI computing GPU to directly acquire real-time game environment snapshots in a zero-copy manner, completely eliminating rendering pipeline blockage caused by cross-card data synchronization. Simultaneously, the system is configured with a local lightweight vector database within the dedicated storage of the coprocessor AI computing GPU. During game runtime, the system uses Retrieval Enhancement Generation (RAG) technology to achieve long-term memory of player behavior habits and tactical intent prediction; during low-load or idle rendering, the scheduling system triggers the coprocessor AI computing GPU to perform low-rank adaptive learning (LoRA) or parameter efficient fine-tuning (PEFT) on the local large language model, generating personalized model weights specific to the player. This invention enables physical-level personalized evolution of local AI characters and significantly reduces the technical threshold for game developers to access highly intelligent multi-agent systems. Attached Figure Description

[0004] Figure 1 This is a diagram of the overall heterogeneous multi-GPU hardware topology and bus communication architecture of the system of this invention. Figure 2 A flowchart for unifying the data interception and serialization workflow of the AI ​​middleware service layer. Figure 3 This is the state machine transition diagram for long-term memory and idle LoRA fine-tuning based on a local vector library. Detailed Implementation

[0005] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0006] I. Implementation Process Reference for Heterogeneous Hardware Zero-Copy Communication Figure 1 and Figure 2 The system of this invention performs the following steps during the initialization phase to establish a low-level hardware communication channel: Step S101: The operating system's low-level driver statically allocates a contiguous memory block of a preset capacity in the physical video memory of the main rendering GPU, defining it as a "game environment shared circular buffer". Step S102: Utilizing the PCIe Root Complex and Base Address Register (BAR) extension mechanism, the page table of the Memory Management Unit (MMU) of the coprocessor AI computing power GPU is rewritten, mapping the physical address of the main rendering GPU to the virtual address space of the coprocessor AI computing power GPU. Step S103: During game engine operation, de-identified data such as the 3D coordinates and NPC viewpoint of the current frame are written to this buffer; the local AI model in the coprocessor AI computing power GPU retrieves this data block as inference context in a lock-free read-only mode through the Direct Memory Access (DMA) controller. This process does not trigger CPU hardware interrupts, achieving microsecond-level zero-copy transmission of environment snapshots.

[0007] II. Reference for Data Parsing and Distribution Process in the Middleware API Layer Figure 2 The unified AI service interface middleware acts as a bridge between game applications and underlying heterogeneous computing power. It integrates the agent-first development philosophy into the underlying layer and executes the following process: Step S201: Receive event requests from the game engine via standard APIs. Step S202: The middleware uses a data serialization subunit to extract a snapshot of the instantaneous environment in the buffer and converts it into a natural language description or multidimensional tensor input. Step S203: The middleware initiates a call instruction to the co-processing AI computing power GPU, completes inference, and returns a decision instruction. Step S204: The middleware reverse-maps the above instructions into behavior tree nodes or action macro code recognizable by the game engine and pushes them back to the rendering pipeline for execution.

[0008] III. Reference for the Evolution Process of Accompanying Learning and Local Fine-tuning Figure 3This invention endows game AI characters with the ability to remember and evolve in a personalized way. The specific state machine flow is as follows: Step S301 (Data Collection State): During game operation, the middleware continuously analyzes the player's operation frequency and tactical preferences. After semanticizing the features (Embedding), they are silently written to the local lightweight vector database. Step S302 (Intent Prediction State): Before inference, the coprocessor AI computing power GPU prioritizes Retrieval Enhancement Generation (RAG) to extract the player's historical behavioral preferences from the vector library as mandatory constraints, enabling the AI ​​to actively respond to the player's tactical intentions. Step S303 (Idle Fine-tuning State): The system-level load monitoring module periodically polls. When it detects that the main rendering GPU load is lower than a preset threshold or the system is in standby mode, it wakes up the high-power mode of the coprocessor AI computing power GPU. Step S304 (Evolution Output State): The high-frequency behavioral feature dataset of players accumulated in the vector database is extracted, and low-rank adaptive (LoRA) fine-tuning is performed on the resident model. The personalized weight tensor file generated by the fine-tuning is automatically mounted by the middleware when the game starts again, causing the AI ​​character to produce specific physical behavior evolution.

[0009] IV. Seamless Switching Mechanism for All-Time Computing Power Roles To avoid idle computing power, this system supports all-time mode switching: When the game is running in the foreground, it enters the companion mode, and the co-processing AI computing power GPU calls the game rule library; when there are no high-load graphics programs in the foreground, the system cuts off the cross-card shared video memory mapping to ensure security, and releases 100% of the co-processing AI computing power GPU resources to the background productivity assistant of the operating system or the local development intelligent agent, so as to realize the all-weather overall scheduling of computing power resources.

Claims

1. A game AI middleware system supporting heterogeneous multi-GPU collaboration and companion learning, characterized in that, The system, applied in a computer device comprising a main rendering GPU and at least one coprocessor AI computing GPU, includes: a physical communication architecture: the main rendering GPU and the coprocessor AI computing GPU are interconnected via a peripheral component interconnect (PCIe) bus supporting a peer-to-peer protocol; the system allocates an isolated circular buffer within the main rendering GPU's video memory, and through a base address register (BAR) mapping mechanism, enables the coprocessor AI computing GPU to have direct physical read-only access to the buffer, achieving zero-copy cross-card transmission of real-time game context data; and a unified AI service interface middleware: operating independently of the game graphics rendering pipeline, used to receive game environment state data sent by the game engine and convert it into inference prompts that the artificial intelligence model in the coprocessor AI computing GPU can parse; simultaneously, it receives inference decisions output by the coprocessor AI computing GPU and maps them inversely to action instructions that the game engine can execute.

2. The system according to claim 1, characterized in that, The unified AI service interface middleware further includes an accompanying learning and memory module, and a local lightweight vector database is deployed in the dedicated video memory of the coprocessor AI computing power GPU. During game operation, the middleware performs semantic tensor quantization (Embedding) on ​​the player's operation sequence and preference features and silently writes them into the local lightweight vector database. Before inference execution, the coprocessor AI computing power GPU uses retrieval augmented generation (RAG) technology to extract the player's historical habits that match the current scene from the local lightweight vector database, as a mandatory pre-context for inference prompts, thereby realizing the memory and prediction of the player's behavioral intentions.

3. The system according to claim 2, characterized in that, The system also includes a physical-level idle evolution fine-tuning module: This module monitors the rendering load status of the main rendering GPU in real time; when the rendering load is lower than a preset threshold or the system is in standby mode, it wakes up the coprocessing AI computing power GPU to enter a high-power computing mode; The coprocessing AI computing power GPU reads the specific player feature dataset accumulated in the local lightweight vector database, performs low-rank adaptive (LoRA) or parameter efficient fine-tuning (PEFT) on the resident local large language model, generates a personalized model weight file, and the middleware automatically mounts the exclusive weight file when the game resumes running.

4. The system according to claim 1, characterized in that, The unified AI service interface middleware provides a seamless switching mechanism for all-time computing power roles: When a specific game process is detected running in the foreground, the system enters the companion mode, and the coprocessing AI computing power GPU runs the game rule library and tactical companion logic; When no high-load graphics program is detected running in the foreground, the system cuts off the cross-card shared video memory mapping mechanism to ensure data isolation and security, and releases the computing resources of the coprocessing AI computing power GPU to the background general artificial intelligence assistant of the operating system or the code development intelligent agent module.

Images