A dual-channel pedigree isolation method and system for player mirror model training data

Abstract

The application relates to the software field and discloses a double-channel pedigree isolation method and system for player mirror model training data, which comprises the following steps: before updating a target model, performing bitwise and preliminary screening on the pedigree signature of a candidate training sample and the pedigree signature of the target model, generating a pedigree graph after a hit, performing reverse traversal, and accurately determining whether the candidate training sample is directly or indirectly derived from the target model or a historical parent model version thereof; according to the detection result, routing the sample to a basic behavior model training channel, a relationship branch training channel or an isolation set, so that mirror model generation data is not mixed into a source player real behavior training channel and different relationship branch data is mutually isolated; after updating, performing pedigree closure recording on a new model version, so that recursive dependence detection can be continuously performed across multiple model versions. The application can identify and isolate cross-version recursive feedback data, and improve the data reliability and behavior stability in the continuous updating process of a player mirror model.

Description

Technical Field

[0001] This invention relates to the field of software, and more particularly to a dual-channel genealogical isolation method and system for player mirror model training data. Background Technology

[0002] With the application of artificial intelligence technology in multiplayer online games, game systems can collect behavioral data on players during gameplay, such as movement, attacks, skill releases, target selection, and team cooperation. This data can then be used to train player mirror models that simulate the operational habits and decision-making characteristics of specific players. When players are offline, these player mirror models can control virtual characters to participate in combat, cooperation, or other game activities.

[0003] In the prior art, Chinese patent CN112169339B, "Customized Model for Mimicking Player Gameplay in Video Games," discloses a technical solution for customizing a general behavior model based on a specific player's game data and using the customized model to simulate that player's game behavior. This solution discloses: obtaining a general player behavior model; designating at least one subset of layers or hierarchies in the general model as frozen layers; generating a customized layer based on a first player's game data; storing the trained customized model; the customized model receiving game state data and outputting automated actions; and providing the customized model to a second player to mimic the first player's gameplay.

[0004] US Patent 10713543B1 discloses a technical solution for training a machine learning system based on automatically generated realistic game information. Specifically, automatically generated or simulated game data is used to train the machine learning model, indicating that model-generated data is generally allowed to enter the training process in the prior art. However, the aforementioned prior art mainly focuses on player behavior imitation, personalized model generation, model feedback updates, version management of the model development process, or the utilization of automatically generated data, and still has the following problems: First, after the player mirror model is put into actual operation, the actions generated by the mirror model will also be recorded as game behavior data. Existing systems usually cannot continuously distinguish whether the behavior data is generated directly by real players or by a certain version of the player mirror model. When this data is used again to update the player mirror model, a recursive feedback process may be formed: "model generates behavior - behavior enters training data - training generates a new model - new model generates behavior again".

[0005] Second, existing source tags typically only record the direct source of the data. After data aggregation, asynchronous battle result write-back, sample replication, or multiple rounds of model updates, direct source tags cannot reflect whether the data is indirectly inherited from data generated by previous version models, making it difficult to identify indirect recursive feedback formed across multiple model versions.

[0006] Third, existing model version management schemes are mainly used to record the correspondence between training tasks, datasets and model files. They do not establish a computable genealogical relationship between a single behavioral event and the model version that generated the behavior, nor can they determine whether candidate training data is a direct or indirect derivative of the model to be updated based on the genealogical relationship before the model is updated.

[0007] Fourth, mirror models of the same source player can be invoked by different associated players. Data generated by interactions between different associated players and their mirror models may be mixed and written into the same training dataset, causing the behavior of one associated player to affect the state of the mirror model of another associated player, resulting in data crosstalk between different relationship branches.

[0008] The aforementioned issues may cause the player mirror model to gradually deviate from the behavioral characteristics of real players after multiple updates. This leads to the repeated learning and amplification of random errors in the model's generated behavior, and increases the computational resources and processing time required for training data cleaning, model retraining, and anomaly tracing. Therefore, a technical solution is needed that can record the multi-generational generation relationship between behavioral data and model versions, identify recursive feedback paths, and isolate different update channels. Summary of the Invention

[0009] This invention primarily addresses the problem that during the continuous operation and iterative updates of a player mirror model, the behavioral data generated by the model may be reused as real player behavioral data for training, thereby creating recursive feedback and causing a shift in the model's behavior.

[0010] Specifically, it includes: This addresses the issue that existing source tags can only represent the direct entity that generates behavioral data, and cannot identify indirect derived relationships formed after data aggregation, replication, asynchronous result write-back, and multiple rounds of model updates.

[0011] This addresses the problem that existing systems fail to establish a computable genealogy between individual behavioral events, training samples, and model versions, making it impossible to identify direct or indirect recursive feedback paths before model updates.

[0012] This addresses the issue of mirror model-generated data being repeatedly fed into the source player model training set, leading to the cyclical learning and amplification of model errors and the gradual distortion of player behavior characteristics.

[0013] This addresses the issue of cross-relationship interference in model parameters or states when a mirror model of a player from the same source is invoked by multiple related players, resulting in data from different relationship branches being mixed together.

[0014] This addresses the problem of difficulty in locating the source of data contamination after it occurs, which necessitates large-scale cleaning of training data or retraining of the model for recovery, resulting in wasted computing resources, storage resources, and processing time.

[0015] To this end, this invention aims to establish a multi-generational genealogy among behavioral events, training samples, and model versions. Before model updates, it detects whether there are recursive dependency paths between candidate training data and the target model, and performs dual-channel acceptance or isolation processing on the training data based on the detection results, thereby improving the data reliability, behavioral stability, and update efficiency during the continuous update process of the player mirror model.

[0016] To address the aforementioned technical problems, this invention provides a dual-channel lineage isolation method for player mirror model training data, comprising: An event genealogy record is generated for behavioral events generated during game operation. The event genealogy record includes an event identifier, an execution subject type, a model version identifier that generated the behavioral event, a source player identifier, a relationship branch identifier, and a set of parent event identifiers. The execution subject type identifies whether the actual execution subject of the behavioral event is a real player or a player mirror model. The model version identifier corresponding to a behavioral event directly generated by a real player is empty. A genealogical graph is constructed, wherein the nodes of the genealogical graph include behavioral event nodes, training sample nodes, and model version nodes. The edges of the genealogical graph include generation edges from model version nodes to behavioral event nodes, processing edges from behavioral event nodes to training sample nodes, update edges from training sample nodes to model version nodes, inheritance edges from parent model version nodes to child model version nodes, and derivation edges from parent event nodes to derived event nodes. Among them, behavioral events generated by the player mirror model are connected to the model version node that generated the behavioral event through generation edges, and behavioral events directly generated by the real player are root events that do not have an upstream model version node. The parent model version is the prior model version connected to the current model version through inheritance edges. Generate a fixed-length binary bit vector signature for each node in the generated genealogy graph. The genealogy signature of a model version is obtained by bitwise ORing the bit vector corresponding to the model version identifier with the genealogy signatures of all parent model versions. The genealogy signature of the root event directly generated by the real player is a zero bit vector. The genealogy signature of the behavior event generated by the player mirror model inherits the genealogy signature of the model version that generated the behavior event. The genealogy signature of the derived event is obtained by bitwise ORing the genealogy signatures of its parent events. The genealogy signature of the training sample is obtained by bitwise ORing the genealogy signatures of all its parent events. Before updating the target model, recursive dependency detection is performed on the candidate training samples: a bitwise AND operation is performed between the genealogical signature of the candidate training sample and the genealogical signature of the target model; when the bitwise AND result is empty, it is determined that the candidate training sample has no recursive dependency relative to the target model; when the bitwise AND result is not empty, the generated genealogical graph is traversed in reverse from the candidate training sample node along the processing edge, derived edge, and generating edge; if the traversal reaches the target model or any historical parent model version of the target model, it is determined that the candidate training sample has a recursive dependency relative to the target model; if the traversal reaches all root events but still does not reach the target model or its historical parent model version, it is determined that the candidate training sample has no recursive dependency relative to the target model. Based on the recursive dependency detection results, execution subject type, sample purpose, source player identifier, and relation branch identifier, candidate training samples are routed to the basic behavior model training channel, relation branch training channel, or isolated set: Candidate training samples that have no recursive dependency relative to the target model, are of the execution subject type (real player), are used as behavior learning samples, and whose source player identifier matches the source player identifier of the target model, enter the basic behavior model training channel; candidate training samples that have no recursive dependency relative to the target model, are used as result evaluation samples or relation state samples, and whose relation branch identifier matches the target relation branch, enter the relation branch training channel; candidate training samples that have recursive dependency relative to the target model enter the isolated set and do not participate in the target model update. The basic behavior model part of the target model is updated using samples from the basic behavior model training channel, and the relation branch model part corresponding to the target relation branch in the target model is updated using samples from the relation branch training channel to generate a new model version. Then, a genealogical closure record is executed: a unique model version identifier is assigned to the new model version, the target model is recorded as the parent model version of the new model version, and an update edge from the training sample node used this time to the new model version node and an inheritance edge from the target model node to the new model version node are established in the generated genealogical graph. The genealogical signature of the target model, the bit vector corresponding to the new model version identifier, and the genealogical signature of the training sample used this time are bitwise ORed and merged as the genealogical signature of the new model version.

[0017] This invention also relates to a dual-channel lineage isolation system for player mirror model training data, comprising: The behavior event collection module is used to collect behavior events generated by real players and player mirror models during game operation, and generate event genealogy records for each behavior event. The event genealogy records include event identifier, execution subject type, model version identifier that generated the behavior event, source player identifier, relationship branch identifier, and parent event identifier set. The model version identifier corresponding to the behavior event directly generated by the real player is empty. A genealogical graph construction module is used to construct and generate a genealogical graph. The nodes of the generated genealogical graph include behavioral event nodes, training sample nodes, and model version nodes. The edges of the generated genealogical graph include generation edges from model version nodes to behavioral event nodes, processing edges from behavioral event nodes to training sample nodes, update edges from training sample nodes to model version nodes, inheritance edges from parent model version nodes to child model version nodes, and derivation edges from parent event nodes to derived event nodes. The genealogical signature calculation module is used to generate a fixed-length binary bit vector form of genealogical signature for the nodes in the generated genealogical graph. The genealogical signature of the model version is obtained by bitwise ORing the bit vector corresponding to the model version identifier and the genealogical signatures of all parent model versions. The genealogical signature of the root event directly generated by the real player is an all-zero bit vector. The genealogical signature of the behavior event generated by the player mirror model inherits the genealogical signature of the model version that generated the behavior event. The genealogical signature of the derived event and the training sample is obtained by bitwise ORing the genealogical signature of its parent event. The recursive dependency detection module is used to perform a bitwise AND operation between the genealogical signature of the candidate training sample and the genealogical signature of the target model before updating the target model. When the bitwise AND result is not empty, the generated genealogical graph is traversed backward from the candidate training sample node along the processing edge, derived edge and generating edge. If the traversal reaches the target model or any historical parent model version of the target model, it is determined that the candidate training sample has a recursive dependency relative to the target model. The data routing module is used to route candidate training samples that have no recursive dependency relative to the target model, are executed by real players, are used for behavioral learning, and whose source player identifiers match the source player identifiers of the target model, to the basic behavioral model training channel; to route candidate training samples that have no recursive dependency relative to the target model, are used for result evaluation or relation state, and whose relation branch identifiers match the target relation branch, to the relation branch training channel; and to route candidate training samples that have recursive dependency relative to the target model to the isolation set. The model update module is used to update the basic behavior model part of the target model using samples from the basic behavior model training channel, update the relation branch model part of the target model corresponding to the target relation branch using samples from the relation branch training channel, generate a new model version, and establish update edges from the training sample nodes used this time to the new model version nodes and inheritance edges from the target model nodes to the new model version nodes. The model version management module is used to assign a unique model version identifier to a new model version, record the target model as the parent model version of the new model version, and combine the lineage signature of the target model, the bit vector corresponding to the new model version identifier, and the lineage signature of the training samples used in this session into the lineage signature of the new model version by bitwise OR.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention establishes a genealogy between behavioral events, training samples, and model versions. It can not only identify the direct data-generating entity, but also trace the indirect sources after aggregation, duplication, processing, asynchronous write-back, and multiple rounds of model updates, thus solving the problem that ordinary source labels cannot reflect multi-generational generation relationships.

[0019] This invention detects whether there is an intersection between the upstream model source of the candidate training samples and the historical version of the target model before the model is updated. It can identify and isolate data directly or indirectly generated by the target model and its historical versions, avoid the model repeatedly learning its own output, and reduce the risk of accidental errors being continuously amplified and player behavior characteristics gradually becoming distorted.

[0020] The basic behavior model is updated only using real player operation data verified by the source. The behavior generated by the model must not be used as the target behavior sample of the source player. This helps the player mirror model to maintain the operation habits, skill preferences and decision-making characteristics of the source player in the long term.

[0021] This invention establishes relationship branch identifiers based on the combination of source players and associated players, and isolates training samples and model parameters according to the relationship branch identifiers, so that data generated by different friends, teams or guilds can only affect the corresponding relationship branches, avoiding the interaction results of one associated player from incorrectly affecting the mirror model corresponding to other associated players.

[0022] This invention employs a two-stage detection method: "initial screening using genealogical signatures + precise tracing of genealogical graphs." It rapidly filters a large number of candidate samples using fixed-length bitmaps, Bloom filters, or hash sets, performing graph traversal only on samples suspected of having recursive dependencies. This reduces unnecessary full genealogical queries and improves the efficiency of processing large-scale training data.

[0023] Existing systems typically require manual review of large amounts of historical data or retraining the entire model after detecting abnormal model behavior. This invention can record the event identifier, upstream model version, recursive dependency path, and isolation reason of isolated samples, thereby quickly locating the affected data range and reducing the processor resources, storage resources, and processing time consumed by repeated training.

[0024] Each model version is associated with its parent model version, training batch, and training sample set. It is possible to look up the data source used by any model version in reverse, and to look up the model version that may be affected by any behavioral event in forward. This is beneficial for the reproduction, auditing, and anomaly localization of model update results.

[0025] This invention does not rely on a specific model structure and can be applied to neural networks, decision trees, behavior trees, state machines, action probability tables, and incremental adaptation models; the relationship branches can also be extended from friend relationships to teams, guilds, or other multi-person relationships, making it easy to deploy in different types of multiplayer online games. Attached Figure Description

[0026] Figure 1 This is a block diagram of the components of the dual-channel genealogical isolation system for player mirror model training data provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of the genealogy diagram provided in Embodiment 2 of the present invention; Figure 3 This is a flowchart of the recursive dependency detection and dual-channel sample isolation method provided in Embodiment 4 of the present invention. Detailed Implementation

[0027] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Equivalent substitutions made by those skilled in the art regarding data fields, model structures, parameter ranges, storage methods, and processing order without departing from the core concept of the present invention are all embodiments of the present invention.

[0028] definition 1. Source player refers to a player whose actual gameplay behavior is used to construct a player mirror model.

[0029] 2. A player mirror model is a model generated based on the historical behavior data of the source player, used to control virtual characters to simulate the behavior of the source player.

[0030] 3. The basic behavior model refers to the model part used to preserve the relatively stable operating habits, combat strategies, and decision-making characteristics of the source player.

[0031] 4. The relationship branch model refers to the part of the model built on the basis of player combinations of "source player - associated player" to save the interaction state or incremental parameters in a specific relationship scenario.

[0032] 5. Behavioral events refer to records of player actions, mirror model actions, interactive behaviors, combat results, or state changes that occur during game operation.

[0033] 6. Model-generated data refers to data directly output by the player mirror model, or data formed by further processing, aggregation, copying, or writing back the output results of the player mirror model.

[0034] 7. Generative lineage refers to the direct or indirect generative relationships formed between behavioral events, training samples, and model versions.

[0035] Construction of Genealogical Diagram The system constructs a genealogical graph G based on event genealogy records, training sample records, and model version records. The genealogical graph G consists of a node set N and a directed edge set L, i.e., G = (N, L). The node set N includes behavioral event nodes, training sample nodes, and model version nodes. The directed edges include at least: The generation edge from the model version node to the behavior event node indicates that the behavior is generated by the corresponding model version; the processing edge from the behavior event node to the training sample node indicates that the training sample is processed by the corresponding behavior event; the update edge from the training sample node to the model version node indicates that the model version uses the corresponding training sample; the inheritance edge from the parent model version node to the child model version node indicates the update relationship between the two model versions; the derivation edge from the parent event node to the derived event node indicates the aggregation, copying, or result write-back relationship between events.

[0036] Action events directly generated by real players serve as root events in the generation hierarchy and do not have upstream model version nodes. Action events generated by mirrored models must be connected to the model version node that actually generated the action through generation edges.

[0037] When an asynchronous battle result is generated by multiple player actions and mirror model actions, the battle result node establishes derived edges with all related action event nodes, enabling the system to trace back from the battle result to the actual generating entity and model version of each action.

[0038] The phylogenetic graph is stored in an append-only manner. Each time a new event, sample, or model version is generated, only new nodes and edges are added, without modifying the existing generation relationships between nodes.

[0039] Incremental generation of genealogical signatures To reduce the processing overhead of large-scale phylogenetic graph queries, this invention generates a fixed-length phylogenetic signature for each node. The phylogenetic signature is in the form of a fixed-length binary bit vector.

[0040] The genealogical signature of a model version is obtained by bitwise ORing the bit vector corresponding to the model version identifier with the genealogical signatures of all parent model versions. The root events directly generated by real players have their phylogenetic signatures set to an all-zero vector. Events directly generated by a mirror model inherit the genealogical signature of the model version that generated the event. An event derived from multiple events has its phylogenetic signature obtained by bitwise ORing and merging the phylogenetic signatures of each parent event; The phylogenetic signature of a training sample is obtained by bitwise ORing and combining the phylogenetic signatures of all its parent events.

[0041] The bit vector corresponding to the model version identifier is generated as follows: multiple independent hash operations are performed on the model version identifier, and the position corresponding to the result of each hash operation is set to 1. The length b of the bit vector can be 1024 to 8192 bits, preferably 4096 bits; the number of hash operations k can be 3 to 7, preferably 4.

[0042] A bitmap, Bloom filter, or hash set can be used to store lineage signatures. A Bloom filter is a data structure that uses multiple hash functions to map set elements to a fixed-length bit array. When using a Bloom filter, multiple hash operations are performed on the model version identifier, setting the corresponding positions to 1; parent lineage signatures are merged using a bitwise OR operation. This method will not miss upstream model versions that have already been written, but may produce suspected hits due to hash collisions. Therefore, this invention adopts a two-level detection method of "initial screening of lineage signatures + precise tracing of lineage graphs".

[0043] Recursive Dependency Detection When the target model M needs to be updated, the system obtains the model lineage set L(M) of the target model M. The model lineage set includes the target model M and all its historical parent model versions. The historical parent model versions of the target model are obtained by traversing the inheritance edges backward.

[0044] For a candidate training sample S, the system obtains its upstream model source set A(S). The upstream model source set includes all model versions that can be reached by tracing back from the candidate training sample S along the processing edge and the derived edge.

[0045] The recursive dependency determination result C(S,M) of candidate training sample S relative to target model M is determined according to the following rules: when the intersection of A(S) and L(M) is not empty, C(S,M)=1, indicating that the candidate training sample is directly or indirectly derived from the target model or its historical version; when the intersection of A(S) and L(M) is empty, C(S,M)=0, indicating that the recursive dependency relationship was not found.

[0046] The specific testing process includes: Perform a bitwise AND operation between the genealogical signature of the candidate training sample S and the model genealogical signature of the target model M; if the bitwise AND result is empty, it is determined that the candidate training sample and the target model do not have a common model source; if the bitwise AND result is not empty, add the candidate training sample to the set to be confirmed; starting from the node of the candidate training sample, traverse backwards along the processing edge, derived edge, and generation edge to generate the genealogical graph; if the target model M or any historical parent model version of the target model M is found, it is determined that there is a recursive dependency; if the model version in the genealogical system of the target model is not found after traversing all root events, it is determined that there is no recursive dependency.

[0047] Reverse traversal can employ either depth-first search or breadth-first search, and avoids the same node being processed repeatedly by using the set of visited nodes; reverse traversal can also use the reverse adjacency index to read the upstream node pointing to the current node.

[0048] For mixed samples formed by multiple parent events, a conservative rule can be applied: if any parent event has a recursive dependency, the entire sample is considered a recursive sample. Alternatively, the recursive contamination ratio can be calculated based on the contribution weight of each parent event in the sample: Recursive contamination ratio = Sum of contribution weights of recursive source events ÷ Sum of contribution weights of all events. When the recursive contamination ratio reaches a preset threshold, the sample is rejected; if it does not reach the preset threshold, the data portion corresponding to the recursive source event can be deleted, and a new training sample can be generated.

[0049] Dual-channel directional routing and isolation of training data The system routes candidate training samples to different data channels based on the purpose of the training samples, the type of the executing entity, the source player identifier, the relation branch identifier, and the recursive dependency determination result. The data channels include two training channels: a basic behavior model training channel and a relation branch training channel, as well as an isolated set.

[0050] Basic Behavioral Model Training Channel Only samples that simultaneously meet the following conditions can enter the source player's basic behavior model training channel: (1) The sample originates from real operations directly generated by the source player; (2) The source player identifier of the sample is consistent with the source player identifier of the target model; (3) The sample does not have an upstream model version; (4) The sample has no recursive dependency on the target model; (5) The behavioral data summary and parent event relationship of the sample pass the verification. Any behavioral data directly or indirectly generated by the mirror model shall not be allowed to enter the training channel of the basic behavioral model.

[0051] Relationship branch training channel The relationship branch training channel uses the combination of source player identifier and associated player identifier as the isolation unit. Only samples whose relationship branch identifier is completely consistent with the target relationship branch are allowed to enter the corresponding relationship branch training channel.

[0052] The combat results generated by the mirror model can be used as relationship status or combat evaluation data, but cannot be used as labels for the source player's actual behavior. The system distinguishes between behavior learning samples, result evaluation samples, and relationship status samples by the sample purpose field, and restricts samples of different purposes to only updating the corresponding parameter areas. Behavior learning samples only update the parameter area corresponding to the basic behavior model, while result evaluation samples and relationship status samples only update the parameter area corresponding to the corresponding relationship branch model.

[0053] Separate sets Samples that meet any of the following conditions are included in the segregated set: (1) Missing execution subject type; (2) Missing or unverifiable model version identifier; (3) Incomplete parent event; (4) Inconsistent genealogical signature and precise tracing result; (5) Detected recursive dependency; (6) Inconsistent relationship branch identifier and target model; (7) Inconsistent source player identifier and target model. Samples in the isolated set do not participate in the target model update, but retain the event identifier, isolation reason, and detection path for subsequent auditing or reprocessing.

[0054] Model update and lineage closure record The model update module obtains training samples from the detected data channels and performs parameter updates on the target model to obtain a new model version. Specifically, it updates the basic behavior model portion of the target model using samples from the basic behavior model training channel, and updates the relationship branch model portion of the target model corresponding to the target relationship branch using samples from the relationship branch training channel. Parameter updates can be neural network parameter training, decision tree node updates, behavior probability updates, rule weight updates, or incremental adaptation parameter updates; this invention does not limit the specific model type of the player mirror model.

[0055] After the new model version is generated, the system performs the following genealogical closure record operation: Assign a unique model version identifier to the new model version; record the target model as the parent model version of the new model version; record the training batch and training sample set used in this update; establish update edges from training samples to the new model version and inheritance edges from the target model to the new model version in the generated genealogy graph; combine the genealogy signature of the target model, the bit vector corresponding to the new model version identifier, and the genealogy signature of the training samples used in this update bitwise or by combining them to form the genealogy signature of the new model version; write the new model version and its genealogy record into the model version library in an overwriteable manner; when the new model version generates behavior in the future, write the new model version identifier into the genealogy record of the corresponding behavior event.

[0056] Before releasing a new model version, double-check that the actual sample set summary used for training tasks matches the summary of the sample set that passed detection. If they do not match, stop the release to prevent undetected data from being mixed into the training task after detection.

[0057] Through the above processing, any behavior generated by the new model can still be traced back to the model version that actually generated the behavior, even after multiple aggregations, copies, or asynchronous write-backs, thus ensuring that recursive dependency detection can continue across multiple model versions.

[0058] Systems that implement the above methods include: The behavior event collection module connects to the game server and is used to collect behavior events generated by real players and player mirror models during game operation, and generate event genealogy records for each behavior event; The source confirmation module is used to determine the actual executor of the behavior based on the control status in the game server, and write the source player identifier, executor type and model version identifier. The genealogy record module is used to generate event genealogy records, training sample records, and model version records, and to append them in a non-overwriteable manner. The phylogenetic graph construction module is used to establish generating edges, processing edges, derived edges, updating edges, and inherited edges between behavioral events, training samples, and model versions. The genealogical signature calculation module is used to incrementally encode the upstream model source of a node and complete the initial screening of recursive dependencies. The recursive dependency detection module is used to perform a reverse traversal of the genealogical graph for candidate training samples that are suspected of being hits. The data routing module is used to write candidate training samples into the basic behavior model training channel, the relation branch training channel, or the isolated set, respectively. The model update module is used to generate a new model version using the training samples that have passed the detection. The model version management module is used to save the correspondence between model version, parent model version, training batch, training sample set, and model lineage signature. The audit module is used to query recursive dependency paths and reasons for sample isolation based on event identifiers, sample identifiers, or model version identifiers.

[0059] The above modules can be deployed on the same game server, or they can be deployed separately on the game server, data processing server, and model training server, and connected through a network communication interface.

[0060] Example 1: System Composition and Data Processing Procedure like Figure 1 As shown, this embodiment provides a genealogy isolation system for generating player mirror models. The system is deployed on the server side of a multiplayer online game and includes a behavior event acquisition module 100, a source confirmation module 200, a genealogy recording module 300, a genealogy graph construction module 400, a genealogy signature calculation module 500, a recursive dependency detection module 600, a data routing module 700, a model update module 800, a model version management module 900, and an auditing module 1000.

[0061] Each module can be deployed on the same server, or it can be deployed separately on the game server, data processing server, model training server, and data storage server, and exchange data through network interfaces.

[0062] The behavior event acquisition module 100 is used to acquire behavior events generated by real players and player mirror models during game operation. The behavior events include at least: Unique event identifier; game session identifier; time of action; subject of action identifier; type of source of action; source player identifier; source model version identifier; relationship branch identifier; game environment state; action and parameters; parent event identifier; event summary value.

[0063] Among them, the types of behavior sources include at least real player behavior, mirror model generated behavior, and hybrid control behavior.

[0064] The game environment status may include information such as character position, movement speed, health, skill cooldown status, equipment status, nearby virtual objects, map area, mission objectives, and combat phase. Actions may include moving, attacking, releasing skills, selecting a target, using items, sending text, sending emoticons, and performing one or more interactive actions.

[0065] The source confirmation module 200 is used to determine the true source of a behavioral event based on the execution context on the server side. The source is not directly determined by the source field uploaded by the game client, but rather by the match identifier, controller identifier, model deployment instance identifier, and model version identifier generated by the server.

[0066] When a behavior is directly triggered by a real player client, the behavior source type is marked as a real player behavior; when a behavior is triggered by a mirror model inference service, the behavior source type is marked as a mirror model generated behavior, and the model version that generated the behavior is recorded; when a behavior is formed by both real player input and mirror model output, the behavior source type is marked as a mixed control behavior, and the corresponding player identifier and model version identifier are recorded respectively.

[0067] Verifying the source of behavior on the server side can prevent inaccurate determination of the source of generated data due to missing, forged, or transmitted fields on the client side.

[0068] The genealogy record module 300 is used to generate immutable genealogy records. Previously written genealogy records are not overwritten or modified; when a state change occurs, a new state record is appended and associated with it using a unique event identifier.

[0069] In a preferred embodiment, each lineage record includes at least: a unique event identifier, a node type, a source node identifier, a target node identifier, a relationship type, a model version identifier, a player identifier, a relationship branch identifier, a generation time, a processor version, a data digest value, and a record status.

[0070] Example 2: Construction of Genealogical Chart like Figure 2 As shown, the genealogy graph construction module 400 constructs a directed genealogy graph based on genealogy records. The generated genealogy graph includes at least model version nodes 201, behavior event nodes 202, and training sample nodes 203.

[0071] Model version node 201 is used to represent a player mirror model version that can be deployed and invoked independently. Its attributes include at least the model version identifier, the player identifier, the parent model version identifier, the model parameter summary, the training task identifier, the training sample set summary, the creation time, and the deployment status.

[0072] Behavior event node 202 is used to represent behavioral events generated by real players or mirror models during the game.

[0073] Training sample node 203 is used to represent training samples formed by cleaning, segmenting, feature extraction and labeling of one or more behavioral events.

[0074] The edges in the phylogenetic graph include at least: Edge 301 is generated, indicating that the model version node generates the behavior event node; Processing edge 302 indicates that the behavior event node is processed into a training sample node; Update edge 303 to indicate that the training sample node participates in the generation of the new model version node; Inheritance edge 304 indicates that the new model version is inherited from the original model version; Derived edge 305 indicates that one behavioral event participates in generating another behavioral event.

[0075] For example, if the mirror model M(v) generates a mirror action E(m) during a game, then a generating edge 301 is established from M(v) to E(m). The mirror action E(m) interacts with the real player's action E(h) to produce a derived event E(d), then derived edges 305 are established from E(m) and E(h) to E(d), respectively. When the derived event E(d) is processed into a candidate training sample S(d), a processing edge 302 is established from E(d) to S(d).

[0076] When real player behavior E(h) is processed into real training sample S(h) and used to generate a new model M(v+1), a processing edge 302 from E(h) to S(h), an update edge 303 from S(h) to M(v+1), and an inheritance edge 304 from M(v) to M(v+1) are established.

[0077] The phylogenetic graph is stored in an append-only manner. Each time a new event, sample, or model version is generated, only new nodes and edges are added, without modifying the existing generation relationships between nodes.

[0078] In one implementation, the genealogy graph is stored in a graph database, with separate indexes for node identifiers, model versions, player identifiers, and relationship branches. In another implementation, the genealogy graph is stored in a relational database or a key-value database, and genealogy queries are implemented using node tables, edge tables, and reverse adjacency indexes.

[0079] Example 3: Generation of Genealogical Signatures To reduce the computational and data reading overhead of traversing the complete genealogy graph before each training session, the genealogy signature calculation module 500 generates genealogy signatures for model version nodes, behavior event nodes, and training sample nodes.

[0080] In this embodiment, the genealogical signature uses a binary bit vector of length b. b can be 1024 to 8192 bits, preferably 4096 bits. k independent hash operations are performed on the model version identifier, k can be 3 to 7, preferably 4, and the corresponding positions are set to 1.

[0081] The following formulas are all represented using plain text that can be directly copied.

[0082] The basic bit vector corresponding to the model version identifier x is represented as: B(x) = bit(h1(x)) OR bit(h2(x)) OR ... OR bit(hk(x)) Here, h1 to hk represent different hash functions, bit means setting the corresponding position to 1, and OR means bitwise OR operation.

[0083] The lineage signature of model version M(v) is represented as: L(M(v)) = B(M(v)) OR L(M(p1)) OR ... OR L(M(pn)) Here, M(p1) to M(pn) represent the parent model versions connected to M(v) via inheritance edges. When M(v) does not have a parent model version, L(M(v)) equals B(M(v)).

[0084] For a behavioral event E generated by model version M(s), its upstream source signature is represented as: A(E) = B(M(s)) OR L(M(s)) OR A(E(p1)) OR ... OR A(E(pm)) Here, E(p1) to E(pm) represent the parent events of the behavior event pointed to by derived edges. For behavior events that are generated entirely by real players and have no model source, their upstream source signature is initialized as an all-zero vector.

[0085] The upstream source signature of the training sample S, which is formed by processing multiple behavioral events E1 to Er, is represented as follows: A(S) = A(E1) OR A(E2) OR ... OR A(Er) Through the above processing, the upstream source signature of the training sample contains compressed tokens of its entire traceable model source.

[0086] Lineage signatures are used to quickly eliminate candidate samples that do not have recursive dependencies. Since different model versions may have the same bits after hashing, the intersection of lineage signatures can only indicate the possibility of recursive dependencies, and the final result is still determined by the exact traversal of the generated lineage graph.

[0087] In other implementations, lineage signatures can also be achieved using a sorted set of model version identifiers, a compressed bitmap, a segmented bitmap, or a Bloom filter.

[0088] Example 4: Recursive Dependency Detection and Sample Isolation like Figure 3As shown, before updating the player mirror model using candidate training samples, the recursive dependency detection module 600 performs the following steps.

[0089] S301, obtain candidate training samples and target model version.

[0090] Let the candidate training sample be S(d), and the target model version to be updated be M(t). Read the upstream source signature A(S(d)) of S(d), and read the lineage signature L(M(t)) of M(t).

[0091] S302, Perform a bitwise AND operation on the two genealogical signatures: D = A(S(d)) AND L(M(t)) AND stands for bitwise AND operation.

[0092] When D equals a vector of all zeros, it means that the upstream model source of the candidate training sample does not have a common label with the target model and its historical versions. Therefore, a complete recursive graph traversal can be skipped, and the process proceeds to step S304.

[0093] When D is not equal to a vector of all zeros, it indicates that the candidate training samples may contain data generated by the target model or a historical version of the target model, and proceed to step S303.

[0094] S303, starting from the candidate training sample nodes, performs a reverse traversal of the generated genealogy graph.

[0095] The reverse traversal first searches for the behavior event that generates candidate training samples along processing edge 302, and then recursively searches for the parent event of each behavior event along derived edge 305. When a behavior event has a generating edge 301, the search continues for the model version node that generated that behavior event.

[0096] At the same time, starting from the target model version M(t), we search back along inheritance edge 304 to find all historical versions of the target model, forming the target historical version set H(t).

[0097] The target historical version set is represented as: H(t) = {M(t), M(t-1), M(t-2), ...} If any model version reached from the upstream path of the candidate training sample belongs to H(t), then it is determined that there is a recursive generation dependency between the candidate training sample and the target model.

[0098] The precise detection process can be implemented using the following pseudocode: Input: Candidate samples S, target model M Output: Does recursive dependency exist? Path hit? 1. Obtain the target model M and all its historical versions, forming a set H. 2. Add the candidate sample S to the queue to be accessed Q. 3. Establish the set of visited nodes V. 4. When Q is not empty: 4.1 Retrieve node N from Q 4.2 If N already exists in V, then continue processing the next node. 4.3 Add N to V 4.4 If N is a model version node, and N belongs to H: Returns the existence of recursive dependencies and the current hit path. 4.5 Obtain all upstream nodes that point to N through processing edges, derived edges, or generated edges. 4.6 Add unvisited upstream nodes to Q 5. Returns an error indicating no recursive dependency exists. Using the set of visited nodes during traversal avoids repeatedly visiting the same node due to the convergence of multiple derived paths. The reverse adjacency index is used to directly read the upstream node pointing to the current node, thereby reducing the number of genealogical queries.

[0099] S304. When no exact recursive dependency is detected, further verify the source entity, sample purpose, and relation branch identifier of the candidate training samples.

[0100] Source subject verification confirms whether the player to whom the sample belongs and the player corresponding to the target model meet the preset data usage rules. Sample purpose verification confirms whether the sample is allowed to be used for model training, and not just for replay, evaluation, or auditing. Relationship branch verification prevents data belonging to other friend relationships, other mirror instances, or other experimental branches from being incorrectly written into the current model.

[0101] S305, when a recursive dependency is detected, or any of the validations for the source subject, sample purpose, or relation branch fails, the data routing module 700 writes the candidate training sample into an isolated set, disallowing the sample from participating in the parameter update of the current target model.

[0102] The quarantine record should include at least: Candidate training sample identifiers; Target model version identifier; The upstream model version identifier that was hit; The complete lineage path of the hit; Reason for quarantine; Detection strategy version; Detection time; Under quarantine status.

[0103] S306, when a candidate training sample has no recursive dependency and all validations pass, the sample is routed to the training set of the target player and the corresponding relation branch.

[0104] S307, the model update module 800 uses the detected training samples to update the player mirror model, and generates a new model version node after the model update is completed. At the same time, it adds training samples to the update edge 303 of the new model version and the inheritance edge 304 from the original model version to the new model version.

[0105] Before releasing a new model version, double-check that the actual sample set summary used for training tasks matches the summary of the sample set that passed detection. If they do not match, stop the release to prevent undetected data from being mixed into the training task after detection.

[0106] Example 5: Player Mirror Model Training Process In one specific embodiment, the player mirror model is used to learn how players operate in multiplayer online combat scenarios.

[0107] A training window of 32 consecutive sampling moments is used, with a time interval of 100 milliseconds between adjacent sampling moments. The input data for each sampling moment includes player position, movement direction, movement speed, health, skill status, target object, nearby object status, and combat phase.

[0108] Continuous numerical values ​​are normalized, and discrete data such as skill number, role number, and target type are vector-encoded to obtain the state feature vector at each sampling time.

[0109] The model consists of an input encoding layer, a temporal feature extraction layer, and an action output layer. Among them: The input coding layer consists of two fully connected layers, each with 128 to 512 neurons. The temporal feature extraction layer uses two gated recurrent units, and the hidden state dimension can be from 128 to 512. The action output layer is used to output the movement direction, action type, skill type, and probability of the target object.

[0110] The English name for a gated recurrent unit is Gated Recurrent Unit, abbreviated as GRU.

[0111] During model training, cross-entropy loss is used for discrete actions, while mean squared error loss is used for continuous action parameters. The optimizer can be the Adam optimizer, with an initial learning rate of 0.0001, a batch size of 64 to 512, and 3 to 20 training epochs.

[0112] The model structure and parameters described above are only one preferred implementation. The core of this invention is not limited to a specific neural network type; recurrent neural networks, long short-term memory networks, Transformer networks, decision trees, reinforcement learning models, or other models capable of outputting game actions based on player behavior data can also be used.

[0113] Regardless of the model used, the genealogical signature pre-detection, genealogical graph generation accurate detection, and data routing processing described in this invention should be performed before the training samples enter the model update program.

[0114] Example 6: Recursive Sample Isolation Example Player P's current mirror model version is M12, which is derived from the historical versions M10 and M11.

[0115] M12 participates in an asynchronous battle while the player is offline, generating a mirrored behavior event E1001. E1001, together with the behavior event E1002 of another real player, produces a battle outcome event E1003. The data processing program processes E1003 into candidate training sample S8001.

[0116] The following paths exist in the genealogical diagram: M12 → E1001 → E1003 → S8001 Among them, M12 to E1001 are generating edges, E1001 to E1003 are derived edges, and E1003 to S8001 are processing edges.

[0117] When the system prepares to update M12 using S8001, the bitwise AND result of the upstream source signature of S8001 and the genealogical signature of M12 is not empty. The recursive dependency detection module 600 then traverses backwards to generate the genealogical graph and reaches M12. Therefore, the system determines that S8001 contains data generated by M12 itself, writes S8001 into the isolation set, and does not allow it to participate in the update of M12.

[0118] Another real player behavior event, E2001, was confirmed by the game server to be a real player behavior. E2001 was processed into training sample S8002. Since S8002 has no model generation source, its upstream source signature is a vector of all zeros. After verification of the source subject, sample purpose, and relational branch, S8002 was routed to the training set corresponding to M12.

[0119] The model update module 800 uses S8002 to generate a new model version M13 and adds the following lineage relationships: E2001 → S8002 S8002 → M13 M12 → M13 Subsequently, if the data generated by M13 or its subsequent versions is processed again as training samples, the system can continue to perform recursive dependency detection based on the newly added phylogenetic nodes and edges.

[0120] Example 7: Auditing and Anomaly Handling The audit module 1000 is used to save operation records for genealogy writing, signature calculation, recursive detection, sample isolation, data release, and model updates.

[0121] When an event record is submitted repeatedly, the system performs an idempotency check based on the event's unique identifier and its summary. If both the unique identifier and summary are identical, the original processing result is returned, and nodes and edges are not created again. If the unique identifiers are the same but the summaries are different, the event is marked as an abnormal event, and the training process is stopped.

[0122] When a training sample lacks a source event identifier, source model version identifier, or relation branch identifier, the system adopts a default rejection strategy and writes it into the pending confirmation set or the isolation set.

[0123] When the pedigree chart service is temporarily unavailable, samples that have not undergone accurate testing should not be directly included in the model training set. The system can re-perform the testing once the pedigree chart service is restored.

[0124] When the genealogy signature calculation program, data processing program, or detection strategy changes, the corresponding program version and strategy version are saved in the genealogy record for recalculation and verification.

[0125] Example 8: System Hardware Implementation The present invention also provides an electronic device, including a processor, a memory, a network communication interface, and a computer program stored in the memory.

[0126] When the processor executes the computer program, it performs behavioral event acquisition, source confirmation, genealogy recording, genealogy graph construction, genealogy signature calculation, recursive dependency detection, data routing, model updating, model version management, and auditing.

[0127] This invention can also be implemented using a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the aforementioned method for isolating the data lineage of the player mirror model generation.

[0128] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dual-channel lineage isolation method for player mirror model training data, characterized in that, include: Generate an event genealogy record for behavioral events generated during game operation. The event genealogy record includes an event identifier, an execution subject type, a model version identifier that generated the behavioral event, a source player identifier, a relationship branch identifier, and a set of parent event identifiers. The execution subject type identifies whether the actual execution subject of the behavioral event is a real player or a player mirror model. A genealogical graph is constructed, wherein the nodes of the genealogical graph include behavioral event nodes, training sample nodes, and model version nodes, and the edges of the genealogical graph include generation edges from model version nodes to behavioral event nodes, processing edges from behavioral event nodes to training sample nodes, update edges from training sample nodes to model version nodes, inheritance edges from parent model version nodes to child model version nodes, and derivation edges from parent event nodes to derived event nodes. The parent model version is the prior model version connected to the current model version via inheritance edges. Generate a fixed-length binary bit vector signature for each node in the generated genealogy graph. The genealogy signature of a model version is obtained by bitwise ORing the bit vector corresponding to the model version identifier with the genealogy signatures of all parent model versions. The genealogy signature of the root event directly generated by the real player is a zero bit vector. The genealogy signature of the behavior event generated by the player mirror model inherits the genealogy signature of the model version that generated the behavior event. The genealogy signature of the derived event is obtained by bitwise ORing the genealogy signatures of its parent events. The genealogy signature of the training sample is obtained by bitwise ORing the genealogy signatures of all its parent events. Before updating the target model, perform recursive dependency detection on the candidate training samples; Based on the recursive dependency detection results, the type of the executing entity, the purpose of the sample, the source player identifier, and the relation branch identifier, the candidate training samples are routed to the basic behavior model training channel, the relation branch training channel, or the isolated set: the candidate training samples that have no recursive dependency relative to the target model, are of the type of real player, are used as behavior learning samples, and whose source player identifier is consistent with the source player identifier of the target model, enter the basic behavior model training channel. Candidate training samples that are not recursive dependencies in the target model, whose purpose is to evaluate results or state of relations, and whose relation branch identifiers are consistent with the target relation branch are entered into the relation branch training channel. Candidate training samples that have recursive dependencies on the target model are placed into an isolated set and do not participate in the target model update. The basic behavior model part of the target model is updated using samples from the basic behavior model training channel, and the relation branch model part of the target model corresponding to the target relation branch is updated using samples from the relation branch training channel, generating a new model version, and performing genealogical closure record.

2. The method according to claim 1, characterized in that, The type of the executing entity for the behavioral event is determined by the server side based on the control status of the game server. The control status includes the controller identifier and the model deployment instance identifier. The game client only submits the original operation instructions and does not specify the type of executing entity, the model version identifier, or the relationship branch identifier.

3. The method according to claim 1, characterized in that, The bit vector corresponding to the model version identifier is generated in the following way: multiple independent hash operations are performed on the model version identifier, and the position corresponding to the result of each hash operation is set to 1.

4. The method according to claim 1, characterized in that, When a behavior event is formed by the aggregation, replication, or asynchronous result writing back of multiple parent events, a derived edge pointing from each parent event node to the behavior event node is established in the generated genealogy graph.

5. The method according to claim 4, characterized in that, For a mixed training sample formed by multiple parent events, the recursive contamination ratio is calculated based on the contribution weight of each parent event in the mixed training sample. The recursive contamination ratio is the ratio of the sum of the contribution weights of the recursive source events to the sum of the contribution weights of all events. When the recursive contamination ratio reaches a preset threshold, the mixed training samples are rejected. When the recursive contamination rate does not reach the preset threshold, delete the data portion corresponding to the recursive source event and regenerate the training sample.

6. The method according to claim 1, characterized in that, The relationship branch training channel uses the combination of source player identifier and associated player identifier as the isolation unit. The sample purpose is distinguished into behavior learning samples, result evaluation samples and relationship state samples by the sample purpose field. Samples with different sample purposes only update the corresponding parameter area in the player mirror model. The battle results generated by the player mirror model are used as result evaluation samples or relationship state samples and entered into the relationship branch training channel.

7. The method according to claim 1, characterized in that, Candidate training samples with any of the following conditions are included in the isolation set: missing execution subject type, missing or unverifiable model version identifier, incomplete parent event, inconsistent genealogy signature with reverse traversal result, inconsistent relation branch identifier with target model, and inconsistent source player identifier with target model; the samples in the isolation set retain event identifier, isolation reason and detection path.

8. The method according to claim 1, characterized in that, Before releasing a new model version, the summary of the sample set actually used in the training task is compared with the summary of the sample set that passed the recursive dependency detection. If the two are inconsistent, the release of the new model version is stopped.

9. A dual-channel lineage isolation system for player mirror model training data, characterized in that, include: The behavior event collection module is used to collect behavior events generated by real players and player mirror models during game operation, and generate event genealogy records for each behavior event; The phylogenetic tree construction module is used to build and generate phylogenetic trees; The genealogical signature calculation module is used to generate a fixed-length binary bit vector form of genealogical signature for the nodes in the generated genealogical graph; The recursive dependency detection module is used to perform a bitwise AND operation between the genealogical signature of the candidate training sample and the genealogical signature of the target model before updating the target model. When the bitwise AND result is not empty, the generated genealogical graph is traversed backward from the candidate training sample node along the processing edge, derived edge and generating edge. If the traversal reaches the target model or any historical parent model version of the target model, it is determined that the candidate training sample has a recursive dependency relative to the target model. The data routing module is used to route candidate training samples that have no recursive dependency relative to the target model, are executed by real players, are used for behavioral learning, and whose source player identifiers match the source player identifiers of the target model, to the basic behavioral model training channel; to route candidate training samples that have no recursive dependency relative to the target model, are used for result evaluation or relation state, and whose relation branch identifiers match the target relation branch, to the relation branch training channel; and to route candidate training samples that have recursive dependency relative to the target model to the isolation set. The model update module is used to update the basic behavior model part of the target model using samples from the basic behavior model training channel, update the relation branch model part of the target model corresponding to the target relation branch using samples from the relation branch training channel, generate a new model version, and establish update edges from the training sample nodes used this time to the new model version nodes and inheritance edges from the target model nodes to the new model version nodes. The model version management module is used to assign a unique model version identifier to a new model version, record the target model as the parent model version of the new model version, and combine the lineage signature of the target model, the bit vector corresponding to the new model version identifier, and the lineage signature of the training samples used in this session into the lineage signature of the new model version by bitwise OR.

10. The system according to claim 9, characterized in that, It also includes a genealogy recording module and an auditing module; the genealogy recording module appends event genealogy records, training sample records and model version records in a non-overwrite manner, and performs idempotent verification on repeated submissions of the same event identifier. When the event content summary is inconsistent, the event is marked as an abnormal event and the training process is stopped. The audit module queries the recursive dependency path and the reason for sample isolation based on the event identifier, sample identifier, or model version identifier.

Images