Game task demand intelligent generation method and system based on knowledge graph
By representing the details of the wargame deduction task as a knowledge graph and using neural networks for prediction, the existing deep reinforcement learning methods have been solved, and fast, accurate and interpretable task generation is achieved.
Patent Information
- Application Number
- CN202411583040.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-07
- Publication Date
- 2025-02-18
AI Technical Summary
The existing deep reinforcement learning methods are long training time in wargame deduction, consume a lot of computing resources, and have low interpretability in task generation process, making it difficult to meet the needs of rapid generation and high accuracy.
Using a knowledge graph-based method, the task details in wargame deduction are represented as knowledge graphs, and the graph nodes and relationships are predicted in time through neural networks to generate the next task.
A task generation method with short training time, high model interpretability and high accuracy is realized, reducing training costs and hardware resource requirements.
Smart Images
Figure CN119476489A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of intelligent game technology, and in particular relates to a method and system for intelligently generating game task requirements based on a knowledge graph. Background Art
[0002] With the rapid development of artificial intelligence, intelligent game has become one of the hot topics of current research, among which the research on intelligent game in the field of strategic deduction is also gradually rising. The research of this invention focuses on the intelligent generation method of task requirements in the scenario of war game deduction. The game problems in war game deduction are far more difficult than those in Go. The difficulties mainly lie in the complexity of deduction state, the diversity of task combinations and the long deduction time. Intelligent game helps people or machines make better decisions in the face of uncertainty and competitors by simulating complex decision-making environments. This can improve the quality of decision-making in real life, such as business, military, finance and other fields.
[0003] Existing methods mostly use deep reinforcement learning or multi-agent methods to learn from the replay of war games. However, methods based on deep reinforcement learning require a lot of computing resources, have high training costs and long training times, and the task generation process is less interpretable.
[0004] Therefore, there is an urgent need for a task generation method that requires short training time, high accuracy, and explainability. Summary of the invention
[0005] In response to the above problems, the present invention proposes a method and system for intelligently generating game task requirements based on knowledge graphs, which represents the task details in war game simulations as knowledge graphs, and uses neural networks to perform temporal predictions on graph nodes and relationships to obtain the next task. This not only shortens the training time, but also makes the model highly interpretable and accurate.
[0006] To achieve the above objectives, the technical solution of the present invention includes the following contents.
[0007] A method for intelligently generating game task requirements based on a knowledge graph, the method comprising:
[0008] Based on the current game state of the war game simulation, a knowledge graph of the game state at the current moment is generated; wherein the nodes in the game state knowledge graph include: frame nodes, entity nodes and task operation nodes;
[0009] By aggregating the information around the node, we can get the node vector in the knowledge graph of the current game status.
[0010] The current game state knowledge graph and the previous game state knowledge graph are regarded as a temporal sequence, and the sequence is encoded based on a recurrent neural network to obtain the global representation of the current moment, the local representation of the node, and the local representation of the relationship;
[0011] Based on the global representation, node local representation, relationship local representation and node vectors in the current game status knowledge graph, the execution task at the current moment is generated.
[0012] Furthermore, the game status and execution operations in the war game simulation process are used to generate a knowledge graph of the game status at the current moment, including:
[0013] Obtaining the game state of the war game at the current moment; wherein the game state includes the entity of the war game at the current moment and the corresponding entity state;
[0014] Set the valid operation set A between the previous moment and the current moment t , as the execution operation of the war game at the current moment; wherein the valid operation set A t The first action is the execution task at the previous moment;
[0015] Generate the frame node of the current moment;
[0016] The entity of the war game at the current moment is used as an entity node in the game state knowledge graph at the current moment, and the entity node is connected to the frame node based on the entity state of the war game at the current moment;
[0017] Generate a task operation node based on the task category, and connect the task operation node to the frame node based on the execution operation of the war game at the current moment;
[0018] Connect adjacent frame nodes in the time dimension to obtain the knowledge graph of the game status at the current moment.
[0019] Furthermore, the entities include: units, buildings, technologies and resources; and the entity states include: owned, observed, trained, constructed and researched.
[0020] Furthermore, the task categories include: constructing buildings, training units, researching technology, building deformation and empty tasks.
[0021] Furthermore, the node vector in the current game state knowledge graph is obtained by aggregating the information around the node, including:
[0022] Generate an adjacency matrix of the game status knowledge graph;
[0023] Based on the adjacency matrix, a game state knowledge graph triple (s, r, e) is generated; wherein s represents the head node, r represents the relationship, and e represents the tail node;
[0024] The learnable embedding vector v based on the head node s , the learnable embedding vector v of the relation s and the learnable embedding vector v of the tail node e , calculate the attention value α passed from the tail node e to the head node s e ; Wherein, the learnable embedding vector v s , the learnable embedding vector v s and the learned embedding vector v e Initialize based on Xavier uniform distribution;
[0025] Based on the attention value α e Aggregate learnable embedding vector v e , get the node vector of the head node s.
[0026] Furthermore, encoding the sequence based on a recurrent neural network to obtain a global representation and a local representation includes:
[0027] The global graph structure f(G t ) and the global representation U of the knowledge graph of the game state at the previous moment t-1 , calculate the global representation U of the knowledge graph of the current game state t ;
[0028] Based on the node vector of the current game state knowledge graph and the global representation U of the current game state knowledge graph t And the local representation of the relationship between the knowledge graph of the game state at the previous moment l t-1 (s, r), the local representation of the relationship between the game state knowledge graph at the time before calculation l t (s, r); wherein the local representation l t (s, r) focuses on the local information of the edge in the knowledge graph of the current game state;
[0029] Based on the node vector of the current game state knowledge graph and the global representation U of the current game state knowledge graph t And the local representation of the node in the knowledge graph of the game state at the previous moment l t-1 (s), the local representation of the node in the knowledge graph of the game state at the time before calculation l t (s); wherein the local representation l t (s) Focus on the local information of the nodes in the knowledge graph of the game status at the current moment.
[0030] Furthermore, based on the global representation, the local representation of the node, the local representation of the relationship, and the node vector in the knowledge graph of the game state at the current moment, the execution task at the current moment is generated, including:
[0031] Get the event G of the previous h steps t-h:t-1 ;
[0032] The event G is calculated based on the global representation t-h:t-1 The corresponding head entity s;
[0033] The node vector v based on the head entity s s and node local representation l t-1 (s), get the event G t-h:t-1 The corresponding relationship r;
[0034] The node vector v based on the head entity s s and the edge vector v of the relation r r and the local representation of the relationship t-1 (s,r), get the event G t-h:t-1 The corresponding tail node;
[0035] According to the event G t-h:t-1 The corresponding task operation node obtains the execution task at the current moment.
[0036] A knowledge graph-based intelligent generation system for game task requirements, the system comprising:
[0037] The knowledge graph generation module is used to generate a knowledge graph of the current game state based on the current game state of the war game; wherein the nodes in the game state knowledge graph include: frame nodes, entity nodes and task operation nodes;
[0038] The node vector calculation module is used to obtain the node vector in the current game status knowledge graph by aggregating the information around the node;
[0039] The global and local representation generation module is used to regard the game state knowledge graph at the current moment and the game state knowledge graph at the previous moment as a temporal sequence, and encode the sequence based on a recurrent neural network to obtain the global representation of the current moment, the local representation of the node, and the local representation of the relationship;
[0040] The task generation module is used to generate the execution task at the current moment based on the global representation, node local representation, relationship local representation and node vectors in the knowledge graph of the game status at the current moment.
[0041] An electronic device, characterized in that the electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements any of the above-mentioned methods for intelligently generating game task requirements based on knowledge graphs.
[0042] A computer-readable storage medium, characterized in that computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, any of the above-mentioned methods for intelligently generating game task requirements based on knowledge graphs is implemented.
[0043] Compared with the prior art, the present invention has the following beneficial effects:
[0044] 1) Representing the action details of the tasks in the war game as a knowledge graph and using a neural network to predict the time of the nodes can effectively enhance the interpretability of the model, be transparent to players, and enable them to understand the specific source and detailed execution plan of the next task;
[0045] 2) The model parameters are much smaller than those of deep reinforcement learning, which can reduce training time and hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 This is a flow chart of the method for intelligently generating game task requirements based on knowledge graph. DETAILED DESCRIPTION
[0047] The method is further described below through specific implementation modes in conjunction with the accompanying drawings.
[0048] like Figure 1 As shown, the present invention screens the training videos obtained from the war game, obtains high-quality playback videos, and uses the pysc2 package to extract the game status and execution operations of each video every 8 frames to obtain a json file. The task operation and unit type are mapped to a real number for easy training and representation. Then, a knowledge graph is constructed based on the obtained json file, with the frame node as the core, and the game details in the corresponding frame are connected to the frame node, and the frame nodes are also connected to each other to indicate the progress of the game. After that, the knowledge graph in the training set is used for training to obtain a task generation model. Finally, the task generation model obtained by training is used to complete the generation of the game task.
[0049] 1. Automatically obtain game status and execute operations.
[0050] The operation of obtaining the game status and executing the game in one embodiment of the present invention includes the following sub-steps:
[0051] 1a) Get the battle video and obtain the observation current_observation of the current state of the game interface every n frames t The operation set A corresponding to the role t This operation set A t Includes all valid actions that occurred from n frames ago to time t.
[0052] In a preferred embodiment, based on the battle video, the following conditions are used for screening:
[0053] ●The video file has no errors;
[0054] ● The duration of the simulation (calculated by the number of frames) is greater than or equal to 10,000;
[0055] ●There are only two types of deduction roles in the video file;
[0056] ●The deduction result of each character is victory or failure;
[0057] ●The average operations per minute (APM) of each character is greater than or equal to the set minimum APM10;
[0058] ●Each character's matching rating (MMR) is greater than or equal to the set minimum MMR of 1000.
[0059] 1b) In A t In the current_observation t-1 If there is no action in A, the empty task α0 is used as the target action by default.
[0060] 1c) When parsing to the end of the playback file, all (observation, action) pairs and deduction results (whether winning or not) are stored in a json data file.
[0061] 2. Represent the obtained game status and performed operations as a knowledge graph.
[0062] In the training preparation phase of this step, the obtained game status and performed operations are represented in the form of a knowledge graph using automated or manual methods, and represented according to the rules in the war game simulation.
[0063] An embodiment of the present invention includes the following sub-steps:
[0064] 2a) The video frame is used as the node Frame (frame number) and is connected to the previous frame node with the relationship Lapse (time elapsed) to indicate the progress of the game;
[0065] 2b) Connect the entities in the war game (such as Units, Buildings, Tech, and Resource) as corresponding nodes (entities in the knowledge graph) and frame nodes (entities in the knowledge graph) with the relationships Own, Observe, Train, Build, and Research (entity relationships in the knowledge graph) to indicate the information that the party owns and observes the enemy;
[0066] 2c) Connect the executed task operation Action to the frame node with the relationship Act, indicating the task operation executed in the current frame. The task operation includes: building a building, training a unit, researching a technology, building deformation and empty task.
[0067] 3. The trained model is used to generate tasks for the knowledge graph of the war game status.
[0068] This step generates tasks for the war game that needs to generate tasks. After the game state is represented as a knowledge graph according to the above steps, the knowledge graph is passed into the model in the form of an adjacency matrix. First, the attention pooling method is used to aggregate the information around the node in the knowledge graph at the current moment, and then the information of all time is passed into the recurrent neural network. The global representation and local representation of the moment are calculated and updated, and finally a linear softmax classifier is used to obtain the generated task. In general, the model consists of a linear layer, two GRU-based RNN layers, and three linear layers. In the experiment, the node dimension is set to 200. The dimension of the input feature vector of the first linear layer is 3*200 (experimental setting value), and the dimension of the output feature vector is 200. The purpose is to calculate the attention value around the node, and then use softmax to make the sum of the attention values around the node 1. After determining the node vector, two side-by-side RNN layers are connected to calculate the global representation and local representation respectively. The three linear layers that follow use the global representation and local representation to calculate the probability of the knowledge graph nodes and relationships at future moments, and finally the task operations can be obtained. The number of neurons in the fully connected layer that generate nodes and relationships is the total number of nodes and relationships at the current moment, respectively. The probability of dropout in the model is 0.5, the learning rate is 0.001, the training epochs is 40, the batch size is 128, and the optimizer used in the algorithm is Adam.
[0069] The process of generating intelligent game tasks based on knowledge graph in one embodiment of the present invention includes the following sub-steps:
[0070] 3a) After the game state is represented as a knowledge graph according to the above steps, the knowledge graph is passed into the model in the form of an adjacency matrix;
[0071] 3b) Use Xavier uniform distribution (xavier_uniform_) to initialize node vectors, relationship vectors, and attention weights. Xavier uniform distribution initializes the weights to be uniformly distributed in [-a, a], where In the specific implementation: for the initialization of the node vector, fan in is the dimension of the node vector (here 200), fan out is the total number of nodes; for the initialization of the relationship vector, fan in is the dimension of the relationship vector (here 200), fan out is twice the total number of relations (including forward and reverse relations); for the initialization of the attention weight vector, fan in is 1, fan out , is 200, which is the dimension of the attention weight vector. Xavier’s specific formula is as follows:
[0072]
[0073] 3c) Use the linear layer to learn the initialized knowledge graph. The result of the linear layer is the attention value provided by the surrounding nodes to the target node, and the learned vector is updated based on this. The knowledge graph triple is represented as (s, r, e) (head node, relationship, tail node). After a fully connected layer, the tanh activation function is used. Finally, the vector v of node s is s It is the result of adding all its neighboring nodes after attention calculation. Represents the set of nodes connected to node s by relationship r. is the learnable embedding vector of the node, where d is the dimension of the node, which is 200 as set above. is the trainable weight matrix, α e is the attention value that node e wants to pass to node s. The specific formula is as follows:
[0074] α e =softmax(w T tanh(W(v s ;v r ;v e )))
[0075]
[0076] 3d) After learning the node and relationship vectors, the RNN layer uses a gated recurrent unit to t and local variable l t (s) is learned and updated. The global representation Ut It captures the global information of the entire graph from the beginning to the time point t, which reflects the global trend of the upcoming events and can represent the distribution and association of all entities in the graph. In contrast, the local representation focuses more on each head node and relationship (s, r), which captures the specific knowledge closely related to these entities and relationships. t (s) and l t (s,r) represent the local representation of the head node s and (s,r) respectively. The local representation focuses on each node and its relationship, so the information is mainly obtained from the nodes adjacent to the node. The knowledge between the global representation and the local representation obtained from the knowledge graph complement each other, so that the present invention can more efficiently predict the generation process of the graph after time t. are all nodes related to s at the current time t, f is the attention pooling aggregator, f(G t ) is the global graph structure, which is the aggregation of all nodes at time t, and let The specific formula is as follows:
[0077] U t =RNN(f(G t ),U t-1 )
[0078]
[0079] 3e) After three linear layers, the probability of the head node is calculated, and then the relationship and corresponding tail node are generated according to the head node. is the learnable embedding vector of the head node s and the relation r, is the local representation of (s,r) at time t-1, l t-1 (s) is the local information of s at time t-1. is the global representation at time t-1, and are the parameters of the linear softmax classifier, where we assume that the event G at a certain time t t The probability of G depends on the event G in the previous h steps t-h:t-1 Given all past events G t-h:t-1 The present invention firstly analyzes the subject entity s t According to p(s t |G t-h:t-1 ) is sampled. Then according to p(r t |s,G t-h:t-1 ) generates a relation r t , and finally object e t By p(e t |s,r,G t-h:h-1) is generated. Finally, the task operation node obtained in the generated knowledge graph is the task to be executed. The specific formula is as follows, where : in the formula represents the splicing operation:
[0080]
[0081] 3f) Finally, if the task to be obtained is the task at the current moment, the task operation node is directly extracted from the generated knowledge graph. If it is necessary to continue to generate tasks at later moments, the task generation algorithm is re-executed using the generated knowledge graph until the moment when the task needs to be generated.
[0082] Although the specific contents, implementation algorithms and drawings of the present invention are disclosed for the purpose of illustration, the purpose is to help understand the contents of the present invention and implement them accordingly, it can be understood by those skilled in the art that various substitutions, changes and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. The present invention should not be limited to the contents disclosed in the best embodiment of this specification and the drawings, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.
Claims
1. A method for intelligently generating game task requirements based on knowledge graph, characterized in that: The method comprises: Based on the current game state of the war game simulation, a knowledge graph of the game state at the current moment is generated; wherein the nodes in the game state knowledge graph include: frame nodes, entity nodes and task operation nodes; By aggregating the information around the node, we can get the node vector in the knowledge graph of the current game status. The current game state knowledge graph and the previous game state knowledge graph are regarded as a temporal sequence, and the sequence is encoded based on a recurrent neural network to obtain the global representation of the current moment, the local representation of the node, and the local representation of the relationship; Based on the global representation, node local representation, relationship local representation and node vectors in the current game status knowledge graph, the execution task at the current moment is generated.
2. The method according to claim 1, characterized in that The method of generating a current game state knowledge graph based on the game state and execution operation in the war game simulation process includes: Obtaining the game state of the war game at the current moment; wherein the game state includes the entity of the war game at the current moment and the corresponding entity state; Set the valid operation set A between the previous moment and the current moment t , as the execution operation of the war game at the current moment; wherein the valid operation set A t The first action is the execution task at the previous moment; Generate the frame node of the current moment; The entity of the war game at the current moment is used as an entity node in the game state knowledge graph at the current moment, and the entity node is connected to the frame node based on the entity state of the war game at the current moment; Generate a task operation node based on the task category, and connect the task operation node to the frame node based on the execution operation of the war game at the current moment; Connect adjacent frame nodes in the time dimension to obtain the knowledge graph of the game status at the current moment.
3. The method according to claim 2, characterized in that The entities include: units, buildings, technologies and resources; the entity states include: owned, observed, trained, constructed and researched.
4. The method according to claim 2, characterized in that: The mission categories include: constructing buildings, training units, researching technology, building transformations, and empty missions.
5. The method according to claim 1, characterized in that The node vector in the current game state knowledge graph is obtained by aggregating the information around the node, including: Generate an adjacency matrix of the game status knowledge graph; Based on the adjacency matrix, a game state knowledge graph triple (s, r, e) is generated; wherein s represents the head node, r represents the relationship, and e represents the tail node; The learnable embedding vector v based on the head node s , the learnable embedding vector v of the relation s and the learnable embedding vector v of the tail node e , calculate the attention value α passed from the tail node e to the head node s e ; Wherein, the learnable embedding vector v s , the learnable embedding vector v s and the learned embedding vector v e Initialize based on Xavier uniform distribution; Based on the attention value α e Aggregate learnable embedding vector v e , get the node vector of the head node s.
6. The method according to claim 5, characterized in that The sequence is encoded based on a recurrent neural network to obtain a global representation and a local representation, including: The global graph structure f(G t ) and the global representation U of the knowledge graph of the game state at the previous moment t-1 , calculate the global representation U of the knowledge graph of the current game state t ; Based on the node vector of the current game state knowledge graph and the global representation U of the current game state knowledge graph t And the local representation of the relationship between the knowledge graph of the game state at the previous moment l t-1 (s,r), the local representation of the relationship between the game state knowledge graph at the time before calculation l t (s, r); wherein the local representation l t (s, r) focuses on the local information of the edge in the knowledge graph of the current game state; Based on the node vector of the current game state knowledge graph and the global representation U of the current game state knowledge graph t And the local representation of the node in the knowledge graph of the game state at the previous moment l t-1 (s), the local representation of the node in the knowledge graph of the game state at the time before calculation l t (s); wherein the local representation l t (s) Focus on the local information of the nodes in the knowledge graph of the game status at the current moment.
7. The method according to claim 5, characterized in that Based on the global representation, node local representation, relationship local representation, and node vectors in the current game state knowledge graph, the execution task at the current moment is generated, including: Get the event G of the previous h steps t-h:t-1 ; The event G is calculated based on the global representation t-h:t-1 The corresponding head entity s; The node vector v based on the head entity s s and node local representation l t-1 (s), get the event G t-h:t-1 The corresponding relationship r; The node vector v based on the head entity s s and the edge vector v of the relation r r and the local representation of the relationship t-1 (s,r), get the event G t-h:t-1 The corresponding tail node; According to the event G t-h:t-1 The corresponding task operation node obtains the execution task at the current moment.
8. A knowledge graph-based intelligent generation system for game task requirements, characterized in that: The system comprises: The knowledge graph generation module is used to generate a knowledge graph of the current game state based on the current game state of the war game; wherein the nodes in the game state knowledge graph include: frame nodes, entity nodes and task operation nodes; The node vector calculation module is used to obtain the node vector in the current game status knowledge graph by aggregating the information around the node; The global and local representation generation module is used to regard the game state knowledge graph at the current moment and the game state knowledge graph at the previous moment as a temporal sequence, and encode the sequence based on a recurrent neural network to obtain the global representation of the current moment, the local representation of the node, and the local representation of the relationship; The task generation module is used to generate the execution task at the current moment based on the global representation, node local representation, relationship local representation and node vectors in the knowledge graph of the game status at the current moment.
9. An electronic device, characterized in that: The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method for intelligently generating game task requirements based on knowledge graph as described in any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method for intelligently generating game task requirements based on a knowledge graph as described in any one of claims 1 to 7.