Game decision model training method and device, storage medium and electronic equipment

Abstract

The invention discloses a game decision model training method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a historical game video of a sample player, carrying out the data extraction of the historical game video, determining the game data of the sample player in a specified time period, and taking the game data as a training sample; and determining first decision information corresponding to a decision executed by the sample player in the game state corresponding to the training sample, and taking the first decision information as a first label of the training sample. And determining a first prompt text corresponding to the training sample, inputting the first prompt text and the training sample into the general large language model, and determining first information output by the general large language model. And taking the first label and the first information as the second label of the training sample, and training the to-be-trained game decision model according to the training sample and the second label, so that the trained game decision model can be used for determining the game decision according to the game data of the to-be-decided player, and the generation speed and accuracy of the game decision are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, storage medium and electronic device for training a game decision model. Background Art

[0002] With the continuous development of technology, machine learning is being used more and more widely, especially in decision-making games, such as real-time strategy games and card games.

[0003] At present, when playing decision-making games, players can directly select machine learning models to play games, that is, pre-trained machine learning models directly replace players. The machine learning model can determine the game decision at the next moment based on the current game environment, and directly control the game character corresponding to the player based on the game decision. Generally, the performance of the machine learning model will affect the speed and accuracy of the generation of game decisions, thereby affecting the development and results of the game. Therefore, how to train the game decision model to improve the speed and accuracy of the generation of game decisions is a very important issue.

[0004] Based on this, this specification provides a game decision model training method. Summary of the invention

[0005] This specification provides a game decision model training method, device, storage medium and electronic device to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions: This specification provides a game decision model training method, including: Get historical game videos of sample players; Extracting data from the historical game video, determining the game data of the sample player within a specified time period and using the data as a training sample, and determining first decision information corresponding to a decision executed by the sample player in a game state corresponding to the training sample and using the first label of the training sample; Determine a first prompt text corresponding to the training sample, input the first prompt text and the training sample into a universal large language model, and determine first information output by the universal large language model; Using the first annotation and the first information as a second annotation of the training sample; The game decision model to be trained is trained according to the training samples and the second annotations; the trained game decision model is used to determine the game decision according to the game data of the player to be decided; wherein the game decision model is a large language model, and the parameter amount of the game decision model is smaller than the parameter amount of the general large language model.

[0007] Optionally, extracting data from the historical game video, determining the game data of the sample player within a specified time period and using it as a training sample, and determining first decision information corresponding to a decision executed by the sample player in a game state corresponding to the training sample and using it as a first annotation of the training sample, specifically includes: Using a pre-set data extraction tool, extract data from the historical game video according to a preset time length, and determine the game data of the sample players in several time periods; From each game data, the game data of the sample player within a specified time period is determined and used as a training sample, and the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample is determined and used as a first annotation of the training sample.

[0008] Optionally, determining the game data of the sample player within a specified time period from each game data and using it as a training sample, and determining first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample, specifically includes: Determine the information content corresponding to each game data; According to the content of each information and the preset filtering rules, the game data are filtered; From the filtered game data, the game data of the sample player within a specified time period is determined and used as a training sample, and the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample is determined and used as a first annotation of the training sample.

[0009] Optionally, inputting the first prompt text and the training sample into a general large language model, and determining the first information output by the general large language model specifically includes: According to a preset template, the training sample is converted to determine a first text; Concatenate the first prompt text and the first text to obtain a second text; The second text is input into a general large language model, and first information output by the general large language model is determined.

[0010] Optionally, using the first annotation and the first information as a second annotation of the training sample specifically includes: Determining decision information included in the first information and using the decision information as second decision information; Determining a difference between the first annotation and the second decision information; When the difference is greater than a preset threshold, using the information in the first information except the second decision information and the first annotation as the second annotation of the training sample; When the difference is not greater than a preset threshold, the first information and the first annotation are used as a second annotation of the training sample.

[0011] Optionally, the first prompt text includes at least a game development status prompt and a game decision prompt; Determining a first prompt text corresponding to the training sample specifically includes: Displaying the training sample to an operator; In response to the input operation of the operator, a first prompt text corresponding to the training sample is determined.

[0012] Optionally, the method further comprises: Obtaining game data of the player to be decided and using it as the first data; Determine a second prompt text corresponding to the first data; Inputting the first data and the second prompt text into a trained game decision model to determine prediction information output by the game decision model; Determining game decision instructions based on the prediction information; According to the game decision instruction, the game character corresponding to the player to be decided is controlled.

[0013] This specification provides a game decision model training device, including: An acquisition module is used to obtain historical game videos of sample players; An extraction module, configured to extract data from the historical game video, determine the game data of the sample player within a specified time period, and use the data as a training sample, and determine first decision information corresponding to a decision executed by the sample player in a game state corresponding to the training sample, and use the first annotation of the training sample; A first determination module, configured to determine a first prompt text corresponding to the training sample, and input the first prompt text and the training sample into a universal large language model to determine first information output by the universal large language model; A second determining module, configured to use the first annotation and the first game information as a second annotation of the training sample; A training module is used to train the game decision model to be trained according to the training samples and the second annotations; the trained game decision model is used to determine the game decision according to the game data of the player to be decided; wherein the game decision model is a large language model.

[0014] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned game decision model training method is implemented.

[0015] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned game decision model training method when executing the program.

[0016] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects: The game decision model training method provided in this specification obtains the historical game video of the sample player, extracts data from the historical game video, determines the game data of the sample player within a specified time period, and uses it as a training sample, and determines the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample, and uses it as the first annotation of the training sample. Determine the first prompt text corresponding to the training sample, and input the first prompt text and the training sample into the universal large language model to determine the first information output by the universal large language model. The first annotation and the first information are used as the second annotation of the training sample, and the game decision model to be trained is trained according to the training sample and the second annotation.

[0017] It can be seen from the above method that when training the game decision model, the present application obtains the historical game video of the sample player, extracts data from the historical game video, determines the game data of the sample player within a specified time period, and uses it as a training sample, and determines the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample, and uses it as the first annotation of the training sample. Determine the first prompt text corresponding to the training sample, and input the first prompt text and the training sample into the general large language model to determine the first information output by the general large language model. Use the first annotation and the first information as the second annotation of the training sample, and train the game decision model to be trained based on the training sample and the second annotation, so that the trained game decision model can be used to determine the game decision based on the game data of the player to be decided, thereby improving the speed and accuracy of generating game decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings: Figure 1 A flowchart of a game decision model training method provided in this specification; Figure 2A schematic diagram of a game decision model to be trained provided in this specification; Figure 3 A schematic diagram of a training process of a game decision model provided in this specification; Figure 4 A schematic diagram of a game decision model application provided in this specification; Figure 5 A schematic diagram of a game decision model training device provided in this specification; Figure 6 A method corresponding to the Figure 1 Schematic diagram of the structure of an electronic device. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0020] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0021] Figure 1 A flowchart of a game decision model training method provided in this specification includes the following steps: S100: Obtain historical game videos of sample players.

[0022] In this specification, the device for training the game decision model can obtain the historical game videos of the sample players. The device for training the game decision model can be a server or an electronic device such as a desktop computer or a laptop computer. For the sake of ease of description, the following describes the game decision model training method provided in this specification with the server as the execution subject.

[0023] The above-mentioned sample players are players of decision-making type games, which may be real-time strategy type games, card type games, and of course, other decision-making type games, which are not specifically limited in this specification. The above-mentioned sample players may be real players, such as ordinary players, professional players, etc., and may also be agents used to execute decision-making type games, that is, any existing machine learning models for executing decision-making type games, which are not specifically limited in this specification. The above-mentioned historical game videos are historical replay data of sample players when playing decision-making type games, that is, historical replay files, which may be replay data of any game in the history of sample players, and which may be generated by the client of the decision-making type game played by the sample players. Of course, the historical game video may also be video data of sample players when playing decision-making type games, such as screen recording data, which are not specifically limited in this specification.

[0024] The above historical game videos can be pre-collected by the server. Specifically, the server can pre-collect the players' historical game videos from data sources such as official websites, clients, and forums of decision-making games. Of course, the server can also pre-collect the players' historical game videos from other data sources including historical game videos, which is not specifically limited in this specification. The data volume of the above historical game videos can be 10,000, and of course, it can also be other volumes, which is not specifically limited in this specification.

[0025] S102: Extract data from the historical game video, determine the game data of the sample player within a specified time period, and use it as a training sample; and determine first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample, and use it as a first annotation of the training sample.

[0026] In this specification, the server can extract data from historical game videos, determine the game data of sample players within a specified time period, and use it as a training sample, and determine the first decision information corresponding to the decision made by the sample player in the game state corresponding to the training sample, and use it as the first annotation of the training sample. Among them, the specified time period is pre-set, and the game data is all game-related data of the sample player within the specified time period. The game data may include game time and game resource information of the sample player, and may also include historical decision information, and may also include other information of the other party who is playing against the sample player in the game, which is not specifically limited in this specification. The above-mentioned game resource information may be information about resources owned by the sample player. For example, taking a real-time strategy game as an example, the game resource information may be information about minerals, gases, worker supplies, worker units, buildings, population, and technology owned by the sample player. The above historical decision information may be decision information corresponding to the decision that the sample player has executed within the specified time period. The decision is the decision executed by the sample player in the game state corresponding to the game data of the previous time period of the specified time period. The decision information may include resource management decisions. For example, taking a real-time strategy game as an example, the decision information may include information corresponding to decisions such as resource collection, economic expansion, building construction, scientific and technological research and development, unit production, and base defense. The game state is the game environment, that is, the game data corresponds to the game environment. The above other information of the other player who plays against the sample player may be information obtained by the sample player as the game develops. The other information may include the game resource information and historical decision information of the other player.

[0027] Specifically, the server may use a pre-set data extraction tool to extract data from historical game videos according to a preset time length, and determine the game data of sample players within a certain time period. Then, from each game data, the game data of the sample player within the specified time period is determined and used as a training sample, and the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample is determined and used as the first annotation of the training sample. Among them, the above-mentioned data extraction tool may be a pre-set algorithm or program, and of course, it may also be any existing data extraction tool, which is not specifically limited in this specification. The above-mentioned time length is pre-set, and the time length may be 1 second or 5 seconds, which is not specifically limited in this specification.

[0028] The above-mentioned designated time period may be any time period among the above-mentioned several time periods. Therefore, when determining the game data of the sample player in the designated time period from each game data and using it as a training sample, the server may randomly select a time period from each game data and use the game data of the sample player in the selected time period as a training sample. Subsequently, the server may use the first decision information corresponding to the decision made by the sample player in the game state corresponding to the game data of the sample player in the selected time period as the first annotation of the training sample. The first decision information is generally obtained from the game data of the next time period of the above-mentioned selected time period, that is, the game data of the next time period of the selected time period includes the decision made by the sample player in the game state corresponding to the game data in the selected time period.

[0029] In addition, the above-mentioned specified time period can also be any continuous preset number of time periods among the above-mentioned several time periods, and the preset number is pre-set, and the preset number can be 5. Therefore, when determining the game data of the sample player in the specified time period from each game data and using it as a training sample, the server can first divide each game data according to the preset number to obtain several data groups, and then randomly select a data group from each data group. The game data included in the selected data group is used as a training sample. Subsequently, the server can use the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the game data included in the selected data group as the first annotation of the training sample, and the first decision information is generally obtained from the game data of the next data group of the above-mentioned selected data group, that is, the game data of the next data group of the selected data group includes the decision executed by the sample player in the game state corresponding to the game data included in the selected data group. Of course, the server can also directly randomly determine the game data in a continuous preset number of time periods from each game data and use it as a training sample. Subsequently, the server can use the decision information included in the game data of the next time period of the determined continuous preset number of time periods, that is, the first decision information, as the first annotation of the training sample.

[0030] In this specification, in order to reduce the use of repeated training samples to train the game decision model to be trained, save training time, and improve training efficiency, so that the game decision model to be trained can learn more knowledge, thereby improving the accuracy of the trained game decision model. Therefore, the server can determine the information content corresponding to the game data in each time period, and the information content refers to the difference between the game data in any time period and the game data in the previous time period of the time period, that is, the difference between the game data in any two consecutive time periods can be the information content corresponding to the game data in the later time period of any two consecutive time periods. When the information content of the game data in a certain time period is low, it means that the difference between the game data in the time period and the game data in the previous time period is small. On the contrary, when the information content of the game data in a certain time period is high, it means that the difference between the game data in the time period and the game data in the previous time period is large.

[0031] Based on this, when determining the game data of the sample players within the specified time period from each game data and using it as a training sample, and determining the first decision information corresponding to the decision executed by the sample players in the game state corresponding to the training sample, the server can determine the information content corresponding to each game data, and filter each game data according to the information content and the preset filtering rules. From the filtered game data, determine the game data of the sample players within the specified time period and use it as a training sample, and determine the first decision information corresponding to the decision executed by the sample players in the game state corresponding to the training sample, and use it as the first label of the training sample.

[0032] The above information content represents the difference between the game data in any two consecutive time periods. Therefore, when determining the information content corresponding to each game data, the server can determine the game data in the previous time period of each time period in turn for the game data in each time period, and use it as the first game data, determine the difference between the game data in the time period and the first game data, and use it as the information content corresponding to the game data in the time period. The information content can be determined based on the similarity between the game data in the time period and the first game data, that is, the information content can be 1 minus the difference of the above similarity.

[0033] The above-mentioned filtering rules can be used to filter game data whose information content is less than a preset value. The preset value is a pre-set value. Therefore, when filtering each game data according to the preset filtering rules based on the information content, the server can filter each game data when the information content of the game data is less than the preset value, and when the information content of the game data is not less than the preset value, the game data is used as the filtered game data.

[0034] S104: Determine a first prompt text corresponding to the training sample, and input the first prompt text and the training sample into a general large language model to determine first information output by the general large language model.

[0035] In this specification, the server may determine the first prompt text corresponding to the training sample, and input the first prompt text and the training sample into the general large language model to determine the first information output by the general large language model. Among them, the first prompt text may be pre-set, and the first information is the analysis result obtained by the general large language model analyzing the training sample under the prompt of the first prompt text. The content included in the first information is related to the prompt content included in the first prompt text. The general large language model may be a model pre-trained by the server, or any existing large language model. The general large language model may be a GPT (Generative Pre-trained Transformer) model. Of course, it may also be a large language model of other architectures, which is not specifically limited in this specification. The general large language model may be a model obtained by pre-training on data from multiple fields.

[0036] The first prompt text at least includes a game development status prompt and a game decision prompt. The game development status prompt is used to prompt the general large language model to analyze the training sample to determine the current game development status. Therefore, the first information may include the game development status, which is obtained by analyzing the training sample by the general large language model. The game development status may include a game overview, a game stage, a game status, and important resources. The game overview may be a brief overview of the game status corresponding to the training sample. The game stage may be one of the early game, the middle game, and the late game. The game status may include resource status. For example, taking a real-time strategy game as an example, the game status may include the status of resources such as units, buildings, economy, and technology. The important resources may be important resources among the resources owned by the sample players.

[0037] The game decision prompt is used to prompt the general large language model to analyze the training sample to determine the decision that the sample player can execute in the game state corresponding to the training sample. Therefore, the first information may also include game decision information, which may include potential decisions that the sample player can execute and potential decisions of the other player who plays against the sample player. The game decision information can be obtained by analyzing the training sample by the general large model.

[0038] In addition, the first prompt text may also include game background prompts and game task prompts. The game background prompts may include the background and gameplay rules of the decision-making type game, and the game task prompts may include game task information such as task name and task content. Of course, the first prompt text may also include game character information corresponding to the sample player, and the game character information may include character type, name, gameplay, and other information.

[0039] In this specification, the language of the first prompt text and the first information may be Chinese, English, or other types of languages, which are not specifically limited in this specification. For example, assuming that the first prompt text is in Chinese, the first prompt text may be "You are a trained AI that can analyze and summarize game A. You understand the nuances and strategies of the Protoss.

[0040] Based on the summary of multiple rounds in the game, we hope that you can analyze the game process in a structured way. Your analysis should include the following aspects: 1. Game Overview: Provides a brief overview of the current situation based on all turns.

[0041] 2. Current game stage: Determine the game stage based on information from all rounds. Is it early, mid, or late game? 3. Our Current Situation: Describe our current situation in the following ways: 3.1 Units and Buildings: Analyze the status of our units and buildings. Ensure sufficient assimilators are built for gas collection, especially in expansions and bases with unused gas geysers.

[0042] 3.2 Economy: Assess our economy, including the gathering and use of resources. As a Protoss unit, prioritize gas gathering, and advanced technologies are highly dependent on gas.

[0043] 3.3 Technology: Describe the current status of our technology research, what technologies we have unlocked so far. Analyze our technology tree, indicating available and potential upgrades or units. Ensure that gas-intensive technologies are supported by sufficient assimilation structures.

[0044] 4. Our strategy: Infer our potential strategy based on our current situation and information from all parties.

[0045] 5. Enemy Strategy: Infer the enemy's potential strategy based on the available information.

[0046] 6. Key information: highlight the most important aspects of all rounds that have a significant impact on the game. "Of course, the first prompt text can also be in English, so I will not repeat it here.

[0047] In this specification, when determining the first prompt text corresponding to the training sample, the server may display the training sample to the operator, and determine the first prompt text corresponding to the training sample in response to the operator's input operation. Of course, the server may also directly respond to the operator's input operation to determine the first prompt text corresponding to the training sample, and the first prompt text is directly input by the operator and sent to the server.

[0048] In this specification, when the first prompt text and the training sample are input into the general large language model to determine the first information output by the general large language model, the server can directly splice the first prompt text and the training sample, and then input the spliced ​​result into the general large language model to determine the first information output by the general large language model.

[0049] S106: Use the first annotation and the first information as the second annotation of the training sample.

[0050] S108: Training the game decision model to be trained according to the training samples and the second annotations; the trained game decision model is used to determine the game decision according to the game data of the player to be decided; wherein the game decision model is a large language model, and the number of parameters of the game decision model is smaller than the number of parameters of the general large language model.

[0051] In this specification, the server may use the first annotation and the first information as the second annotation of the training sample. Then, according to the training sample and the second annotation, the game decision model to be trained is trained. Among them, the above-mentioned second annotation may include all the contents of the first annotation and the first information. When the first information only includes the game development status, the second annotation includes the first annotation and the first information. However, when the first information only includes the game decision information, or the first information includes the game development status and the game decision information, the server may determine the decision information included in the first information (i.e., the above-mentioned game decision information) and use it as the second decision information. Then determine the difference between the first annotation and the second decision information. When the difference is greater than a preset threshold, the information in the first information other than the second decision information and the first annotation are used as the second annotation of the training sample. When the difference is not greater than the preset threshold, the first information and the first annotation are used as the second annotation of the training sample. The preset threshold is pre-set. By taking the first decision information in the first annotation as the basis and the content included in the first information as the supplement, the game decision model to be trained is trained, so that the game decision model can better learn how to generate game decisions and improve the accuracy of the output results of the game decision model. In addition, due to the high degree of professionalism in professional fields, the proportion of data about specific scenarios in the training data of the general large language model is relatively low, resulting in low decision effectiveness of the general large language model. Compared with the general large language model, the above-mentioned game decision model is trained based on training samples determined by historical game videos of decision-making games. The trained game decision model can learn knowledge in specific fields, and the game decisions output by the game decision model are more accurate.

[0052] In addition, the game decision model to be trained may be a pre-trained large language model or an untrained large language model, which is not specifically limited in this specification. If the game decision model to be trained is a pre-trained large language model, the process of the server training the game decision model to be trained based on the training samples and the second annotations is actually a process of fine-tuning the game decision model to be trained based on the training samples and the second annotations.

[0053] When the game decision model to be trained is trained based on the training samples and the second annotation, Figure 2 As shown, Figure 2This is a schematic diagram of a training game decision model to be trained provided in this specification. The server can input the training sample and the first prompt text into the game decision model to be trained to determine the first result output by the game decision model to be trained. Then, according to the difference between the first result and the second annotation, the game decision model to be trained is trained. Among them, the first result may include the game development status and may also include game decision information, which is not specifically limited in this specification. The content included in the first result is related to the prompt content included in the first prompt text. The above process of inputting the training sample and the first prompt text into the game decision model to be trained and determining the first result output by the game decision model to be trained is similar to the process of inputting the first prompt text and the training sample into the general large language model in the above step S104 to determine the first information output by the general large language model, and will not be repeated here.

[0054] In this specification, the game decision model to be trained or the trained game decision model is a large language model, and the parameter amount of the game decision model to be trained or the trained game decision model is smaller than the parameter amount of the general large language model. In addition, the parameter amount of the game decision model to be trained or the trained game decision model can be 7B, the game decision model to be trained or the trained game decision model Qwen2 model, of course, can also be a large language model of other architectures, which is not specifically limited in this specification. In addition, since the parameter amount of the above-mentioned trained game decision model is smaller than the parameter amount of the general large language model, this makes the game decision model easier to deploy and apply than the general large language model, and at the same time, it also improves the speed at which the game decision model generates game decisions.

[0055] It can be seen from the above method that when training the game decision model, the server can obtain the historical game video of the sample player, extract data from the historical game video, determine the game data of the sample player within the specified time period, and use it as a training sample, and determine the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample, and use it as the first annotation of the training sample. Determine the first prompt text corresponding to the training sample, and input the first prompt text and the training sample into the general large language model to determine the first information output by the general large language model. Use the first annotation and the first information as the second annotation of the training sample, and train the game decision model to be trained according to the training sample and the second annotation, so that the trained game decision model can be used to determine the game decision according to the game data of the player to be decided. The training sample determined by the historical game video of the game based on the decision type is used to train the game decision model to be trained, so that the trained game decision model can learn the knowledge of a specific field, and then the game decision output by the game decision model is more accurate. At the same time, the game decision model is a large language model, and the number of parameters of the game decision model is smaller than that of the general large language model, so that the game decision model is easy to deploy and the speed of generating game decisions by the game decision model is improved.

[0056] In addition, the above-mentioned trained game decision model can also be used as an opponent of the game player, and the game level of the game player can be improved by playing against the game player, that is, the game decision model can be used to play against the game player, and the game player can be an ordinary player or a professional player. Of course, the trained game decision model can also be used as a game player to play against other game players, that is, the real game player can choose the game decision model to replace himself to play against other game players, so as to save the time of the real game player and improve the game level of the real game player. In addition, the trained game decision model can also be used to play against other machine learning models to assist other machine learning models in training or fine-tuning, which helps to improve the accuracy of the output results of other machine learning models.

[0057] Furthermore, the number of parameters of the game decision model is smaller than that of the general large language model, making it easy to deploy the game decision model on the client side of the game, and quickly generate game decisions to compete with game players, thereby improving the game level of game players, and also facilitating the replacement of real game players to compete with other game players. At the same time, the game decision model with a small number of parameters is easy to deploy, and the speed of generating game decisions is fast and the accuracy is high, so that the speed of training or fine-tuning other machine learning models based on the game decision model is fast, and the accuracy of the results output by other machine learning models after training or fine-tuning is high.

[0058] In this specification, in order to better splice the first prompt text and the training sample, so that the game decision model to be trained can better learn the characteristics of the training sample and the first prompt text, so as to better train the game decision model to be trained. Therefore, in the above step S104, when the first prompt text and the training sample are input into the general large language model to determine the first information output by the general large language model, the server can also convert the training sample according to the preset template to determine the first text. The first prompt text and the first text are spliced ​​to obtain the second text. The second text is input into the general large language model to determine the first information output by the general large language model. Among them, the above preset template can be pre-set, and the preset template is used to convert the training sample into text. The language of the above training sample, preset template, first text and second text can be Chinese, English, or other types of languages, which are not specifically limited in this specification. The above preset template may include fixed template content and several slots to be filled in. Specifically, the preset template may be "At game time [ ], our current game situation is as follows: Resources: [ ] - Game time: [ ] - Worker supply: [ ] - Minerals: [ ] - Gas: [ ] - Remaining supply: [ ] - Supply limit: [ ] - Use supply: [ ].", wherein the text portion in the above preset template is the fixed template content, and the above "[ ]" is the slot to be filled in, and the content in the slot to be filled in needs to be determined by the server based on the training sample. The server can convert the training sample according to the preset template to determine the first text, which can be generated by the server based on the content corresponding to the slot to be filled in and the preset template, that is, the text obtained after the content corresponding to the slot to be filled in is filled in the preset template. The content corresponding to the slot to be filled in is determined by the server based on the training sample. Therefore, the first text may be "At game time [00:00], our current game situation is as follows: Resources: [n] - Game time: [00:00] - Worker supply:

[12] - Minerals:

[50] - Gas: [0] - Remaining supply: [3] - Supply limit:

[15] - Used supply:

[12] ." It should be noted that the above preset model and the first text are only examples, and this specification does not limit the specific content of the template and the first text.

[0059] In this manual, if Figure 3 As shown, Figure 3 This is a schematic diagram of a training process of a game decision model provided in this specification. The server may first obtain historical game videos of sample players. Figure 3In the example, historical game videos are used as historical replay files. Data is extracted from historical game videos to determine the game data of sample players within a specified time period and used as training samples, and the first decision information corresponding to the decision made by the sample players in the game state corresponding to the training samples is determined and used as the first annotation of the training samples. The first prompt text corresponding to the training sample is determined, and the first prompt text and the training sample are input into the general large language model to determine the first information output by the general large language model. The first annotation and the first information are used as the second annotation of the training sample. The game decision model to be trained is trained according to the training sample, the first prompt text and the second annotation.

[0060] In this manual, the above Figure 1 The game decision model trained by the training method shown is a large language model with macro decision-making, that is, the game decision model can be used as a commander in the game to generate macro decisions in real time based on current game data, rather than micro instructions. For example, macro decisions can be upgrading buildings, developing population, and attacking, etc., and micro instructions can be attacking building A, advancing to point B, etc. The above game can be a simulation environment.

[0061] In this specification, after the game decision model is trained, the server can obtain the game data of the player to be decided and use it as the first data. Determine the second prompt text corresponding to the first data. Input the first data and the second prompt text into the trained game decision model to determine the prediction information output by the game decision model. According to the prediction information, determine the game decision instruction. According to the game decision instruction, control the game character corresponding to the player to be decided. Among them, the player to be decided can be a real game player, or a game decision model that replaces the real game player, or other machine learning models, which is not specifically limited in this specification. The second prompt text can be pre-set, and the second prompt text is the same as the content included in the first prompt text, and the process of determining the second prompt text can be similar to the process of determining the first prompt text, which will not be repeated here. The prediction information can include the game development status and the game decision information. The game decision instruction is an instruction generated according to the game decision information in the prediction information. After obtaining the game decision instruction, the server can directly control the game character corresponding to the player to be decided.

[0062] The process of inputting the first data and the second prompt text into the trained game decision model and determining the prediction information output by the game decision model can be as follows: Figure 4 As shown, Figure 4This is a schematic diagram of a game decision model application provided in this specification. Of course, the process of inputting the first data and the second prompt text into the trained game decision model to determine the prediction information output by the game decision model is also similar to the process of inputting the training sample and the first prompt text into the game decision model to be trained to determine the first result output by the game decision model to be trained, and will not be repeated here.

[0063] In this specification, after obtaining the trained game decision model, the performance of the game decision model can also be tested, and the performance may include the winning rate of the game decision model, that is, the winning rate of the game decision model when competing against other machine learning models. Specifically, the server can use the game's built-in AI (Artificial Intelligence) model and the general large language model to compete against the game decision model at different game difficulties to evaluate the game decision model to obtain an evaluation result, which is the winning rate of the game decision model and may also include the duration of the confrontation, that is, the duration of the game. The above-mentioned difficulty refers to the difficulty of the game, which may include very simple, simple, medium and difficult. It should be noted that the above-mentioned built-in AI model, that is, the AI ​​model built into the game, is not limited to any type of machine learning model, and the above-mentioned general large language model may be a GPT model.

[0064] The above evaluation results can be presented in the form of a table, as shown in Table 1. It should be noted that Table 1 is only an example. Table 1 shows the game time (i.e., time length) when the built-in AI model competes with the game decision model at very simple, simple, medium, and difficult difficulty levels, as well as the winning rate of the game decision model. In addition, Table 1 also shows the game time (i.e., time length) when the general large language model competes with the game decision model without limiting the game difficulty, as well as the winning rate of the game decision model.

[0065] Table 1

[0066] Table 1 above shows that the game time of the built-in AI model when playing against the game decision model at a very simple difficulty is 15 minutes, and the winning rate of the game decision model is 100%. The game time of the built-in AI model when playing against the game decision model at a simple difficulty is 15 minutes, and the winning rate of the game decision model is 100%. The game time of the built-in AI model when playing against the game decision model at a medium difficulty is 20 minutes, and the winning rate of the game decision model is 69%. The game time of the built-in AI model when playing against the game decision model at a difficult difficulty is 20 minutes, and the winning rate of the game decision model is 67%. The game time of the general large language model when playing against the game decision model without limiting the game difficulty is 20 minutes, and the winning rate of the game decision model is 65%. It should be noted that the above-mentioned general large language model takes GPT-4o as an example.

[0067] After obtaining the evaluation results, the server can continue to train the game decision model based on the evaluation results. Of course, the server can also directly deploy the game decision model to the target device. The target device can be the client or server where the game is located. In addition, the game data when the game decision model is applied can be obtained later, and the game decision model can be trained based on the game data.

[0068] The above are one or more implementation methods of this specification. Based on the same idea, this specification also provides a corresponding game decision model training device, such as Figure 5 shown.

[0069] Figure 5 A schematic diagram of a game decision model training device provided in this specification includes: An acquisition module 200 is used to acquire historical game videos of sample players; An extraction module 202 is used to extract data from the historical game video, determine the game data of the sample player within a specified time period, and use it as a training sample, and determine the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample, and use it as a first label of the training sample; A first determination module 204 is used to determine a first prompt text corresponding to the training sample, and input the first prompt text and the training sample into a universal large language model to determine first information output by the universal large language model; A second determining module 206, configured to use the first annotation and the first game information as a second annotation of the training sample; The training module 208 is used to train the game decision model to be trained according to the training samples and the second annotations; the trained game decision model is used to determine the game decision according to the game data of the player to be decided; wherein the game decision model is a large language model.

[0070] Optionally, the extraction module 202 is specifically used to use a pre-set data extraction tool to extract data from the historical game video according to a preset time length, and determine the game data of the sample player in several time periods; determine the game data of the sample player in a specified time period from each game data, and use it as a training sample; and determine the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample, and use it as the first annotation of the training sample.

[0071] Optionally, the extraction module 202 is specifically used to determine the information content corresponding to each game data; filter the game data according to the information content and preset filtering rules; determine the game data of the sample player within a specified time period from the filtered game data, and use it as a training sample, and determine the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample, and use it as the first annotation of the training sample.

[0072] Optionally, the first determination module 204 is specifically used to convert the training sample according to a preset template to determine the first text; concatenate the first prompt text and the first text to obtain a second text; input the second text into a universal large language model to determine the first information output by the universal large language model.

[0073] Optionally, the second determination module 206 is specifically used to determine the decision information included in the first information and use it as the second decision information; determine the difference between the first annotation and the second decision information; when the difference is greater than a preset threshold, use the information in the first information except the second decision information and the first annotation as the second annotation of the training sample; when the difference is not greater than the preset threshold, use the first information and the first annotation as the second annotation of the training sample.

[0074] Optionally, the first prompt text includes at least a game development status prompt and a game decision prompt; Optionally, the first determining module 204 is specifically configured to display the training sample to an operator; and determine a first prompt text corresponding to the training sample in response to an input operation of the operator.

[0075] Optionally, the device further comprises: The application module 210 is used to obtain the game data of the player to be decided and use it as the first data; determine the second prompt text corresponding to the first data; input the first data and the second prompt text into the trained game decision model to determine the prediction information output by the game decision model; determine the game decision instruction according to the prediction information; and control the game character corresponding to the player to be decided according to the game decision instruction.

[0076] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A game decision model training method is provided.

[0077] This manual also provides Figure 6 The one shown corresponds to Figure 1 A schematic diagram of the electronic device. Figure 6 As shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The game decision model training method.

[0078] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0079] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0080] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0081] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0082] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0083] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0087] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0088] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0089] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0090] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0091] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0093] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0094] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for training a game decision model, characterized in that: The method comprises: Get historical game videos of sample players; Extracting data from the historical game video, determining the game data of the sample player within a specified time period and using the data as a training sample, and determining first decision information corresponding to a decision executed by the sample player in a game state corresponding to the training sample and using the first label of the training sample; Determine a first prompt text corresponding to the training sample, input the first prompt text and the training sample into a universal large language model, and determine first information output by the universal large language model; Using the first annotation and the first information as a second annotation of the training sample; The game decision model to be trained is trained according to the training samples and the second annotations; the trained game decision model is used to determine the game decision according to the game data of the player to be decided; wherein the game decision model is a large language model, and the parameter amount of the game decision model is smaller than the parameter amount of the general large language model.

2. The method according to claim 1, characterized in that Extracting data from the historical game video, determining the game data of the sample player within a specified time period and using the data as a training sample, and determining first decision information corresponding to a decision executed by the sample player in a game state corresponding to the training sample and using the first annotation of the training sample, specifically includes: Using a pre-set data extraction tool, extract data from the historical game video according to a preset time length, and determine the game data of the sample players in several time periods; From each game data, the game data of the sample player within a specified time period is determined and used as a training sample, and the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample is determined and used as a first annotation of the training sample.

3. The method according to claim 2, characterized in that Determining the game data of the sample player within a specified time period from each game data as a training sample, and determining first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample, specifically includes: Determine the information content corresponding to each game data; According to the content of each information and the preset filtering rules, the game data are filtered; From the filtered game data, the game data of the sample player within a specified time period is determined and used as a training sample, and the first decision information corresponding to the decision executed by the sample player in the game state corresponding to the training sample is determined and used as a first annotation of the training sample.

4. The method according to claim 1, characterized in that Inputting the first prompt text and the training sample into a universal large language model, and determining first information output by the universal large language model specifically includes: According to a preset template, the training sample is converted to determine a first text; Concatenate the first prompt text and the first text to obtain a second text; The second text is input into a general large language model, and first information output by the general large language model is determined.

5. The method according to claim 1, characterized in that Using the first annotation and the first information as the second annotation of the training sample specifically includes: Determining decision information included in the first information and using the decision information as second decision information; Determining a difference between the first annotation and the second decision information; When the difference is greater than a preset threshold, using the information in the first information except the second decision information and the first annotation as the second annotation of the training sample; When the difference is not greater than a preset threshold, the first information and the first annotation are used as a second annotation of the training sample.

6. The method according to claim 1, characterized in that The first prompt text at least includes a game development status prompt and a game decision prompt; Determining a first prompt text corresponding to the training sample specifically includes: Displaying the training sample to an operator; In response to the input operation of the operator, a first prompt text corresponding to the training sample is determined.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining game data of the player to be decided and using it as the first data; Determine a second prompt text corresponding to the first data; Inputting the first data and the second prompt text into a trained game decision model to determine prediction information output by the game decision model; Determining game decision instructions based on the prediction information; According to the game decision instruction, the game character corresponding to the player to be decided is controlled.

8. A game decision model training device, characterized in that: include: An acquisition module is used to obtain historical game videos of sample players; An extraction module, configured to extract data from the historical game video, determine the game data of the sample player within a specified time period, and use the data as a training sample, and determine first decision information corresponding to a decision executed by the sample player in a game state corresponding to the training sample, and use the first annotation of the training sample; A first determination module, configured to determine a first prompt text corresponding to the training sample, and input the first prompt text and the training sample into a universal large language model to determine first information output by the universal large language model; A second determining module, configured to use the first annotation and the first game information as a second annotation of the training sample; A training module, used for training the game decision model to be trained according to the training samples and the second annotations; The trained game decision model is used to determine the game decision according to the game data of the player to be decided; wherein the game decision model is a large language model.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 7 is implemented.

Images