Regional gaming method, device, electronic device and storage medium for games
By setting game characters in the preset area and updating the AI model, the problem of inefficient training of AI models in asymmetric competitive games is solved, and resource conservation and personification are improved.
Patent Information
- Application Number
- CN202111479504.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-06
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-12-06
AI Technical Summary
In the prior art, AI model training for all scenarios consumes high performance, is inefficient, and has a long training cycle, making it difficult to effectively apply in asymmetric competitive games.
By setting the game characters in the preset area, using artificial intelligence models to play games, determining the action scores and execution probability of the target action, updating the game character model, using deep neural network to fit candidate actions, and determining the target action based on area feature information , and train within the specified range.
Reduces resource consumption for model training, improves the efficiency of model training, and enhances the degree of anthropomorphism of game AI in fixed scenarios.
Smart Images

Figure CN114159792B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of network technology, and in particular to a regional game method for a game, a regional game device for a game, an electronic device and a storage medium. Background Art
[0002] With the development of AI (Artificial Intelligence) technology, game AI is now common and mature in symmetrical competitive games such as Honor of Kings and DOTA, but is rarely used in asymmetrical competitive games.
[0003] Since an important way of competing in asymmetric competitive games is that the two sides compete in a specific area, the traditional AI model training method for the entire scene consumes a lot of performance, easily causes waste of resources, and has a long training cycle. Summary of the invention
[0004] An embodiment of the present invention provides a regional game method for a game to solve the problem of high performance consumption and low efficiency in training AI models for the entire scene in the prior art.
[0005] Correspondingly, an embodiment of the present invention also provides a regional gaming device for a game, so as to ensure the implementation and application of the above method.
[0006] In order to solve the above problems, an embodiment of the present invention discloses a regional gaming method for a game, wherein the method comprises:
[0007] The first game character and the second game character are arranged in a preset area of the game scene of the game; wherein the first game character plays a game with the second game character in a preset manner in the preset area; the second game character has a corresponding second game character model, and the second game character model is an artificial intelligence model that controls the action behavior of the second game character;
[0008] Determining an action score obtained by a target action of the second game character model, the target action having a corresponding execution probability;
[0009] Determine a sample action performed by the second game character controlled by the second game character model, wherein the sample action has a corresponding symbol parameter;
[0010] The second game character model is updated according to the action score, execution probability and symbol parameter of the target action.
[0011] Optionally, obstacles are placed in the area, and the first game character and the second game character are arranged in a preset area of the game scene of the game, including:
[0012] The first game character and the second game character are arranged in a designated range corresponding to the obstacle.
[0013] Optionally, the specified range is calculated as follows:
[0014] Taking the obstacle as the center of the circle, a first range is determined by a first preset radius;
[0015] Acquire a second radius, and use the obstacle as the center of the circle, where the second radius determines a second range;
[0016] Analyze the historical data of the game, and count the game coordinate points when the first game character and the second game character play the game in the historical data of the game;
[0017] Calculating a ratio between the number of the gaming coordinate points in the second range and the number of the gaming coordinate points in the first range;
[0018] When the ratio is less than a preset threshold, increasing the second radius, and returning to the step of determining the second range with the second radius taking the obstacle as the center of the circle;
[0019] When the ratio is greater than or equal to a preset threshold, the second range is determined as a designated range.
[0020] Optionally, the method further comprises:
[0021] Acquire regional characteristic information of the preset area;
[0022] Obtaining candidate actions of the second game character model;
[0023] A target action is determined from the candidate actions according to the regional feature information.
[0024] Optionally, determining a target action from the candidate actions according to the regional feature information includes:
[0025] Using a deep neural network to fit candidate actions of the second game character model;
[0026] The regional feature information is input into the deep neural network so that after the deep neural network obtains the execution probability of the candidate actions, a target action is determined from the candidate actions according to a preset action selection strategy and the execution probability.
[0027] Optionally, the area feature information includes at least one of environmental features, game character model features and image features; the environmental features include the state of the preset obstacles, the game character model features include at least one of skill cooldown time, attack cooldown time, movement speed, position, direction, talent and game character status; the image features include game character field of view information.
[0028] Optionally, updating the second game character model according to the action score, execution probability and symbol parameter of the target action includes:
[0029] When the game between the first game character and the second game character ends, the second game character model is updated according to the action score, execution probability and symbol parameter of the target action.
[0030] Optionally, the end of the game is determined by:
[0031] When the first game character hits the second game character, or the second game character triggers the obstacle to hit the first game character, it is determined that the game between the first game character and the second game character is over.
[0032] Optionally, updating the second game character model according to the action score, execution probability and symbol parameter of the target action includes:
[0033] Weighting the action score of the target action to obtain a weighted action score;
[0034] Adding the weighted action scores to obtain a total action score;
[0035] Constructing a loss function according to the total action score, the execution probability and the symbolic parameter;
[0036] Using a gradient descent algorithm to sequentially calculate the loss value of the target action according to the loss function;
[0037] Update the model parameters of the second game character model according to the loss value.
[0038] The embodiment of the present invention further discloses a regional gaming device for a game, the device comprising:
[0039] An area setting module, used to set the first game character and the second game character in a preset area of the game scene of the game; wherein the first game character plays a game with the second game character in a preset manner in the preset area; the second game character has a corresponding second game character model, and the second game character model is an artificial intelligence model that controls the action behavior of the second game character;
[0040] an action score determination module, used to determine an action score obtained by a target action of the second game character model, wherein the target action has a corresponding execution probability;
[0041] A sampling action determination module, used to determine a sampling action that the second game character model controls the second game character to perform, wherein the sampling action has a corresponding symbol parameter;
[0042] The role model updating module is used to update the second game role model according to the action score, execution probability and symbol parameter of the target action.
[0043] An embodiment of the present invention discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the regional game method of the game as described above when executed by the processor.
[0044] An embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the regional game method of the game described above are implemented.
[0045] The embodiments of the present invention include the following advantages:
[0046] In an embodiment of the present invention, by setting the first game character and the second game character in a preset area of the game scene, controlling the first game character and the second game character to play a game in a preset manner, recording the execution probability and action score corresponding to the target action of the second game character model, and determining the symbol parameters corresponding to the sampled action executed by the second game character model to control the second game character, and then updating the second game character model according to the action score, execution probability and symbol parameters of the target action. The second game character model can be trained in a specified area, reducing the resource consumption of model training and improving the efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 It is a flowchart of steps of an embodiment of a regional gaming method of a game of the present invention;
[0048] Figure 2 It is a schematic diagram of obstacles of an embodiment of a regional gaming method of a game of the present invention;
[0049] Figure 3 is a schematic diagram of gaming coordinate points for analyzing historical data according to an embodiment of the present invention;
[0050] Figure 4 is a schematic diagram of determining a designated gaming range according to an embodiment of the present invention;
[0051] Figure 5 is a schematic flow chart of determining a target action according to an embodiment of the present invention;
[0052] Figure 6 It is a structural block diagram of an embodiment of a regional gaming device for a game of the present invention. DETAILED DESCRIPTION
[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0054] The characteristic of asymmetric competitive games is that the number of competitors is unequal, and both sides compete in the same scene. For example, in an asymmetric competitive game, there are two roles, the supervisor and the survivor, and the game is played in 1v4, that is, 1 supervisor against 4 survivors. When matching games, if the number of supervisors and survivors is the same, it will inevitably lead to insufficient number of survivors. Therefore, AI survivors are needed to make up for the missing number of survivors. Since usually in asymmetric competitive games, the two sides of the competition mainly play around certain fixed objects and areas, the current conventional game AI implementation method will have the following problems: For example, the full-scene reinforcement learning algorithm used in symmetric competitive games has high computing power requirements and a long construction period; the use of supervised learning not only requires a lot of manual annotation, but also lacks a large number of high-quality player board game samples; and the current game AI has a relatively simple performance for fixed scenes and is not highly anthropomorphic.
[0055] In view of the above problems, an embodiment of the present invention provides a regional game method for a game to solve the problems in the prior art that the game AI has a single performance for fixed scenes, a low degree of anthropomorphism and a large configuration workload.
[0056] For ease of understanding, the asymmetric competitive game described above is taken as an example in the following description of the embodiments of the present invention, wherein the first game character is a supervisor and the second game character is a survivor.
[0057] Reference Figure 1 , shows a flowchart of a regional gaming method embodiment of a game of the present invention, and the embodiment of the present invention may include the following steps:
[0058] Step 101: arranging a first game character and a second game character in a preset area of the game scene; wherein the first game character plays a game with the second game character in a preset manner within the preset area; the second game character has a corresponding second game character model, and the second game character model is an artificial intelligence model that controls the action behavior of the second game character;
[0059] Among them, the first game character and the second game character are game characters that play games in the game scene, and specifically can be game characters that perform corresponding actions according to instructions in the game scene; the second game character model is an AI (artificial intelligence) model that controls the second game character to perform specific actions.
[0060] In a specific implementation, the first game character and the second game character can be set in advance in any area of the game scene, and the first game character can be controlled in a preset manner to play a game with the second game character in the area, wherein the preset method can be the use of AI, pre-set operation instructions or historical player operation instructions, etc.; the game method can include the first game character chasing the second game character in an area; the second game character model can be used to fit the actions of the second game character, and in the process of playing the game with the first game character, the second game character determines and executes the action through the second game character model.
[0061] In one example, there are cipher machines and obstacles in the game scene, and the game is played in a 1v4 mode, that is, one supervisor plays against four survivors. The way of the game is that the survivors need to repair all cipher machines in the shortest time, and the supervisor needs to catch all the survivors. The time for the survivors to repair the cipher machines is won by other survivors who are in direct confrontation with the supervisor. Therefore, the survivors who are directly confronting the supervisor need to delay as much time as possible, and making good use of obstacles to increase the distance between the supervisor and the survivors is the most effective way to delay time.
[0062] Among them, the preset AI method in the game can be used to control the supervisor, allowing it to automatically play games with the survivors; the AI model can be used to control the survivors, so that they can predict the supervisor's next behavior through the supervisor's behavior and perform corresponding actions to deal with the supervisor as much as possible. In the process of the game, the survivor's AI model is updated.
[0063] Step 102: determining an action score obtained by a target action of the second game character model, wherein the target action has a corresponding execution probability;
[0064] Among them, the AI model can be trained through reinforcement learning. Reinforcement learning is a machine learning method in which the reinforcement signal provided by the environment is used to evaluate the quality of the generated action.
[0065] In the specific implementation, a deep neural network can be used to fit all the actions of the survivors. A deep neural network (DNN) is a neural network with multiple hidden layers. According to the positions of different layers, the neural network layers inside the DNN can be divided into: input layer, hidden layer and output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. After fitting the actions of the survivors, the deep neural network can convert the actions of the survivors into execution probabilities according to the input parameters, and then determine a target action from the actions of the survivors and execute it according to the action selection strategy and execution probability.
[0066] Since reinforcement learning judges the quality of an action based on rewards, each action will be assigned an action score through parameters and preset rules to evaluate the quality of the action. The action score can be set and modified according to actual needs. A setting method for the action score is shown in the following table:
[0067]
[0068] In the above table, Reward is a reward item. It should be noted that in reinforcement learning, rewards are not only positive rewards, but also reverse rewards (i.e., negative incentives); Weight is the weight value (action score) corresponding to the reward item, and Desc is the description corresponding to the reward item. In a specific implementation, if the target action performed by the survivor successfully delays time, the specific action score can be calculated based on the delayed time. When the survivor is hit and injured by the supervisor while performing a target action, the action score of the target action is the Weight value corresponding to the hurt -1. It should be noted that the action score calculation can be a multiple of the Weight value, such as the action score for being hit and injured is -5. The specific score calculation method can be set according to actual needs, and the embodiment of the present invention does not limit this.
[0069] Step 103: determining a sampled action that the second game character model controls to be executed by the second game character, wherein the sampled action has a corresponding symbol parameter;
[0070] During the training process of the AI model, it is necessary to measure the gap between the AI model and the expected effect based on reference samples, and then update and optimize the AI model.
[0071] Among them, the sampling action is the action performed by the second game character in the actual scene. In the specific implementation, for different states, the second game character will perform a corresponding action based on the previously trained model and action selection strategy. We will collect the action as the sampling action.
[0072] The sampling action has a corresponding symbolic parameter, where the symbolic parameter may include a number, a symbol or other form of parameter. Specifically, 0 and 1 may be used to represent the execution status of the sampling action. For example, if the sampling action is for the survivor to move backward, that is, the action performed by the survivor is backward, then the symbolic parameter of the backward action is marked as 1, and the other actions that have not been executed are marked as 0.
[0073] Step 104: Update the second game character model according to the action score, execution probability and symbol parameter of the target action.
[0074] In the specific implementation, all action scores and execution probabilities of the survivor AI model during the game, as well as the symbolic parameters corresponding to the sampled actions, can be recorded. When the game is over, the total action score of all action scores can be calculated, and then the survivor AI model can be updated based on the total action score, execution probability and sampled actions.
[0075] In an embodiment of the present invention, by setting the first game character and the second game character in a preset area of the game scene, controlling the first game character and the second game character to play a game in a preset manner, recording the execution probability and action score corresponding to the target action of the second game character model, and determining the symbol parameter corresponding to the sampled action performed by the second game character, and then updating the second game character model according to the action score, execution probability and symbol parameter of the target action. The second game character model can be trained in a specified area, reducing the resource consumption of model training and improving the efficiency of model training.
[0076] In an exemplary embodiment, an obstacle is placed in the area, and the step 101 of placing the first game character and the second game character in a preset area of the game scene of the game includes:
[0077] The first game character and the second game character are arranged in a designated range corresponding to the obstacle.
[0078] In a specific implementation, obstacles may be placed in certain areas of the game scene, wherein the obstacles may have interactive functions, and the second game character may perform some specific operations on the obstacles through interaction, such as placing obstacles to block the first game character model, or climbing over the obstacles. Before the game starts, the first game character and the second game character may be set in advance within a specified range of a certain distance from the obstacle, so that the first game character and the second game character can play the game around the obstacle.
[0079] In one example, as in the above-mentioned asymmetric competitive game, the obstacle can be a baffle, referring to Figure 2 , showing an obstacle diagram of an embodiment of a regional gaming method of a game of the present invention; Figure 2 As shown, the supervisor and the survivors are set in a specified range at a certain distance from the baffle to play the game. The survivors can delay time around the obstacles and baffles in the middle, or they can put the baffle down in one direction when passing by to block the supervisor, while the supervisor can attack the survivors when approaching the survivors.
[0080] Since the positions of obstacles in different areas of the game are different, if the first game character and the second game character are placed at the same distance from the obstacles every time, the trained second game character will appear stiff and less humanized. Therefore, the designated range of the game between the first game character and the second game character can be set according to actual conditions.
[0081] In an exemplary embodiment, the specified range is calculated as follows:
[0082] Taking the obstacle as the center of the circle, a first range is determined by a first preset radius;
[0083] Acquire a second radius, and use the obstacle as the center of the circle, where the second radius determines a second range;
[0084] Analyze the historical data of the game, and count the game coordinate points when the first game character and the second game character play the game in the historical data of the game;
[0085] Calculating a ratio between the number of the gaming coordinate points in the second range and the number of the gaming coordinate points in the first range;
[0086] When the ratio is less than a preset threshold, increasing the second radius, and returning to the step of determining the second range with the second radius taking the obstacle as the center of the circle;
[0087] When the ratio is greater than or equal to a preset threshold, the second range is determined as a designated range.
[0088] In a specific implementation, the specified range can be determined by analyzing the historical player data of the game, wherein the first preset radius and the preset threshold are both preset fixed values, and the game coordinate point can be the coordinate point when the first game character attacks the second game character.
[0089] Reference Figure 3 , shows a schematic diagram of the gaming coordinate points of analyzing historical data according to an embodiment of the present invention; the points in the figure are gaming coordinate points, and the circle in the figure is determined by taking the obstacle as the center and the first preset radius as the radius, thus determining an initial range. Since the initial range is large, there may be many blank areas in it. Therefore, after determining an initial range, it is necessary to further determine the designated range with the most gaming coordinate points in the initial range, because the more gaming coordinate points there are, the more gaming occurs in this range. Therefore, determining the designated range can improve the degree of anthropomorphism of the second game character model training.
[0090] Reference Figure 4 , shows a schematic diagram of determining a designated gaming range according to an embodiment of the present invention; as shown in the figure, a point is a gaming coordinate point, an outer circle is a first range determined by a first preset radius with an obstacle as the center, and an inner circle is a second range determined by a second radius with an obstacle as the center. One way to determine the designated range is as follows:
[0091]
[0092] Among them, sum() is an addition operation, that is, the data in the brackets are added, and θ is a preset threshold. In the specific implementation, the first range is first determined by the first preset radius, and then the second radius is obtained to determine the second range, the historical data of the game is analyzed, and the game coordinate points when the first game character and the second game character play the game in the historical data of the game are counted. The ratio between the number of game coordinate points in the second range and the number of game coordinate points in the first range is calculated. If the ratio is less than the preset threshold θ, the value of the second radius is increased, the second range and the calculated ratio are re-determined until the ratio is greater than or equal to the preset threshold θ, and the second range is determined as the specified range. Among them, the preset threshold θ is a value less than or equal to 1, so the specified range is the same size as the first range when it is at its maximum. By adjusting the preset values of the first preset radius and the preset threshold, the size of the specified range can be adjusted.
[0093] In an embodiment of the present invention, by analyzing game history data, determining a designated range for the game between the first game character and the second game character based on the game history data and obstacles, and setting the first game character and the second game character to play the game in the designated range corresponding to the obstacle, the second game character model can be trained in a local area, avoiding excessive performance consumption of the model training in the entire scene, and can also make the second game character model training closer to reality and more humanized.
[0094] In an exemplary embodiment, the method further comprises:
[0095] Acquire regional characteristic information of the preset area;
[0096] Obtaining candidate actions of the second game character model;
[0097] A target action is determined from the candidate actions according to the regional feature information.
[0098] Among them, the regional feature information includes at least one of the environmental features, game character model features and image features; the environmental features include the state of the preset obstacles, the game character model features include at least one of the skill cooling time, attack cooling time, movement speed, position, direction, talent and game character status; the image features include the game character field of view information.
[0099] The candidate actions of the second game character model include at least one of forward, backward, left, right, left front, right front, left back, right back, interacting with obstacles, and climbing over obstacles.
[0100] Wherein, determining the target action from the candidate actions according to the regional feature information includes:
[0101] Using a deep neural network to fit candidate actions of the second game character model;
[0102] The regional feature information is input into the deep neural network so that after the deep neural network obtains the execution probability of the candidate actions, a target action is determined from the candidate actions according to a preset action selection strategy and the execution probability.
[0103] The following combination Figure 5 The target action is determined in the embodiment of the present invention, referring to Figure 5 , shows a schematic flow chart of determining a target action according to an embodiment of the present invention, which may specifically include the following steps:
[0104] Step 501: Input digital features and image features into a deep neural network;
[0105] First, a deep neural network is used to fit the candidate actions of the second game character model, and digital features and image features are input, wherein the digital features are the above-mentioned regional feature information, and the image features include the field of view information of the second game character model;
[0106] Step 502: Use CNN to process image features;
[0107] Since directly processing image features consumes a lot of resources and has poor effects, in the embodiment of the present invention, CNN (Convolutional Neural Networks) is used to identify and process image features; CNN can represent local images of an image as information, which can reduce the amount of calculation and improve the image processing effect;
[0108] Step 503, concatenating the digital features and the processed image features in the concat layer;
[0109] Step 504: Process the data in the fully connected layer;
[0110] Step 505: Use the softmax layer to output the target action probability;
[0111] Through the softmax layer, the execution probability of the candidate actions of the second game character model can be calculated, and then the target action can be determined from the candidate actions according to the preset action selection strategy and execution probability.
[0112] The deep neural network will calculate which candidate action should be executed under the current features based on the input features and parameters. For example, when the first game character model is identified from the image features as being in front, it should be moved away. At this time, the probability of executing the backward candidate action should be increased. Specifically, the candidate action with the highest probability of execution may be determined as the target action, or the target action may be determined according to a preset action selection strategy, wherein the action selection strategy may be a pre-set method for selecting the target action, or the target action may be selected according to a preset probability, for example, the candidate action with the highest probability of execution may be determined as the target action with a probability of 90%, and an action may be randomly determined from the candidate actions as the target action with a probability of 10%; or the target action may be selected according to a preset number of times, for example, in every 5 action selections, 4 times the candidate action with the highest probability of execution may be selected, and 1 time a random candidate action may be selected as the target action; the embodiment of the present invention does not limit this.
[0113] In an embodiment of the present invention, the candidate actions of the second game character model are fitted through a deep neural network, and after the execution probability of the candidate actions is obtained based on the acquired regional feature information, the target action is determined from the candidate actions according to the preset action selection strategy and execution probability. This can enable the second game model to execute a more suitable target action and improve the degree of anthropomorphism of the second game character.
[0114] In an exemplary embodiment, the step 104, updating the second game character model according to the action score, execution probability and symbol parameter of the target action, comprises:
[0115] When the game between the first game character and the second game character ends, the second game character model is updated according to the action score, execution probability and symbol parameter of the target action.
[0116] In a specific implementation, when each game ends, the second game character model can be updated. The end of the game is determined by:
[0117] When the first game character hits the second game character, or the second game character triggers the obstacle to hit the first game character, it is determined that the game between the first game character and the second game character is over.
[0118] In an embodiment of the present invention, a first game character and a second game character play a game in an area. When the first game character hits the second game character, or the second game character triggers an obstacle to hit the first game character, it is determined that the game between the first game character and the second game character is over. After each game, the second game character model is updated, which can avoid excessive performance consumption of the training model of the entire scene and improve the efficiency of model training.
[0119] In an exemplary embodiment, the step 104, updating the second game character model according to the action score, execution probability and symbol parameter of the target action, comprises:
[0120] Weighting the action score of the target action to obtain a weighted action score;
[0121] Adding the weighted action scores to obtain a total action score;
[0122] Constructing a loss function according to the total action score, the execution probability and the symbolic parameter;
[0123] Using a gradient descent algorithm to sequentially calculate the loss value of the target action according to the loss function;
[0124] Update the model parameters of the second game character model according to the loss value.
[0125] After the game is over, the second game character model will be updated. Assuming that the current state is state1, where state1 is a certain environmental feature information, the execution probability of the second game character model for the candidate actions through the deep neural network is (0.2, 0.7, ..., 0.03), and the target action with the highest execution probability is determined, as shown in the following table:
[0126]
[0127] The sampled action is the action actually performed by the second game character in the state 1. As shown in the table above, when the sampled action is backward, the symbol parameter of backward is 1, and other actions are 0. The symbol parameter is used to mark the sampled action. For example, when the sampled action is forward, forward is marked as 1, and other actions are marked as 0, which can be expressed as 1000.
[0128] In one example, as shown in the table above, assuming that the candidate actions are forward, backward, left and right, and the current sampling action of the second game character is backward, the symbol parameters of the four actions are 0100, and the execution probabilities of the model output for the above four actions are 0.2, 0.7, 0.3 and 0.03 respectively; if you need to evaluate the quality of this action, you need to calculate the total action score G (ie G value) corresponding to the action after the game is over, and then use the G value to measure whether the execution probability of the model output should approach or move away from the symbol parameters.
[0129] In the specific implementation, for example, in a certain state s1, action a is executed, but the goodness of executing action a in state s1 can only be evaluated after the game is over. After executing action a, the second game character may also execute multiple target actions, and action a will affect subsequent actions. Therefore, it is necessary to calculate the total action score of the action scores corresponding to action a and its subsequent actions, and evaluate whether executing action a in state s1 is a good choice based on the total action score of action a. One way to calculate the total action score is as follows:
[0130] G t =R t +γR t+1 +γ 2 R t+2 +...+γ N R t+N
[0131] Among them, G t is the total action score, R t is the action score corresponding to each target action, γ is the discount factor, γ∈(0,1), Rt+N It is the action score corresponding to the target action N steps after the current target action. Since the influence of the currently executed action on the later executed actions will become smaller and smaller, the discount factor γ is used to weight the later executed actions. The specific calculation method is to calculate the total action score in the order of target action execution, and calculate the total action score from the back to the front.
[0132] After calculating the total action score, a loss function can be constructed based on the total action score, execution probability, and symbolic parameters. The loss function is an operational function used to measure the degree of difference between the model's predicted value (execution probability) and the true value (expected value). The loss function is mainly used in the training phase of the model. After each batch of training data is fed into the model, the predicted value is output through forward propagation, and then the loss function calculates the difference between the predicted value and the true value, that is, the loss value. A loss function in an embodiment of the present invention is as follows:
[0133] loss=-G*∑y*log(p)
[0134] Among them, G is the total action score, y is the symbolic parameter, and p is the execution probability output by the model.
[0135] The gradient descent algorithm is used to calculate the loss value of the target action in turn according to the loss function. After obtaining the loss value, the second game role model updates each model parameter through back propagation to reduce the loss between the true value and the predicted value, that is, when the G of an action is positive in a certain state, the execution probability of the target action output by the second game role model is increased, and when G is negative, the execution probability of the target action output by the second game role model is reduced. Finally, the predicted value output by the second game role model is brought closer to the true value, thereby achieving the purpose of training the second game role model. Specifically, the value of increasing or decreasing the execution probability of the target action output by the second game role model can be: when the G value is positive and the larger it is, the larger the value of the execution probability of the target action output by the second game role model is, and the closer the execution probability of the action is to the symbol parameter; when the G value is negative and the negative value is larger, the smaller the value of the execution probability of the target action output by the second game role model is, and the farther the execution probability of the action is from the symbol parameter.
[0136] In the embodiment of the present invention, by setting the first game character model and the second game character model in the area, controlling the first game character model and the second game character model to play a game in a preset manner, recording the execution probability and action score corresponding to the target action of the second game character model, and determining the symbol parameter corresponding to the sampled action performed by the second game character, and then updating the second game character model according to the action score, execution probability and symbol parameter of the target action. The second game character model can be trained in the specified area, reducing the resource consumption of model training and improving the efficiency of model training.
[0137] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0138] Reference Figure 6 , shows a structural block diagram of an embodiment of a regional gaming device for a game of the present invention, and the embodiment of the present invention may specifically include the following modules:
[0139] The area setting module 601 is used to set the first game character and the second game character in a preset area of the game scene of the game; wherein the first game character plays a game with the second game character in a preset manner in the preset area; the second game character has a corresponding second game character model, and the second game character model is an artificial intelligence model that controls the action behavior of the second game character;
[0140] An action score determination module 602 is used to determine an action score obtained by a target action of the second game character model, wherein the target action has a corresponding execution probability;
[0141] A sampling action determination module 603 is used to determine a sampling action that the second game character model controls the second game character to perform, and the sampling action has a corresponding symbol parameter;
[0142] The role model updating module 604 is used to update the second game role model according to the action score, execution probability and symbol parameter of the target action.
[0143] In an exemplary embodiment, the area has an obstacle, and the area setting module 601 is used to set the first game character and the second game character in a specified range corresponding to the obstacle.
[0144] In an exemplary embodiment, the device also includes: a designated range determination module, which is used to determine the first range with the obstacle as the center and a first preset radius; obtain a second radius, and determine the second range with the obstacle as the center and the second radius; analyze the historical data of the game, and count the game coordinate points when the first game character and the second game character play the game in the historical data of the game; calculate the ratio between the number of the game coordinate points in the second range and the number of the game coordinate points in the first range; when the ratio is less than a preset threshold, increase the second radius, and return to execute the step of determining the second range with the obstacle as the center and the second radius; when the ratio is greater than or equal to the preset threshold, determine the second range as the designated range.
[0145] In an exemplary embodiment, the device further includes: a target action determination module, which is used to obtain regional characteristic information of the area; obtain candidate actions of the second game character model; and determine a target action from the candidate actions based on the regional characteristic information.
[0146] In an exemplary embodiment, the action score determination module 602 is used to fit the candidate actions of the second game character model using a deep neural network; the regional feature information is input into the deep neural network so that after the deep neural network obtains the execution probability of the candidate actions, the target action is determined from the candidate actions according to a preset action selection strategy and the execution probability.
[0147] In an exemplary embodiment, the area feature information includes at least one of environmental features, game character model features and image features; the environmental features include the state of the preset obstacles, the game character model features include at least one of skill cooldown time, attack cooldown time, movement speed, position, direction, talent and game character status; the image features include game character field of view information.
[0148] In an exemplary embodiment, the role model updating module 604 is used to update the second game role model according to the action score, execution probability and symbol parameter of the target action when the game between the first game role and the second game role ends.
[0149] In an exemplary embodiment, the device further includes: a game end determination module, which is used to determine that the game between the first game character and the second game character is over when the first game character hits the second game character, or the second game character triggers the obstacle to hit the first game character.
[0150] In an exemplary embodiment, the character model update module 604 is used to weight the action score of the target action to obtain a weighted action score; add the weighted action scores to obtain a total action score; construct a loss function based on the total action score, the execution probability and the symbol parameter; use a gradient descent algorithm to calculate the loss value of the target action in sequence according to the loss function; and update the model parameters of the second game character model according to the loss value.
[0151] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0152] An embodiment of the present invention discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the steps described in the regional game method embodiment of the above game are implemented.
[0153] An embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps described in the regional game method embodiment of the above game are implemented.
[0154] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0155] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0156] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0157] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0158] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0159] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0160] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0161] The regional game method of a game, the regional game device of a game, the electronic device and the storage medium provided by the present invention are introduced in detail above. The principle and implementation mode of the present invention are explained in this article by using specific examples. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation mode and the scope of application. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A regional gaming method of a game, characterized in that: The method comprises: The first game character and the second game character are arranged in a preset area of the game scene of the game; wherein the first game character plays a game with the second game character in a preset manner in the preset area; the second game character has a corresponding second game character model, and the second game character model is an artificial intelligence model that controls the action behavior of the second game character; Determining an action score obtained by a target action of the second game character model, the target action having a corresponding execution probability; Determine a sample action performed by the second game character controlled by the second game character model, wherein the sample action has a corresponding symbol parameter; Update the second game character model according to the action score, execution probability and symbol parameter of the target action; Wherein, obstacles are placed in the area, and the first game character and the second game character are arranged in a preset area of the game scene of the game, including: The first game character and the second game character are arranged in a designated range corresponding to the obstacle.
2. The method according to claim 1, characterized in that The specified range is calculated as follows: Taking the obstacle as the center of the circle, a first range is determined by a first preset radius; Acquire a second radius, and use the obstacle as the center of the circle, where the second radius determines a second range; Analyze the historical data of the game, and count the game coordinate points when the first game character and the second game character play the game in the historical data of the game; Calculating a ratio between the number of the gaming coordinate points in the second range and the number of the gaming coordinate points in the first range; When the ratio is less than a preset threshold, increasing the second radius, and returning to the step of determining the second range with the second radius taking the obstacle as the center of the circle; When the ratio is greater than or equal to a preset threshold, the second range is determined as a designated range.
3. The method according to claim 1, characterized in that The method further comprises: Acquire regional characteristic information of the preset area; Obtaining candidate actions of the second game character model; A target action is determined from the candidate actions according to the regional feature information.
4. The method according to claim 3, characterized in that The determining a target action from the candidate actions according to the regional feature information includes: Using a deep neural network to fit candidate actions of the second game character model; The regional feature information is input into the deep neural network so that after the deep neural network obtains the execution probability of the candidate actions, a target action is determined from the candidate actions according to a preset action selection strategy and the execution probability.
5. The method according to claim 1, characterized in that The regional feature information includes at least one of environmental features, game character model features and image features; the environmental features include the state of the obstacles, the game character model features include at least one of skill cooldown time, attack cooldown time, movement speed, position, direction, talent and game character state; the image features include game character field of view information.
6. The method according to claim 1, characterized in that The updating of the second game character model according to the action score, execution probability and symbol parameter of the target action comprises: When the game between the first game character and the second game character ends, the second game character model is updated according to the action score, execution probability and symbol parameter of the target action.
7. The method according to claim 6, characterized in that The end of the game is determined by: When the first game character hits the second game character, or the second game character triggers the obstacle to hit the first game character, it is determined that the game between the first game character and the second game character is over.
8. The method according to claim 1, characterized in that The updating of the second game character model according to the action score, execution probability and symbol parameter of the target action comprises: Weighting the action score of the target action to obtain a weighted action score; Adding the weighted action scores to obtain a total action score; Constructing a loss function according to the total action score, the execution probability and the symbolic parameter; Using a gradient descent algorithm to sequentially calculate the loss value of the target action according to the loss function; Update the model parameters of the second game character model according to the loss value.
9. A regional gaming device for a game, characterized in that: The device comprises: An area setting module, used to set the first game character and the second game character in a preset area of the game scene of the game; wherein the first game character plays a game with the second game character in a preset manner in the preset area; the second game character has a corresponding second game character model, and the second game character model is an artificial intelligence model that controls the action behavior of the second game character; an action score determination module, used to determine an action score obtained by a target action of the second game character model, wherein the target action has a corresponding execution probability; A sampling action determination module, used to determine a sampling action that the second game character model controls the second game character to perform, wherein the sampling action has a corresponding symbol parameter; A role model updating module, used for updating the second game role model according to the action score, execution probability and symbol parameter of the target action; The area has an obstacle, and the area setting module is used to set the first game character and the second game character in a specified range corresponding to the obstacle.
10. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the regional gaming method of the game as claimed in any one of claims 1 to 8 are implemented.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the regional gaming method of the game according to any one of claims 1 to 8 are implemented.
