A method, device, equipment and medium for controlling a sports vehicle in a game

Abstract

The application provides a game motion carrier control method, a control device, equipment and a medium. The control method comprises the following steps: acquiring a direction adjustment instruction issued by a target player for a motion carrier and operation data of the motion carrier in real time; inputting the direction adjustment instruction and the operation data into a trained action prediction model to predict a reward value of each action performed by the motion carrier; selecting a target action from a plurality of actions according to a historical operation habit of the target player and the reward value of each action; and controlling the motion carrier to perform the target action in a game scene.

Description

Technical Field

[0001] The present application relates to the field of game control, and in particular to a method, a control device, a device, and a medium for controlling a moving vehicle in a game. Background Art

[0002] With the advancement of technology and the improvement of mobile phone performance, online games have gradually become part of people's lives. Among them, online racing games are gaining popularity due to their thrilling speed. However, racing games place a high premium on reaction speed, making failure easy without prior training. Therefore, to help novice players quickly adapt to racing games, games offer automatic modes, providing more training and improving their adaptability. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method, device, equipment and medium for controlling a sports vehicle in a game, so as to solve the problem in the prior art that the target action of the sports vehicle controlled by the target player cannot be accurately predicted.

[0004] In a first aspect, embodiments of the present application provide a method for controlling a sports vehicle in a game. A terminal device provides a graphical user interface, wherein the graphical user interface includes at least a portion of a game scene and a sports vehicle controlled by a target player within the game scene. The control method includes:

[0005] Real-time acquisition of direction adjustment instructions issued by the target player to the sports vehicle and operation data of the sports vehicle;

[0006] Inputting the direction adjustment instruction and the operation data into a trained action prediction model to predict the reward value of each action performed by the motion vehicle;

[0007] Selecting a target action from the plurality of actions based on the target player's historical operating habits and the reward value of each action;

[0008] Control the motion vehicle to perform the target action in the game scene.

[0009] Optionally, the action prediction model is trained through the following steps:

[0010] Obtaining a set of action training samples; the set of action training samples includes training samples obtained from at least two players; each training sample includes historical operating data of a motion vehicle controlled by the player and historical direction adjustment instructions issued by the player for the motion vehicle;

[0011] For each training data, the historical operation data and the historical direction adjustment instructions are input into an offline reinforcement learning network in the action prediction model to determine the reward value of the action represented by the training data; the historical operation data and the historical direction adjustment instructions in the training data are then used as inputs to a tree model to be trained in the action prediction model, and the reward values ​​in the training data are used as outputs of the tree model to be trained in the action prediction model, so as to train the tree model to be trained in the action prediction model;

[0012] The offline reinforcement learning network in the action prediction model and the trained tree model in the action prediction model constitute a trained action prediction model.

[0013] Optionally, the action training sample set is obtained through the following steps:

[0014] Acquiring, in manual mode, first historical operating data of a motion vehicle controlled by the player and a first historical direction adjustment instruction issued by the player for the motion vehicle;

[0015] Acquire second historical running data of the sports vehicle controlled by the player in automatic mode.

[0016] Optionally, selecting a target action from a plurality of actions based on the historical operation habits of the target player and the reward value of each action includes:

[0017] Determining a sensitivity function that matches the target player based on the target player's historical operating habits;

[0018] A target action is selected from the plurality of actions using the sensitivity function and the reward value of each action.

[0019] Optionally, determining a sensitivity function that matches the target player based on the target player's historical operating habits includes:

[0020] Acquire a plurality of consecutive third historical operation data of the sports vehicle controlled by the target player in the manual mode and a real action label corresponding to each third historical operation data;

[0021] determining a plurality of candidate parameter combinations from a sensitivity parameter range of the sensitivity function to be adjusted based on a plurality of third consecutive historical operating data;

[0022] Determining a target parameter combination from a plurality of candidate parameter combinations using the third historical operation data and a real action label corresponding to each third historical operation data;

[0023] The target parameter combination is used to adjust the sensitivity parameters of the sensitivity function to be adjusted, and a sensitivity function that matches the target player is determined.

[0024] Optionally, the determining a target parameter combination from a plurality of candidate parameter combinations by using the third historical operation data and the real action label corresponding to each third historical operation data includes:

[0025] For each candidate parameter combination, using the third historical operation data and the candidate sensitivity function set with the candidate parameter combination, determine a virtual action label corresponding to each third historical operation data;

[0026] For each candidate parameter combination, calculating a plurality of virtual action labels corresponding to the candidate parameter combination and a real action label corresponding to each third historical operation data, and determining a degree of match between each candidate parameter combination and the historical operation habits of the target player;

[0027] The parameter combination with the highest matching degree is determined as the target parameter combination.

[0028] Optionally, the control method further includes:

[0029] Obtaining route data of the position of the sports vehicle in the game scene;

[0030] Controlling the motion vehicle to perform the target action in the game scene includes:

[0031] If the target action satisfies the execution condition under the route data, controlling the sports vehicle to execute the target action in the game scene;

[0032] If the target action does not meet the execution conditions under the route data, the sports vehicle is controlled to run according to the original action in the game scene.

[0033] In a second aspect, embodiments of the present application provide a device for controlling a sports vehicle in a game. The device provides a graphical user interface (GUI) via a terminal device. The GUI includes at least a portion of a game scene and a sports vehicle controlled by a target player within the game scene. The device includes:

[0034] An acquisition module, configured to acquire, in real time, the direction adjustment instruction issued by the target player to the sports vehicle and the operation data of the sports vehicle;

[0035] a prediction module, configured to input the direction adjustment instruction and the operation data into a trained action prediction model to predict the reward value of each action performed by the vehicle;

[0036] A selection module, configured to select a target action from a plurality of actions based on the historical operation habits of the target player and the reward value of each action;

[0037] A control module is used to control the sports vehicle to perform the target action in the game scene.

[0038] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are executed.

[0040] The embodiment of the present application proposes a method for controlling a sports vehicle in a game. First, the direction adjustment instructions issued by the target player for the sports vehicle and the operating data of the sports vehicle are obtained in real time; second, the direction adjustment instructions and the operating data are input into a trained action prediction model to predict the reward value of each action performed by the sports vehicle; third, a target action is selected from a plurality of actions based on the historical operating habits of the target player and the reward value of each action; finally, the sports vehicle is controlled to perform the target action in the game scene.

[0041] In some embodiments, a trained action prediction model is used to predict the reward value of each action of a sports vehicle controlled by a player. The predicted reward value of each action is more in line with the operating habits of most players. In order to make the prediction result more accurate, the predicted reward value of each action is personalized based on the historical operating habits of a single target player, so that the final target action is more in line with the operating intention of the target player, thereby improving the accuracy of determining the target action.

[0042] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0044] Figure 1A flowchart of a method for controlling a moving vehicle in a game provided by an embodiment of the present application;

[0045] Figure 2 A flowchart of a method for training an action prediction model provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a control device for a sports vehicle in a game provided by an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0049] In the current manual mode of online racing games, the terminal device's graphical user interface provides directional controls (left and right) and drift controls. Players can touch the directional controls to issue directional adjustment commands to the vehicle, causing it to adjust its direction within the game scene. Alternatively, they can touch the drift controls to issue drift commands to the vehicle, causing it to drift within the game scene. In the automatic mode of online racing games, the terminal device's graphical user interface only provides directional controls. Players can touch the directional controls to issue directional adjustment commands to the vehicle, causing it to adjust its direction accordingly. Whether the vehicle needs to drift is determined by the system's behavior tree based on the vehicle's current operating data and the directional adjustment commands sent by the player.

[0050] In the prior art, whether a vehicle needs to perform drifting in automatic mode can be determined using a behavior tree algorithm in the system. However, traditional game control methods using behavior tree algorithms lack interactive features with the player because the parameters and structure in the behavior tree are manually set. This makes it impossible to accurately identify the player's intentions, resulting in inaccurate vehicle control. The present application provides a method, control device, equipment, and medium for controlling a vehicle in a game, which are described below through examples.

[0051] In one embodiment of the present disclosure, the method for controlling a moving vehicle in a game can be executed on a local terminal device or a server. When the method for controlling a moving vehicle in a game is executed on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0052] In an optional embodiment, various cloud applications can be run under the cloud interaction system, such as cloud games. Taking cloud games as an example, cloud games refer to a gaming method based on cloud computing. In the cloud game operation mode, the operating body of the game program and the main body of the game screen presentation are separated. The storage and operation of the control method of the sports vehicle in the game are completed on the cloud game server. The role of the client device is to receive and send data and present the game screen. For example, the client device can be a display device with data transmission function close to the user side, such as a mobile terminal, TV, computer, PDA, etc.; but the cloud game server in the cloud is responsible for information processing. When playing the game, the player operates the client device to send operation instructions to the cloud game server. The cloud game server runs the game according to the operation instructions, encodes and compresses the game screen and other data, and returns it to the client device through the network. Finally, the client device decodes and outputs the game screen.

[0053] In an optional embodiment, taking a game as an example, a local terminal device stores a game program and is used to present the game screen. The local terminal device is used to interact with the player through a graphical user interface, that is, conventionally downloading and installing the game program through an electronic device and running it. The local terminal device can provide the graphical user interface to the player in a variety of ways, for example, it can be rendered and displayed on the terminal's display screen, or provided to the player through holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen is used to present the graphical user interface, the graphical user interface includes the game screen, and the processor is used to run the game, generate the graphical user interface, and control the display of the graphical user interface on the display screen.

[0054] This application is described using a method for controlling a moving vehicle in a game running on a terminal device, where the terminal device is an electronic device with touch functionality, such as a smartphone. A graphical user interface is provided by the terminal device, which refers to an interface rendered on the display screen of the terminal device for human-computer interaction. The game of this application includes a game scene and a target virtual character and / or moving vehicle located in the game scene. The content displayed in the graphical user interface also includes a game screen, which can be a game screen of a portion of the game scene. The terminal device can respond to touch operations applied to the operating area of ​​the graphical user interface to control the target virtual character and / or moving vehicle to perform virtual actions in the game scene, such as moving in the game scene or releasing skills. The game scene can include any one or more of the following game elements: landforms, rocks, flowers, rivers, buildings, virtual objects, game props, etc. Generally speaking, a virtual camera is provided in the game scene, and the game screen displayed on the graphical user interface is the portion of the game scene content captured by the virtual camera. For example, in a first-person game, a virtual camera can be set at the head (e.g., eye position) of a target virtual character controlled by a player or at the front window position (e.g., rearview mirror position) of a sports vehicle. The virtual camera moves with the movement of the target virtual character and / or sports vehicle, and the virtual camera's orientation rotates with the rotation of the virtual subject (the target virtual character and / or sports vehicle). Therefore, the game screen presented on the graphical user interface is a portion of the game scene within a preset range in front of the target virtual character and / or sports vehicle, and the graphical user interface includes a graphical user interface. For another example, in a third-person game, the virtual camera can be set directly above or above the rear of the target virtual character and / or sports vehicle controlled by the player. Therefore, the game screen presented on the graphical user interface is a portion of the game scene including the target virtual character and / or sports vehicle. If the target player's account is logged in to the terminal device, the target virtual character and / or sports vehicle is controlled by the terminal device. A virtual object refers to a dynamic object that can be controlled in a virtual scene. Optionally, the dynamic object can be a virtual person, a virtual animal, an anime character, etc. The target virtual character is a character controlled by the player through a terminal device, or an artificial intelligence (AI) set through training to compete in a virtual environment, or a non-player character (NPC) set to compete in a virtual scene. Optionally, the target virtual character is a virtual person competing in a virtual scene.

[0055] Based on the defects in the prior art, the embodiment of the present application provides a method for controlling a sports vehicle in a game, which provides a graphical user interface through a terminal device, wherein the terminal device can be the local terminal device mentioned above or a client device. The graphical user interface includes at least a portion of the game scene, a sports vehicle controlled by a target player in the game scene, and a target player in the game scene. Figure 1 As shown, the following steps are included:

[0056] S101, obtaining in real time the direction adjustment instruction issued by the target player to the sports vehicle and the operation data of the sports vehicle;

[0057] S102, inputting the direction adjustment instruction and the operation data into a trained action prediction model to predict the reward value of each action performed by the sports vehicle;

[0058] S103, selecting a target action from the plurality of actions based on the historical operation habits of the target player and the reward value of each action;

[0059] S104: Control the sports vehicle to perform the target action in the game scene.

[0060] In the above step S101, the sports vehicle is a prop in the game scene that is used to carry a virtual character and can be controlled by the player to move in the game scene, or it can be a prop that does not carry a virtual character and can be controlled by the player to move in the game scene. For example, the sports vehicle is a car prop. The operation data of the sports vehicle includes any one or more of the following data: position in the game scene, movement speed, movement angular velocity, movement acceleration, etc. The direction adjustment instruction is an instruction for controlling the movement direction of the sports vehicle generated after the target player touches the direction adjustment control in the graphical user interface. The direction adjustment control includes a left turn direction adjustment control and a right turn direction adjustment control, and the direction adjustment instruction includes a left turn direction adjustment instruction and a right turn direction adjustment instruction.

[0061] In specific implementations, after entering an automatic game scene and the target player begins controlling a vehicle to move within the scene, the terminal device will acquire the target player's direction adjustment commands and vehicle operation data in real time. The vehicle may move straight within the game scene, without requiring the target player to control its direction. Therefore, while the target player's direction adjustment commands are acquired in real time, they may not always be received.

[0062] In the step S102, the action includes any one of the following actions: drifting and non-drifting. The drifting is a driving skill of the sports vehicle in the game scene. The reward value of the sports vehicle performing each action refers to the reward or value that the sports vehicle can bring after performing the action in the game scene. The greater the reward value of the action, the greater the probability of the sports vehicle performing the action, and the smaller the reward value of the action, the smaller the probability of the sports vehicle performing the action. The trained action prediction model is trained by a large amount of training data. Since the historical data of a player in playing the game is relatively small, in order to improve the prediction accuracy, a large amount of training data is needed to train the action prediction model, and therefore, the large amount of training data can be composed of the historical data of multiple players in playing the game. In different game maps, the environmental factors in the game scene will affect the running data of the sports vehicle, and will also affect the direction adjustment instruction issued by the target player to the sports vehicle, so the trained action prediction model corresponding to each game scene is different.

[0063] In the step S103, the historical operation habit of the target player is the operation habit of the target player in controlling the sports vehicle in the historical time period in the manual mode. The historical operation habit can include the following data: the third historical running data of the sports vehicle at each time point in the historical time period, the third historical direction adjustment instruction of the target player to the sports vehicle at each time point, and the action performed by the sports vehicle at each time point. The third historical running data includes any one or more of the following data: the position of the historical time point in the game scene, the moving speed, the angular velocity of the moving speed, the acceleration of the moving speed, etc. The third historical direction adjustment instruction is generated by the target player touching the direction adjustment control in the graphical user interface to control the moving direction of the sports vehicle. The direction adjustment control includes left and right direction adjustment controls, and the direction adjustment instruction includes left and right direction adjustment instructions.

[0064] In the implementation, the reward value of the sports vehicle performing each action is predicted by the step S102. Since the action prediction model is trained according to the data of multiple players playing the game, the predicted reward value is matched with the general habit of most players, but each player still has his own small habit. Therefore, in order to improve the accuracy of determining the target action that the target player wants to achieve, the reward value of each action is personalized adjusted according to the operation habit of the target player when playing the game before, and the action with the largest adjusted reward value is determined as the action that the target player is more inclined to. Further, the target action determined in this way is more in line with the operation habit of the target player and the intention of the target player, and the accuracy of determining the target action is improved.

[0065] In the above step S104, after determining the target action, the terminal device controls the sports vehicle to perform the target action in the game scene, and displays the game screen including the sports vehicle in the graphical user interface.

[0066] The method for controlling a motion vehicle in a game provided by this application predicts the reward value of each action of the motion vehicle controlled by the player through a trained action prediction model. The predicted reward value of each action is more in line with the operating habits of most players. In order to make the prediction result more accurate, the predicted reward value of each action is personalized adjusted based on the historical operating habits of a single target player, so that the final target action is more in line with the operating intention of the target player, thereby improving the accuracy of determining the target action.

[0067] Since the solution of this application is mainly used in the technical field of game prediction, it may not be possible to obtain a large amount of training data when training the action prediction model. Therefore, in order to improve the training accuracy of the action prediction model and improve the utilization rate of the training data that can be obtained, the action prediction model of this application adopts a combination of offline reinforcement learning network and tree model. The computational efficiency of the tree model itself is high, which is better than the computational efficiency of the convolutional network fully connected network in the traditional reinforcement learning algorithm. In order to introduce the tree model into the reinforcement learning algorithm, and based on the currently obtained training data, it is impossible to determine the true reward value of the action corresponding to the training data, so this solution adopts a value-based discrete reinforcement learning network combined with a tree model to form an action prediction model. Figure 2 As shown, the following details the training process of the action prediction model:

[0068] S105, obtaining a set of motion training samples; the set of motion training samples includes training samples obtained from at least two players; each training sample includes historical operating data of a motion vehicle controlled by the player and historical direction adjustment instructions issued by the player for the motion vehicle;

[0069] S106, for each training data, inputting the historical operation data and the historical direction adjustment instructions into the offline reinforcement learning network in the action prediction model to determine the reward value of the action represented by the training data; then using the historical operation data and the historical direction adjustment instructions in the training data as inputs to the tree model to be trained in the action prediction model, and using the reward values ​​in the training data as outputs of the tree model to be trained in the action prediction model, to train the tree model to be trained in the action prediction model;

[0070] S107, the offline reinforcement learning network in the action prediction model and the trained tree model in the action prediction model form a trained action prediction model.

[0071] In step S105, a large number of training samples are required to train the motion prediction model in order to improve training accuracy. Therefore, the motion training sample set includes training samples obtained from multiple players. However, if only the training data in automatic mode is used, it will be relatively small. Therefore, in order to increase the quantity and diversity of training data, training data of players in manual mode is also obtained. Specifically, the motion training sample set is obtained through the following steps:

[0072] Acquiring, in manual mode, first historical operating data of a motion vehicle controlled by the player and a first historical direction adjustment instruction issued by the player for the motion vehicle;

[0073] Acquire second historical running data of the sports vehicle controlled by the player in automatic mode.

[0074] In a specific implementation, the historical operation data is the data in the training sample, and the historical operation data includes the first historical operation data and the second historical operation data. The historical direction adjustment instruction is the data in the training sample, and the historical direction adjustment instruction includes the first historical direction adjustment instruction. The first historical operation data and the second historical operation data include any one or more of the following data: the position in the game scene at the historical moment, the moving speed, the moving angular velocity, the moving acceleration, etc. The first historical direction adjustment instruction is an instruction for controlling the moving direction of the motion vehicle generated after the target player touches the direction adjustment control in the graphical user interface at the historical moment. Of course, in addition to the historical data obtained from the player in automatic mode and manual mode, the training data can also be expanded to increase the amount of training data. For example, the expansion method includes swapping the direction adjustment instructions in the original historical data left and right.

[0075] In step S106, since both the first historical operation data and the second historical operation data in the training data represent the corresponding operation state of the sports vehicle at a historical moment and cannot intuitively represent the movement of the sports vehicle at that time, it is necessary to use an offline reinforcement learning network to perform data analysis based on the historical operation data to determine a reward value that can be used to represent the movement corresponding to the historical operation data, and then use this reward value to train the tree model. During the training process, each piece of training data is also divided into good and bad. Some training data may be obtained when the player has good control of the sports vehicle, while some training data may be obtained when the player has poor control of the sports vehicle. Therefore, in order to distinguish these training data and obtain more accurate movement reward values, the wall collision labels and completion times of the sports vehicle during the operation of the training data are also obtained. The fewer the number of wall collisions and the shorter the completion time, the better the training data, while the more the number of wall collisions and the longer the completion time, the worse the training data.

[0076] In the above step S107, the offline reinforcement learning network and the trained tree model will constitute a trained action prediction model.

[0077] After obtaining the reward value of each action through the trained action prediction model, personalized adjustment will be made according to the historical operation habits of the target player. This adjustment process is realized through a sensitivity function. Specifically, step S103 includes:

[0078] Step 1031, determining a sensitivity function matched with the target player according to the historical operation habits of the target player.

[0079] Step 1032, selecting a target action from a plurality of actions using the sensitivity function and the reward value of each action.

[0080] In the above step 1031, the sensitivity function can adjust the reward value of the action output by the trained action prediction model to be more inclined to the operation habits of the target player. The sensitivity function is as follows:

[0081]

[0082] Where x1 and x2 are the reward values corresponding to drift and non-drift respectively, and a and b are sensitivity parameters matched with the historical operation habits of the target player.

[0083] In specific implementation, the sensitivity parameters in the sensitivity function can be adjusted using the historical operation habits of the target player, and finally a sensitivity function matched with the target player is determined.

[0084] Specifically, the detailed steps of determining a sensitivity function matched with the target player are as follows:

[0085] Step 10311, obtaining a plurality of continuous third historical running data of a motion vehicle controlled by the target player in manual mode and a real action label corresponding to each third historical running data.

[0086] Step 10312, determining a plurality of candidate parameter combinations from the sensitivity parameter range of the sensitivity function to be adjusted according to the plurality of continuous third historical running data.

[0087] Step 10313, determining a target parameter combination from the plurality of candidate parameter combinations using the third historical running data and the real action label corresponding to each second historical running data.

[0088] Step 10314, adjusting the sensitivity parameters of the sensitivity function to be adjusted using the target parameter combination, and determining a sensitivity function matched with the target player.

[0089] In the above step 10311, since the sensitivity function needs to match the operating habits of the target player, the real action instructions issued by the target player to the sports vehicle need to be taken into account. Therefore, what is obtained in this step is the historical data of the target player in manual mode, that is, the third historical operation data of the sports vehicle controlled by the target player and the real action label corresponding to each third historical operation data are obtained, wherein the real action label is matched with the real action instruction issued by the target player to the sports vehicle. For example, if the real action label is drift, the real action instruction of the target player to the sports vehicle is a drift instruction.

[0090] In the above step 10312, the sensitivity parameter range is manually set, that is, the α and β mentioned above both have corresponding sensitivity parameter ranges.

[0091] Specifically, the candidate parameter combination is determined in the following manner:

[0092] Divide the sensitivity parameter range corresponding to α into preset intervals to obtain multiple interval values;

[0093] The reward value of each action outputted in step S102 is used as a fixed parameter, and multiple interval values ​​of α are used as variables and input into the formula f(x1, x2, α, β) = 0. The β value corresponding to each interval value of α is calculated, and each interval value of α and the β value corresponding to the interval value of α are combined into a candidate parameter combination.

[0094] In the above step 10313, the specific detailed steps for determining the target parameter combination are as follows:

[0095] Step 103131: for each candidate parameter combination, using the second historical operation data and the candidate sensitivity function set with the candidate parameter combination, determine the virtual action label corresponding to each second historical operation data;

[0096] Step 103132: For each candidate parameter combination, calculate the multiple virtual action labels corresponding to the candidate parameter combination and the real action labels corresponding to each third historical operation data to determine the matching degree of each candidate parameter combination;

[0097] Step 103133: determine the parameter combination with the highest matching degree as the target parameter combination.

[0098] In the above step 103131, each candidate parameter combination is substituted into Formula 1, and multiple consecutive third historical operation data are input into Formula 1 with the candidate parameter combination, so as to obtain multiple virtual action labels for each candidate parameter combination.

[0099] In step 103132, for each candidate parameter set, the precision and recall rates are calculated using multiple virtual action labels and multiple real action labels. The F-score is then calculated based on the precision and recall rates. F-score = Precision × Recall × 2 / (Precision + Recall). The higher the F-score, the more closely the candidate parameter set matches the target player's historical operating habits. In other words, in step 103133, the parameter set with the highest degree of match is determined as the target parameter set.

[0100] In the above step 10314, the target parameter combination replaces the original sensitivity parameters in the sensitivity function to be adjusted to obtain a sensitivity function that matches the target player. The sensitivity parameters in the adjusted sensitivity function are determined based on the real data of the target player in manual mode and personalized adjustments are made to suit the target player, thereby improving the accuracy of the sensitivity function in determining the target action.

[0101] In step 1032, after the target parameter combination is determined, the target parameter combination is entered into the sensitivity function, the parameters in the sensitivity function are adjusted, and the reward value of each action output by the trained action prediction model is then input into the sensitivity function with the adjusted sensitivity parameters. The reward value of each action is then personalized, resulting in an adjusted reward value for each action. Finally, the action with the largest reward value is determined as the target action. The sensitivity function is determined based on the historical data of the target player, and the target action determined using the sensitivity function is more consistent with the target player's operating habits, thereby improving the accuracy of determining the target action.

[0102] Finally, even the target motion determined using the trained motion prediction model and sensitivity function is not necessarily 100% accurate. It is very likely that the motion vehicle will drift when performing linear motion using the above scheme. This is very unreasonable. Therefore, this application will delete abnormal motions that do not conform to common sense and allow the motion vehicle to maintain its original motion. In other words, the control method of this application also includes:

[0103] Step 108, obtaining route data of the position of the sports vehicle in the game scene;

[0104] Step S104 includes:

[0105] Step 1041: If the target action satisfies the execution condition under the route data, controlling the sports vehicle to execute the target action in the game scene;

[0106] Step 1042: If the target action does not meet the execution conditions under the route data, control the sports vehicle to run according to the original action in the game scene.

[0107] In step 108, the route data of the position of the sports vehicle in the game scene refers to the moving route of the sports vehicle in the game scene from the initial position to the end position, and the moving route includes a curve and / or a straight line.

[0108] In steps 1041 and 1042, the target action meets the execution condition of the route data, that is, the target action is a non-abnormal action under the current route, such as a non-drifting action under a straight line route and a drifting action at a turning. The target action does not meet the execution condition of the route data, that is, the target action is an abnormal action under the current route, such as a drifting action under a straight line route and a non-drifting action at a turning. If it is found that the target action is abnormal, the target action needs to be rejected in time, so that the sports vehicle runs according to the original action, reduces the failure of determining the target action, and improves the user experience.

[0109] The application is described by taking two actions of drifting and non-drifting as an example. Of course, other actions, such as spraying and non-spraying, braking and non-braking, can also be applicable to the scheme. However, the parameters in the action prediction model and the parameters in the sensitivity function corresponding to different actions are different, such as a set of action prediction models and sensitivity functions corresponding to spraying and non-spraying, and a set of action prediction models and sensitivity functions corresponding to braking and non-braking. Spraying is an acceleration skill of the sports vehicle in the game scene, and braking is a driving means of the sports vehicle in the game scene. Non-drifting, non-spraying, and non-braking all refer to normal driving of the sports vehicle in the game scene.

[0110] The embodiment of the application provides a control device for a sports vehicle in a game, as shown in Figure 3 The control method comprises the following steps:

[0111] The acquisition module 301 is configured to acquire, in real time, a direction adjustment instruction issued by the target player for the sports vehicle and running data of the sports vehicle.

[0112] The prediction module 302 is configured to input the direction adjustment instruction and the running data into a trained action prediction model to predict a reward value of each action performed by the sports vehicle.

[0113] The selection module 303 is configured to select a target action from a plurality of actions according to a historical operation habit of the target player and the reward value of each action.

[0114] The control module 304 is configured to control the sports vehicle to perform the target action in the game scene.

[0115] Optionally, the control device includes:

[0116] A sample acquisition module is configured to acquire a set of motion training samples; the set of motion training samples includes training samples acquired from at least two players; each training sample includes historical operating data of a motion vehicle controlled by the player and historical direction adjustment instructions issued by the player for the motion vehicle;

[0117] A training module is configured to input the historical operation data and the historical direction adjustment instructions into an offline reinforcement learning network in an action prediction model for each training data, and determine a reward value for the action represented by the training data; then use the historical operation data and the historical direction adjustment instructions in the training data as inputs to a tree model to be trained in the action prediction model, and use the reward values ​​in the training data as outputs of the tree model to be trained in the action prediction model, to train the tree model to be trained in the action prediction model;

[0118] A combination module is used to combine the offline reinforcement learning network in the action prediction model and the trained tree model in the action prediction model to form a trained action prediction model.

[0119] Optionally, the sample acquisition module includes:

[0120] A first acquiring unit is configured to acquire, in a manual mode, first historical operating data of a sports vehicle controlled by the player and a first historical direction adjustment instruction issued by the player for the sports vehicle;

[0121] The second acquisition unit is used to acquire second historical operation data of the sports vehicle controlled by the player in the automatic mode.

[0122] Optionally, the selection module includes:

[0123] A first determining unit is configured to determine a sensitivity function that matches the target player based on the historical operation habits of the target player;

[0124] A selection unit is used to select a target action from the multiple actions by using the sensitivity function and the reward value of each action.

[0125] Optionally, the determining unit includes:

[0126] A third acquisition unit is configured to acquire a plurality of consecutive third historical operation data of the sports vehicle controlled by the target player in the manual mode and a real action label corresponding to each third historical operation data;

[0127] a second determining unit, configured to determine, based on a plurality of consecutive third historical operating data, a plurality of candidate parameter combinations within a sensitivity parameter range of the sensitivity function to be adjusted;

[0128] a third determining unit, configured to determine a target parameter combination from a plurality of candidate parameter combinations by using the third historical operation data and a real action label corresponding to each third historical operation data;

[0129] The fourth determining unit is configured to adjust the sensitivity parameters of the sensitivity function to be adjusted by using the target parameter combination to determine a sensitivity function that matches the target player.

[0130] Optionally, the third determining unit includes:

[0131] a first determining subunit, configured to determine, for each candidate parameter combination, a virtual action label corresponding to each third historical operation data by using the third historical operation data and a candidate sensitivity function set with the candidate parameter combination;

[0132] a second determining subunit, configured to calculate, for each candidate parameter combination, a plurality of virtual action labels corresponding to the candidate parameter combination and a real action label corresponding to each third historical operation data, and determine a degree of match between each candidate parameter combination and the historical operation habits of the target player;

[0133] The third determining subunit is configured to determine the parameter combination with the highest matching degree as the target parameter combination.

[0134] Optionally, the control device further includes:

[0135] A route acquisition module, used to obtain route data of the position of the sports vehicle in the game scene;

[0136] Control module, including:

[0137] A judgment unit is used to control the sports vehicle to perform the target action in the game scene if the target action meets the execution conditions under the route data; if the target action does not meet the execution conditions under the route data, control the sports vehicle to run according to the original action in the game scene.

[0138] Corresponding to Figure 1 The embodiment of the present application further provides a computer device 400, such as Figure 4 As shown, the device includes a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402. When the processor 402 executes the computer program, the method for controlling the moving vehicle in the game is implemented. The control device includes:

[0139] Real-time acquisition of direction adjustment instructions issued by the target player to the sports vehicle and operation data of the sports vehicle;

[0140] Inputting the direction adjustment instruction and the operation data into a trained action prediction model to predict the reward value of each action performed by the motion vehicle;

[0141] Selecting a target action from the plurality of actions based on the target player's historical operating habits and the reward value of each action;

[0142] Control the motion vehicle to perform the target action in the game scene.

[0143] Optionally, the action prediction model is trained through the following steps:

[0144] Obtaining a set of action training samples; the set of action training samples includes training samples obtained from at least two players; each training sample includes historical operating data of a motion vehicle controlled by the player and historical direction adjustment instructions issued by the player for the motion vehicle;

[0145] For each training data, the historical operation data and the historical direction adjustment instructions are input into an offline reinforcement learning network in the action prediction model to determine the reward value of the action represented by the training data; the historical operation data and the historical direction adjustment instructions in the training data are then used as inputs to a tree model to be trained in the action prediction model, and the reward values ​​in the training data are used as outputs of the tree model to be trained in the action prediction model, so as to train the tree model to be trained in the action prediction model;

[0146] The offline reinforcement learning network in the action prediction model and the trained tree model in the action prediction model constitute a trained action prediction model.

[0147] Optionally, the action training sample set is obtained through the following steps:

[0148] Acquiring, in manual mode, first historical operating data of a motion vehicle controlled by the player and a first historical direction adjustment instruction issued by the player for the motion vehicle;

[0149] Acquire second historical running data of the sports vehicle controlled by the player in automatic mode.

[0150] Optionally, selecting a target action from a plurality of actions based on the historical operation habits of the target player and the reward value of each action includes:

[0151] Determining a sensitivity function that matches the target player based on the target player's historical operating habits;

[0152] A target action is selected from the plurality of actions using the sensitivity function and the reward value of each action.

[0153] Optionally, determining a sensitivity function that matches the target player based on the target player's historical operating habits includes:

[0154] Acquire a plurality of consecutive third historical operation data of the sports vehicle controlled by the target player in the manual mode and a real action label corresponding to each third historical operation data;

[0155] determining a plurality of candidate parameter combinations from a sensitivity parameter range of the sensitivity function to be adjusted based on a plurality of third consecutive historical operating data;

[0156] Determining a target parameter combination from a plurality of candidate parameter combinations using the third historical operation data and a real action label corresponding to each third historical operation data;

[0157] The target parameter combination is used to adjust the sensitivity parameters of the sensitivity function to be adjusted, and a sensitivity function that matches the target player is determined.

[0158] Optionally, the determining a target parameter combination from a plurality of candidate parameter combinations by using the third historical operation data and the real action label corresponding to each third historical operation data includes:

[0159] For each candidate parameter combination, using the third historical operation data and the candidate sensitivity function set with the candidate parameter combination, determine a virtual action label corresponding to each third historical operation data;

[0160] For each candidate parameter combination, calculating a plurality of virtual action labels corresponding to the candidate parameter combination and a real action label corresponding to each third historical operation data, and determining a degree of match between each candidate parameter combination and the historical operation habits of the target player;

[0161] The parameter combination with the highest matching degree is determined as the target parameter combination.

[0162] Optionally, the control method further includes:

[0163] Obtaining route data of the position of the sports vehicle in the game scene;

[0164] Controlling the motion vehicle to perform the target action in the game scene includes:

[0165] if the target action meets the execution condition under the route data, controlling the motion vehicle to perform the target action in the game scene;

[0166] if the target action does not meet the execution condition under the route data, controlling the motion vehicle to run according to the original action in the game scene.

[0167] Specifically, the above-mentioned memory 401 and processor 402 can be general memory and processor, which are not specifically limited here. When the processor 402 runs the computer program stored in the memory 401, the above-mentioned control method of the motion vehicle in the game can be executed, solving the problem that the target action of the motion vehicle controlled by the target player cannot be accurately predicted in the prior art. The present application predicts the reward value of each action of the motion vehicle controlled by the player through the trained action prediction model. The predicted reward value of each action is more in line with the operation habits of most players. In order to make the prediction result more accurate, the reward value of each action predicted will be individually adjusted in combination with the historical operation habits of the single target player, and then the finally determined target action is more in line with the operation intention of the target player, thereby improving the accuracy of determining the target action.

[0168] Corresponding to the control method of the motion vehicle in the game in Figure 1 , the present application embodiment further provides a computer readable storage medium, the computer readable storage medium has a computer program stored thereon, the computer program is run by a processor to execute the above-mentioned control method of the motion vehicle in the game, and the control method comprises:

[0169] obtaining in real time the direction adjustment instruction issued by the target player for the motion vehicle and the running data of the motion vehicle;

[0170] inputting the direction adjustment instruction and the running data into the trained action prediction model to predict the reward value of each action performed by the motion vehicle;

[0171] selecting a target action from a plurality of actions according to the historical operation habits of the target player and the reward value of each action;

[0172] controlling the motion vehicle to perform the target action in the game scene.

[0173] Optionally, the action prediction model is obtained by the following steps:

[0174] obtaining an action training sample set; the action training sample set includes training samples obtained from at least two players; each training sample includes historical running data of the motion vehicle controlled by the player and historical direction adjustment instructions issued by the player for the motion vehicle;

[0175] For each training data, the historical operation data and the historical direction adjustment instructions are input into an offline reinforcement learning network in the action prediction model to determine the reward value of the action represented by the training data; the historical operation data and the historical direction adjustment instructions in the training data are then used as inputs to a tree model to be trained in the action prediction model, and the reward values ​​in the training data are used as outputs of the tree model to be trained in the action prediction model, so as to train the tree model to be trained in the action prediction model;

[0176] The offline reinforcement learning network in the action prediction model and the trained tree model in the action prediction model constitute a trained action prediction model.

[0177] Optionally, the action training sample set is obtained through the following steps:

[0178] Acquiring, in manual mode, first historical operating data of a motion vehicle controlled by the player and a first historical direction adjustment instruction issued by the player for the motion vehicle;

[0179] Acquire second historical running data of the sports vehicle controlled by the player in automatic mode.

[0180] Optionally, selecting a target action from a plurality of actions based on the historical operation habits of the target player and the reward value of each action includes:

[0181] Determining a sensitivity function that matches the target player based on the target player's historical operating habits;

[0182] A target action is selected from the plurality of actions using the sensitivity function and the reward value of each action.

[0183] Optionally, determining a sensitivity function that matches the target player based on the target player's historical operating habits includes:

[0184] Acquire a plurality of consecutive third historical operation data of the sports vehicle controlled by the target player in the manual mode and a real action label corresponding to each third historical operation data;

[0185] determining a plurality of candidate parameter combinations from a sensitivity parameter range of the sensitivity function to be adjusted based on a plurality of third consecutive historical operating data;

[0186] Determining a target parameter combination from a plurality of candidate parameter combinations using the third historical operation data and a real action label corresponding to each third historical operation data;

[0187] Adjust the sensitivity parameter of the to-be-adjusted sensitivity function by using the target parameter combination, and determine a sensitivity function matched with the target player.

[0188] Optionally, the determining the target parameter combination from the plurality of candidate parameter combinations by using the third historical running data and the virtual action label corresponding to each third historical running data comprises:

[0189] For each candidate parameter combination, the virtual action label corresponding to each third historical running data is determined by using the third historical running data and the candidate sensitivity function with the candidate parameter combination set.

[0190] For each candidate parameter combination, the matching degree of each candidate parameter combination with the historical operation habit of the target player is determined by calculating the virtual action label corresponding to each third historical running data in the candidate parameter combination and the real action label corresponding to each third historical running data.

[0191] The candidate parameter combination with the highest matching degree is determined as the target parameter combination.

[0192] Optionally, the control method further comprises:

[0193] Obtaining route data of a position of the motion carrier in the game scene;

[0194] Controlling the motion carrier to perform the target action in the game scene, comprising:

[0195] If the target action meets the execution condition under the route data, controlling the motion carrier to perform the target action in the game scene;

[0196] If the target action does not meet the execution condition under the route data, controlling the motion carrier to run according to the original action in the game scene.

[0197] Specifically, the storage medium can be a general storage medium such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the game motion carrier control method described above can be executed, solving the problem that the target action of the motion carrier controlled by the target player cannot be accurately predicted in the prior art. The present application predicts the return value of each action of the motion carrier controlled by the player by using the trained action prediction model. The return value of each action predicted is more in line with the operation habit of most players. In order to make the prediction result more accurate, the return value of each action predicted is further personalized adjusted in combination with the historical operation habit of the single target player, and then the target action determined finally is more in line with the operation intention of the target player, improving the accuracy of determining the target action.

[0198] In the embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0199] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0200] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0201] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0202] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0203] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the same. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any skilled person in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features, within the technical scope disclosed by the present application. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. All should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for controlling a moving vehicle in a game, characterized in that: A graphical user interface is provided through a terminal device, wherein the graphical user interface includes at least a portion of a game scene and a sports vehicle controlled by a target player within the game scene. The control method includes: Real-time acquisition of direction adjustment instructions issued by the target player to the sports vehicle and operation data of the sports vehicle; Inputting the direction adjustment instruction and the operation data into a trained action prediction model to predict the reward value of each action performed by the sports vehicle; wherein the predicted reward value of each action matches the player's general operating habits; Determining a sensitivity function that matches the target player based on the target player's historical operating habits; Adjusting the reward value of each action using the sensitivity function, and determining the action with the largest adjusted reward value from the multiple actions as the target action; Control the motion vehicle to perform the target action in the game scene.

2. The control method according to claim 1, characterized in that: The action prediction model is trained by the following steps: Obtaining a set of action training samples; the set of action training samples includes training samples obtained from at least two players; each training sample includes historical operating data of a motion vehicle controlled by the player and historical direction adjustment instructions issued by the player for the motion vehicle; For each training data, input the historical operation data and the historical direction adjustment instructions into the offline reinforcement learning network in the action prediction model to determine the reward value of the action represented by the training data; Then, the historical operation data and the historical direction adjustment instructions in the training data are used as inputs of the tree model to be trained in the motion prediction model, and the reward values ​​in the training data are used as outputs of the tree model to be trained in the motion prediction model, and the tree model to be trained in the motion prediction model is trained; The offline reinforcement learning network in the action prediction model and the trained tree model in the action prediction model constitute a trained action prediction model.

3. The control method according to claim 2, characterized in that: The action training sample set is obtained through the following steps: Acquiring, in manual mode, first historical operating data of a motion vehicle controlled by the player and a first historical direction adjustment instruction issued by the player for the motion vehicle; Acquire second historical running data of the sports vehicle controlled by the player in automatic mode.

4. The control method according to claim 1, wherein: Determining a sensitivity function that matches the target player based on the target player's historical operating habits includes: Acquire a plurality of consecutive third historical operation data of the sports vehicle controlled by the target player in the manual mode and a real action label corresponding to each third historical operation data; determining a plurality of candidate parameter combinations from a sensitivity parameter range of the sensitivity function to be adjusted based on a plurality of third consecutive historical operating data; Determining a target parameter combination from a plurality of candidate parameter combinations using the third historical operation data and a real action label corresponding to each third historical operation data; The target parameter combination is used to adjust the sensitivity parameters of the sensitivity function to be adjusted, and a sensitivity function that matches the target player is determined.

5. The control method according to claim 4, characterized in that: The method of determining a target parameter combination from a plurality of candidate parameter combinations by using the third historical operation data and the real action label corresponding to each third historical operation data includes: For each candidate parameter combination, using the third historical operation data and the candidate sensitivity function set with the candidate parameter combination, determine a virtual action label corresponding to each third historical operation data; For each candidate parameter combination, calculating a plurality of virtual action labels corresponding to the candidate parameter combination and a real action label corresponding to each third historical operation data, and determining a degree of match between each candidate parameter combination and the historical operation habits of the target player; The parameter combination with the highest matching degree is determined as the target parameter combination.

6. The control method according to claim 1, characterized in that: The control method further includes: Obtaining route data of the position of the sports vehicle in the game scene; Controlling the motion vehicle to perform the target action in the game scene includes: If the target action satisfies the execution condition under the route data, controlling the sports vehicle to execute the target action in the game scene; If the target action does not meet the execution conditions under the route data, the sports vehicle is controlled to run according to the original action in the game scene.

7. A control device for a sports vehicle in a game, characterized in that: A graphical user interface is provided through a terminal device, wherein the graphical user interface includes at least a portion of a game scene and a sports vehicle controlled by a target player located in the game scene, and the control device includes: An acquisition module, configured to acquire, in real time, the direction adjustment instruction issued by the target player to the sports vehicle and the operation data of the sports vehicle; a prediction module, configured to input the direction adjustment instruction and the operation data into a trained action prediction model to predict a reward value for each action performed by the vehicle; wherein the predicted reward value for each action matches the general operating habits of the player; A selection module, configured to determine a sensitivity function that matches the target player based on the target player's historical operating habits; Adjusting the reward value of each action using the sensitivity function, and determining the action with the largest adjusted reward value from the multiple actions as the target action; A control module is used to control the sports vehicle to perform the target action in the game scene.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.

Images