Artificial intelligence for racing lines in racing games

Abstract

For example, dynamic driving aids such as driving lines, turn indicators, braking indicators, acceleration indicators, etc. can be provided to players participating in a racing game. Typically, driving lines are provided for each class of car. However, even for cars of the same class, each car is sufficiently different that the ideal driving line and braking points may change. Thus, using agents trained by reinforcement learning, the ideal line and other driving aids can be established for each individual car. These guides can also be modified in response to changes in weather and other track conditions.

Description

Technical Field

[0001]

[0001] Embodiments of the present invention generally relate to training artificial intelligence agents. Specifically, the present invention relates to methods and systems for providing dynamic driving lines and other driver aids in racing video games.

Background Art

[0002]

[0002] The following background information can present examples of specific aspects of the prior art (e.g., without limitation, techniques, facts, or general notions), and these examples are expected to help convey further aspects of the prior art to the reader, but should not be construed as limiting the present invention or any of its embodiments to any matter mentioned or implied, or conjectured therefrom.

[0003]

[0003] Usually, racing video games provide one or more driving aides to the player. For example, it is possible to provide one ideal driving line for each class of vehicle. This driving line is established by a professional driver who works for the company and attempts to drive a perfect line representing that class of vehicle. Further, other driving aids can be displayed on the track to provide the driver with additional recommendations such as where to brake, turn, accelerate, etc.

[0004]

[0004] However, even vehicles of the same class can be quite different from each other, so the ideal driving line and braking points can vary.

Summary of the Invention

Problems to be Solved by the Invention

[0005]

[0005] Therefore, there is a need for methods and systems for providing dynamic driving aids that can vary according to various parameters.

Means for Solving the Problems

[0006]

[0006] Embodiments of the present invention provide a method for providing driving assistance in a racing game, comprising the steps of: training a neural network so that a trained agent can achieve an optimal race result in a particular car on a particular track; recording the trajectory of the trained agent on the track; converting the trajectory into course overlay information; and storing the course overlay information for the particular car and the particular track.

[0007]

[0007] Embodiments of the present invention provide a method for providing driving assistance in a racing game, further comprising: training a neural network to enable a trained agent to achieve an optimal race result in a particular car on a particular track; operating the trained agent in parallel with a player racing in the racing game, wherein the trained agent is provided with state information of the player during the racing game; having the trained agent look ahead to determine a recommended course of action based on the state information of the player, wherein the recommended course of action is determined by the trained agent to achieve an optimal race result; and providing course overlay information based on the recommended course of action.

[0008]

[0008] Embodiments of the present invention also provide a method for providing driving assistance in a racing game, comprising the steps of: training a neural network to enable a trained agent to achieve optimal race results in a plurality of specific cars on a plurality of specific tracks; recording the trajectory of the trained agent in each of the plurality of specific cars on the plurality of specific tracks; converting the trajectory into course overlay information; displaying the course overlay information on a player's display during a race in which the player of the racing game uses one selected car on one of the plurality of specific tracks; and storing the course overlay information for the specific cars and the specific tracks.

[0009]

[0009] These and other features, aspects and advantages of the present invention will be better understood by referring to the following drawings, description and claims.

[0010]

[0010] Several embodiments of the present invention are shown by illustration of the accompanying drawings, where the same reference numerals may indicate similar elements, not as limitations but as examples. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows a typical screenshot of a racing game with driving assistance according to an exemplary embodiment of the present invention. [Figure 2] This figure shows an exemplary method according to an exemplary embodiment of the present invention. [Modes for carrying out the invention]

[0012]

[0013] Unless otherwise specified, the figures within the diagrams are not necessarily drawn to scale.

[0013]

[0014] The present invention and its various embodiments can be better understood by referring to the following detailed description of the illustrated embodiments. It should be clearly understood that the illustrated embodiments are for illustrative purposes only and do not limit the invention to the final claims.

[0014]

[0015] The terms used herein are for the purpose of describing specific embodiments and are not intended to limit the invention. The terms “and / or” as used herein include any combination of one or more of the items relating to the description. The singular forms “a, an” and “the” as used herein are intended to include the plural form as well as the singular form unless otherwise explicitly indicated in the context. Furthermore, the terms “comprises and / or comprising” as used herein indicate the presence of the features, steps, actions, elements and / or components referred to, but should be understood not to exclude the presence or addition of one or more other features, steps, actions, elements, components and / or groups thereof.

[0015]

[0016] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as those generally understood by those skilled in the art to which the present invention pertains. Furthermore, terms defined in commonly used dictionaries should be interpreted to have the meanings corresponding to those meanings in the context of the relevant art and this disclosure, and should not be interpreted in an ideal or overly formal sense unless explicitly defined herein.

[0016]

[0017] It will be understood that the description of this invention discloses multiple techniques and steps. Each of these has its own individual benefit and can be used in conjunction with one or more, or possibly all, of the other techniques disclosed. Therefore, for clarity, this description avoids unnecessarily repeating all possible combinations of the individual steps. However, this specification and the claims should be read with the understanding that such combinations are fully included within the scope of the invention and the claims.

[0017]

[0018] The following description provides numerous specific details for illustrative purposes to provide a complete understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be carried out without these specific details.

[0018]

[0019] This disclosure should be considered illustrative of the present invention and is not intended to limit the invention to specific embodiments shown by the following figures or description.

[0019]

[0020] Devices or system modules that communicate with each other using general communication do not need to communicate with each other continuously unless otherwise specified. Furthermore, devices or system modules that communicate with each other using general communication can communicate directly or indirectly through one or more intermediary devices.

[0020]

[0021] The description of embodiments including multiple components that communicate with each other does not mean that all such components are necessary. Rather, various optional components are described to illustrate a wide range of possible embodiments of the present invention.

[0021]

[0022] "Computer" or "computer device" can mean one or more devices and / or one or more systems that accept structured input, process the structured input according to prescribed rules, and produce the results of the processing as output. Examples of computers or computer devices include computers, fixed and / or portable computers, computers having a single processor, multiple processors, or multiple core processors that can operate in parallel and / or non-parallel, supercomputers, mainframes, superminicomputers, minicomputers, workstations, microcomputers, servers, clients, interactive televisions, web appliances, communication devices with internet access, hybrid combinations of computers and interactive televisions, portable computers, tablet personal computers (PCs), personal digital assistants (PDAs), mobile phones, application-specific hardware that emulates computers and / or software, such as digital signal processors (DSPs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific instruction set processors (ASIPs), chips, multiple chips, systems on a chip or chipsets, data acquisition devices, optical computers, quantum computers, biocomputers, and devices that generally accept data, process the data according to one or more stored software programs, produce results, and typically include input devices, output devices, memory devices, arithmetic units, logic units and control units.

[0022]

[0023] "Software" or "application" can mean a set of rules for operating a computer. Examples of software or applications include code segments in one or more computer-readable languages, graphic and / or text instructions, applets, precompiled code, interpreted code, compiled code, and computer programs.

[0023]

[0024] Also, by storing these computer program instructions, which can be used to instruct a computer, other programmable data processing apparatus, or other devices to function in a specific manner, in a computer-readable medium, it is possible to create an article of manufacture in which the instructions stored in the computer-readable medium include instructions for implementing the functions / operations specified within one or more blocks of a flowchart and / or block diagram.

[0024]

[0025] Further, process steps, method steps, algorithms, etc. can be described in a certain order, but such processes, methods, and algorithms can also be configured to function in a different order. In other words, any order or sequence of steps that can be described does not necessarily indicate that these steps must be executed in this order. The process steps described herein can be executed in any practical order. Further, several steps can also be executed simultaneously.

[0025]

[0026] It will be readily apparent that the various methods and algorithms described herein can be implemented, for example, by a suitably programmed general-purpose computer and computer devices. Usually, a processor (e.g., a microprocessor) receives instructions from memory or a similar device and executes these instructions to execute the process defined by these instructions. Further, programs implementing such methods and algorithms can be stored and transmitted using various known media.

[0026]

[0027] As used herein, the term "computer-readable medium" means any medium that participates in providing data (e.g., instructions) that can be read by a computer, processor, or similar device. Such media can take many forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks and other permanent memories. Volatile media typically includes dynamic random access memory (DRAM) that constitutes main memory. Transmission media includes coaxial cables, copper wire, and fiber optics, including wires that include a system bus coupled to a processor. Transmission media can include, or convey, acoustic waves, light waves, and electromagnetic radiation such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, any other magnetic medium, CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASHEEPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium that can be read by a computer.

[0027]

[0028] Various forms of computer-readable media can participate in carrying a series of instructions to a processor. For example, a series of instructions can be supplied to the processor from (i) RAM, can be carried via a wireless transmission medium, and / or can be formatted according to numerous formats, standards, or protocols such as Bluetooth, TDMA, CDMA, 3G, 4G, 5G, etc.

[0028]

[0029] Embodiments of the invention can include an apparatus that performs the operations disclosed herein. The apparatus can include a device specially configured for a desired purpose or can include a general-purpose device selectively actuated or reconfigured by a program stored therein.

[0029]

[0030] Unless otherwise stated, and as may become apparent from the following description and claims, throughout this specification, any use of terms such as “process,” “calculate,” “calculate,” or “determine” should be understood to mean the operation and / or process of a computer, computer system, or similar electronic computer device that manipulates data represented as physical quantities, such as electron quantities in the registers and / or memory of a computer system, and / or transforms it into other data similarly represented as physical quantities in the memory, registers, or other such information storage, transmission, or display device of a computer system.

[0030]

[0031] Similarly, the term “processor” can mean any device or part of a device that processes electronic data from registers and / or memory and converts this electronic data into other electronic data that can be stored in registers and / or memory or transmitted to an external device, so as to cause a physical change or operation of an external device.

[0031]

[0032] The terms “agent,” “intelligent agent,” “artificial agent,” or “artificial intelligence agent” are intended to mean any artificial entity that chooses actions in response to observation. “Agent” can mean, without limitation, a robot, a simulated robot, a software agent or “bot,” an adaptive agent, an internet or webbot.

[0032]

[0033] In general, embodiments of the present invention provide dynamic driving assistance to players participating in racing games. These dynamic driving assistance may include, for example, driving lines, turn signals, braking signals, steering targets, and acceleration indicators. As mentioned above, even cars of the same class are quite different, so ideal driving lines and braking points can vary. Therefore, using reinforcement learning-trained agents, it is possible to establish ideal lines and other driving assistance for each individual car. These guides can also be modified in response to changes in weather and other track conditions.

[0033]

[0034] While the following description focuses on racing games, it should be understood that embodiments of the present invention can be applied beyond car racing games. Embodiments of the present invention can be applied to automobiles, motorcycles, airplanes, other aerial racing games, space racing, skiing, snowboarding, and the like.

[0034]

[0035] Referring to Figure 1, the driving line 10 is simply an overlay on the driver view 12 of the track, indicating the recommended path. Often this is a yellow dotted line, but other indicators can also be used. In some embodiments, as shown in Figure 1, a pair of cones 14 on the right side may indicate where the player should begin braking, and a single cone 16 shown further up the road may indicate where the player should begin accelerating again. A floating circle 18 in the center of the screen indicates a target that the player should aim the race car at. In conventional racing games, these indicators are fixed relative to the track, regardless of the type of car the player is driving. In conventional racing games, these guides are essentially static information superimposed on the track display.

[0035]

[0036] However, according to embodiments of the present invention, these driving aids may be static and may be configured for each type of vehicle or for each individual vehicle. Different vehicles have different traction, weight, braking capabilities, etc. The actual line that the player should follow can be a function of the vehicle's friction limits, weight distribution, and other attributes. Therefore, embodiments of the present invention can provide driving aids that can be varied according to the type and / or attributes of the vehicle.

[0036]

[0037] Furthermore, according to embodiments of the present invention, driving assistance such as a driving line 10 that changes as a function of the player's current position and speed can be provided. For example, if the player is moving slower than the ideal speed or maximum speed, the driving line 10 can automatically shift inward on the track. According to embodiments of the present invention, an "ideal" speed can also be determined for each vehicle or vehicle type and / or track. Such an ideal speed can be determined, for example, by reinforcement learning through training an agent on a given vehicle on a given track. Also, the proposed braking point (indicated by a pair of cones 14 in Figure 1) can change depending on whether the player is moving on the outer part of the track at a speed higher than the ideal speed or whether the player is moving at a speed lower than the ideal speed or is on the inner part of the track.

[0037]

[0038] According to embodiments of the present invention, the braking point can be varied according to one or more characteristics. Such characteristics may include the attributes of the vehicle, the current speed, the type and current wear of the tires being used, the weight of the fuel on board, the weather, the track conditions, and so on.

[0038]

[0039] According to a further embodiment of the present invention, the target to be aimed at (indicated by the floating circle 18 in Figure 1) can be a function of the current position and velocity, and can be changed based on these or other attributes.

[0039]

[0040] Various technologies can be used to incorporate dynamic driving assistance into games. For example, referring to Figure 2, method 24 for providing dynamic driving assistance in a racing game may include an action 26 to train a neural network based on operating a specific car on a specific track. For example, reinforcement learning can be used to train an agent to achieve an optimal race result with a specific car on a specific track. Such an optimal race result could be, for example, the best course time to complete the race. Method 24 may further include an action 28 to record the trajectory of the trained agent on the track. Method 24 may further include an action 30 to convert the trajectory into course overlay information that can be optionally turned on on the player's display. Finally, method 24 may include an action 32 to store car-track specific overlay information in a database. This information can be stored in the game itself, a cloud computing device, etc., so that the game can optionally load this information when the player is preparing for a race. The course overlay information may include details such as the driving line, braking area, acceleration area, and / or aiming point, as described above.

[0040]

[0041] The above provides driving assistance that can be provided based on vehicle and / or truck information, but other elements can be included in the training of the neural network. For example, since the neural network is trained on various levels of tire tread wear, this feature can be taken into account as to how the driving assistance changes based on the actual tread wear of the player's tires.

[0041]

[0042] Furthermore, by running a neural network trained within the game in parallel with the user's actual driving, driving assistance can be dynamically calculated during gameplay. This trained neural network looks ahead for a predetermined time (e.g., n seconds), calculates what to do in the current player driving state, and provides driving assistance based on this decision. Thus, not only can driving assistance be provided based on the car specifications and / or track state, but the driving assistance can also be further refined by the actual state of the player's car at any given time. As mentioned above, if the driver's speed is slower than the ideal speed (the speed at which initial car-track based driving assistance can be determined), the trained neural network running in parallel with the game player can determine what action will produce the best result for the player in a given state. Such information can be provided as driving assistance such as an accelerometer, or by changing the driving line from what can be calculated before the start of the race. Thus, this feature can be used in combination with the initial calculation of driving assistance based solely on the car and / or track state. In this case, the initial calculation of driving assistance is modified based on the trained neural network running in parallel with the player's actual driving. In other embodiments, this feature can be used solely to provide driving assistance, in which case the original driving assistance may not be used as a reference, but the driving assistance can be generated during gameplay based solely on the "live" use of a trained neural network.

[0042]

[0043] In some embodiments, the neural network can be trained not only on individual cars, trucks, and weather conditions, but also on specific traffic conditions. Therefore, in some embodiments, driving assistance can be modified based on the presence of other players' vehicles to the side or in front of the player. For example, a driving line can be provided to help the player navigate around other players' vehicles. Furthermore, suggestions on when to accelerate or brake can be provided to the driver. Such details are dynamically provided by a trained neural network operating in parallel with the player's actual driving, as described above, and driving assistance can be provided to the player based on lookahead to determine the course of action the trained neural network will take to achieve the greatest reward.

[0043]

[0044] Typically, a trained neural network can be trained to maximize rewards based on the best course time. In some embodiments, the reward function of the trained neural network can be varied or selected by the user. For example, the reward function can be optimized to prevent collisions, in which case the driving line can be adjusted to provide more space between the player's car and the opponent's car. The trained neural network can be trained in a variety of ways, such as those disclosed in U.S. Provisional Application No. 63 / 267,136, which are incorporated herein by reference.

[0044]

[0045] Those skilled in the art can make numerous changes and modifications without departing from the spirit and scope of the present invention. Therefore, the illustrated embodiments are merely examples and should not be interpreted as limiting the present invention as defined by the following claims. For example, while the elements of the claims are shown below in specific combinations, it should be clearly understood that the present invention includes other combinations of elements, fewer, more, or different from those disclosed.

[0045]

[0046] The words used herein to describe the present invention and its various embodiments should be understood to include not only their generally defined meanings but also, by their specific definitions herein, comprehensive structures, materials, or actions that represent a single kind.

[0046]

[0047] Therefore, in this specification, the definitions of words or elements in the following claims are not limited to combinations of elements explicitly stated in the text. In this sense, it is conceivable that one of any of the elements in the following claims may be replaced by two or more equivalent substitutes of elements, or a single element may be replaced by two or more elements of the claims. While elements are described above as functioning in specific combinations, and claims may initially be made in this manner, it should be clearly understood that, in some cases, one or more elements resulting from the claimed combinations may be removed from these combinations, and the claimed combinations may be directed towards lower combinations or variations of lower combinations.

[0047]

[0048] It is explicitly assumed that any minor changes from the subject matter, as seen by those skilled in the art, whether currently known or later devised, are equally included in the claims. Therefore, obvious substitutions, both currently known and hereafter known to those skilled in the art, are also defined as being within the scope of the specified elements.

[0048]

[0049] Therefore, the claims should be understood to include those specifically illustrated and described above, those that are conceptually equivalent, those that are clearly substituted, and those that incorporate the fundamental ideas of the present invention. [Explanation of Symbols]

[0049] 10 Driving Line 12 Driver View 14. A pair of cones 16 Single cone 18 Floating Circle 24. Method for providing dynamic driving assistance in racing games. 26. Train a neural network based on operating a specific car on a specific track. 28. Record the trajectory of a trained agent on a track. 30. Convert the trajectory to course overlay information. 32. Store specific car-track course overlay information in a database.

Claims

1. A method for providing driving aids in a racing game, The steps include training a neural network so that a trained agent can achieve optimal race results in a specific car on a specific track, and A step of recording the trajectory of the trained agent on the track, The steps include converting the aforementioned trajectory into course overlay information, The steps include saving the course overlay information for the specified automobile and the specified track, Includes, The method is characterized in that the course overlay information includes a braking suggestion, the braking suggestion is provided as a function of the attributes of the particular vehicle.

2. The method according to claim 1, further comprising the step of displaying the course overlay information on the player's display during a race in which the player of the racing game uses the specific car on the specific track.

3. The method according to claim 1, characterized in that the method is repeated while changing the specific car or the specific track, thereby training the trained agent for each possible combination of car and track in the racing game.

4. The method according to claim 1, characterized in that the course overlay information is stored within the racing game.

5. The method according to claim 1, characterized in that the course overlay information is stored in cloud-based storage and retrieved before the start of a race in the racing game.

6. The method according to claim 1, characterized in that the course overlay information is based on in-game weather conditions while the player is racing in the racing game.

7. The method according to claim 1, characterized in that the course overlay information is further based on the speed and current position of the player's car in the racing game.

8. The method according to claim 7, characterized in that the course overlay information is dynamically calculated while the player is racing in the racing game.

9. The method according to claim 8, characterized in that the trained neural network is executed within the racing game in parallel with the player racing in the racing game, and the trained neural network looks ahead a predetermined amount of time to determine the course overlay information for the player.

10. The method according to claim 1, characterized in that the course overlay information includes at least one of a driving line, acceleration information, and a trajectory target.

11. The method according to claim 10, characterized in that the braking suggestion is provided as a function of the speed of the particular car in the racing game.

12. The method according to claim 10, characterized in that the braking proposal is provided as a function of the tire type and the amount of tire wear.

13. The method according to claim 1, characterized in that the trained neural network is trained to provide the trajectory based on adjacent or approaching traffic encountered by the player during the racing game.

14. A method for providing driving assistance in a racing game, The steps include training a neural network so that a trained agent can achieve optimal race results with multiple specific cars on multiple specific tracks, and A step of recording the trajectory of the trained agent in each of the multiple specific vehicles on the multiple specific tracks, The steps include converting the aforementioned trajectory into course overlay information, The steps include displaying the course overlay information on the player's display during a race in which the player of the racing game uses one selected car from among the multiple specific cars on one selected track from among the multiple specific tracks, The steps include saving the course overlay information for the specified automobile and the specified track, Includes, The method is characterized in that the course overlay information includes a braking suggestion, the braking suggestion is provided as a function of the attributes of the particular vehicle.

15. The method according to claim 14, characterized in that the course overlay information includes at least one of a driving line, acceleration information, and a trajectory target.

Images