SYSTEM AND METHOD FOR TRAINING ARTIFICIAL INTELLIGENCE MODELS FOR COMPETITIVE MATCHES - Patent application
Training AI models to learn player interactions reduces network traffic and delays in multiplayer games by mimicking player inputs, enhancing gameplay experience and network management.
Patent Information
- Application Number
- JP2023109577
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-03-15
- Filing Date
- 2023-07-03
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-02-19
AI Technical Summary
Network traffic increases during multiplayer video games due to the transfer of large amounts of input data from players' devices to servers, leading to network delays and unpredictable traffic management.
Training artificial intelligence models to learn player interactions and reduce the need for real-time data transfer by analyzing and mimicking player inputs, allowing the AI models to operate game characters and adapt to various scenarios.
Reduces network traffic and delays by allowing AI models to learn from player actions, providing a better gaming experience with more predictable network management and improved gameplay performance.
Smart Images

Figure 0007590500000001 
Figure 0007590500000002 
Figure 0007590500000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to systems and methods for training artificial intelligence (AI) models for competitive matches. [Background technology]
[0002] Modern video games are accessed through computer networks. For example, the game Fortnite™ is played by many players from all over the world. One player controls a first avatar and another player controls a second avatar. Each avatar collects weapons and cuts wood during the game. The avatar is then forced to be trapped inside a virtual circle. If the avatar is left outside the virtual circle, it dies virtually in the game. When both avatars are inside the circle, they seek each other out and fight each other with their weapons. Only one of the two avatars survives.
[0003] However, network traffic increases during game play when the game is being played by millions of players around the world. Summary of the Invention
[0004] Embodiments of the present disclosure provide systems and methods for training artificial intelligence (AI) models for competitive matches.
[0005] Other aspects of the present disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the embodiments described in the present disclosure.
[0006] The systems and methods described in this disclosure allow players to set up matches against other users' artificial intelligence (AI) models. For example, an AI model is built over time to match the player's or user's gaming skills, and other AI models are built for other players or users. A user may want to see if their AI model can beat their friend's AI model, and the user and their friend watch virtual characters representing the AI models play against each other. This provides custom matches that users set up and passively watch.
[0007] In one embodiment, a method for training a character of a game is described. The method includes facilitating display of one or more scenes of the game. The one or more scenes include a character and a virtual object. The method further includes receiving input data for manipulating the character by a user to interact with the virtual object and analyzing the input data to identify an interaction pattern of the character in the one or more scenes. The interaction pattern defines inputs for training an AI model associated with a user account of the user. The method includes enabling the character to interact with a new scene based on the AI model. The method includes tracking interactions of the character with the new scene to perform additional training of the AI model.
[0008] In one embodiment, a server for training a character of a game is described. The server includes a processor configured to facilitate display of one or more scenes of the game. The one or more scenes include a character and a virtual object. The processor receives input data for manipulating the character by a user to interact with the virtual object and analyzes the input data to identify an interaction pattern of the character in the one or more scenes. The interaction pattern defines an input for training an AI model associated with a user account of the user. The processor enables the character to interact with a new scene based on the AI model. The character interacts with the new scene according to the AI model. The processor tracks interactions of the character with the new scene to perform additional training of the AI model. The server includes a memory device coupled to the processor, the memory device configured to store the AI model.
[0009] In one embodiment, a computer readable medium including program instructions is described. Execution of the program instructions by one or more processors of a computer system causes the one or more processors to perform a plurality of operations, the plurality of operations including facilitating display of one or more scenes of a game. The one or more scenes include a character and a virtual object. The plurality of operations further includes receiving input data for manipulating the character by a user to interact with the virtual object, and analyzing the input data to identify an interaction pattern of the character in the one or more scenes. The interaction pattern defines an input for training an AI model associated with a user account of the user. The plurality of operations includes enabling the character to interact with a new scene based on the AI model. The plurality of operations includes tracking an interaction of the character with the new scene to perform additional training of the AI model.
[0010] Some advantages of the systems and methods described herein include that actions performed by a user during game play are monitored to allow the artificial intelligence model to learn from those actions. Also, the artificial intelligence model self-learns during execution of the game program of the game, learns from other artificial intelligence models during execution of the game program, and learns from new scenes of the game. These types of learning by the artificial intelligence model reduce the amount of input data transferred between a client device, such as a handheld controller, a head-mounted display, or a computing device, and one or more servers during game play, while providing a better gaming experience to the user. Reducing the amount of input data reduces the amount of network traffic transferred between the client device and one or more servers. Reducing the amount of network traffic increases the transfer speed of network data between the client and one or more servers. Thus, when the artificial intelligence model is trained, the input data transferred over the computer network is reduced, reducing network latency.
[0011] Also, the generation of input data is less predictable than using an artificial intelligence model during the execution of a game program. For example, a human user may take a long time to select a button on a handheld controller, perform a gesture, or take a break during game play. This makes the generation and transfer of input data from a client device to one or more servers less predictable. This makes the management of network traffic by one or more servers or computer networks less predictable. With an artificial intelligence model, the human factor becomes less important and the network traffic becomes more predictable, resulting in better network traffic management.
[0012] The various embodiments of the present disclosure are best understood by referring to the following description taken in conjunction with the accompanying drawings. [Brief description of the drawings]
[0013] [Figure 1A-1] FIG. 1 is a diagram of one embodiment of a system for illustrating artificial intelligence (AI) training. [Figure 1A-2] FIG. 1 is a diagram of an embodiment of another system for illustrating training of an AI model by a user. [Figure 1B] FIG. 1 is a diagram of one embodiment for explaining training of an artificial intelligence model by a user. [Diagram 2] FIG. 1 is a diagram of one embodiment of a virtual scene to illustrate training of an artificial intelligence model with other artificial intelligence models. [Figure 3-1] FIG. 1 is a diagram of an embodiment of a virtual scene to illustrate how an artificial intelligence model trains itself. [Figure 3-2] FIG. 1 illustrates one embodiment of an analysis by an artificial intelligence model to determine that an artificial intelligence model learned from one or more interaction patterns produces a better outcome or result compared to an AI model learned from another one or more interaction patterns. [Figure 4] FIG. 1 is a diagram of one embodiment of a new virtual scene to illustrate training an artificial intelligence model using the new virtual scene. [Figure 5A] FIG. 13 is a diagram of one embodiment to explain that an artificial intelligence model can apply any skill level during a game match. [Figure 5B] FIG. 2 illustrates an embodiment of a virtual scene in which a character is pitted against other characters during execution of a game program. [Figure 5C] 1 is an embodiment of a virtual scene to illustrate a user watching a battle between characters without controlling the characters. [Figure 6]FIG. 1 is a diagram of one embodiment of a system for illustrating a notification that is displayed on a display device of a head-mounted display (HMD) to indicate to a user that other users cannot play the game and that an artificial intelligence model can play the game with the user instead. [Figure 7] FIG. 1 is a diagram of one embodiment for explaining selection of an artificial intelligence model from multiple artificial intelligence models. [Figure 8A] FIG. 1 illustrates an embodiment of a system to illustrate capturing gestures performed by a user during execution of a game program so that an artificial intelligence model can apply the gestures to a character. [Figure 8B] FIG. 1 illustrates an embodiment of a virtual scene displayed on an HMD showing a character performing actions or gestures similar to those performed by a user while playing a game. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] A system and method for training an artificial intelligence (AI) model for competitive matches is described. It should be noted that various embodiments of the present disclosure may be practiced without some or all of these specific details. In other instances, well-known process operations have not been described in detail in order to avoid unnecessarily obscuring various embodiments of the present disclosure.
[0015] FIG. 1A-1 is a diagram of an embodiment of a system 10 for illustrating an artificial intelligence model AI1A. The system 10 includes a virtual scene 12, another virtual scene 14, an AI model AI1A, and another virtual scene 16. The virtual scenes 12, 14, and 16 shown in FIG. 1A-1 are displayed on a head-mounted display or other display device, such as a television, computer, smartphone, or tablet, during the playing of a video game or interactive game. In the virtual scene 12, a character C1 is swinging a weapon at another character CA. The character C1 moves towards the character CA in a pattern 26 to virtually kill the character CA.
[0016] Similarly, in virtual scene 14, character C1 climbs to the top of a virtual mountain and dives into virtual water on the other side of the virtual mountain. Character C1 follows movement pattern 28 to climb to the top of the virtual mountain and dive into the virtual water.
[0017] The AI model AI1A receives the pattern 26 as the AI input 18 and the pattern 28 as the AI input 20, learns from the patterns 26 and 28, and generates the AI output 22, which is the learned method. The AI output 22 is applied by the AI model AI1A to the virtual scene 16 as an AI input. For example, a character C1 without a virtual weapon in the virtual scene 16 is controlled by the AI model AI1A to climb to the top of a virtual mountain in the virtual scene 16, but does not dive. The character C1 descends the virtual mountain, encounters the character CA, and fights the character CA with his bare hands. In the virtual scene 16, there is no virtual water, but there is a virtual mountain, and the character CA is on the other side of the virtual mountain.
[0018] While interacting with the virtual scene 16 via the character C1, the AI model AI1A determines that a game level, such as a number of virtual coins or a number of game points, will increase when the character C1 climbs a virtual mountain in the virtual scene 16 and defeats the character CA with his bare hands, compared to when the character C1 climbs the virtual mountain but does not defeat the character CA, or compared to when the character C1 does not climb the virtual mountain and goes elsewhere in the virtual scene 16. Based on the determination regarding the game level, the AI model AI1A learns an interaction pattern 24, such as, for example, climbing a virtual mountain and fighting the character CA with his bare hands. The AI model AI1A applies the interaction pattern 24 to itself to learn or train from the interaction pattern for other instances of the video game or interactive game.
[0019] FIG. 1A-2 is a diagram of an embodiment of a system 100 for illustrating training of an AI model AI1A by a user A such that the AI model AI1A reacts in the same or similar manner as the user A reacts while playing a game. As an example, the artificial intelligence model described herein is a neural network of neural nodes. Each neural node may be a server or a processor. The neural nodes are coupled to each other via connections. For example, two adjacent neural nodes are connected to each other via connections. As another example, two server nodes are coupled to each other via a network cable, or two processors are connected to each other via a cable. An AI input is provided to the neural network, and an AI output is generated. The system 100 includes multiple servers A, B, and C. The system 100 also includes a computer network 102 and a computing device 104. The system 100 also includes a head mounted display (HMD) 106 and a handheld controller 108.
[0020] As used herein, a server includes one or more processors and one or more memory devices. For example, server 1 includes processor 118 and memory device 120. Processor 118 is coupled to memory device 120. One or more memory devices of one or more of servers A, B, and C store one or more AI models described herein. Examples of processors include application specific integrated circuits (ASICs), programmable logic devices (PLDs), microprocessors, and central processing units. Examples of memory devices include read only memory devices (ROMs) and random access memory devices (RAMs). Illustratively, memory devices are non-volatile or volatile memory devices. Examples of memory devices include redundant array of independent disks (RAIDs) and flash memory. Examples of handheld controllers described herein include a PlayStation Move controller, a joystick, a gun controller, and a sword controller.
[0021] The computer network 102 is a wide area network such as the Internet, or a local area network, or a combination thereof. Examples of computing devices described herein include a game console, or a computer, e.g., a desktop or laptop, or a smartphone. The HMD 106 is coupled to the computing device 104 via a wired or wireless connection, and the handheld controller 108 is coupled to the computing device 104 or the HMD 106 via a wired or wireless connection. As used herein, examples of wired connections between the handheld controller and the computing device, between the HMD and the computing device, or between the camera and the computing device include a coaxial cable connection or a Universal Serial Bus (USB) cable connection. As used herein, examples of wireless connections between the handheld controller and the computing device, between the HMD and the computing device, between the HMD and the handheld controller, or between the camera and the computing device include a Wi-Fi® connection, a Bluetooth® connection, or a short-range wireless connection. The computing device 104 is coupled to the servers A, B, and C via the computer network 102.
[0022] User A logs into user account 1 assigned to user A to access a game session of a game. For example, a user identification (ID) is authenticated by one or more of servers A, B, and C to allow user A to log into user account 1. Data related to the user accounts is stored in one or more memory devices of one or more of servers A, B, and C. For example, a user ID of user account 1 is stored in memory device 120, a user ID of another user account 2 is stored in memory device 120, and a user ID of another user account 3 is stored in memory device 120. User account 2 is assigned to another user B, and user account 3 is further assigned to another user C. After logging into user account 1, a game program is executed by one or more of servers A, B, and C, for example, by processor 118, during the game session to provide a cloud game. When the game program is executed by one or more of servers A, B, and C, one or more image frames for displaying a virtual scene are generated by one or more of servers A, B, and C. As used herein, an example of a virtual scene includes a virtual reality (VR) scene.
[0023] Image frames are transmitted from one or more of servers A, B, and C via computer network 102 and computing device 104 to HMD 106 to display the virtual scene on a display device of HMD 106. Examples of display devices include liquid crystal displays (LCDs) and light emitting diode displays (LEDs).
[0024] Upon viewing the virtual scene, User A manipulates the handheld controller 108 to select one or more buttons on the handheld controller 108 and / or move one or more joysticks on the handheld controller 108. The handheld controller 108 is manipulated by User A to generate input data 116.
[0025] The input data 116 is transmitted from the handheld controller 108 to the computing device 104 via a wired or wireless connection. The computing device 104 generates one or more packets by packetizing the input data by applying a network communication protocol, such as the Internet Protocol or the Transmission Control Protocol (TCP) / IP protocol, and transmits the one or more packets to one or more of the servers A, B, and C via the computer network 102. The one or more of the servers A, B, and C obtain the input data 116 from the packets by applying the network communication protocol. The one or more of the servers A, B, and C execute a game program to analyze the input data 116, thereby generating one or more image frames of the virtual scene 110 in which the character C1, representing the artificial intelligence model AI1A, is shooting the virtual object 112 but not other virtual objects 114 in the virtual scene 110. For example, when user A generates input data 116 using handheld controller 108, one or more of servers A, B, and C determine that character C1, representing artificial intelligence model AI1A, will shoot virtual object 112 before shooting virtual object 114 in the virtual scene 110, and generate one or more image frames for displaying the virtual scene 110.
[0026] Examples of virtual objects include virtual characters, game characters, virtual weapons, virtual vehicles, virtual aircraft, virtual boxes, virtual balloons, and avatars representing users. Character C1 is a non-player character (NPC) when operated by artificial intelligence model AI1A. Virtual object 112 is a character using a virtual parachute and about to land on the ground in virtual scene 110. Also, virtual object 114 is a character about to shoot character C1. Artificial intelligence model AI1A is associated, e.g., mapped or linked, to user account 1 by one or more of servers A, B, and C. Similarly, another artificial intelligence model AI2 is associated with user account 2 by one or more of servers A, B, and C, and yet another artificial intelligence model AI3 is associated with user account 3 by one or more of servers A, B, and C.
[0027] One or more of the servers A, B, and C analyze the input data 116 to identify or identify one or more interaction patterns 119 of the character C1 associated with the input data 116. As an example, the identified or identified one or more interaction patterns 119 indicate that in the virtual scene 110, the character C1 shoots the virtual object 112 before shooting the virtual object 114. The character C1 shoots the virtual object 112 when the user A aims at the virtual object 112 using the joystick of the handheld controller 108 and selects a button on the handheld controller 108 to shoot the virtual object 112. In the virtual scene 110, the character C1, which represents the artificial intelligence model AI1A, encounters the virtual object 112 that is about to land and also encounters another virtual object 114 that is about to shoot the character C1. As yet another example, the identified one or more interaction patterns 119 may indicate that in a majority of instances of a virtual scene in which the character C1 encounters a flying virtual object and also encounters other virtual objects that are about to shoot the character C1, the character C1 is likely to shoot the flying virtual object before shooting the other virtual objects.
[0028] One or more of the servers A, B, and C store the one or more interaction patterns 119 in one or more memory devices of the one or more servers A, B, and C as a training program for training the artificial intelligence model AI1A. For example, the one or more interaction patterns 119 are provided as inputs to the artificial intelligence model AI1A to enable the artificial intelligence model AI1A to learn from the one or more interaction patterns 119, and the learned method or behavior is applied by the artificial intelligence model AI1A to a new virtual scene different from the virtual scene 110. The new virtual scene is displayed from image frames generated by one or more of the servers A, B, and C executing a game program. The learned method or behavior may also be applied to a virtual scene identical or similar to the virtual scene 110.
[0029] One or more of the servers A, B, and C generate one or more packets by applying a network communication protocol to packetize one or more image frames of a virtual scene 110 in which the character C1 shoots the virtual object 112 but not the virtual object 114, and transmit the packets to the computing device 104 via the computer network 102. The computing device 104 obtains the one or more image frames by applying a network communication protocol to depacketize the one or more packets, and transmits the one or more image frames to the HMD 106 via a wired or wireless connection, thereby displaying the virtual scene 110 showing the shooting on a display device of the HMD 106.
[0030] In one embodiment, one or more of the servers A, B, and C associate one or more interaction patterns 119 with the virtual scene 110 having the character C1, the virtual object 112, and the virtual object 114, or with other virtual scenes similar to the virtual scene 110, in which some virtual objects are flying and other virtual objects are about to land on the virtual ground. For example, one or more of the servers A, B, and C generate identifiers for the one or more interaction patterns 119 and establish a mapping, such as a one-to-one correspondence or link, between the identifiers of the one or more interaction patterns 119 and the identifiers of the virtual scene 110. The mapping is stored in one or more memory devices of one or more of the servers A, B, and C and made accessible to the artificial intelligence model AI1A for its training.
[0031] In one embodiment, the HMD 106 communicates with the computer network 102 via a wireless network interface controller (WNIC), and the computing device 104 is not required for the HMD 106 to communicate with the computer network 102 .
[0032] In one embodiment, the HMD or computing device described herein communicates with one or more of the servers A, B, and C via the computer network 102 using a fifth generation (5G) network protocol. Similar to previous second generation (2G), third generation (3G), and fourth generation (4G) mobile networks, the 5G network is a digital cellular network in which the service area covered by the provider is divided into a collection of small geographic areas called cells. Analog signals representing sounds and images are digitized in the computing device or HMD by being converted by an analog-to-digital converter in the computing device or HMD and transmitted as a stream of bits to one or more of the servers A, B, and C via the computer network 102. All 5G wireless devices, including computers or HMDs in the cell, have transceivers that communicate over the air with local antenna arrays and low-power automatic transceivers in the cell on frequency channels assigned by the low-power automatic transceivers from a common pool of frequencies reused in geographically separated cells. The local antennas are connected to the telephone network and computer network 102 by high-bandwidth optical fiber or wireless backhaul connections. When a user moves from one cell to another, the HMD or computing device is automatically and seamlessly handed off to the new cell's antenna.The advantage is that 5G networks deliver much higher data rates than previous cellular networks, with 5G networks delivering up to 10Gbit / s, 100 times faster than 4G Long Term Evolution (LTE) cellular technology.
[0033] In one embodiment, a computing device 104, such as a smartphone, is part of the HMD 106. For example, a display device of the computing device 104 is used as a display device of the HMD 106.
[0034] In one embodiment, the game is displayed on the computing device 104 instead of the HMD 106, or on a television coupled to the computing device 104 via a wireless or wired connection.
[0035] In one embodiment, one or more glove controllers, or one or more ring controllers, or no controllers are used instead of the handheld controllers. For example, the glove controllers are worn on the user's hands and the ring controllers are worn on the fingers of the user's hands. When no controllers are used, the user makes gestures of one or more parts of his or her body, and the gestures are captured by an image capture device, such as a camera. Examples of cameras include depth cameras, video cameras, and digital cameras. The image capture device is located in the same real-world environment, such as a room, warehouse, building, or house, where the user is located. The camera is coupled to the computing device via a wired or wireless connection to communicate input data, including gesture data identifying the gesture, to the computing device and the servers A, B, and C via the computer network 102. In an embodiment where no computing device is used, the camera includes a network interface controller, such as a network interface card (NIC) or a wireless network interface card, to communicate with the servers A, B, and C.
[0036] In one embodiment, the HMD includes one or more cameras that capture gestures made by the user's hands and output gesture data.
[0037] In one embodiment, the operations described herein performed by one or more of servers A, B, and C are performed by one or more processors in one or more of the servers.
[0038] In one embodiment, all artificial intelligence models, such as artificial intelligence models AI1A, AI2, and AI3 described herein, are executed by one or more of servers A, B, and C. For example, each server is a node of the artificial intelligence model. As another example, two or more servers form a node of the artificial intelligence model. A connection between any two servers is a connection between any two nodes of the artificial intelligence model. In one embodiment, the artificial intelligence described herein is a combination of one or more of servers A, B, and C and computer operations performed by one or more of servers A, B, and C. In one embodiment, the artificial intelligence models described herein learn from one or more interaction patterns, and the learned operations of the artificial intelligence models are applied by one or more of servers A, B, and C to obtain results.
[0039] It should be noted that all operations described herein with respect to the virtual scene 110 apply equally to multiple virtual scenes displayed on the HMD 106.
[0040] FIG. 1B is a diagram of an embodiment for explaining the training of the artificial intelligence model AI1A. One or more of the servers A, B, and C analyze the input data 116 to identify a function associated with the input data 116, such as f1, f2, f3-fn, etc. For example, the input data 116 is analyzed to determine that a button labeled x on the handheld controller 108 was selected by the user A, or the joystick was moved upward by the user A, or the user slid a finger on the touch screen of the handheld controller 108, or a gesture was made by the user A. Examples of functions associated with the input data 116 include a selection of a button on the handheld controller 108 operated by the user A, a finger movement on the touch screen, or a directional or rotational movement of a joystick. To further illustrate, examples of functions associated with the input data 116 include a selection of a button on the handheld controller 108, a movement of the joystick on the handheld controller 108 in a certain direction, an aiming gesture made by the user A, a shooting gesture made by the user A, etc.
[0041] Each of the functions f1-fn associated with the input data 116 is a different function or a different gesture performed by the user A. For example, the function f1 is the selection of a button marked with "O" on the handheld controller 108, and the function f2 is the selection of another button marked with "X" on the handheld controller 108. As another example, the function f1 is an aiming gesture performed by the index finger of the left hand of the user A, and the function f2 is a shooting gesture performed by the index finger and the middle finger of the user's left hand.
[0042] Based on the functions associated with the input data 116 and the virtual scene 110, one or more of the servers A, B, and C determine features of the virtual scene 110, e.g., fe1, fe2-fen, etc., and classify the features to output classifiers, e.g., Cl1, Cl2, Cl3-Cln, etc. An example of a feature of the virtual scene 110 is a character C1 moving and shooting. Another example of a feature of the virtual scene 110 is a virtual object 112 flying with a virtual parachute through the virtual air. Yet another example of a feature of the virtual scene 110 is a virtual object 114 holding a virtual gun. An example of a classifier includes a function, movement, action, or combination thereof performed by the character C1 based on the manipulation of the handheld controller 108 or based on one or more gestures made by the user A. By way of illustration, examples of classifiers include the character C1 shooting a virtual object 112, opening a treasure chest in the game, landing on a virtual building in the game, or shooting a virtual aircraft in the game. To further illustrate, an example of classifier Cl2 is that the character C1 moves towards the virtual object 112. An example of classifier Cl3 is that the character C1 shoots the virtual object 112. An example of classifier Cl4 is that the character C1 does not shoot the virtual object 114, and an example of classifier Cln is that the character C1 shoots the virtual object 114 after the virtual object 114 shoots the virtual object 112. Each classifier identifies a movement or action performed by the character C1 when present in the virtual scene 110. Each classifier Cl1-Cln corresponds to a different function or movement or combination thereof performed by the character C1 in the virtual scene 110. For example, in the virtual scene 110, classifier Cl1 is a shooting function and classifier Cl2 is a jumping function.
[0043] The classifier is provided as an AI input to the artificial intelligence model AI1A by one or more of the servers A, B, and C to train the artificial intelligence model AI1A. For example, the processor 118 identifies one or more interaction patterns 119 in the classifier of the virtual scene 110. By way of illustration, upon determining from the one or more interaction patterns 119 that when the character C1 is placed in the virtual scene 110 in which the virtual object 112 is flying toward the character C1 and the virtual object 114 is trying to shoot the character C1, the character C1 shoots the virtual object 112 instead of shooting the virtual object 114, the one or more of the servers A, B, and C train the AI model AI1A to determine that when the character C1 faces the same or a similar situation in another execution instance of the game program, the character C1 is more likely to shoot or try to shoot the flying virtual object first instead of shooting other virtual objects on the ground. An example of a similar situation is a situation where a virtual object is flying towards the character C1 and another virtual object is on the ground ready to shoot the character C1. Another example of a similar situation is a situation where a predetermined number, such as a majority, of the virtual objects are performing the same function as the function performed by the virtual objects 112 and 114 in the virtual scene 110. As another example, the processor 118 trains the AI model AI1A to determine that when the character C1 is presented with an instance of the game in which the character C1 encounters a virtual aircraft flying towards the character C1 or a virtual weapon on the ground that is about to shoot the character C1, the character C1 is more likely to shoot the virtual aircraft flying towards the character C1 than otherwise.In yet another example, the processor 118 trains the neural network of the AI model AI1A to determine that, given an input of a virtual object flying toward the character C1 and an input of another virtual object trying to shoot the character C1, the output is more likely than not to result in the character C1 first shooting the virtual object flying toward the character C1 before shooting the other virtual object. As another example, one or more of the servers A, B, and C train the AI model AI1A using a classifier to determine that, given a different or new virtual scene in which no virtual objects are flying or trying to shoot the character C1, the character C1 does not shoot any of the virtual objects but dances in front of them or exchanges a virtual weapon with them. An example of a different or new virtual scene is a virtual scene in which a predetermined number, such as a majority, of the virtual objects perform a different function than the function performed in the virtual scene 110. To illustrate, the virtual objects in the new or different virtual scene are collecting wood or dancing and do not have a virtual weapon aimed at the character C1.
[0044] The identified one or more interaction patterns 119 are applied as AI inputs to the AI model AI1A by one or more of the servers A, B, and C. For example, the AI model AI1A is a set of computer operations executed by the processor 118 or one or more of the servers A, B, and C. The set of computer operations is not written or coded by a human user, but is learned by the AI model AI1A from a virtual scene, from other AI models, from operations performed by a user via a handheld controller or gestures, from a match, or from a combination thereof.
[0045] 2 is a diagram of an embodiment of a virtual scene 202 for explaining the training of the artificial intelligence model AI1A by the artificial intelligence model AI2. The virtual scene 202 is generated in a similar manner to the virtual scene 110 being generated by one or more of the servers A, B, and C during game play. For example, image frames for displaying the virtual scene 110 are generated by one or more of the servers A, B, and C, and transmitted to the HMD 106 via the computer network 102 and the computing device 104, and the virtual scene 202 is displayed on the HMD 106. However, it is not necessary to display the virtual scene 202 or generate image frames related to the virtual scene 202 for the artificial intelligence model AI1A to learn from the artificial intelligence model AI2.
[0046] The artificial intelligence model AI1A requests permission from the artificial intelligence model AI2 to train itself based on the artificial intelligence model AI2. For example, the processor 118 executes the artificial intelligence model AI1A to generate a request for permission and sends the request to the artificial intelligence model AI2 via the user account 2. Upon receiving the request for permission via the user account 2, one or more of the servers A, B, and C determine whether the request is granted. For example, the user B uses the handheld controller to send data to one or more of the servers A, B, and C indicating that the artificial intelligence model AI2 is not granted access to other users, is granted access to a selected user group, is granted access to a private user group, is granted access to all users who request access, or is granted access to all users regardless of whether the user requests access to the artificial intelligence model AI2. The data regarding the instruction to access the artificial intelligence model is associated with the user account 2, for example, linked or mapped. The artificial intelligence model AI2 is associated with the user account 2 and is created by the user B in a similar manner as the artificial intelligence model AI1A is created by the user A.
[0047] Upon determining that the request is granted, one or more additional patterns 204 or additional classifiers of the artificial intelligence model AI2 are sent from one or more of the servers A, B, and C running the artificial intelligence model AI2 to one or more of the servers A, B, and C running the artificial intelligence model AI1A. The one or more additional patterns 204 used to train the artificial intelligence model AI2 are interaction patterns and are created by one or more of the servers A, B, and C in a similar manner as the one or more interaction patterns 119 and other interaction patterns of the artificial intelligence model AI1A are created by one or more of the servers A, B, and C. The one or more additional patterns 204 are applied by one or more of the servers A, B, and C to train the artificial intelligence model AI2.
[0048] Upon receiving the one or more additional patterns 204, one or more of the servers A, B, and C provide the one or more additional patterns 204 to the artificial intelligence model AI1A as AI inputs to train the artificial intelligence model AI1A based on the one or more additional patterns 204. The artificial intelligence model AI1A learns from the one or more additional patterns 204. For example, instead of controlling the character C1 to shoot the virtual object 112 flying toward the character C1, the artificial intelligence model AI1A learns to control the character C1 to open a virtual object 206, such as a virtual box or a virtual container, after the virtual object 206 lands on the virtual ground. The virtual object 206 includes virtual weapons, such as guns and bullets, as well as virtual medicines and bandages. The character C2, which is controlled by the artificial intelligence AI2 assigned to the user account 2, also applies the one or more additional patterns 204 to open the virtual object 206. The virtual object 206 is a box attached to a balloon, as shown. When controlled by the artificial intelligence model AI2, the character C2 is an NPC.
[0049] In one embodiment, the artificial intelligence model AI2 performs additional types of training as described herein, such as self-training or training from other artificial intelligence models or new virtual scenes.
[0050] In one embodiment, some characters in the game are controlled by a user, and the remaining characters in the game are NPCs that are controlled by corresponding artificial intelligence models.
[0051] FIG. 3-1 is a diagram of an embodiment of a virtual scene 110 for explaining the artificial intelligence model AI1A training itself. For example, regardless of whether user A is logged in to user account 1, the artificial intelligence model AI1A learns from one or more other patterns 302, which are interaction patterns, to increase the skill level associated with user account 1. For example, when user A is logged out of user account 1, the artificial intelligence model AI1A logs in to user account 1 and requests the processor 118 to execute a game program. When the game program is executed, the artificial intelligence model AI1A determines that the game skill level, for example, the game score, game level, or virtual coin number, etc., will increase when the character C1 is placed in the virtual scene 110 and the character C1 shoots the virtual object 114 first before shooting the virtual object 112. The one or more other patterns 302 indicate that when the character C1 is presented with the virtual scene 110 in which the virtual object 112 is flying towards the character C1 and the virtual object 114 is about to shoot the character C1, in order to increase the skill level associated with the user account 1, the character C1 should shoot the virtual object 114 first before shooting the virtual object 112. The skill level is increased above the other skill levels associated with the user account 1 and is an example of an AI output of the artificial intelligence model AI1A. The other skill levels correspond to training the artificial intelligence model AI1A by applying one or more interaction patterns 119 shown with respect to Figures 1A-2 and 1B. The artificial intelligence model AI1A trains itself by applying the one or more other patterns 302 as an AI input that is an input to the AI model AI1A.
[0052] As another example, when user A is logged into user account 1 and virtual scene 110 is displayed on the display device of HMD 106, user A manipulates handheld controller 108 to shoot virtual object 112 first rather than shooting virtual object 114. Artificial intelligence model AI1A learns from one or more other patterns 302 by applying machine learning to determine that when character C1 shoots virtual object 114 before shooting virtual object 112, the skill level associated with user account 1 increases.
[0053] FIG. 3-2 is a diagram of one embodiment for illustrating an analysis by an artificial intelligence model AI1A to determine that applying one or more other patterns 302 as an AI input results in a better outcome or result compared to applying one or more interaction patterns 119 as an AI input. When the artificial intelligence model AI1A is trained to apply one or more interaction patterns 119, a plurality of outputs O1, O2-On are generated while one or more of the servers A, B, and C are executing a game program. Examples of each output O1, O2-On include the number of points accumulated during the execution of the game program, the number of virtual character kills accumulated during the execution of the game program, and the health level of the character C1.
[0054] Similarly, when the artificial intelligence model AI1A is trained to apply one or more other patterns 302, a plurality of outputs O11, O12 to O11n are generated while one or more of the servers A, B, and C are executing the game program. Examples of each output O11, O12 to O11n include the number of points accumulated during the execution of the game program, the number of virtual character kills accumulated during the execution of the game program, and the health level of the character C1. The artificial intelligence model AI1A learns by applying one or more other patterns 302 during the execution of the game program to obtain outputs O11 to O11n and determining that the outputs O11 to O11n correspond to a higher skill level than the skill level corresponding to the outputs O1 to On. Upon determining that the outputs O11 to O11n correspond to a higher skill level than the skill level corresponding to the outputs O1 to On, the artificial intelligence model AI1A determines to apply one or more other patterns 302 instead of the one or more interaction patterns 119.
[0055] During additional execution instances of the game program, the artificial intelligence model AI1A, instead of applying the interaction pattern 119, applies one or more other patterns 302. For example, during additional execution instances of the game program, when a virtual scenario in which a virtual character is about to shoot a character C1 and another virtual character is about to land in front of the character C1 is displayed on the HMD 106, the artificial intelligence model AI1A controls the character C1 to first shoot the character that is about to shoot the character C1, and then shoots the virtual character that is about to land.
[0056] In one embodiment, the artificial intelligence model AI1A requests permission from the user A before applying one or more interaction patterns described herein. For example, during the display of the virtual scene 110, one or more of the servers A, B, and C generate an image frame including a request for permission, apply a network communication protocol to packetize the image frame to generate one or more packets, and transmit the packets to the computing device 104 via the computer network 102. The computing device 104 obtains the image frame by applying a network communication protocol to depacketize the packets, and transmits the image frame to the HMD 106 via a wired or wireless connection to display the request for permission. The user A uses the handheld controller 108 to indicate whether to grant or reject the request. Upon receiving a positive indication that the request is granted, the positive indication is packetized by the computing device 104 to generate one or more packets, and the packets are transmitted by the computer network 102 to one or more of the servers A, B, and C. One or more servers A, B, and C apply a network communication protocol to the packets to obtain a positive indication, and send the positive indication to the artificial intelligence model AI1A. Upon receiving the positive indication, the artificial intelligence model AI1A applies one or more interaction patterns.
[0057] On the other hand, upon receiving a negative indication that the request has been denied, the negative indication is packetized by the computing device 104 to generate one or more packets, and the packets are transmitted by the computer network 102 to one or more servers A, B, and C. The one or more servers A, B, and C apply a network communication protocol to the packets to obtain the negative indication and transmit the negative indication to the artificial intelligence model AI1A. Upon receiving the negative indication, the artificial intelligence model AI1A does not apply the one or more interaction patterns.
[0058] FIG. 4 is a diagram of an embodiment of a new virtual scene 400 to explain training the artificial intelligence model AI1A using the new virtual scene 400. The new virtual scene 400 is generated when one or more of the servers A, B, and C execute a game program to generate one or more image frames, and apply a network communication protocol to packetize the image frames to generate one or more packets. As an example, the artificial intelligence model AI1A requests one or more of the servers A, B, and C to execute a game program to generate one or more image frames for displaying the new virtual scene 400. The packets are sent by one or more of the servers A, B, and C to the computing device 104 via the computer network 102. The computing device 104 applies a network communication protocol to extract the image frames from the packets, and sends the image frames to the HMD 106 to display the virtual scene 400. The virtual scene 400 includes the character C1, the virtual object 112, and another virtual object 402, but does not include the virtual object 114. Virtual object 402 is an aircraft in which virtual object 404 is seated and shoots character C1. Virtual object 404 shoots character C1 using a virtual gun of virtual object 402. Virtual scene 400 has not been presented to user A during play of the game. For example, user A has not reached a certain level during execution of the game program, which level includes virtual objects 402 and 404.
[0059] The artificial intelligence model AI1A, having learned from any of the interaction patterns described herein, such as the one or more interaction patterns 119, the one or more additional patterns 204, the one or more other patterns 302, or a combination thereof, applies the learned behavior to manipulate the character C1 to shoot the virtual object 402 or the virtual object 404 first, rather than shooting the virtual object 112. For example, the artificial intelligence model AI1A determines that by shooting one or more of the virtual objects 402 and 404 first, a skill level associated with the user account 1 is increased as compared to a skill level achieved by shooting the virtual object 112 first. The increased skill level is an example of the reaction data 406. For example, the increased skill level includes a number of virtual coins corresponding to the user account 1, a virtual score corresponding to the user account 1, or a combination thereof.
[0060] Based on the reaction data 406, the artificial intelligence model AI1A determines or learns that following one or more additional interaction patterns 408 will help achieve a higher skill level. An example of the one or more additional interaction patterns 408 includes shooting the virtual object 402 or the virtual object 404 first and then shooting the virtual object 112. Upon so determining, the artificial intelligence model AI1A trains itself to apply the one or more additional interaction patterns 408 during the next or subsequent execution instance of the game program. For example, when faced with an execution instance of the game program where a virtual flying object is approaching the character C1 and at the same time the virtual character is about to land on the virtual ground, the artificial intelligence model AI1A manipulates the character C1 to shoot the virtual flying object first and then shoot the virtual character currently on the virtual ground.
[0061] When the artificial intelligence model AI1A is presented with a new virtual scene 400, the artificial intelligence model AI1A trains itself in a similar manner as described above with reference to FIG. 3-2. For example, when the new virtual scenario 400 is presented, the artificial intelligence model AI1A determines whether a game level, such as the number of virtual points in the game or the game score, will increase by shooting down the virtual object 402 or the virtual object 404 before shooting the virtual object 112. The increase in the game level is higher compared to the increase in the game level when the virtual object 112 is shot first by the character C1 before shooting the virtual object 402 or 404. When it is determined that the game level will increase, the artificial intelligence model AI1A learns to shoot the flying virtual aircraft first before shooting the flying virtual character attached to the parachute. On the other hand, if it is determined that the increase in game level is lower compared to the amount of increase in game level that would occur if virtual object 112 were shot by character C1 prior to shooting virtual object 402 or 404, then artificial intelligence model AI1A learns to shoot the flying virtual character attached to a parachute prior to shooting the flying virtual aircraft.
[0062] FIG. 5A is a diagram of an embodiment for explaining that the artificial intelligence model AI1A can apply any skill level in the range of 1 to 5 during a game match. As an example, all training of the artificial intelligence model AI1A described herein is performed with the intention of training the artificial intelligence model AI1A to play against the artificial intelligence model AI3 in the game. In the match, one or more virtual objects or virtual characters of the game compete to win in the game, to achieve a predetermined number of kills of other virtual characters in the game, to achieve a predetermined number of virtual points in the game, to survive, or a combination thereof. All of the skill levels 1 to 5 are stored in a data file, and the data file is stored in one or more memory devices of one or more of the servers A, B, and C. An example of skill level 1 is the skill level before user A trains the artificial intelligence model AI1A, as shown in FIG. 1A-2 and FIG. 1B, and an example of skill level 2 is the skill level after user A trains the artificial intelligence model AI1A. An example of skill level 2 is a skill level before the artificial intelligence model AI1A is trained by the artificial intelligence model AI2, as shown in FIG. 2, and an example of skill level 3 is a skill level after the artificial intelligence model AI1A is trained by the artificial intelligence model AI2. Also, an example of skill level 3 is a skill level before the artificial intelligence model AI1A trains itself, as shown in FIG. 3-1, and an example of skill level 4 is a skill level after the artificial intelligence model AI1A trains itself. An example of skill level 4 is a skill level before training the artificial intelligence model AI1A using the new scene 400 shown in FIG. 4A, and an example of skill level 5 is a skill level after training the artificial intelligence model AI1A using the new scene 400. Skill levels 1 to 5 are assigned, for example, mapped or linked, to the artificial intelligence model AI1A by one or more of the servers A, B, and C.
[0063] Skill level 2 is higher than skill level 1. Also, skill level 3 is higher than skill level 2, skill level 4 is higher than skill level 3, and skill level 5 is higher than skill level 4. For example, skill level 1 is a low skill level, skill level 2 is a medium skill level, skill level 3 is a high skill level, skill level 4 is an excellent skill level, and skill level 5 is the highest skill level. The artificial intelligence model AI1A accesses one of skill levels 1 to 5 from the data file and applies it during the execution of the game program. The skill level of the artificial intelligence model AI1A increases with time t.
[0064] In one embodiment, the skill level shown in FIG. 5A is the skill level of artificial intelligence model AI1A, and is also the skill level associated with user account 1.
[0065] In one embodiment, the artificial intelligence model AI1A competes against an artificial intelligence model AI3 controlling a character C3.
[0066] In one embodiment, user account 3 and artificial intelligence model AI3 are assigned to user C other than user B.
[0067] FIG. 5B is a diagram of an embodiment of a virtual scene 508 in which a character C1 plays against a character C3 during execution of a game program. The character C3, operated by an artificial intelligence AI3 assigned to the user account 3, plays against the character C3 by applying one or more interaction patterns similar to those described herein. When operated by the artificial intelligence model AI3, the character C3 is an NPC. User C logs into his user account 3 to access the virtual scene 508. The virtual scene 508 is displayed on the HMD 106 as well as the HMD 510 to provide a shared virtual environment. User C wears the HMD 510 on his head in a similar manner to how user A wears the HMD 106 on his head. In the virtual scene 508, user C uses the handheld controller 512 to operate the character C3, or the artificial intelligence model AI3 operates the character C3, and user A uses the handheld controller 108 to operate the character C1, or the artificial intelligence model AI1A operates the character C1. The characters C1 and C3 compete or play against each other during execution of the game program to achieve a virtual reward, such as a virtual number of coins, a virtual number of points, a number of kills, or a game level within a game.
[0068] User C selects, via handheld controller 512, a skill level similar to character C3 or skill level 3 to be applied by artificial intelligence model AI3. For example, user C selects a skill level of character C3 during a match by selecting a button on handheld controller 512. Data indicative of the selection is transmitted to computing device 514 via a wired or wireless connection between handheld controller 512 and computing device 514. Computing device 514 generates one or more packets by applying a network communication protocol to packetize the selection data, and transmits the packets to one or more servers A, B, and C via computer network 102. One or more of servers A, B, and C receive the packets, apply a network communication protocol to the packets to obtain the selection data, analyze the selection data to determine that user C wishes to apply the skill level of character C3, and one or more interaction patterns corresponding to the skill level selected by user C are selected by artificial intelligence model AI3 to be applied during the match. One or more of servers A, B, and C apply the artificial intelligence model AI3 and further apply one or more interaction patterns corresponding to the skill level selected by user C. For example, when character C3 is in the virtual scene 202 of FIG. 2, character C3 approaches virtual object 206 rather than shooting virtual object 112.
[0069] The artificial intelligence model AI1A sends a request to the artificial intelligence model AI3 to determine the skill level of the character C3. For example, one or more of the servers A, B, and C that apply or execute the artificial intelligence model AI1A send a request to one or more of the servers A, B, and C that apply or execute the artificial intelligence model AI3 together with the ID of the user account 3, the ID of the character C3, and the ID of the game session of the execution of the game program. The artificial intelligence model AI3 executed by one or more of the servers A, B, and C receives the IDs of the user account 3, the character C3, and the game session together with the request and determines that the request corresponds to the user account 3, the character C3, and the game session and is for the ID of the skill level of the character C3. The artificial intelligence model AI3 accesses a database stored in one or more of the memory devices of the one or more servers A, B, and C to determine whether there is permission in the user account 3 to provide the skill level of the character C3 to the artificial intelligence model AI1A. For example, user C uses handheld controller 512 to provide permission to access the skill level of character C3 to all artificial intelligence models or one or more specific artificial intelligence models. The permission is transmitted from handheld controller 512 to computing device 514 via a wired or wireless connection. Computing device 514 generates one or more packets by packetizing the permission by applying a network communication protocol, and transmits the one or more packets to one or more servers A, B, and C via computer network 102. One or more servers A, B, and C associate the permission with artificial intelligence model AI3 or user account 3, for example, establishing a link or one-to-one correspondence or mapping between them.
[0070] When the artificial intelligence model AI1A determines that access to the skill level of the character C3 is permitted, the artificial intelligence model AI3 provides permission to the artificial intelligence model AI1A. Upon receiving permission from the artificial intelligence model AI3, the artificial intelligence model AI1A determines to apply a skill level 3 to the character C1 during a match. The skill level 3 is similar or substantially the same or identical to the skill level of the character C3. As an example, the skill levels are substantially the same or similar when they are within a predetermined range of each other. To illustrate, a skill level 2.5 is within a predetermined range from the skill level 2, and a skill level 3.4 is within a predetermined range from the skill level 3. The artificial intelligence model AI1A accesses a pattern corresponding to the skill level 3 of the character C3, such as one or more additional patterns 204. The artificial intelligence model AI1A applies the pattern corresponding to the skill level 3 to the character C1 during a match in the game.
[0071] In one embodiment, user C selects a skill level for character C3 during a match by performing a gesture. Data indicative of the gesture is captured by a camera disposed in the real-world environment where user C is located and transmitted to computing device 514 via a wired or wireless connection between the camera and computing device 514. Computing device 514 generates one or more packets by packetizing the gesture data by applying a network communication protocol, and transmits the packets to one or more servers A, B, and C via computer network 102. One or more of servers A, B, and C receive the packets, apply a network communication protocol to the packets to obtain the gesture data, analyze the gesture data to determine that user C wants to apply a skill level for character C3, and one or more interaction patterns corresponding to the skill level selected by user C are selected by artificial intelligence model AI3 to be applied during the match.
[0072] In one embodiment, each of the artificial intelligence models AI1A and AI3 collects output data, such as information including game scores or virtual points, during a match, and determines a pattern based on the output data to facilitate training the artificial intelligence model to win the match. For example, during a match, the artificial intelligence model AI1A receives data that the character C3 operated by the artificial intelligence model AI3 jumps when shot in the majority of instances. From this data, the artificial intelligence model AI1A identifies an interaction pattern that causes the artificial intelligence model AI3 to manipulate the character C3 to jump. The artificial intelligence model AI1A learns that it is easy to kill the character C3 by manipulating the character C1 to jump when the character C3 jumps. During the next match, the artificial intelligence model AI1A applies the learned behavior to manipulate the character C1 to jump and shoot the character C3 when the character C3 jumps.
[0073] 5C is an embodiment of a virtual scene 508 for illustrating that users A and B watch a match between characters C1 and C3 without controlling the characters C1 and C3. For example, during the match, an artificial intelligence model AI1A controls the character C1, and an artificial intelligence model AI3 controls the character C3. For example, during the match, one or more interaction patterns 119, one or more additional patterns 202, one or more other patterns 302, one or more additional interaction patterns 408, or a combination thereof are applied by the artificial intelligence model AI1A to control the character C1. Similarly, during the match, one or more interaction patterns corresponding to the user account 3 and the artificial intelligence model AI3 are applied by the artificial intelligence model AI3 to control the character C3.
[0074] There is no manipulation of character C1 by user A via handheld controller 106 or using gestures, and there is no manipulation of character C3 by user C via handheld controller 512 or using gestures. For example, there is no input data sent from handheld controller 106 via computing device 104 and computer network 102 to one or more of servers A, B, and C to manipulate movements or actions performed by character C1, such as shooting, exploring, collecting, acquiring, or searching. Also, there is no input data sent from handheld controller 512 via computing device 514 and computer network 102 to one or more of servers A, B, and C to manipulate movements or actions performed by character C3. Users A and C passively watch the match via their corresponding HMDs 106 and 510 without manipulating the corresponding characters C1 and C3. Characters C1 and C3 are manipulated by corresponding artificial intelligence models AI1A and AI3 without receiving input data. When the user is not using the handheld controller or making gestures, no input data is generated.
[0075] 6 is a diagram of an embodiment of a system 600 to illustrate a notification 510 displayed on a display device of an HMD 602 indicating to User C that User A cannot play the game and that the artificial intelligence model AI1A can play the game with User C. User C selects one or more buttons on a handheld controller 512 to generate input data including a request to send to User A via User Account 1 to play the game and an identification of User Account 3. The input data is transmitted from the handheld controller 512 to a computing device 514 via a wired or wireless connection. The computing device 514 applies a network communication protocol to the input data to generate one or more packets having the input data and transmits the packets over the computer network 102 to User Account 1, whose user account information is stored in one or more memory devices of one or more of Servers A, B, and C.
[0076] When the request is received by user account 1, user A has logged out of user account 1 and is not playing a game. For example, user A logs out of user account 1 by using the handheld controller 108 to select a log out button displayed on the display device of the HMD 106. When the log out button is selected, input data indicating the selection of the log out button is generated by the handheld controller 108 and transmitted from the handheld controller 108 to the computing device 104 via a wired or wireless connection. The computing device 104 packetizes the input data to generate one or more packets and transmits the packets to one or more of the servers A, B, and C via the computer network 102 to indicate to the one or more of the servers A, B, and C that user A has logged out of user account 1. As another example, user A logs out of a user account by selecting a power off button on the computing device 104.
[0077] One or more of servers A, B, and C generate a response to the request received from user account 3 indicating that user A has logged out of user account 1 and that the artificial intelligence model AI1A corresponding to, e.g., linked or mapped to, user account 1 is available to play the game. One or more of servers A, B, and C apply a network communication protocol to the response to generate one or more packets and transmit the packets to computing device 514 over computer network 102. Computing device 514 applies a network communication protocol to the packets to obtain the response from the packets and transmits data for displaying the response to HMD 510 over a wired or wireless connection.
[0078] The HMD 510 displays the response as a notification 610 to indicate to the user C that the user A cannot play the game and the artificial intelligence model AI1A can play the game instead. The notification 610 also includes a question for the user C as to whether the user C wants the artificial intelligence model AI1A to play the game instead of the user A. The user C generates input data by selecting one or more buttons on the handheld controller 512 to provide an answer to the question, and the input data is transmitted from the handheld controller 512 to the computing device 514 via a wired or wireless connection. The computing device 514 generates one or more packets by packetizing the input data including the answer by applying a network communication protocol, and transmits the packets to the user account 1 stored in one or more of the servers A, B, and C via the computer network 102. The one or more of the servers A, B, and C apply a network communication protocol to the packets to obtain the data including the answer, and transmit the answer to the artificial intelligence model AI1A.
[0079] Upon receiving the answer, the artificial intelligence model AI1A plays the game with the user C. For example, one or more servers A, B, and C, trained based on one or more of the interaction patterns 119, one or more additional patterns 204, one or more other patterns 302, one or more additional interaction patterns 408, or a combination thereof, apply the methods they learned from the training to play the game with the user C.
[0080] FIG. 7 is a diagram of an embodiment for explaining the selection of the artificial intelligence model AI1A from multiple artificial intelligence models AI1A, AI2A, and AI3A, all corresponding to the user account 1. For example, the artificial intelligence model AI2A is trained based on one or more interaction patterns to operate another character C11 during the execution of the game program, and the artificial intelligence model AI3A is trained based on one or more interaction patterns to operate yet another character C12 during the execution of the game program. As another example, the artificial intelligence model AI1A operates a character C1 having a skill level 2, the artificial intelligence model AI2A operates a character C1 having a skill level 3 or 4, and the artificial intelligence model AI3A operates a character C1 having a skill level 5. The artificial intelligence models AI2A and AI3A are generated in a similar manner to the artificial intelligence model AI1A is generated, and are executed by one or more of the servers A, B, and C.
[0081] One or more of servers A, B, and C generate one or more image frames including data for displaying the virtual scene 702. One or more of servers A, B, and C apply a network communication protocol to the image frames to generate one or more packets and transmit the packets over computer network 102 to computing device 104. Computing device 104 applies a network communication protocol to the packets to obtain the image frames and transmits the image frames over a wired or wireless connection to HMD 106 to display the virtual scene 702 on a display device of HMD 106.
[0082] Upon viewing the virtual scene 702 while the game program is running, user A selects an artificial intelligence model AI1A for playing the game by selecting one or more buttons using the handheld controller 108. Input data including the selection is transmitted from the handheld controller 108 to the computing device 104 via a wired or wireless connection. The computing device 104 generates one or more packets by applying a network communication protocol to packetize the selection and transmits the packets to one or more of the servers A, B, and C over the computer network 102.
[0083] One or more of the servers A, B, and C apply a network communication protocol to the packets to obtain a selection and decide to apply the artificial intelligence model AI1A for playing the game. Similarly, other artificial intelligence models AI2A and AI3A are selected by the user A via the handheld controller 108. When the artificial intelligence model AI1A is applied, the character C1 is operated by the artificial intelligence model AI1A during the execution of the game program.
[0084] In one embodiment, one of the artificial intelligence models AI1A, AI2A, and AI3A is selected by user A by making a gesture, which is captured by a camera in a real-world environment to generate gesture data. The gesture data is transmitted from the camera to the computing device 104 via a wired or wireless connection between the camera and the computing device 104. The computing device 104 generates and transmits one or more packets containing the gesture data to one or more of the servers A, B, and C via the computer network 102.
[0085] 8A is a diagram of an embodiment of a system 800 for illustrating capturing gestures made by a user A during execution of a game program so that an artificial intelligence model AI1A can apply the gestures to a character C1. The system 800 includes an HMD 106, a handheld controller 108, a camera 802, a computing device 104, a computer network 102, and servers A, B, and C.
[0086] The camera 802 is coupled to the computing device via a wired or wireless connection. During game play, the user A performs one or more gestures. For example, after the user 106 kills a virtual character in a game, the user 106 raises his or her right hand in the air and yells out loud, "Yay! I won!" The camera 802 records one or more gestures performed by the user A during execution of the game program to generate gesture data 804, which is an example of input data, and transmits the gesture data 804 to the computing device 104 via a wired or wireless connection. The gesture data 804 includes audio data and video data.
[0087] The computing device 104 applies a network communication protocol to the gesture data 804 to generate one or more packets and transmits the packets over the computer network 102 to one or more of the servers A, B, and C that execute the game program. The one or more of the servers A, B, and C apply the network communication protocol to obtain the gesture data 804 from the packets and apply the gesture data to train the artificial intelligence model AI1A. For example, the one or more of the servers A, B, and C train the artificial intelligence model AI1A by identifying a pattern 808 from the gesture data 804 and integrating, such as adding, the pattern 808 into its own method learned from the one or more interaction patterns 119, the one or more additional patterns 204, the one or more other patterns 302, and / or the one or more additional interaction patterns 408. For example, one or more of the servers A, B, and C train the artificial intelligence model AI1A to move the right or left hand of the character C1 in the air and move the mouth of the character C1 to say the words "Yay! I won," "I won!", "Yay!", or "Great!" after killing a virtual character in the game, shooting and destroying a virtual flying object in the game, earning virtual points in the game, or increasing a health level during the execution of a game program. As another example, one or more of the servers A, B, and C train the artificial intelligence model AI1A to manipulate the character C1 to dance and manipulate the mouth of the character C1 to say one or more of these words after killing or earning virtual points in the game, or increasing a health level in the game.
[0088] FIG. 8B is a diagram of an embodiment of a virtual scene 820 displayed on the HMD 106 worn by the user A to illustrate the character C1 performing an action similar to that of the user A during the game play. As shown in the virtual scene 820, the character C1 raises his right hand and outputs a sound indicating "I won!" after achieving a game level, virtual points, virtual coins, or health level during the game play. For example, one or more of the servers A, B, and C determine that the character C1 has gained virtual points or achieved a game level or increased its health level during the game play. Upon such determination, one or more of the servers A, B, and C apply the artificial intelligence model AI1A to generate one or more image frames and audio frames similar to the action performed by the user A after a win, such as a kill, gaining points, or achieving a game level or health level. The image frames and audio frames are packetized by one or more of the servers A, B, and C by applying a network communication protocol to generate one or more packets. One or more of the servers A, B, and C transmit packets having image and audio frames to the computing device 104 via the computer network 102. The computing device 104 applies a network communication protocol to obtain the image and audio frames from the packets and transmits the image and audio frames to the HMD 106 via a wired or wireless connection. A display device of the HMD 106 displays the image frames to display a virtual scene 820 including the character C1 raising his right hand in victory. Additionally, an audio device including one or more audio amplifiers coupled to one or more audio speakers of the HMD 106 processes the audio frames to output sounds made by the character C1.
[0089] In one embodiment, any of the interaction patterns described herein are used to train the AI model described herein or to help the AI model learn and generate a learned method. The AI model applies the learned method to output a decision result, such as an AI output, and there is no user control of the AI model while applying the learned method to output a decision result. As an example, the decision result is applied to a new virtual scene.
[0090] Although the embodiments described herein are described with reference to virtual scenes, in one embodiment, these embodiments apply equally to augmented reality (AR) scenes. For example, character C1 is manipulated by artificial intelligence model AI1A in the AR scene displayed on HMD 106, and character C3 is manipulated by artificial intelligence model AI3 in the AR scene displayed on HMD 510. Data including a portion of the AR scene displayed on the HMD is captured by a camera in the same real-world environment as the user wearing the HMD. The portion of the AR scene is a portion of the real-world environment in front of the user. The data is transmitted from the camera to a computing device via a wired or wireless connection. The computing device generates one or more packets by packetizing the data by applying a network communication protocol, and transmits the packets to one or more of servers A, B, and C via computer network 102. One or more of servers A, B, and C apply artificial intelligence model AI1A to generate one or more image frames for overlaying character C1 on the data captured by the camera coupled to computing device 104. Similarly, one or more of the servers A, B, and C apply the artificial intelligence model AI3 to generate one or more image frames for overlaying the character C3 on the data captured by the camera coupled to the computing device 514. The image frames are packetized by one or more of the servers A, B, and C and transmitted by the computer network 102 to the corresponding computing devices 104 and 514. The computing device 104 applies a network communication protocol to obtain the image frames from the packets and transmits the image frames to the HMD 106 to display the AR scene on the HMD 106. The AR scene displayed on the HMD 106 includes a portion of the real-world environment in which the user A is located and the character C1 overlaid on the data of the real-world environment.Similarly, the computing device 514 applies a network communication protocol to obtain image frames from the packets, and transmits the image frames to the HMD 512 to display the AR scene on the HMD 512. The AR scene displayed on the HMD 512 includes a portion of the real-world environment in which the user C is located and a character C3 overlaid on the data of the real-world environment.
[0091] It should be noted that in various embodiments, one or more features of some of the embodiments described herein may be combined with one or more features of one or more of the remaining embodiments described herein.
[0092] The embodiments described in this disclosure may be practiced with a variety of computer system configurations, including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. In one implementation, the embodiments described in this disclosure are practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wire-based or wireless network.
[0093] With the above embodiments in mind, it should be understood that in one implementation, the embodiments described in this disclosure employ various computer-implemented operations involving data stored in computer systems. These operations are operations requiring physical manipulation of physical quantities. Any of the operations described herein that form part of the embodiments described in this disclosure are useful machine operations. Some embodiments described in this disclosure also relate to devices or apparatus for performing these operations. This apparatus may be specially constructed for the required purposes, or this apparatus may be a general-purpose computer selectively activated or configured by a computer program stored in the computer. In particular, in one embodiment, various general-purpose machines are used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
[0094] In one embodiment, some embodiments described in this disclosure are embodied as computer readable code on a computer readable medium. A computer readable medium is any data storage device that stores data, which is subsequently read by a computer system. Examples of computer readable media include hard drives, network attached storage (NAS), read only memory (ROM), random access memory (RAM), compact disc ROM (CD-ROM), CD recordable (CD-R), CD rewriteable (CD-RW), magnetic tape, optical data storage devices, non-optical data storage devices, etc. As an example, the computer readable medium includes computer readable tangible media distributed over a network-coupled computer system such that the computer readable code is stored and executed in a distributed fashion.
[0095] Also, although some of the above embodiments are described with respect to a gaming environment, in some embodiments other environments, such as a video conferencing environment, are used instead of a game.
[0096] Although the operations of the method have been described in a particular order, it should be understood that other maintenance operations may be performed between operations, or operations may be coordinated to occur at slightly different times, or operations may be distributed throughout the system to allow processing operations to occur at various intervals related to processing, so long as the processing of the overlay operation is performed in the desired manner.
[0097] Although the foregoing embodiments described in this disclosure have been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications can be practiced within the scope of the appended claims. Thus, the present embodiments should be considered as illustrative and not restrictive, and the present embodiments should not be limited to the details set forth herein, but may be modified within the scope of the appended claims and their equivalents.
Claims
1. 1. A method for training an artificial intelligence (AI) model, comprising: receiving data regarding interactions of a virtual character controlled by a user when interacting with a virtual scene of a game; training a first AI model based on data relating to the interaction, the first AI model being used when interacting with the virtual scene of the game; and continuing training the first AI model based on data output from a second AI model, the data output from the second AI model including data learned from one or more other users when interacting with a virtual scene of the game; The method comprising:
2. 2. The method of claim 1, wherein the first AI model is accessed through a first user account of the user and the second AI model is accessed through a second user account of another user.
3. requesting permission via the second user account for use of the second AI model; receiving said authorization; and obtaining data output from the second AI model upon receiving the authorization; and The method of claim 2 , further comprising:
4. 2. The method of claim 1 , wherein the data regarding the interactions comprises a plurality of interaction patterns having movement of the virtual character relative to one or more virtual objects in the virtual scene, interactions between the virtual character and the one or more virtual objects, or a combination thereof.
5. The method of claim 4, wherein the AI model is trained to follow the multiple interaction patterns of manipulating the virtual character to move relative to the one or more virtual objects, manipulating the virtual character to interact with the one or more virtual objects, or combinations thereof.
6. The method of claim 1 , wherein the second AI model is trained to apply a plurality of additional interaction patterns between another virtual character and one or more virtual objects.
7. The method of claim 6 , wherein the data output from the second AI model includes the plurality of additional interaction patterns.
8. The method of claim 1 , wherein the first AI model is configured to manipulate the virtual character and the second AI model is configured to manipulate another virtual character.
9. and further comprising continuing the training of the first AI model by the first AI model based on a plurality of additional interaction patterns, wherein continuing the training of the first AI model by the first AI model comprises: obtaining reaction data generated from the training of the first AI model based on data regarding the interaction; acquiring additional response data based on continuing the training of the first AI model with the first AI model; determining whether the additional response data exceeds the response data; applying the plurality of additional interaction patterns in place of data regarding the interaction upon determining that the additional response data exceeds the response data; and The method of claim 1 , comprising:
10. The method of claim 1, wherein the first AI model is used during a battle between the virtual character controlled by the first AI model and another virtual character controlled by the second AI model.
11. 1. A system for training an artificial intelligence (AI) model, comprising: receiving data regarding interactions of a virtual character controlled by a user when interacting with a virtual scene of a game; training a first AI model for use in interacting with the virtual scene of the game based on data relating to the interaction; a processor configured to continue training the first AI model based on data output from a second AI model; a memory device coupled to the processor; Equipped with The system, wherein the data output from the second AI model includes data learned from one or more other users when interacting with a virtual scene of the game.
12. 12. The system of claim 11, wherein the first AI model is accessed through a first user account of the user and the second AI model is accessed through a second user account of another user.
13. The processor, using the first AI model to request permission via the second user account for use of the second AI model; receiving said authorization; 13. The system of claim 12, configured to obtain output data from the second AI model upon receiving the authorization.
14. 12. The system of claim 11, wherein the data regarding the interaction comprises a plurality of interaction patterns having movement of the virtual character relative to one or more virtual objects in the virtual scene, an interaction between the virtual character and the one or more virtual objects, or a combination thereof.
15. 15. The system of claim 14, wherein the first AI model is trained to follow the plurality of interaction patterns, the plurality of interaction patterns being followed by manipulating the virtual character to move relative to the one or more virtual objects, manipulating the virtual character to interact with the one or more virtual objects, or a combination thereof.
16. 12. The system of claim 11, wherein the second AI model is trained to apply a plurality of additional interaction patterns between another virtual character and one or more virtual objects.
17. 20. The system of claim 16, wherein the data output from the second AI model includes the plurality of additional interaction patterns.
18. The system of claim 11 , wherein the first AI model is configured to manipulate the virtual character and the second AI model is configured to manipulate another virtual character.
19. The processor is configured to continue the training of the first AI model based on a plurality of additional interaction patterns, and to continue the training of the first AI model based on the plurality of additional interaction patterns, the processor comprises: obtaining response data generated from the training of the first AI model based on data regarding the interaction; acquiring additional response data based on continuing the training of the first AI model with the first AI model; determining whether the additional response data exceeds the response data; applying the plurality of additional interaction patterns in place of data regarding the interaction upon determining that the additional response data exceeds the response data. The system of claim 11 configured to:
20. The system described in claim 11, wherein the first AI model is used during a battle between the virtual character controlled by the first AI model and another virtual character controlled by the second AI model.
Citation Information
Patent Citations
Controlling a massively multiplayer online role-playing game
JP2016526952A
Program, system, and method for adjusting weighting of neural network using q-learning
JP2018063602A
Systems and methods for training artificial intelligence models for competitive matches
JP2022525411A
Online learning simulator using machine learning
US20180318719A1
Information processing device, information processing method, and program
WO2018150654A1
