Using machine learning to increase or decrease the difficulty level of beating video game opponents

Abstract

Machine learning is used to determine the ability of one or more video game players to defeat a non-player video game opponent, such as a boss. The difficulty level of defeating the video game opponent can then be increased or decreased based on the player's ability to defeat the video game opponent. In some examples, another player can then select the increased or decreased difficulty level, or can choose to play the video game without the increased or decreased difficulty level.

Description

Technical Field

[0001] The present application generally relates to technically creative and unconventional solutions that are necessarily rooted in computer technology and produce specific technical improvements. In particular, the present application relates to using machine learning to increase or decrease the difficulty level of defeating non-player video game opponents. Background Art

[0002] Artificial intelligence based machine learning is becoming increasingly sophisticated and powerful. Additionally, the present application recognizes that consumers may realize increased value in video games if they can replay the video game after beating it, but with variations from the first time they beat the game. Summary of the invention

[0003] Thus, the present application also recognizes that machine learning, sometimes in the form of deep learning artificial neural networks, can be used for a variety of useful applications related to increasing or decreasing the difficulty level of defeating non-player video game opponents (such as bosses) at the end of a video game. This can enhance the player's experience of the video game and provide a more powerful and skilled experience, while also giving the player more motivation to replay the video game, even after beating it the first time.

[0004] Thus, in one aspect, an apparatus includes at least one processor and at least one computer storage device that is not a transient signal. The computer storage device includes instructions executable by the at least one processor to receive data related to the ability of one or more video game players to defeat a video game opponent and change the difficulty level of defeating the video game opponent based on the data. The video game opponent can be a virtual video game opponent that is controlled by a computer rather than a human during game play, and the difficulty level can be changed by increasing or decreasing the difficulty level.

[0005] In some implementations, the instructions may also be executable to transmit first data indicating a changed difficulty level to one or more video game consoles or other consumer electronics (CE) devices, and transmit second data indicating a reward for defeating a video game opponent to one or more video game consoles or other consumer electronics (CE) devices. The reward may vary with the difficulty level of defeating the video game opponent. In some examples, the reward may vary so that a higher difficulty level for defeating a video game opponent may result in a higher reward, while a lower difficulty level for defeating a video game opponent may result in a lower reward. The reward may include, for example, a specific trophy, a specific medal, a specific amount of video game currency, a specific amount of video game points, and / or a specific amount of currency consumed via an electronic or e-commerce store.

[0006] If desired, the instructions may be executable to use artificial intelligence machine learning to analyze data related to the ability of one or more video game players to defeat video game opponents to identify successful and / or unsuccessful methods used by the one or more video game players to defeat the video game opponents. The instructions may then be executable to change the difficulty level based on the analysis.

[0007] In another aspect, a method includes receiving data related to one or more video game players attempting to defeat a video game opponent, and determining different ways to defeat the video game opponent in the future based on the data. The method also includes changing the game for the video game opponent based on the different ways to defeat the video game opponent in the future. The video game opponent can be a non-player character of the video game, such as a boss that is engaged at the end of a level or stage of the video game.

[0008] Furthermore, in some examples, different ways to defeat a video game opponent in the future can be customized for a specific video game player to play with the specific video game player to circumvent a specific attack or attack pattern launched by the specific video game player against the video game opponent. Additionally or alternatively, different ways to defeat a video game opponent in the future can be customized to be played by multiple video game players in a separate game instance to circumvent a specific attack or attack pattern launched by the multiple video game players against the video game opponent.

[0009] Furthermore, in some implementations, different ways to defeat video game opponents in the future can be customized based on a specific subset of video game players. The specific subset can be e-sports video game players, first-person shooter video game players, simulation video game players, and / or role-playing game video game players.

[0010] The different ways to beat video game opponents in the future can also be customized for increasing or reducing the joystick difficulty when beating video game opponents. Additionally or alternatively, the different ways to beat the video game opponents in the future can be customized for the button combination difficulty that increases or decreases when beating the video game opponents.

[0011] In yet another aspect, an apparatus includes at least one computer-readable storage medium that is not a transient signal. The computer-readable storage medium includes instructions executable by at least one processor to initially install a version of a video game on a computer and receive data from a server indicating one or more increased or decreased difficulty levels for defeating one or more opponents of the video game. The instructions are also executable to execute the video game according to the one or more increased or decreased difficulty levels for defeating the one or more opponents.

[0012] In some examples, the instructions may also be executable to present a graphical user interface (GUI) on a display accessible to the at least one processor. The GUI may include an option to permit a player to select to play the video game using one or more increased or decreased difficulty levels to defeat one or more opponents relative to an initially installed version of the video game.

[0013] The details of the present application, both as to its structure and operation, may be best understood by reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which: BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram of an example system consistent with the present principles;

[0015] Figure 2 is a flow chart of example logic that may be executed by a server or server farm in accordance with present principles to increase or decrease the difficulty level of defeating a non-player video game opponent;

[0016] Figure 3 is a flow chart of example logic executable by a video game console or other consumer electronic device for operating a video game according to one or more increasing or decreasing difficulty levels for defeating non-player video game opponents; and

[0017] Figure 4 and Figure 5 An example graphical user interface (GUI) for configuring game settings in accordance with the present principles is shown. DETAILED DESCRIPTION

[0018] The present disclosure generally relates to computer ecosystems that include aspects of consumer electronics (CE) device networks, such as, but not limited to, distributed computer game networks, augmented reality (AR) networks, virtual reality (VR) networks, video broadcasting, content delivery networks, virtual machines, and artificial neural network and machine learning applications. The systems herein may include server and client components that are connected via a network so that data can be exchanged between the client and server components. The client components may include one or more computing devices, including a gaming console (such as a Sony or by or or game consoles manufactured by other manufacturers), virtual reality (VR) headsets, augmented reality (AR) headsets, game controllers, portable televisions (e.g., smart TVs, Internet-enabled televisions), portable computers (such as laptops and tablets), and other mobile devices including smartphones and other examples discussed below. These client devices can operate in a variety of operating environments. For example, some client computers can use, for example, Operating system, from operating system, or operating system, or by or These operating environments can be used to execute one or more browsing programs, such as those provided by or or A browser or other browser program that can access a website hosted by an Internet server discussed below. In addition, an operating environment according to the present principles can be used to execute one or more computer game programs according to the present principles.

[0019] The server and / or gateway may include one or more processors that execute instructions that configure the server to receive and transmit data over a network such as the Internet. Alternatively, the client and server may be connected via a local intranet or a virtual private network. The server or controller may be powered by a game console such as a Sony personal computer, etc.) is instantiated.

[0020] Information may be exchanged between the client and the server over the network. To this end and for security, the server and / or client may include firewalls, load balancers, temporary storage devices and proxies, and other network infrastructure for reliability and security. One or more servers may form an apparatus that implements a method according to the present principles to provide network users with a secure community such as an online social networking site or a video game site for communication.

[0021] As used herein, instructions refer to computer-implemented steps for processing information in a system. Instructions may be implemented in software, firmware, or hardware, and include any type of programmed steps undertaken by components of the system.

[0022] The processor may be any conventional general purpose single chip processor or multi-chip processor capable of executing logic with the aid of various lines such as address, data and control lines, as well as registers and shift registers.

[0023] The software modules described by the flowcharts and user interfaces herein may include various subroutines, programs, etc. Without limiting the present disclosure, the logic stated as being performed by a particular module may be redistributed to other software modules and / or combined together in a single module and / or made available in a shareable library.

[0024] The present principles described herein may be implemented as hardware, software, firmware, or a combination thereof; therefore, illustrative components, blocks, modules, circuits, and steps are described in terms of their functionality.

[0025] The functions and methods described below, when implemented in software, may be written in an appropriate language such as, but not limited to, Java, C#, or C++, and may be stored on or transmitted through a computer-readable storage medium, such as a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a compact disk read-only memory (CD-ROM) or other optical disk storage devices (such as a digital versatile disk (DVD)), a magnetic disk storage device, or other magnetic storage devices including a removable thumb drive, etc. A connection may establish a computer-readable medium. Such a connection may include, for example, hard-wired cables, including optical fiber and coaxial cables, as well as digital subscriber lines (DSL) and twisted pairs. Such a connection may include wireless communication connections, including infrared and radio.

[0026] The components included in one embodiment may be used in other embodiments in any appropriate combination. For example, any of the various components described herein and / or depicted in the accompanying drawings may be combined, interchanged or excluded from other embodiments.

[0027] “A system having at least one of A, B, and C” (similarly, “a system having at least one of A, B, or C” and “a system having at least one of A, B, C”) includes the following systems, which: have only A; have only B; have only C; have both A and B; have both A and C; have both B and C; and / or have A, B, and C, etc.

[0028] Now specifically refer to Figure 1, an exemplary system 10 is shown, which may include one or more of the exemplary devices mentioned above and further described below according to the present principles. The first of the exemplary devices included in the system 10 is a consumer electronics (CE) device, such as an audio video device (AVD) 12, such as but not limited to an Internet-enabled TV with a TV tuner (equivalently, a set-top box that controls the TV). However, the AVD 12 may alternatively be an appliance or household item, such as a computerized Internet-enabled refrigerator, washer, or dryer. Alternatively, the AVD 12 may also be a computerized Internet-enabled (“smart”) phone, a tablet computer, a laptop computer, an augmented reality (AR) headset, a virtual reality (VR) headset, Internet-enabled or “smart” glasses, another type of wearable computerized device such as, for example, a computerized Internet-enabled watch, a computerized Internet-enabled bracelet, other computerized Internet-enabled devices, a computerized Internet-enabled music player, a computerized Internet-enabled headset, a computerized Internet-enabled implantable device (such as an implantable skin device), etc. In any event, it should be understood that the AVD 12 is configured to implement the principles of the present invention (e.g., communicate with other CE devices to implement the principles of the present invention, execute the logic described herein, and perform any other functions and / or operations described herein).

[0029] Therefore, in order to implement such a principle, the AVD 12 can be composed of Figure 1Some or all of the components shown are established. For example, the AVD 12 may include one or more displays 14, which may be implemented by a flat screen of high definition or ultra high definition ("4K") or higher, and may be touch-enabled for receiving user input signals through touch on the display. The AVD 12 may include: one or more speakers 16, which are used to output audio according to the present principles; and at least one additional input device 18 (such as, for example, an audio receiver / microphone), which is used to input audible commands to the AVD 12 to control the AVD 12, for example. The example AVD 12 may also include one or more network interfaces 20 for communicating on at least one network 22 (such as the Internet, a wide area network, a local area network, etc.) under the control of one or more processors 24 (e.g., a central processing unit). A graphics processor 24A may also be included. Therefore, the interface 20 may be, but is not limited to, a Wi-Fi transceiver, which is an example of a wireless computer network interface, such as, but not limited to, a mesh network transceiver. It should be understood that processor 24 controls AVD 12 to implement the present principles, including other elements of AVD 12 described herein, such as, for example, controlling display 14 to present images on the display and receive input from the display. In addition, it should be noted that network interface 20 can be, for example, a wired or wireless modem or router or other suitable interface (such as, for example, a wireless telephone transceiver or a Wi-Fi transceiver as mentioned above, etc.).

[0030] In addition to the foregoing, the AVD 12 may also include one or more input ports 26 (such as, for example, a high-definition multimedia interface (HDMI) port or a USB port) for physically connecting (e.g., using a wired connection) to another CE device and / or a headphone port for connecting headphones to the AVD 12 for presenting audio from the AVD 12 to the user through the headphones. For example, the input port 26 may be connected to a cable or satellite source 26a of audio-video content by wire or wirelessly. Thus, the source 26a may be, for example, a separate or integrated set-top box or satellite receiver. Alternatively, the source 26a may be a game console or disc player containing content such as computer game software and databases. When implemented as a game console, the source 26a may include some or all of the components described below with respect to the CE device 44, and may implement some or all of the logic described herein.

[0031] The AVD 12 may also include one or more computer memories 28 that are not transient signals, such as disk-based storage devices or solid-state storage devices, which are in some cases embodied as independent devices in the housing of the AVD, or as personal video recorders (PVRs) or video disk players for playing back AV programs inside or outside the housing of the AVD, or as removable storage media. In addition, in some embodiments, the AVD 12 may include a position or location receiver (such as, but not limited to, a cell phone receiver, a GPS receiver, and / or an altimeter 30), which is configured to, for example, receive geographic location information from at least one satellite or cell phone tower and provide the information to the processor 24 and / or determine the altitude at which the AVD 12 is set in conjunction with the processor 24. However, it should be understood that according to the present principles, another suitable position receiver other than a cell phone receiver, a GPS receiver, and / or an altimeter can be used, for example, to determine the location of the AVD 12 in all three dimensions.

[0032] Continuing with the description of the AVD 12, in some embodiments, the AVD 12 may include one or more cameras 32, which may be, for example, a thermal imaging camera, a digital camera such as a webcam, an infrared (IR) camera, and / or a camera integrated into the AVD 12 and controllable by the processor 24 to collect pictures / images and / or videos in accordance with the present principles.

[0033] The AVD 12 may also include a Bluetooth transceiver 34 and other near field communication (NFC) elements 36 for communicating with other devices using Bluetooth and / or NFC technology, respectively. An exemplary NFC element may be a radio frequency identification (RFID) element.

[0034] Further, the AVD 12 may include one or more auxiliary sensors 37 (e.g., motion sensors such as accelerometers, gyroscopes, gyroscopes, or magnetic sensors, infrared (IR) sensors, optical sensors, speed and / or cadence sensors, gesture sensors (e.g., for sensing gesture commands), etc.) that provide input to the processor 24. The AVD 12 may include an over-the-air television broadcast port 38 for receiving OTA television broadcasts that provide input to the processor 24. In addition to the foregoing, it should be noted that the AVD 12 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 42, such as an IR data association (IRDA) device. A battery (not shown) may be provided for powering the AVD 12.

[0035] Still refer to Figure 1In addition to the AVD 12, the system 10 may also include one or more other CE device types. In one example, a first CE device 44 may be used to send computer game audio and video to the AVD 12 via commands sent directly to the AVD 12 and / or via a server described below, while a second CE device 46 may include similar components to the first CE device 44. In the example shown, the second CE device 46 may be configured as a VR headset worn by a player 47 as shown, or a handheld game controller with one or more joysticks, one or more buttons, etc., manipulated by the player 47. In the example shown, only two CE devices 44, 46 are shown, and it should be understood that fewer or more devices may be used.

[0036] In the example shown, all three devices 12, 44, 46 may be members of an entertainment network, for example, in a home / residence or distributed over several locations. Therefore, the present principles are not limited to a specific location shown by dashed line 48 unless otherwise expressly stated.

[0037] An example non-limiting first CE device 44 may be established by any of the above devices, for example, a portable wireless laptop or notebook computer, a video game console or a video game controller, and thus may have one or more of the components described below. The first CE device 44 may be a remote control (RC) for issuing AV play and pause commands to the AVD 12, for example, or it may be a more complex device, such as a tablet computer, a game controller, a personal computer, a wireless phone, etc., that communicates with the AVD 12 via a wired or wireless link.

[0038] Thus, the first CE device 44 may include one or more displays 50, which may be touch-enabled for receiving user input signals via touch on the display(s) 50. Additionally or alternatively, the display(s) 50 may be at least partially transparent displays configured for presenting AR and / or VR images, such as an AR headset display or a "smart" glasses display or a "heads-up" display, as well as a VR headset display or other displays.

[0039] The first CE device 44 may also include one or more speakers 52 for outputting audio according to the present principles, and at least one additional input device 54, such as, for example, an audio receiver / microphone, for inputting audible commands to the first CE device 44 to control the device 44. The exemplary first CE device 44 may also include one or more network interfaces 56 for communicating over the network 22 under the control of one or more CE device processors 58. A graphics processor 58A may also be included. Thus, the interface 56 may be, but is not limited to, a Wi-Fi transceiver, which is an example of a wireless computer network interface, including a mesh network interface. It should be understood that the processor 58 controls the first CE device 44 to implement the present principles, including other elements of the first CE device 44 described herein, such as, for example, controlling the display 50 to present images on the display and receiving input from the display. In addition, it should be noted that the network interface 56 may be, for example, a wired or wireless modem or router, or other suitable interface, such as, for example, a wireless telephone transceiver, or a wireless network transceiver as described above, etc.

[0040] In addition to the foregoing, the first CE device 44 may also include one or more input ports 60 (such as, for example, an HDMI port or a USB port) for physically connecting (e.g., using a wired connection) to another CE device and / or a headphone port for connecting headphones to the first CE device 44 for presenting audio from the first CE device 44 to the user through the headphones. The first CE device 44 may also include one or more tangible computer-readable storage media 62, such as a disk-based storage device or a solid-state storage device. In addition, in some embodiments, the first CE device 44 may include a position or location receiver (such as, but not limited to, a cell phone and / or a GPS receiver and / or an altimeter 64) configured to receive geographic location information from at least one satellite and / or cell phone tower using, for example, triangulation and provide the information to the CE device processor 58 and / or determine, in conjunction with the CE device processor 58, the altitude at which the first CE device 44 is located. However, it should be understood that another suitable position receiver other than a cell phone and / or a GPS receiver and / or an altimeter may be used in accordance with the present principles to determine the location of the first CE device 44, for example, in all three dimensions.

[0041] Continuing with the description of the first CE device 44, in some embodiments, according to the present principles, the first CE device 44 may include one or more cameras 66, which may be, for example, thermal imaging cameras, digital cameras (such as webcams), and / or cameras integrated into the first CE device 44 and controllable by the CE device processor 58 to capture pictures / images and / or videos. A Bluetooth transceiver 68 and other near field communication (NFC) elements 70 may also be included on the first CE device 44, which are used to communicate with other devices using Bluetooth and / or NFC technology, respectively. An exemplary NFC element may be a radio frequency identification (RFID) element.

[0042] Further, the first CE device 44 may include one or more auxiliary sensors 72 (e.g., motion sensors such as accelerometers, gyroscopes, gyroscopes or magnetic sensors, infrared (IR) sensors, optical sensors, speed and / or cadence sensors, gesture sensors (e.g., for sensing gesture commands), etc.) that provide input to the CE device processor 58. The first CE device 44 may include other sensors that provide input to the CE device processor 58, such as, for example, one or more climate sensors 74 (e.g., barometers, humidity sensors, wind sensors, light sensors, temperature sensors, etc.) and / or one or more biometric sensors 76. In addition to the foregoing, it should be noted that in some embodiments, the first CE device 44 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 78, such as an IR data association (IRDA) device. A battery (not shown) may be provided to power the first CE device 44, and / or the CE device 44 may be powered via a wired connection to a wall outlet. The CE device 44 may communicate with the AVD 12 via any of the above-mentioned communication modes and related components.

[0043] The second CE device 46 may include some or all of the components shown for the CE device 44. One or both CE devices may be powered by one or more batteries and / or a wall outlet connection.

[0044] Referring now to the aforementioned at least one server 80, it includes at least one server processor 82, at least one tangible computer-readable storage medium 84 (such as a disk-based storage device or a solid-state storage device), and at least one network interface 86 that allows communication with the server over the network 22 under the control of the server processor 82. Figure 1 The network interface 86 may be a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface (such as a wireless telephone transceiver, for example).

[0045] Thus, in some embodiments, server 80 may be an Internet server or an entire server "farm" and may include and perform "cloud" functions, such that the devices of system 10 may access the "cloud" environment via server 80 in an example embodiment of, for example, a network gaming application. Alternatively, server 80 may be provided by a server that is associated with Figure 1 The other devices shown are implemented in the same room or nearby one or more game consoles or other computers.

[0046] In addition to what has been mentioned above, the logic blocks, modules and circuits described below can be implemented or executed with a general-purpose processor, a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic devices designed to perform the functions described herein (such as application-specific integrated circuits (ASICs), discrete gate or transistor logic, discrete hardware components or any combination thereof). The processor can be implemented by a combination of a controller or a state machine or a computing device. Therefore, the method herein can be implemented as software instructions executed by a processor, a properly configured application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA) module, or any other convenient manner that will be understood by those skilled in the art. Where adopted, the software instructions can be embodied in a non-temporary device (such as a hard drive, a CD ROM or a flash drive). The software code instructions can also be downloaded on the Internet.

[0047] Continue to Figure 2 Before describing, it should be understood that according to the present principles, a machine learning algorithm (established by an artificial intelligence neural network with an input layer and an output layer and multiple hidden layers therebetween) can be used to change the behavior of a boss or other non-player controlled video game opponent. As understood herein, machine learning can be a serialized process that involves identifying, acquiring and preparing raw data sets, feature engineering, and generating extraction, transformation, and loading (ETL) jobs that convert raw data into features of a machine learning model, and designing and encoding a machine learning model. The model can be "trained" in computationally intensive tasks, which may involve a distributed computing architecture running on a large number of CPUs, GPUs, and emerging optimized chips to run specific algorithms such as deep learning. The performance and latency of the model can even be evaluated. After the model is trained, the quality of its predictions and the time it takes for the model to provide the predictions can be evaluated. Then, a "good" model can be deployed in a production environment to provide inferences about how to circumvent certain attacks, attack patterns, or habits of human video game players against non-player opponents, or if the player has difficulty defeating the opponent, the opponent is easier to defeat.

[0048] To this end, reinforcement learning algorithms may be used with dynamic programming and Markov decision processes (MDPs). However, supervised and / or unsupervised learning algorithms may also be used. In any case, a model may be constructed by using as training data certain human player behavior data indicating various moves and strategies that various human video game players have used in the past to defeat a given non-player opponent. The model may then be used to infer / predict how the opponent may successfully avoid or counter the moves, or even acquiesce to the moves, thereby making it harder or easier to defeat the boss accordingly.

[0049] Reference now Figure 2 , Figure 2 An example logic that can be executed by a device or a group of devices, such as a server or server farm, according to the principles of the present invention is shown to use machine learning to change behavior and therefore change the difficulty level of defeating a non-player video game opponent, which is like a boss controlled by the video game system itself rather than a person. Beginning at box 200, the device can receive data from various video game consoles or other computers executing a certain video game. The data can be related to the ability of various video game players to defeat a certain non-player video game opponent (e.g., a boss encountered at the end of a certain level of the video game or at the final level / stage of the video game). The opponent can also be another type of character, object or entity that has no common interest in one or more goals that the human player is trying to achieve in the video game. For example, the opponent can be a rival racing driver, a neighbor in a simulated environment competing for resources with a human player, an opposing competitive team composed of multiple virtual athletes controlled by a game console, a hostile warrior or opponent space force, a group of zombies, etc.

[0050] Then, at block 202, the apparatus may use the data as training data to develop a machine learning model for changing the difficulty level of defeating bosses in future game instances based on successful and / or unsuccessful methods or strategies previously used by the player when controlling the player's video game character. As an example, methods or strategies may include specific attacks, such as flying punches, low kicks, etc. Certain attack patterns or sequences may also establish methods / strategies, as well as certain player movements and location changes within a level or combat environment that the player tends to maintain while fighting a boss / opponent.

[0051] The logic may then move from block 202 to block 204, where the device may access data relating to various rewards that may be offered to the player for defeating the boss. The data may be preset by the developer of the video game, and the rewards themselves may relate to, for example, a trophy or medal awarded for defeating the boss, an amount of video game currency or points awarded for defeating the boss, or even an amount of currency that is awarded for defeating the boss and is spent in an electronic store and may be used by the player to make additional real-life purchases (e.g., in-game purchases of the game, purchases of other games, purchases of gaming hardware such as additional video game controllers, etc.).

[0052] After box 204, the logic can then move to box 206. At box 206, the device can use the model developed at box 202 to determine the new behavior of the boss. The new behavior may make it more or less difficult to defeat the boss (e.g., kill the boss), and thus continue to the next video game level. The data used to develop the model at box 202 may be specific to only a certain player, or may be data from multiple players in the overall population playing the video game, or data from a subset of the population. Therefore, if the model is built based on the data of the player only, the new behavior determined at box 206 may be specific to only a certain player, or if the model is built based on the data of multiple players, it may be a behavior determined based on trends or patterns identified from multiple players. In any case, the new behavior may make it more difficult or easier for the player to defeat the boss using various joystick manipulations on the video game remote controller used by the player and / or using various buttons or button combinations on the video game remote controller.

[0053] For example, in terms of making it harder for the player to defeat a boss or other opponent, machine learning can be used to identify that the boss is not moving fast enough to dodge a punch, and accordingly infer that if the boss moves in certain directions faster than the boss's currently set movement speed to dodge a punch, then the punch will not hit the boss. As another example, if the player uses a certain weapon to defeat the boss and the weapon is successful, the machine learning model can infer that the boss should be assigned a shield or other object that has not been assigned to the boss in the past in the future so that the boss cannot be harmed by the weapon in future games.

[0054] In terms of making it easier for players to defeat bosses or other opponents, as an example, when a boss uses a certain weapon or initiates a certain attack sequence, machine learning can be used to identify that the player or group of players is always defeated (or defeated above a threshold percentage of total game instances), and accordingly infer that the boss should stop using that weapon or attack sequence.

[0055] Then, at box 208, the device can assign different rewards for defeating the boss, which changes with the new behavior and thus increases or decreases the difficulty level of defeating the boss. The output of the model indicates whether the boss is becoming more or less difficult to defeat and can be used for this purpose.

[0056] Thus, when a player or group of players defeats a boss relatively easily or above a threshold percentage of the total instances in which the player attempts to defeat the boss, a greater reward may be assigned for defeating the boss as the difficulty level of the boss increases in response to the player or group of players defeating the boss. The threshold percentage may be set by the developer of the video game, where, for example, the developer may want people to defeat the boss less than fifty percent of the number of times they attempt to defeat the boss. Thus, a greater reward may be assigned for defeating a boss that is customized to exhibit new behaviors (e.g., one or more specific moves or countermoves) that are more sophisticated or complex to make it more difficult to defeat the boss. A greater reward may include more points or in-game currency earned for defeating the boss relative to the points or currency earned for defeating the boss before the boss's behavior was changed, as well as a better trophy or medal, such as a gold trophy or medal, rather than a silver or bronze trophy or medal that might have been earned before the boss's behavior was changed.

[0057] If a boss is customized to exhibit less sophisticated or less complex behavior to make it easier to defeat the boss, a smaller reward may also be assigned for defeating the boss. This may occur if a player or group of players cannot defeat the boss or has difficulty defeating the boss (e.g., the number of times the boss is defeated is less than the threshold percentage of the instances mentioned above), as this may result in a negative gaming experience. Smaller rewards may include fewer points or in-game currency awarded for defeating the boss relative to the points or currency awarded for defeating the boss before the boss behavior was changed, as well as smaller trophies or medals, such as bronze trophies or medals instead of gold or silver trophies or medals that may have been awarded before the boss behavior was changed.

[0058] The logic may then proceed from block 208 to block 210, where the apparatus may transmit the data to one or more consoles or other computers that have installed a copy of the video game to update the video game with new behaviors for the bosses and different rewards assigned to defeating the changed bosses. In some examples, the apparatus may transmit data to one or more consoles only in response to a corresponding pull request from a console requesting the data, such as during a game update performed at the respective consoles. However, in other examples, the apparatus may "push" the data to one or more consoles even without receiving a corresponding request for the data from the one or more consoles. As in Figure 3 The console / computer can then be updated with the data as reflected in the

[0059] Therefore, now refer to Figure 3 , which may be executed by a device such as a video game console or other computer configured to execute a video game for a person to play. Beginning at block 300, a device may install certain versions of a video game on the device. For example, the version may be region specific (e.g., a U.S. version as opposed to a European version). The version may also be a base version or other original version of the video game prior to any subsequently released updates that may be downloaded over the Internet.

[0060] Then, the logic may proceed from block 300 to block 302, where the device may allow the player to choose to defeat one or more bosses using increased and / or decreased difficulty levels in accordance with the present principles. Figure 4 Further examples of how players can opt-in are described. However, reference Figure 3 , the logic may then proceed from block 302 to block 304 .

[0061] At block 304, the apparatus may facilitate a game in which a human player may play a video game and attempt to advance through various levels or stages of the video game. Then, at block 306, the apparatus may identify or otherwise collect data about successful or unsuccessful methods that may have been used to defeat a certain boss in the game, as well as any weapons that may have been used to attempt to defeat the boss, locations in the virtual environment of the game where the player engaged the boss, etc. For example, if the video game is designed so that the player engages the boss within a preset virtual geographic area within a particular level of the game (such as a virtual room or even a virtual arena), and the player uses a sword to attempt to defeat the boss within the area, then the location within the area where the player engaged the boss and data indicating the use of the sword may be collected.

[0062] Then, at box 308, the device may transmit the data identified or collected at box 306 to a server or server farm in accordance with present principles. Thereafter, the logic may proceed to box 310, where the device may re-receive data from the server or server farm indicating one or more increased or decreased difficulty levels for defeating a boss in accordance with the principles of the present invention. For example, for increased difficulty, the device may re-receive a game update that includes additional combat moves to be performed by the boss that make defeating the boss more difficult, or the game update may be specifically tailored to counter-moves that players have used to defeat bosses in the past. As another example, to reduce difficulty, the device may re-receive a game update that specifies that the boss moves slower and avoids using combat moves that were previously effective in defeating the player.

[0063] Additionally, it should be noted that in some examples of block 310, the device may transmit a pull request to the server or server farm requesting the data before actually receiving the data at block 310. However, in other examples, if there is no request from the execution Figure 3 In the case of a pull request from a server in a server farm, the device may simply receive the data at block 310 .

[0064] From block 310, Figure 3 The logic of the apparatus may then proceed to block 312, where the apparatus may execute the video game in accordance with the increased or decreased difficulty level. For example, block 312 may be executed immediately upon receiving an update from a server or server farm, such that the next time a player encounters a boss, the boss exhibits an increased or decreased difficulty level. However, block 312 may also be executed at other times, such as in response to the console / computer being started after the player has stopped playing the video game, in response to the game application being started again after the game application has been shut down, or even in response to the player restarting the game, and not updating the boss from the beginning for subsequent attempts to defeat the boss until the player plays and beats the entire game for the first time (or at least starts a "new game").

[0065] As described above, a player may be given the opportunity to opt-in to a video game, increasing or decreasing the difficulty level for defeating one or more bosses or other opponents. The opportunity may appear when a user first installs or otherwise launches the video game on his or her device, and the option may be provided as part of the game's initial tutorial process, which acclimates the player to the game, the various controller moves to use when playing the game, the game's objectives, etc. The player may also access the video game's settings at a later time after playing some video games to opt-in, or opt-out after opting-in.

[0066] Therefore, in Figure 4 An example settings graphical user interface (GUI) 400 is shown in FIG. 400 for a player to provide input to opt-in and configure other settings. It should be understood that each of the options or sub-options to be described below can be selected by providing a touch or cursor input to an adjacent check box to select the option or sub-option.

[0067] like Figure 44, option 402 may be presented on GUI 400 and selected to configure the device / video game to opt-in to increase or decrease the difficulty level of defeating a boss or other opponent based on the player's own joystick and button habits or movement, weapon use, etc. Option 404 may also be presented to opt-in to use a boss / opponent that has been altered based on the game data of a group of other people. Thus, sub-options may be presented under option 404 for selecting a boss / opponent that has been altered based on the game play of a particular set or subset of people. For example, sub-option 406 may be selected to select a general group of people, while sub-option 408 may be selected to select people from the player's own friends list who have also played the same video game.

[0068] Other sub-options 410 may also be selected for selecting the use of altered bosses / opponents based on games for subsets of different types of video game players that have grouped their gaming habits together. For example, a subset of players that primarily play electronic sports (e-sports) game types may be selected, and subsets of people who play first person shooter (FPS) game types, simulation game types, and role playing game (RPG) game types may also be selected.

[0069] GUI 400 may also include other options that may be configured by the player. For example, options 412, 414, and 416 may be presented for selection to implement a boss change whenever a change is received in the game (option 412), to implement the change in response to the next console startup, game update, or game application launch (option 414), or to implement the change when a new game is started from the starting level or stage of the game (option 416).

[0070] Options 418 and 420 may also be presented. Option 418 may be selected for the device or video game to perform changes specifically only on bosses, while option 410 may be selected to perform difficulty level changes on all non-player opponents that a person may encounter while playing the video game.

[0071] In some examples, GUI 400 may even include options 422-428 for selecting one or more specific bosses or other non-player opponents to implement a change in difficulty level in accordance with the principles of the present invention. This may be done so that a player may choose to play against certain non-player opponents whose difficulty level has been adjusted, while also playing against other non-player opponents of the same game whose difficulty level has not been adjusted but whose difficulty level is maintained at a default or initial level. Thus, option 422 may be selected to configure the device or video game to perform a change in difficulty level for all bosses or all non-player opponents. Option 424 may be selected to configure the device or video game to perform a change in difficulty level for a boss only at the end of a first level of a video game, and option 426 may be selected to configure the device or video game to perform a change in difficulty level for a boss only at the end of a second level of a video game. In addition, option 428 may be selected to configure the device or video game to perform a change in difficulty level for a final boss only at the end of a video game (e.g., the last or final level of a video game).

[0072] Figure 5 Yet another example opt-in GUI 500 is shown. GUI 500 may be presented as part of GUI 400, or may be presented separately. In any case, GUI 500 may include various options that may select, via touch or cursor input, a version of a boss for a given video game that has had its difficulty level altered based on the play of a particular team of players participating together, such as a video game demonstration or competition. Bosses customized for these groups of players may typically exhibit a difficulty level that makes the boss more difficult to defeat in response to the team's more sophisticated gaming abilities.

[0073] As shown, a boss that changes based on the game of the "California Esports" team can be selected via option 502. Option 502 can even be accompanied by an interesting message 504 from the team itself, for example, if the team is given the option to make its version of the boss available to others. The message in this case indicates, "Come try to beat our boss!"

[0074] Another example option 506 may also be presented for selecting a boss version that is based on gameplay of the same video game but altered by the "Bay Area Sim Pros" team. A message 508 from the team may also be included, such as "We challenge you!"

[0075] It can now be appreciated that machine learning can be used to make non-player opponents that a person might face in a video game harder to beat if one or other person beats the opponent too easily, or easier to beat if one has difficulty beating the opponent. Training data from people who have faced a given non-player opponent can be used to build a model specific to that non-player opponent, which can then be used to update the opponent with new or different strategies or moves. In some examples, a boss or other non-player opponent might even become more capable of defending against attack patterns of a specific player, or against attack patterns of the general population.

[0076] Furthermore, some bosses may have their evolution suppressed relative to other bosses, making the former evolve faster and therefore more difficult to defeat relatively quickly, while other bosses may evolve at a slower rate. This may be configured by the developer of the video game so that, for example, a boss encountered in the first level of the video game is still relatively easy to defeat even after many people have defeated that boss, while a boss in the final level may evolve faster and therefore may become more difficult to defeat if many people have defeated it.

[0077] In some examples, the developer of a video game may even set a threshold for how often a given boss is actually defeated relative to the total number of attempts to defeat the boss. For example, the developer of a video game may set a threshold for people to defeat a certain boss 50% of the time on average, and thus the machine learning model may increase or decrease the difficulty level of the boss over time until the most recent X attempts reach an average of 50% where the boss was defeated by a subset of the overall population or even a specific player.

[0078] For example, the present principles can also be used in simulation games where players are thrown into an unknown virtual world and try to understand the world and simulate living and interacting within the world. In such examples, as more and more people play the video game and more training data becomes available, the video game can subsequently become "smarter", e.g., more difficult for the player to advance.

[0079] Different rewards can even be given to players who beat leaders at different difficulty levels, relative to the rewards given by default based on the default difficulty level set by the developer of the video game for the leader. Different rewards can even be triggered based on the time threshold and / or training data volume threshold for the game for the leader reaching a certain amount. For example, if a player wants to beat a leader that is changed based on the ability of ordinary people to beat the leader, but only two weeks after the video game is released to the public, and / or after only twenty people's training data is available, although the leader has changed, a more moderate reward increase can also be provided. In contrast, a person beats the same leader six months after the video game is released, and the data is available through the training of 100,000 people, and the leader is changed at a more refined level due to the increase in the amount of training data, and he may instead receive a more important reward for defeating the leader. Similarly, the reward can involve trophies, video game points, video game currency, currency for e-commerce, etc. Other threshold types can also be used in relation to the available training data and the increased reward, such as the total number of hours played by a person, a group of people, or the overall population, and / or the time spent on each specific level of the leader to be changed.

[0080] Additionally, as discussed above, a certain boss may be made smarter or more sophisticated for a certain demographic, such that the nuances required to defeat the boss are a function of the characteristics or tendencies of said demographic. For example, if many people who play esports have become accustomed to using certain techniques (e.g., controller joystick sequences or button combinations) that prove very useful in defeating a certain boss, then the difficulty level of the boss may be more difficult for people who are classified as part of the esports demographic. However, for uninformed or casual gamers, the same boss may actually maintain a default difficulty level, or actually exhibit a reduced difficulty level, because defeating an esports-customized boss may be too difficult for said group.

[0081] As also described above, bosses or other non-player opponents can be customized through machine learning based on a number of factors. These factors may include the sequence and complexity of joystick manipulations used to defeat the boss, which buttons and / or which button combinations are used to defeat the boss, which (one or more) weapons are used to defeat the boss, where the human player's character is located in the virtual environment when the player defeats the boss, etc. In some examples, the server or server farm performing machine learning can even analyze the movement of the human player's or her virtual character throughout the battle with the boss, and generate a heat map to identify where the human player's character is most often located when engaging with the boss, and then customize the boss to avoid those areas, or even focus on appearing in those areas if the human player is otherwise evasive, so that the server can develop counterattacks that may be effective in those areas.

[0082] It will be appreciated that while the present principles have been described with reference to certain exemplary embodiments, these embodiments are not intended to be limiting and that various alternative arrangements may be used to implement the subject matter claimed herein.

Claims

1. A device, comprising: at least one processor; as well as at least one computer storage device that is not a transient signal and includes instructions executable by the at least one processor to: receiving data related to an ability of one or more video game players to defeat a video game opponent; Using the data as training data to train a machine learning model; and utilizing the trained machine learning model to predict new behaviors of the video game opponent to change the difficulty level for the one or more video game players to defeat the video game opponent, wherein the machine learning model includes at least one machine learning algorithm, the machine learning algorithm including a serialization process, the serialization process involving identifying, obtaining and preparing a raw data set, feature engineering and generating extraction, transformation and loading (ETL) jobs that convert the raw data set into features for the machine learning algorithm, the machine learning algorithm being generated by evaluating multiple models evaluated for performance and latency, wherein the machine learning algorithm is selected from models whose prediction quality is better than other models, and the machine learning algorithm provides inferences about how to circumvent certain attacks, or attack patterns, or habits of human video game players, or make the video game opponent easier to defeat if the video game player has difficulty defeating the video game opponent.

2. The apparatus of claim 1, wherein the video game opponent is a virtual video game opponent that is controlled by a computer rather than a human during the game.

3. The apparatus of claim 1 , wherein the instructions are executable to: First data indicative of the changed difficulty level is transmitted to one or more video game consoles or other consumer electronics (CE) devices.

4. The apparatus of claim 3, wherein the instructions are executable to: Second data indicative of a reward for defeating a video game opponent is transmitted to the one or more video game consoles or other CE devices, the reward varying with the difficulty level for defeating the video game opponent.

5. The apparatus of claim 4, wherein the reward varies such that defeating a higher difficulty level of the video game opponent results in a higher reward and defeating a lower difficulty level of the video game opponent results in a lower reward.

6. The apparatus of claim 4, wherein the reward comprises a specific trophy, a specific medal, a specific amount of video game currency, a specific amount of video game points, and / or a specific amount of currency to be spent via an electronic store. The apparatus of claim 1 , wherein the difficulty level is changed by increasing the difficulty level. The apparatus of claim 1 , wherein the difficulty level is changed by decreasing the difficulty level.

9. The apparatus of claim 1, wherein the instructions are executable to: using artificial intelligence machine learning to analyze the data related to the ability of the one or more video game players to defeat the video game opponents to identify successful and / or unsuccessful methods used by the one or more video game players to defeat the video game opponents; and The difficulty level is changed based on the analysis.

10. A method comprising: receiving data related to one or more video game players attempting to defeat a video game opponent; determining different ways to defeat the video game opponent in the future based on the data; and Altering the game against the video game opponent based on the different ways to defeat the video game opponent in the future, wherein the determining and altering are performed by at least one computer using at least one machine learning algorithm, the machine learning algorithm including a serialization process, the serialization process involving identifying, obtaining and preparing a raw data set, feature engineering and generating extraction, transformation and loading (ETL) jobs that convert the raw data set into features for the machine learning algorithm, the machine learning algorithm being generated by evaluating a plurality of models evaluated for performance and latency, wherein the machine learning algorithm is selected from models whose predictive quality outperforms other models, the machine learning algorithm providing inferences about how to circumvent certain attacks, or attack patterns, or habits of human video game players, or make the video game opponent easier to defeat if the video game player has difficulty defeating the video game opponent.

11. The method of claim 10, wherein the video game opponent is a non-player character of the video game.

12. The method of claim 11, wherein the non-player character is a boss that participates at the end of a level or stage of the video game.

13. The method of claim 10, wherein the different ways to defeat the video game opponent in the future are customized for a specific video game player to play with the specific video game player so as to circumvent a specific attack or attack pattern launched by the specific video game player against the video game opponent.

14. The method of claim 10, wherein the different ways of defeating the video game opponent in the future can be customized to be played by multiple video game players in separate game instances to circumvent specific attacks or attack patterns launched by multiple video game players against the video game opponent.

15. The method of claim 10, wherein the different ways of defeating the video game opponents in the future are customized based on a particular subset of video game players.

16. The method of claim 15, wherein the specific subset is selected from the group consisting of: e-sports video game players, first person shooter video game players, simulation video game players, role playing game video game players.

17. The method of claim 10, wherein the different ways of defeating the video game opponent in the future are tailored to increase or decrease joystick manipulation difficulty in defeating the video game opponent.

18. The method of claim 10, wherein the different ways of defeating the video game opponent in the future are tailored to increase or decrease the difficulty of button combinations in defeating the video game opponent.

19. A device, comprising: At least one computer-readable storage medium that is not a transient signal and that includes instructions executable by at least one processor to: Identify the version of the video game on the computer; receiving data from a server indicating one or more increasing or decreasing difficulty levels for defeating one or more opponents of the video game; executing the video game according to the one or more increasing or decreasing difficulty levels for defeating the one or more opponents; as well as Presenting at least one user interface UI, the UI comprising: At least one first selector for selecting to increase or decrease difficulty based on play of a player of the video game on the computer, and at least one second selector for selecting to use a boss / opponent of the altered video game based on a set of game data from a person other than the player.

20. The apparatus of claim 19, wherein the second selector comprises: A first sub-option is used to select the game data set from a first group of people other than the player, and a second sub-option is used to select the game data set from a second group of people.

Images