Method for customizing in-game audio using ai

Abstract

Techniques for customizing computer game audio using artificial intelligence (AI) are described. Game audio is sent to a pre-trained machine learning (ML) model to identify specified audio objects (such as speech, footsteps, and gunshots), to enhance the audio objects by amplifying the audio objects and / or shifting frequencies and / or improving their clarity, and to insert the enhanced objects back into the game audio in real time.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Application No. 18 / 623,930, filed April 1, 2024, which is incorporated herein by reference in its entirety. Technical Field

[0003] This application relates to technological innovations and unconventional solutions that are necessarily rooted in computer technology and produce specific technological improvements, and more specifically, to the use of artificial intelligence to customize computer game audio. Background Technology

[0004] Computer games can be played on a variety of platforms, including home video game consoles, PCs, and cloud servers with emulated consoles or PCs. As understood in this article, gamers may want to enhance certain audio objects. For example, a gamer might want to gain an advantage in stealth games by better hearing the footsteps of their opponents. Or, someone with hearing difficulties in the low-frequency range might want to be able to hear those sounds more clearly. Summary of the Invention

[0005] As understood in this article, simply using EQ or other simple signal manipulation to enhance game audio and amplify a specific frequency band will result in all audio objects within that band being amplified, rather than just a single audio object that the gamer might be particularly interested in.

[0006] Therefore, a device includes at least one processor assembly configured to receive at least one signal from at least one microphone during the rendering of a computer game. The processor assembly is configured to input the signal into at least one machine learning (ML) model and receive at least one predicted audio object from the ML model. The processor assembly is also configured to modify the predicted audio object received from the ML model to render a modified audio object, and to replace the audio object from the computer game represented by the signal with the modified audio object, such that at least one speaker plays the modified audio object instead of the audio object from the computer game represented by the signal.

[0007] For example, the modified audio object has a larger amplitude than the audio object from the computer game represented by the signal, and / or a different frequency than the audio object from the computer game represented by the signal, and / or higher acoustic intelligibility than the audio object from the computer game represented by the signal.

[0008] If needed, the processor assembly can be configured to receive predicted audio objects from the ML model in response to input signals, without receiving other audio objects.

[0009] The modified audio object can include footsteps, weapon noise, or speech, but is not limited to these.

[0010] In the example implementation, the signal from the microphone represents only the initial portion of the audio object from the computer game that the signal represents.

[0011] On the other hand, the method includes sending a microphone signal to at least one machine learning (ML) model during the presentation of at least one computer simulation. The method includes receiving a predicted audio object in response to the sending, and enhancing the predicted audio object received from the ML model to render the enhanced audio object. The method also includes playing audio from the computer simulation using the enhanced audio object on at least one speaker.

[0012] On the other hand, an apparatus includes at least one computer storage device that is not a transient signal, and further includes instructions executable by at least one processor assembly to receive audio from a computer game during game playback and send the audio to at least one machine learning (ML) model. These instructions are executable to render a modified audio object using the output of the ML model and play the modified audio object during game playback, instead of playing the audio object from the computer game's audio.

[0013] The details of this disclosure regarding both its structure and operation can be best understood by referring to the accompanying drawings, in which the same reference numerals denote the same parts, and in the drawings: Attached Figure Description

[0014] Figure 1 It is a block diagram of an example system that includes examples conforming to this principle; Figure 2 An example game system conforming to this principle is shown; Figure 3 An example game system architecture conforming to this principle is shown; Figure 4 Example logic for training a machine learning (ML) model that conforms to this principle is shown in the form of an example flowchart. Figure 5 The overall logic of the example is shown in the format of an example flowchart; Figure 6 Example logic for the first audio object enhancement technique is shown in the example flowchart format; Figure 7 Example logic for the second audio object enhancement technique is shown in the example flowchart format; Figure 8Example logic for third-audio object enhancement techniques is shown in the example flowchart format; Figure 9 An example user interface is shown for selecting the desired audio object to be enhanced; and Figure 10 An example user interface for selecting desired audio object enhancements is shown. Detailed Implementation

[0015] This disclosure generally relates to the computer ecosystem, including various aspects of consumer electronics (CE) device networks (such as, but not limited to, computer gaming networks). Systems described herein may include server and client components that are network-connected, enabling the exchange of data between client and server components. Client components may include one or more computing devices, including game consoles (such as PlayStation® or game consoles manufactured by Microsoft, Nintendo, or other manufacturers), extended reality (XR) headsets (such as virtual reality (VR) headsets), augmented reality (AR) headsets, portable televisions (such as smart TVs, internet-enabled TVs), portable computers (such as laptops and tablets), and other mobile devices (including smartphones and other examples discussed below). These client devices may operate in a variety of operating system environments. For example, some client computers may run Linux, an operating system from Microsoft, or a Unix operating system, or an operating system manufactured by Apple or Google, or a Berkeley Software distribution or Berkeley Standard Distribution (BSD) OS (including descendants of BSD). These operating environments may be used to execute one or more browsing programs, such as browsers developed by Microsoft, Google, or Mozilla, or other browser programs that can access websites hosted by the internet servers discussed below. Furthermore, an operating environment based on this principle can be used to execute one or more computer game programs.

[0016] A server and / or gateway may be used, which may include one or more processors that execute instructions to configure the server to receive and send data over a network, such as the Internet. Alternatively, the client and server may connect via a local intranet or virtual private network. The server or controller may be instantiated from a game console (such as a Sony PlayStation®), a personal computer, or the like.

[0017] Clients and servers can exchange information over a network. For this purpose, and for security reasons, servers and / or clients may include firewalls, load balancers, temporary storage devices, proxies, and other network infrastructure for reliability and security. One or more servers can form a device that implements methods for providing secure communities (such as online social networking sites or gaming networks) to network members.

[0018] A processor can be a single-chip or multi-chip processor that executes logic via various lines, such as address lines, data lines, control lines, registers, and shift registers. A processor that includes a digital signal processor (DSP) can be an implementation of a circuit. A processor assembly can include one or more processors.

[0019] Components included in one embodiment may be used in any suitable combination in other embodiments. For example, any of the various components described herein and / or shown in the figures may be combined, interchanged, or excluded from other embodiments.

[0020] "A system having at least one of A, B and C" (similarly, "a system having at least one of A, B or C" and "a system having at least one of A, B and C") includes systems having only A, only B, only C, both A and B, both A and C, both B and C, and / or both A, B and C.

[0021] Now for reference Figure 1 The figure illustrates an example system 10, which may include one or more of the example devices mentioned above, based on the present principles, and is further described below. The first example device included in system 10 is a consumer electronics (CE) device, such as an audio-visual device (AVD) 12, such as, but not limited to, a projector-based cinema display system, or an internet-enabled TV with a TV tuner (equivalent to a set-top box for controlling the TV). Alternatively, the AVD 12 may also be an internet-enabled computerized (“smart”) phone, tablet computer, laptop computer, head-mounted device (HMD) and / or headset (such as smart glasses or VR headsets), another wearable computerized device, an internet-enabled computerized music player, an internet-enabled computerized headset, an internet-enabled computerized implantable device (such as an implantable skin device), etc. In any case, it should be understood that the AVD 12 is configured to implement the principles (e.g., communicate with other CE devices to implement the principles, perform the logic described herein, and perform any other functions and / or operations described herein).

[0022] Therefore, to implement these principles, the AVD 12 can be constructed by showing some or all of the components. For example, the AVD 12 may include one or more touch-enabled displays 14, which may be implemented using high-definition or ultra-high-definition "4K" or higher resolution flat screens. The touch-enabled displays 14 may include, for example, a capacitive or resistive touch-sensing layer with an electrode grid for touch sensing, conforming to these principles.

[0023] AVD 12 may also include one or more speakers 16 for outputting audio according to these principles, and at least one additional input device 18, such as an audio receiver / microphone, for inputting audible commands to control AVD 12. For example, AVD 12 may also include one or more network interfaces 20 for communicating over at least one network 22 (such as the Internet, WAN, LAN, etc.) under the control of one or more processors 24. Thus, interface 20 may be, but is not limited to, a Wi-Fi transceiver, an example of a wireless computer network interface, such as, but not limited to, a mesh network transceiver. It should be understood that processor 24 controls AVD 12 to implement these principles, including other elements of AVD 12 as described herein, such as controlling display 14 to present images thereon and receiving input from that display. Furthermore, note that network interface 20 may be a wired or wireless modem or router, or other suitable interfaces, such as a wireless telephone transceiver or a Wi-Fi transceiver as described above.

[0024] In addition to the above, the AVD 12 may also include one or more input and / or output ports 26, such as an HDMI port or a USB port, for physically connecting to another CE device, and / or a headphone port for connecting headphones to the AVD 12 to present audio from the AVD 12 to the user via headphones. For example, input port 26 may be connected via a cable or satellite source 26a to audio / video content, either wired or wirelessly. Therefore, source 26a may be a standalone or integrated set-top box, or a satellite receiver. Alternatively, source 26a may be a game console or disk player containing content. Source 26a is implemented as a game console (such as the Sony PlayStation). ® Or Microsoft Xbox ® It may include some or all of the components described below with respect to CE device 48.

[0025] AVD 12 may also include one or more computer memory / computer-readable storage media 28, such as disk-based or solid-state storage devices, which are not transient signals and, in some cases, are embodied within the AVD chassis as a standalone device or as a personal video recording device (PVR) or video disk player (inside or outside the AVD chassis) for playing AV programs, or as removable storage media or a server as described below. Furthermore, in some embodiments, AVD 12 may include a position / location receiver, such as, but not limited to, a mobile phone receiver, a GPS receiver, and / or an altimeter 30, configured to receive geographic location information from a satellite or mobile phone base station and provide this information to processor 24 and / or, in conjunction with processor 24, determine the altitude set by AVD 12.

[0026] Continuing the description of AVD 12, in some embodiments, AVD 12 may include one or more cameras 32, which may be thermal imaging cameras, digital cameras (such as webcams), IR sensors, event-based sensors, and / or cameras integrated into AVD 12 and controllable by processor 24, to collect pictures / images and / or videos in accordance with these principles. Furthermore, AVD 12 may include a Bluetooth® transceiver 34 and other near-field communication (NFC) elements 36 for communicating with other devices using Bluetooth and / or NFC technologies, respectively. An example NFC element may be a radio frequency identification (RFID) element.

[0027] In addition, the AVD 12 may also include one or more auxiliary sensors 38 that provide input to the processor 24. For example, the one or more auxiliary sensors 38 may include one or more pressure sensors that form a layer of the touch-enabled display 14 itself, and may be, but are not limited to, piezoelectric pressure sensors, capacitive pressure sensors, piezoresistive strain gauges, optical pressure sensors, electromagnetic pressure sensors, etc. Other sensor examples include pressure sensors, motion sensors (such as accelerometers, gyroscopes, odometers, or magnetometers), infrared (IR) sensors, optical sensors, speed and / or cadence sensors, event-based sensors, and gesture sensors (e.g., for sensing gesture commands). Thus, the sensor 38 may be implemented by one or more motion sensors, such as individual accelerometers, gyroscopes, and magnetometers, and / or inertial measurement units (IMUs), which typically include a combination of accelerometers, gyroscopes, and magnetometers, to determine the position and orientation of the AVD 12 in three dimensions, or by event-based sensors (such as event detection sensors (EDS)). The EDS conforming to this disclosure provides an output indicating a change in light intensity sensed by at least one pixel in the light sensing array. For example, if the light sensed by the pixel decreases, the output of the EDS may be -1; if the light increases, the output of the EDS may be +1. When the change in light intensity is below a certain threshold, the output binary signal is 0, which can indicate no change.

[0028] The AVD 12 may also include an over-the-air (OTA) television broadcast port 40 for receiving OTA TV broadcasts to provide input to the processor 24. In addition to the foregoing, it should be noted that the AVD 12 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 42, such as an IR data association (IRDA) device. A battery (not shown) may be provided to power the AVD 12, and a kinetic energy harvester may also be provided to convert kinetic energy into electricity to charge the battery and / or power the AVD 12. A graphics processing unit (GPU) 44 and a field-programmable gate array (FPGA) 46 may also be included. One or more tactile / vibration generators 47 may be provided to generate tactile signals that can be sensed by a person holding or touching the device. Therefore, the haptic generator 47 can use a motor connected to an eccentric and / or unbalanced weight via a rotatable shaft of an electric motor to vibrate all or part of the AVD 12, so that the shaft can rotate under the control of the motor (which can in turn be controlled by a processor (such as processor 24)), thereby generating vibrations of various frequencies and / or amplitudes as well as force simulations in various directions.

[0029] It may also include light sources, such as projectors, such as infrared (IR) projectors.

[0030] In addition to AVD 12, system 10 may also include one or more other CE device types. For example, the first CE device 48 may be a computer game console that can send computer game audio and video to AVD 12 via sending commands directly to AVD 12 and / or via a server described below, while the second CE device 50 may include components similar to the first CE device 48. In the example shown, the second CE device 50 may be configured as a computer game controller operated by a player, or as a head-mounted display (HMD) or headset worn by the player. When embodied as an HMD, the second CE device 50 may include a transparent or opaque head-up display for presenting AR / MR content or VR content (more generally, extended reality (XR) content), respectively. The HMD may be configured as a glasses-style display or as a bulky VR-type display sold by a computer game equipment manufacturer.

[0031] In the example shown, only two CE devices are illustrated; however, it should be understood that fewer or more devices may be used. The devices described herein can implement some or all of the components shown for AVD 12. Any component shown in the following figures can be incorporated with some or all of the components shown in the case of AVD 12.

[0032] Now, referring to the aforementioned at least one server 52, it includes at least one server processor 54, at least one tangible computer-readable storage medium 56 (such as a disk-based or solid-state storage device), and at least one network interface 58, which, under the control of the server processor 54, allows communication with other illustrated devices via network 22 and, in practice, facilitates communication between server and client devices according to this principle. Note that the network interface 58 may be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface, such as, for example, a wireless telephone transceiver.

[0033] Therefore, in some implementations, server 52 may be an internet server or an entire server "cluster" and may include and perform "cloud" functionality, enabling devices of system 10 to access a "cloud" environment via server 52, for example, in an example implementation for a network gaming application. Alternatively, server 52 may be implemented by one or more game consoles or other computers in the same room or vicinity as the other devices shown.

[0034] The components shown in the diagram below may include some or all of the components described herein. Any user interface (UI) described herein may be merged and / or extended, and UI elements may be mixed and matched between different UIs.

[0035] This principle can be applied to various machine learning models, including deep learning models. Machine learning models conforming to this principle can be trained using a variety of algorithms, including supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, feature learning, self-learning, and other forms of learning. Examples of such algorithms that can be implemented using computer circuits include one or more neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and an RNN called a long short-term memory (LSTM) network. Large language models (LLMs), such as generative pre-trained transducers (GPTTs) and stable diffusion (SDs), can also be used. Support vector machines (SVMs) and Bayesian networks can also be considered examples of machine learning models. In addition to the types of networks mentioned above, the models in this paper can also be implemented using classifiers.

[0036] Therefore, as understood in this paper, performing machine learning may involve accessing training data and training a model on it so that the model can process further data to make inferences. Thus, an artificial neural network / AI model trained via machine learning may include an input layer, an output layer, and multiple hidden layers in between, which are configured and weighted to make inferences on the appropriate output.

[0037] Figure 1 The system, and subsequent graphs, uses neural networks (NNs) (which can be implemented in machine learning (ML) models) and existing data to enhance game audio. Neural weights are then used to separate certain sounds (such as footsteps) and enhance them. In a non-limiting implementation, a chipset with a built-in general-purpose neural network accelerator can be used, and pre-trained data (e.g., footsteps from the game) can be loaded into the NN accelerator to enhance game objects.

[0038] Therefore, this principle involves using artificial intelligence (AI) and machine learning (ML) to customize and / or enhance in-game audio. Current technology identifies specific audio objects within a game's soundtrack, such as, but not limited to, footsteps, gunshots, and voice, and adjusts their prominence to provide players with a competitive advantage or assist those with hearing impairments. The versatility of current technology extends to a variety of applications, including but not limited to enhancing game immersion, improving accessibility, and personalizing the gaming experience based on player preferences or needs.

[0039] As further explained below, current technology utilizes AI-driven game audio element / object extraction, where AI is trained to accurately identify and separate specific sound elements from complex game audio mixes without human input. Dynamic audio enhancement is then implemented, where the volume and clarity of specific audio elements are adjusted in real time, such as amplifying footsteps in stealth games to provide a tactical advantage. Customization options are provided, allowing players to customize which audio elements are enhanced, thus enabling a personalized gaming experience. Customized audio enhancements also help players with hearing impairments, promoting game accessibility and making the game more inclusive and accessible. This targeted approach, especially in real-time processing, distinguishes it from general audio processing techniques.

[0040] Figure 2 A speaker device 200 is shown, configured as an audio headset for playing audio during computer simulations, such as while playing a computer game. In some examples, one or more microphones 202 may be provided on the device 200 to pick up analog audio and send signals indicative of that analog audio to the AI / ML components described herein, which may be executed by a processor assembly in the device 200. Furthermore, a game console 204 with one or more optional microphones 206 can provide for presenting a computer game on one or more displays 208, which may also optionally have one or more microphones 210. Any of the microphones described herein can provide signals representing game audio to the AI / ML components described herein to perform current techniques. Thus, both the microphones and the processor implementing the AI / ML technology can be located in the headphone speaker device 200 and / or the game console 204 and / or the display 208.

[0041] Figure 3 It shows that it can be made by Figure 2 Further details of the architecture used by any component are provided below. A computer simulation source (such as a computer game engine 300) can send game video to a display 302 for rendering during game playback. The game engine 300 can also send audio to one or more speakers 304 (such as headphone speakers) for playing game audio during game playback. The game audio played on the speakers 304 can be picked up by one or more microphones 306 and provided to one or more processor assemblies 308 executing one or more ML models 310. Based on the microphone signal input, the ML model 310 outputs predicted audio objects, which can be enhanced according to desired enhancement 312 as further described below, thereby creating enhanced audio objects 314, which are then played on the speakers 304 in place of the original audio objects.

[0042] Figure 4 The diagram illustrates that the ML model 310 can be pre-trained before game playback by inputting a microphone signal at state 400 along with a realistic indication of the audio object represented by the signal. Note that the realistic microphone signal may only represent the initial portion of the audio object, allowing the ML model to be trained at state 402 to predict the audio object using only the first segment of the waveform generated by the microphone in response to the audio object. In this way, the pre-trained AI / ML can predict, amplify, and sufficiently insert the audio object back into the audio in real-time to replace the original audio object, making it virtually imperceptible to the player listening to the game audio.

[0043] Now for reference Figure 5 At state 500, the pre-trained ML model described above is provided to a processor assembly, such as a speaker processor assembly found in a chipset in a headphone device or other speaker device. Moving to state 502, the desired audio object enhancements are identified. Examples of these desired enhancements are discussed further below and may include developer or end-user instructions on the specific type of audio object to be enhanced (such as footsteps, gunshots, and speech), and (if necessary) the specific form of enhancement.

[0044] Move to state 504, during computer-simulated playback (such as computer game playback), from one or more microphones (such as... Figure 2 The microphone 202 shown receives a signal. The signal from the microphone represents analog audio played on one or more speakers, such as the speakers of headphones 200. State 506 instructs the signal to be provided to an ML model, which outputs a prediction of the specific audio represented in the signal based on the identifier of the desired audio object to be detected / predicted at state 502. At state 508, enhancement is applied to the predicted audio object according to the desired enhancement in state 502; and at state 510, the enhanced audio object is inserted into a computer-simulated audio stream in place of the full original audio object played in real time.

[0045] therefore, Figure 5 The technology identifies and trains suitable machine learning models, and uses these trained models to perform real-time analysis of game audio feeds to identify different sounds, including but not limited to footsteps and gunshots. Once identified, these sounds are processed according to user settings, which may involve amplification, enhanced clarity, or other modifications. The processed audio is then seamlessly integrated back into the game's audio output, ensuring the enhancement feels natural and blends into the game environment. Note that... Figure 5 The logic and other logic in this paper can be implemented in software or hardware (such as GPU) to reduce latency.

[0046] Figures 6 to 8 This demonstrates a technique for enhancing predicted audio objects output by an ML model. Figure 6 Starting at state 600, one or more audio objects are received from the ML model disclosed above. Proceeding to state 602, the volume (signal amplitude) of the predicted audio object is increased relative to the volume of the audio object to be replaced that was already present in the audio stream. Then, during game playback, at state 604, the enhanced predicted audio object is inserted into the game's audio stream in real time to replace the original audio object (i.e., replace the unplayed portion of the original audio object, which in most cases constitutes the majority of the original audio object).

[0047] exist Figure 7 Beginning at state 700, one or more audio objects are received from the ML model disclosed above. Proceeding to state 702, the frequency / band of the predicted audio object is shifted to a different frequency / band than that represented by the microphone signal from the audio object in the computer game. For example, the frequency of the predicted speech audio object can be shifted to a higher register than the original speech object to help people who may have difficulty hearing low-pitched speech. Then, during game playback, at state 704, the enhanced predicted audio object is inserted into the game's audio stream in real time to replace the original audio object (i.e., replace the unplayed portion of the original audio object, which in most cases constitutes the majority of the original audio object).

[0048] exist Figure 8 Beginning at state 800, one or more audio objects are received from the ML model disclosed above. Proceeding to state 802, the acoustic intelligibility of the predicted audio object is made sharper than the acoustic intelligibility of the audio object from the computer game as represented by the signal. For example, this might involve smoothing the waveform of the predicted audio object compared to the waveform of the original audio object. Then, during game playback, at state 804, the enhanced predicted audio object is inserted into the game's audio stream in real time to replace the original audio object (i.e., replace the unplayed portion of the original audio object, which in most cases constitutes the majority of the original audio object).

[0049] Figure 9 A user interface (UI) 900 is shown that can be visually and / or audibly and / or tactilely presented to select one or more desired audio objects to be enhanced. In the example shown, UI 900 is visually presented on display 902. UI 900 may include prompts 904 for selecting desired audio objects to be enhanced and / or selecting listening requirements that may be related to audio enhancement.

[0050] In the example shown, UI 900 includes a list 906 of audio objects from which the user can select one or more audio objects to enhance. Example list 906 includes, for example, footsteps audio objects that can be amplified to give a player an advantage in stealth computer games. Example list 906 may include voice audio objects to allow players to hear whispers in stealth games, thus gaining an advantage, and / or assisting those who might not hear voices in computer games. Furthermore, example list 906 may include weapon noise audio objects, such as gunshot audio objects.

[0051] In addition, such as Figure 9 As shown at 908, List 906 may include one or more hearing requirement descriptions. In the example shown, the user has difficulty hearing audio objects at lower frequencies within the human hearing range or audio objects at higher frequencies within the human hearing range, so that the predicted frequencies of the audio objects are shifted away from the frequencies of the corresponding original audio objects accordingly.

[0052] Figure 10 A UI 1000 is shown that can be visually and / or audibly and / or tactilely presented to select one or more desired audio objects to be enhanced. In the example shown, UI 1000 is visually presented on display 1002. UI 1000 may include prompts 1004 for selecting specific enhancements to the audio objects, such as using... Figure 9 The UI 900 selects an audio object.

[0053] like Figure 10 As shown, a candidate enhancement list 1006 can be presented; in the example shown, candidate enhancements are A through C, along with a prompt 1008 for selecting one of the candidate enhancements. The candidate enhancements may be enhancements pre-programmed by developers based on enhancements that have been determined to be effective or popular. Each candidate enhancement can be implemented on the selected audio object and / or test tone, and when clicked, played on the speaker, allowing the user to select each individual candidate enhancement sequentially and listen to each candidate enhancement individually to subjectively determine which candidate they prefer. For example, candidate enhancements may include their own corresponding volume and / or frequency ranges. The user then... Figure 5 Box 508 indicates the selection of one of the candidate enhancements to be implemented during gameplay. It should be understood that... Figure 10 The examples provided are not limiting, and other techniques for enhancing audio objects can be employed.

[0054] The imaging of game videos can also be used to identify audio objects that need to be enhanced to reduce latency.

[0055] While specific techniques are shown and described in detail herein, it should be understood that the subject matter covered by this application is limited to the claims.

Claims

1. An apparatus comprising: At least one processor assembly, said at least one processor assembly being configured to: Receive at least one signal from at least one microphone during the presentation of a computer game; The signal is input into at least one machine learning (ML) model; Receive at least one predicted audio object from the ML model; Modify the predicted audio object received from the ML model to render the modified audio object; and The audio object from the computer game represented by the signal is replaced with the modified audio object, such that at least one speaker plays the modified audio object instead of the audio object from the computer game represented by the signal.

2. The device of claim 1, wherein the altered audio object has a larger amplitude than the audio object from the computer game represented by the signal.

3. The device of claim 1, wherein the modified audio object has a different frequency than the audio object from the computer game represented by the signal.

4. The device of claim 1, wherein the altered audio object has greater acoustic intelligibility than the audio object from the computer game represented by the signal.

5. The device of claim 1, wherein the processor assembly is configured to receive the predicted audio object from the ML model in response to input of the signal, without receiving other audio objects.

6. The device of claim 1, wherein the modified audio object includes a footsteps object.

7. The device of claim 1, wherein the modified audio object includes a weapon noise object.

8. The device of claim 1, wherein the modified audio object includes speech.

9. The device of claim 1, wherein the signal represents only the initial portion of the audio object from the computer game represented by the signal.

10. A method comprising: Send microphone signals to at least one machine learning (ML) model during the presentation of at least one computer simulation; In response to the sending, a predicted audio object is received; Enhance the predicted audio object received from the ML model to render the enhanced audio object; as well as The enhanced audio object is used to play audio from the computer simulation on at least one speaker.

11. The method of claim 10, further comprising: The enhanced audio object is used instead of the audio object that causes the microphone to generate the microphone signal to play the audio from the computer simulation.

12. The method of claim 10, further comprising: The enhanced audio object is generated, at least in part, by increasing the amplitude relative to the amplitude of the audio object represented by the microphone signal.

13. The method of claim 10, further comprising: The enhanced audio object is generated, at least in part, by changing the frequency relative to the frequency of the audio object represented by the microphone signal.

14. The method of claim 10, further comprising: The enhanced audio object is generated, at least in part, by increasing the clarity relative to the clarity of the audio object represented by the microphone signal.

15. The method of claim 10, further comprising: At least one user interface is presented, which is configured to receive input indicating desired enhancements to be implemented on the audio object.

16. The method of claim 10, further comprising: At least one user interface is presented, which is configured to receive input indicating a desired video object to be enhanced.

17. An apparatus comprising: At least one computer storage device, which is not a transient signal, and includes instructions executable by at least one processor assembly to: Receive audio from the computer game while playing the game; The audio is sent to at least one machine learning (ML) model; The modified audio object is rendered using the output of the ML model; and The modified audio object is played during gameplay, instead of the audio object from the audio of the computer game.

18. The apparatus of claim 17, wherein the instructions are executable to: At least one user interface is presented, the at least one user interface being configured to receive input indicating desired enhancements to be implemented on the audio object; and The modified audio object is generated based at least in part on the input.

19. The apparatus of claim 17, wherein the instructions are executable to: At least one user interface is presented, the at least one user interface being configured to receive input indicating a desired video object to be enhanced; and The modified audio object is rendered based at least in part on the input.

20. The apparatus of claim 17, wherein the instructions are executable to: The modified audio object is rendered using audio from the computer game that represents only the initial portion of the audio object.

Images