User preference customization generation of game music

Abstract

Systems, processes, and device configurations are provided for user preference customization generation of game music. A method may include detecting at least one user preference for game music; determining a game state; and generating game music based on the at least one user preference and the game state. Customization of audio and game music can have a great influence on game experience. Music preferences may be collected for a user to generate dynamic music and improve the gaming system ability to generate a customized experience. Music preferences may include one or more of genre, particular song titles, indications of preferred music types, music identified for certain gaming campaigns, user feedback, and native countries. Music may be generated based on emotional cues of the game scene. The music may be controlled to be more prominent or less obtrusive to the game state.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. patent application Ser. No. 17 / 871,273, filed on July 22, 2022, entitled “USER PREFERENCE CUSTOMIZED GENERATION OF GAMING MUSIC,” U.S. patent application Ser. No. 17 / 871,279, filed on July 22, 2022, entitled “USER INTERACTION CUSTOMIZED GENERATION OF GAMING MUSIC,” U.S. patent application Ser. No. 17 / 871,294, filed on July 22, 2022, entitled “CUSTOMIZED AUDIO SPECTRUM GENERATION OF GAMING MUSIC,” U.S. patent application Ser. No. 17 / 871,303, filed on July 22, 2022, entitled “GAME ENVIRONMENT CUSTOMIZED GENERATION OF GAMING MUSIC,” and U.S. patent application Ser. No. 17 / 871,304, filed on July 22, 2022, entitled “INTERFACE CUSTOMIZED GENERATION OF GAMING MUSIC” The disclosures herein are based on priority to U.S. Patent Application No. 17 / 871,307 entitled "CUSTOMIZED GENERATION OF GAMINGMUSIC", the specifications of which are expressly incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to generating audio for electronic games and customizing game audio and game music, including music arrangement, music learning, game design, and game generation. Background Art

[0004] Music in electronic games (e.g., video games, computer games, gaming platforms, app-based games, etc.) is often pre-recorded and played as a loop to accompany the video component of the game. Games can often include multiple music recordings for output that are generated during the development of the game. In addition to pre-recorded loops, it is desirable to increase the personalization of the music and audio of the game and game content. For many electronic games, the user does not have the option or ability to modify the music or audio accompaniment except for controlling the volume. Muting the game volume can also mute the sound effects and other audio output of the game. There is a desire for personalization of game content.

[0005] Another problem with gaming systems is the need to format game elements for different gaming markets. Games have a global fan base that serves many game titles and different regions. Certain features of a game, such as a musical style, may not be popular with gamers. Development of additional markets and regions may be expensive and impose additional data requirements on storage. Another difficulty with personalized games and game content is identifying features that will enhance the user experience. There is a need and desire to customize game music to supplement the user experience. Summary of the invention

[0006] Systems, methods, and configurations for user preference customized generation of game music are disclosed and described herein. In one embodiment, a method is provided for detecting, by a device, at least one user preference for game music, determining, by the device, a game state, and generating, by the device, game music based on the at least one user preference and the game state, wherein the game music is dynamically generated to include at least one music theme, the at least one theme being selected based on the at least one user preference and the game state. The method also includes outputting, by the device, the game music.

[0007] In one embodiment, the at least one user preference for the game music includes at least one of a music genre, a music style, a song title, an identification of a music artist, a composer, user feedback, and music preferences.

[0008] In one embodiment, the at least one user preference for gaming music includes at least one of country of origin, nationality, geographic region, religious beliefs, and language preference.

[0009] In one embodiment, detecting at least one user preference for gaming music includes displaying a user interface including a feedback display element for at least one of input of a user preference and feedback on gaming music generated by the device.

[0010] In one embodiment, detecting at least one user preference for gaming music includes determining the user preference from user profile data.

[0011] In one embodiment, the game state is the current game situation within the electronic game.

[0012] In one embodiment, generating game music includes selecting a music theme for a game element of the game state, wherein selection of the theme is guided by at least one user preference.

[0013] In one embodiment, generating the game music includes selecting a music library based on user preferences and generating the game music from one or more themes in the music library.

[0014] In one embodiment, generating the game music includes selecting a music style based on at least one of a cultural parameter and a language parameter.

[0015] In one embodiment, outputting the game music includes output of dynamic music coordinated with output of the game content and controlling prominence of the game music relative to the output of the game content.

[0016] Another embodiment relates to a device configured for user preference customized generation of game music. The device includes a memory storing executable instructions and a controller coupled to the memory. The controller is configured to detect at least one user preference for game music, determine a game state, and generate game music based on the at least one user preference and the game state, wherein the game music is dynamically generated to include at least one music theme, the at least one theme being selected based on the at least one user preference and the game state. The controller is also configured to output the game music.

[0017] Other aspects, features, and techniques will be apparent to persons skilled in the relevant arts in view of the following detailed description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The features, objects and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the accompanying drawings, in which like reference numerals are identified accordingly throughout, and in which:

[0019] Figure 1 is a graphical representation of customized music generation according to one or more embodiments;

[0020] Figure 2 A process for user preference customized music generation according to one or more embodiments is shown;

[0021] Figure 3 shows a graphical representation of user preference customization according to one or more embodiments;

[0022] Figure 4 shows a graphical representation of dynamic music generation according to one or more embodiments;

[0023] Figure 5 shows a graphical representation of a user interface according to one or more embodiments;

[0024] Figure 6 A process for user interactive customized music generation according to one or more embodiments is shown;

[0025] Figure 7 shows a graphical representation of user interaction and game music training according to one or more embodiments;

[0026] Figure 8 A process for user customized audio generation according to one or more embodiments is shown;

[0027] Fig. 9A showing a graphical representation of an audio spectrum customization of gaming music according to one or more embodiments; and

[0028] Fig. 9B A graphical representation of frequency range customization for game music is shown in accordance with one or more embodiments. DETAILED DESCRIPTION

[0029] Overview and Terminology

[0030] One aspect of the present disclosure relates to generating customized game audio, such as game music and game sound effects. Through customized operation and audio generation, game audio can be more personalized, and the output can be customized for users, user environments and user conditions. A process and framework for enabling devices (such as game consoles) and systems to generate game audio are provided. The features related to game music discussed herein can be applied to other types of audio, including but not limited to sound effects. In addition, generating game music as discussed herein can include generating music and / or non-musical audio accompaniment for electronic games. It should be understood that game music can include one or more sound pronunciations (voicing), such as human voices and / or instrument sounds in a manner that produces sounds that can include sound arrangements with melody, rhythm and harmony in some cases. Game music can be generated as one or more tracks for output. According to an embodiment, game music is dynamically generated and output so that the audio output is determined and output by the device during the game.

[0031] Embodiments relate to user preference customization generation of game music. Systems and methods may include determining at least one user preference for game music, determining a game state, and generating music based on at least one user preference and the game state. Customization of audio and game music may have a significant impact on the game experience. Music preferences may be collected for users to generate dynamic music and improve the ability of the game system to generate customized experiences. Music preferences may include one or more identifications of genres, specific song titles, indications of preferred music types, music identified for certain game activities, user feedback, spoken language, seed music, and countries of origin. Music preferences may include music styles, tracks, volume, frequency, and playback rankings or orders of output. According to embodiments, game music may be generated based on emotional cues of a game scene. Music may be controlled to be more prominent or less obtrusive to the game state.

[0032] Embodiments relate to user-interaction customized generation of game music. Systems and methods for generating customized musical accompaniment for electronic games are provided. The method may include determining at least one user interaction with a gaming system for generated music, updating parameters for music generation based on the at least one user interaction, and generating music components for the gaming system based on the parameters. Music interactions with the gaming system may be collected during gaming to train and retrain a dynamic music generator. Dynamic music generation of outputs that interfere with sound effects of the electronic game may be retrained. Inputs may be provided by sensors of the gaming system to detect interactions and control of dynamic music during gaming. Inputs may include detecting the presence of a viewer and user environment to modify at least one output characteristic of the dynamically generated music, and thereby generating user-presence customized generation of gaming music.

[0033] Embodiments relate to frequency range customization generation of game music. Systems and methods for generating customized musical accompaniment for electronic games are provided. The method may include detecting user interactions with a gaming system for frequency components of the generated music, updating dynamic music generated based on the user interaction to modify at least one frequency output characteristic, and outputting the modified music. The frequency or pitch of the generated music that interferes with the game sound effects may be adjusted to improve the game. In some cases, the volume of the sound elements may be changed in volume and / or pitch to indicate the presence of a game element or opportunity during the game. Similarly, the frequency range may be customized to avoid one or more frequencies for the user, such as a frequency range identified as being associated with hearing loss or causing disturbances (such as vibrations).

[0034] As used herein, the term "a" or "an" shall mean one or more than one. The term "plurality" shall mean two or more than two. The term "another" is defined as the second or more. The terms "including" and / or "having" are open ended (e.g., comprising). The term "or" as used herein shall be interpreted as inclusive or meaning any one or any combination. Thus, "A, B, or C" means "any of: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur when a combination of elements, functions, steps, or actions are inherently mutually exclusive in some way.

[0035] Reference throughout this document to "one embodiment," "some embodiments," "an embodiment," or similar terms means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of such phrases in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments without limitation.

[0036] Exemplary Embodiments

[0037] Figure 1 is a graphical representation of customized music generation according to one or more embodiments. According to an embodiment, customized music generation can be provided by a device and / or system. A system can include one or more components and devices. Figure 1 Shown is a system 100 that may include device 105 according to an embodiment. Device 105 may be one or more of a game console, a computing device, and an electronic device that are typically configured to output game content 106 to a display. With regard to game content, device 105 may be configured to output data for presentation by a display or visual output device, typically including graphics (2D and 3D), sound, and data. Device 105 may also be configured to output non-game content, such as video content, visual content, audio content, etc. Embodiments describe the generation of dynamic game music for electronic games and games, but it should be understood that the principles of the present disclosure may be applied to other forms of media and non-game content. Therefore, device 105 and system 100 may be used to generate custom music for one or more applications.

[0038] According to an embodiment, the device 105 is configured to output data 106 and / or content to a display 110. According to an embodiment, the display 110 may be separate from the device 105 or a part of the device 105. The device 105 may be configured to communicate with one or more servers 120 via a network 115. 1-n The device 105 may be configured to receive data from the server 120 1-n Receive one or more of user data, game data and music data. Server 120 1-n Can be configured to store one or more of user data, game data and music data.

[0039] Figure 1 Device configurations according to embodiments are depicted. Device 105 may relate to a console, a media device, and / or a handheld device configured to generate game music. According to embodiments, device 105 includes a controller 125 and a memory 126. Device 105 may also include a network communication module 127 and a music generation module 128. Device 105 may also optionally include game content 129 and one or more sensors 130.

[0040] The controller 125 may refer to a processor or control device configured to perform one or more operations stored in the memory 126, such as a process for generating game music. The controller 105 may be coupled to the memory 126, the network communication module 127, and the music generation module 128. The memory 126 may be a non-transitory memory configured to provide data storage and working memory operations for the device 105. The memory 126 may be configured to store computer-readable instructions for execution by the controller 125 for one or more processes described herein. The network communication module 127 may be a communication module configured to receive and send network communication data. According to one embodiment, the controller 125 may be configured to determine one or more parameters and user preferences of the game music. One or more of the user preferences, interactions, and parameters for dynamic music generation may be determined and used by the controller 125. According to an embodiment, the music generation module 128 may perform music generation operations. The music generation module 128 may include pronunciations, music themes, music patterns, and parameters to generate music for the game, including before, during, and after playing the game. In addition to gaming, dynamic music generation associated with presenting menus and / or operating the device 105 as a gaming console may be performed. The music generation module 128 may be configured to perform one or more learning or training operations to allow music generation. The music generation module 128 may also store one or more data parameters for generating gaming music.

[0041] Device 105 may be configured to receive gaming media (e.g., cards, cartridges, disks, etc.) and output the visual and audio content of the gaming media to display 110. Device 105 may be configured to receive gaming media (e.g., cards, cartridges, disks, etc.) and output the visual and audio content of the gaming media to display 110. 1-n One or more of the server 120 receives data to present and output game content, which may be stored in the memory 126. For network games, the device 105 may receive data from a server such as the server 120. 1-n The network source receives the game data and can be configured to generate game music for the game data.

[0042] Depending on the embodiment, the device 105 may include one or more optional sensors 130. The optional sensors 130 may include one or more of audio (e.g., sound), optical (e.g., camera), and spatial (e.g., presence, proximity, etc.) sensors for collecting user feedback, sensing one or more of the user's environment, and generally for generating game music. The device may include one or more speakers to generate audio output, as shown by audio 131. When connected to an external device (such as display 110), one or more speakers of the external device may be used to output audio.

[0043] Device 105 may be configured to receive data from one or more peripheral devices (such as input device 1351-n ) receives input. Input device 135 1-n 105 and a gaming console. The device 105 may be configured to operate with one or more external sensors, such as optional sensor 140. Optional sensor 140 may be configured to detect audio and / or the presence of a viewer in a user's location. Display 110 may include a sensor, such as optional sensor 140.

[0044] System 100 may include a display 110 for output of visuals and audio of an electronic game and / or output of device 105. According to an embodiment, display 110 may display control elements of device 105, such as selectable menus 150, which may be used to interact with device 105. Display 110 may output audio 155 based on data 106 received from device 105.

[0045] According to an embodiment, device 105 may be configured to generate game music based on user preferences. According to other embodiments, device 105 may be configured to generate game music based on user interaction with a gaming system. In other embodiments, device 100 may generate game music based on parameters for customizing audio spectrum generation. Device 100 may utilize one or more operations of processes 200, 600, and 800 and the processes generally described herein. It should also be understood that system 100 and its components may be configured to perform the operations of device 105.

[0046] This article describes a system, device and process for generating game audio (such as game music).According to an embodiment, game audio can be dynamically generated so that it is not pre-constructed.Dynamic game music can be generated to include audio with one or more of melody, rhythm and harmony.Dynamic game music can be generated to include one or more tones (e.g., notes), pitch progression, rhythm beats and percussion elements provided by one or more music pronunciations (e.g., musical instruments, synthesizers, etc.).Different from pre-recorded traditional game music, dynamic game music can be generated by system 100 or device 105 for current game situation, player and playing game environment.One or more themes trained to include one or more of melody, rhythm and harmony components can be used to generate dynamic music so that device 105 can use theme to generate the audio output of the game based on game situation and / or one or more parameters.In the context of user preference customization generation of game music, device 105 can generate music for the startup screen of the game title so that one or more music themes are selected, and the device selects one or more pronunciations for the theme based on the game.According to an embodiment, the selection of theme, the selection of pronunciation and the output of theme can be based on user preference. As discussed herein, video games are a global activity, and thus popular music in one region of the world may favor music with a strong rhythmic component, while another region may favor loud ambient sounds. Furthermore, user preferences may not match the musical style associated with a region. Embodiments allow for the customization and modification of dynamic music to take into account a specific user, without requiring game developers to generate an almost unlimited number of pre-recorded soundtracks.

[0047] By providing dynamic game music, system 100 and device 105 can provide various forms of music output. According to an embodiment, user preference customization generation of game music is provided to customize the dynamically generated game music to provide a more personalized experience. Technically, the dynamically generated game music does not need to pre-record music for each user's preferences. As discussed herein, user preferences can include one or more preferences for music style, genre, specific title, indication of preferred music type, music identified for certain game activities, user feedback, country of origin, global region, language preference and even user environment.

[0048] Figure 2 200 can generate game music using one or more user preferences. Figure 1The device 105 and / or controller 125 of the present invention is adopted to generate game music based on at least one user preference of one or more users (e.g., players) of the game. Process 200 can also be performed to help generate dynamically generated music. Dynamically generated game audio can be generated for a current game situation that is not pre-built, and thus flexibility can be provided for one or more customized audios in the current game situation and the players who will hear the audio. Process 200 provides a solution to prevent the provision of content that users do not expect. Process 200 can be initiated by a device (e.g., device 105) detecting at least one user preference of game music at box 205. One or more user preferences can be determined for the user. The gaming device, console, or system can identify the user by the selection of a user profile. In the absence of a user profile, the device can associate the user with one or more of an account, a login, and an identifier (e.g., an email address, a call sign, etc.). According to an embodiment, detecting at least one user preference for game music includes determining the user preference according to user profile data. The profile data may include one or more of a preferred language, a preferred region, a user location, and a user selection of the gaming system.

[0049] Detecting the user's gaming music preferences and information about the user can be actively and / or passively collected by the device. Passive collection of information can be achieved through music selections made by players on the same system or device where the game is played (such as songs played or playlists created by players). When determining user preferences, more attention can be paid to songs played by players when playing games of similar styles. Passive collection can also include collecting data from user profiles. Process 200 can include detecting player preferences from one or more devices and network systems through an application programming interface (API).

[0050] Actively collecting player preferences at frame 205 may include allowing players to specify their music expectations through a user interface. Detecting game music preferences in process 200 may include controlling the display of a user interface (e.g., menu 150) including one or more graphical elements to obtain user preferences. The interface may include elements that request and obtain user preferences for genres or songs. The user interface may request information about aspects of preferred music, such as musical instruments to be used and / or not used. According to embodiments, information may be collected through a game console user interface or a game interface. When the user interface is provided by the system, user / player preferences may be used for multiple games. Players may set default music preferences, and then customize these preferences on a game-by-game basis. Players may have multiple preferences, and may select different preferences based on the mood they were in at the time when playing games.

[0051] According to an embodiment, detecting user preferences may include receiving a user's opinion of a set of music clips played by the device. A user interface of the device may obtain the user's opinion of each music clip, an indication of the emotion the user player associates with the music clip, and a ranking of one or more ratings describing the feeling of the music (such as energy level, complexity, and game type). Detecting at least one user preference for game music may include displaying a user interface including a feedback display element for at least one of input of a user preference and feedback on the game music generated by the device.

[0052] According to an embodiment, preferences may include that the generated music will be less obtrusive in order to focus on other aspects of playing the game or more prominent in order to increase user feedback of the emotions or feelings evoked by the music in the game. These user preferences may be incorporated into the game music by controlling aspects of the music generated for the game other than the overall music volume.

[0053] According to an embodiment, user preferences for game music may be detected at block 205 for at least one of music genre, music style, song title, identification of music artist, composer, user feedback, and music preferences. The generation of game music may include generating one or more music components, such as melodies, ambient noises, and synthesized sounds. According to an embodiment, user preferences may be employed to customize game music for user preferences and / or desired output styles. This will also allow users to have music genres and styles that they like when playing games, while still having music that conveys the feel of the game and different aspects of playing the game. This customization of music to users may make the same game more attractive to a wider audience, where the genres and styles of music differ based on the preferences of the user playing the game. Directing the music generated for gameplay to the preferences of the player will differentiate gameplay streams and videos between players, rather than just differences in game play styles. This can help players attract more dedicated fans who prefer to watch their gameplay relative to other players' gameplay. This diversity of music may allow games to attract a loyal fan base from a more diverse range of backgrounds.

[0054] According to an embodiment, user preferences for game music may include at least one of country of origin, nationality, geographic region, religious beliefs, and language preference. Different sounds may have different meanings in different cultures or to speakers of different languages. Music and sound generation may take sounds into account using the language setting of the game and the region or country in which the game is being played. In an embodiment, the user may override the location setting.

[0055] At box 210, process 200 may include determining a game state. The game state may be a current game situation within an electronic game, such as a control or menu screen, a portion of a gameplay, and a transition to a gameplay. In order to connect with the user, the device may be configured to generate appropriate music to accompany the game state. Data from the game may provide metadata or game vector data in the form of a game state, including one or more of a mood or parameter associated with the gameplay. The game state may also include sound elements generated for the game, such as sound effects and alarms. According to embodiments, the device may be configured to determine the game state based on the game medium, such as determining a stage of gameplay, a menu screen, or a transition from gameplay to game controls based on a graphical output.

[0056] At frame 215, process 200 includes generating game music. According to an embodiment, generating game music at frame 215 of process 200 may include generating one or more of audio, sound and music (such as game music) based on at least one user preference. Generating game music at frame 215 may also be based on game state. Whether there is emotion or action in the game, the game state will have information that can be used by the device. For example, a driving game with high speed may have a game state indicating cheerful music to be generated for the game. Therefore, music may be generated by process 200 to include cheerful mode. Another game situation may require that the output may be more related to the pronunciation of the synthesizer (e.g., synthesized sound) style. According to an embodiment, generating game music at frame 215 includes using at least one music theme, which is selected based on at least one user preference and game state. Generating music may include different forms and styles. The theme may be combined with different pronunciation or sound databases to generate audio output. As an example, the theme may include the arrangement of melody and rhythm, so that generating music may include modifying and / or selecting the output of rhythm based on user preference. The first theme can be used to generate the game music of the first form and the second form, and the game music of the first form and the second form has at least one distinguishing characteristic. When the user preference, for example, includes genres (such as rock music), generating music can include defining rhythm patterns (for example, four-beat floor rhythms), bass lines and electric guitars for the theme. If the user preference includes the genre of dance music, different pronunciation selections and rhythms can be applied to the theme. Alternatively or in combination, different themes can be selected based on user preferences. In addition, music generation can be dynamically created such as by equipment on the scene.

[0057] According to an embodiment, the selection of a music theme can be based on one or more of the game elements of a game state, and the selection can be directed by at least one user preference. According to an embodiment, a device can identify one or more themes for the output of a game for a game state, and the selection of at least one theme can be directed by a user preference. According to an embodiment, music generation can be based on information (e.g., game media, game code, game data, etc.) about what emotions the game attempts to evoke in the player during the game state from the game, and the information can be used to create music pointing to the profile of the current player, to best cause the desired emotions in the player. Process 200 can be seeded with music marked with the emotions that music usually evokes in people. Such emotional tags can be different based on factors such as culture, geography, music taste, gender or age. The process according to these embodiments can consider these differences together with the profile information about the player, to best align the tag set that it pays most attention to with the player.

[0058] Generating the game music may include selecting a music library based on user preferences and generating the game music according to one or more themes of the music library. The music library may involve one or more of music stored on the device, music associated with the user, and a music library of a network service. Generating the game music includes selecting a music style based on at least one of a cultural parameter and a language parameter.

[0059] Process 200 may optionally include detecting interference at block 211. Depending on the embodiment, process 200 may be configured to detect audio (e.g., game music) output by a device and one or more sounds relative to the device. For example, detecting interference at block 211 may detect audio output of game music that interferes with game sound effects. When the game music is too similar to the sound effects of the game, generating game music at block 215 may adjust the game music to not interfere. Similarly, when the game music generates sounds that cause disturbances such as vibrations, generating game music at block 215 may adjust the game music to reduce and / or eliminate the disturbance by modifying output characteristics.

[0060] At box 220, process 200 includes outputting game music. Outputting game music includes output of dynamic music coordinated with the output of game content and controlling the prominence of the output of game music relative to the game content. At box 220, process 200 may include outputting the game music to a terminal coupled to a display or media device. According to an embodiment, the output of the game music at box 220 may be to a speaker of the device. According to other embodiments, the output may be to a network connection. According to an embodiment, process 200 may optionally include retraining parameters for generating game music at box 221. Retraining at box 221 may include modifying one or more parameters for generating game music based on one or more of user preferences, user interactions, and feedback from the user.

[0061] Process 200 may include an optional box 225 for controlling the output of game music. The output of game music may include controlling one or more of melody, rhythm, and output style based on the gameplay. For example, a situation of increased stress in the game, such as approaching completion of a task or defeating an obstacle, may be detected by the device based on a game vector, and one or more characteristics of the game music may be adjusted at box 225 accordingly. According to an embodiment, controlling the output of game music at box 225 may include modifying one or more selected themes in response to a change in the game state. According to an embodiment, controlling the output of game music at box 225 may include terminating the output of game music. Controlling the output of game music may be based on one or more user inputs to the device.

[0062] Figure 3 A graphical representation of user preference customization according to one or more embodiments is shown. According to an embodiment, generating game music and generating game audio can be based on parameters that define one or more attributes of the music and / or audio to be output. The parameters can be used to define themes that can be combined. The device can store parameters 300 for outputting music and audio. The device can also store parameters for the user, including one or more of preferences, rankings, game-specific selections, music history played, user interactions, and user feedback. The parameters can be stored in a library 301. 1-n According to an embodiment, the country / region library 301 1 One or more music patterns, rhythms, and sounds may be stored for each of a plurality of regions and countries. As an example, the North American region may include music styles including, but not limited to, rock, country, hip-hop, and classical music styles. Alternatively, the European region may include music styles including, but not limited to, dance music, traditional music, rock, and pop music. Country / Region Library 301 1 Parameters may be included to identify music and audio preferences of one or more specific countries, regions of countries, and geographic regions. Country / Region Library 301 1It is also possible to include a global category that contains all the musical styles of the system. Language Library 301 2 The language library 301 may include one or more terms or sounds associated with a language. 2 One or more sounds to be avoided may be identified, such as unpleasant sounds or offensive language associated with one or more user preferences for language and / or nationality. Thus, generating game music may avoid one or more sounds associated with the user preferences for sounds to avoid mapping to one or more languages ​​and nationalities.

[0063] According to an embodiment, the music genre library 301 3 One or more characteristics of music associated with a music genre may be stored, including chord progressions, rhythmic patterns, articulations, fills, and song structure. User database 301 n One or more preferences or parameters associated with a user may be stored. According to an embodiment, the user database 301 n One or more dynamically generated tracks or playlists may be stored for user playback. n One or more preferences or parameters associated with sounds to be avoided and sound outputs to be avoided may be stored. As an example, user database 301 n One or more user preferences (which may be indicated by the user and / or detected by the device) for various sounds or sound categories may be stored to avoid including one or more of high pitched sounds and sounds with sudden changes in volume (e.g., increases, decreases, etc.). User database 301 n One or more preferences or parameters associated with the user's religious beliefs may be stored.

[0064] The device may also use a theme database 302 including one or more themes that can be used to generate music. Emotions may be associated with one or more of the emotions and situations associated with the game. For example, a game including a chase scene may include music accompanying the chase and one or more themes associated with the risk level or stage of the chase. The theme may also provide a building block of song elements that may be used to generate music that is not an arbitrary sound collection. According to an embodiment, the theme database may include song elements generated from machine learning to generate music and / or audio elements for the game. According to an embodiment, the device may optionally obtain or identify music from a user music database 303 that may be stored in one or more of a device or network location. The device and system described herein may be configured to pull or read one or more of the music artists, songs, and playlists associated with one or more accounts that may be linked to the user.

[0065] Parameters 300 may be stored by a device (e.g., device 105) and / or stored at a network location accessible to the device (e.g., server 120). 1-n ). Dynamic music generation may be based on parameters 300 and one or more user preferences. According to an embodiment, the user preferences detected for one or more users may be stored by a device (e.g., device 105). When the device generates game music or game audio, one or more interactions with the device may be used to identify, set, and store one or more of the parameters to generate game music and audio.

[0066] Figure 3 User interaction 305 associated with game music 310 output by the device is shown 1-n During game play and / or presentation of game menus, game music 310 may be output and the device may detect one or more user interactions, such as user interaction 305. 1-n The device may be configured to update the user profile, such as updating profile 315 1-n According to an embodiment, one or more user interface selections may be detected, such as user interface selection 305. 1 and can update the user's profile, in response to the selection to update the profile 315 1 For example, user interface selections indicating preferences for country / region, language, music genre, and game music may generally result in the device determining / updating parameters for generating music. The user interface selection may be one or more of a menu selection, a volume control, a mute action, and a game control. User feedback 305 2 It can be one or more of the rating, comment, acceptance and cancellation of the game music output by the device. As user feedback 305 2 As a result, an update profile 315 may be performed 2 . Multiplayer Profile 305 3 It can be one or more preferences for a multi-player gaming environment for game music. As a multi-player profile 305 3 As a result, an update profile 315 may be performed 3 According to an embodiment, the detected user preference 305 n The device may determine one or more preferences for game music, including but not limited to the presence of a user's viewer, user preferences from a playlist, seed music, and ambient noise. User preferences detected by the device 305 n As a result, an update profile 315 may be performed n .

[0067] Figure 4A graphical representation of dynamic music generation according to one or more embodiments is shown. According to an embodiment, the generation of game music can be performed dynamically so that the music and / or audio of an electronic game is determined and output during the operation of the game. The dynamic music generation can employ the use of one or more themes that can be used to generate the output. According to an embodiment, dynamic game music generation 400 is performed by a music generator 405. The music generator 405 can be one or more components of a device, such as a controller or processor of a game console or electronic media player. The music generator 405 can be configured to output game music 410 based on one or more of a game vector 406, a user preference 407, a user music library 408, and a game music theme 409. According to an embodiment, a game console can generate and / or receive a game vector from a game medium, which describes one or more operations and states of gameplay. The game vector can include a description of a gameplay scene, a gameplay type, or an emotion associated with one or more game states. The game vector and one or more game states can be used to evaluate the type of music to be generated. Reference above Figure 3 User preferences 407 are described, and user preferences 407 can provide one or more parameters for generating game music. User music library 408 can provide one or more music titles that can be used as references for generating music. For example, if the music library includes a specific type of music (such as techno), the type of music can be used as a reference for generating similar music. Game music theme 409 can be provided by one or more of the game media and the server to provide one or more music elements that can be used to generate music. In some embodiments, game music 410 can be generated independently of pre-recorded music for a specific title. In other embodiments, game music 410 can modify or combine one or more game-specific themes.

[0068] According to an embodiment, dynamic game music generation 400 may optionally include training 415 of one or more parameters for generating music. Although user parameters can be used to select, modify and output variations of themes, parameters and building blocks for device-generated music may require training. As an example, the merging of musical styles may require training rhythmic patterns when combined. Alternatively, parameter combinations of certain tones and frequencies may require training to modify tones or melody processes. Therefore, music generator 405 may receive parameters updated according to training 415 for generating game music. According to an embodiment, training 415 may be performed on a device (e.g., a second device, a matching device, a network device, etc.) different from the device (e.g., a first device) that runs the operation of music generator 405. For example, parameters for training 415 may be received by the device running music generator 405 from one or more communications (e.g., network communications) so that the received training data and parameters may be used by music generator 405.

[0069] According to an embodiment, the music generator 405 may generate game music 410 as an output stream to another device (e.g., a display, TV, etc.) or one or more speakers. According to an embodiment, the generated game music may include one or more musical phrases that may be output in a loop. The game music 410 may include one or more tracks (e.g., a leading track, a bass track, a rhythm track, etc.) that may be independently controlled.

[0070] Figure 5 A graphical representation of a user interface according to one or more embodiments is shown. A device (eg, device 105) may be configured to display one or more graphical elements for detecting game music preferences (including user preferences, user interactions, and audio spectrum preferences). Figure 5 A display 500 is shown which may include a user interface 501. The user interface 501 may include a graphical element 505. 1-n , which can be selected and / or controlled to provide sample or seed music for the game music generator. User playlist 505 1 Can be selected and / or interacted with to provide the Game Music Generator with access to song titles of interest to the user. User Media Source 505 2 Can be selected and / or interacted with to provide the game music generator with access to one or more media files or media including music or audio (such as an external media player). User Media Rating 505 3 Can be selected and / or interacted with to provide the game music generator with access to song titles, artists, composers, tracks, and audio sounds that the user likes or dislikes. Ratings can be a reflection of the user's liking or disliking of certain music or sounds. Social Media Stream 505 n Can be selected and / or interacted with to provide the game music generator with access to one or more network services that include audio and music responses.

[0071] According to an embodiment, user interface 501 may include a display of menu 510 for accessing functions including feedback element 515. 1-n and one or more display elements of one or more of the music controller 520 to control the playback of the music. The elements of the user interface 501 can be displayed as an overlay and / or separately from the game content. Feedback elements 515 1-nThe music controller 520 may be selected to provide feedback on the game music output while playing the game, and may include elements for approving, changing, and disapproving the game music. The music controller 520 may include playback controls 525 to advance, reverse, stop, and play the music. The music controller 520 may include feedback elements 530 to provide feedback on the game music. The music controller 520 may operate as an in-game element (such as a fake box) with which the user can interact to obtain information about the music elements and the user's opinion of the composition. The feedback elements 515 from the music controller 520 and / or the feedback elements 515 may be used to provide feedback on the game music. 1-n The feedback can be used to tune the music generation to the player. The elements in the game can be seeded with AI-generated music clips that the AI ​​can most benefit from player feedback. The elements of the user interface 501 can be accessed using an input device such as a controller.

[0072] Figure 6 A process for user interactive customized music generation according to one or more embodiments is shown. According to an embodiment, one or more interactions may be detected to modify the generation of game music. The interaction may include user input to a gaming system for controlling the operation of the game. According to an embodiment, the user interaction may include features detected by a device (e.g., device 105) or a system (e.g., system 100), such as the presence of one or more users, the gaming environment, and sound interference.

[0073] Process 600 may be initiated by a device (e.g., device 105) that detects game music interactions at block 610. According to an embodiment, at least one user interaction with a game system may be collected during game play. The interaction between a user and a game system or a game device (e.g., device 105) may be used to train the selection of a piece of music. In addition, the sound output during game play may be used to train / retrain music generation, especially when the game music or generated sound interferes with the game sound output. Process 600 may use sensors so that information obtained during game play (such as a friend starting a game or a song in the background) may be used as a parameter for generating game music.

[0074] According to an embodiment, user interaction can be obtained by displaying a user interface, which includes a feedback display element for at least one of input of user preferences and feedback to game music generated by the device. According to other embodiments, detecting at least one user interaction includes receiving audio data from at least one sensor for sound elements in the user environment and / or receiving visual data from at least one sensor for the presence of viewers in the user environment. Detecting an interaction can include receiving music seed material. Process 600 can optionally include generating music at box 605 so that game music is output when an interaction is detected. Generating game music at box 605 can include outputting dynamic music based on training of a dynamic music generator and at least one user interaction.

[0075] According to other embodiments, detecting at least one user interaction includes detecting the presence of a user and one or more viewers in the environment of the device. Input may be provided by a sensor of the gaming system to detect interaction and control of dynamic music during game play. According to an embodiment, input may include detecting the presence of a viewer and a user environment to modify at least one output characteristic of dynamically generated music. By considering the audience (such as the audience in the living room), the generated music may be output to suit all ages. For example, when a child walks into the room while the father kills a zombie, the sound of the monster may change to reduce the fear factor. In addition, the sound generated for the audience of the person watching may take into account their music preferences. The audio generated to attract the audience may avoid certain compensations specific to the player, which may distract the attention of other people watching the game. When the audience is present in the gaming environment, or if the content is being captured for streaming or sharing, the AI ​​game audio may ensure that the generated audio attracts a wide audience in consideration of the player's preferences and audio needs. For example, when targeting players and spectators, AI audio generation may avoid significantly increasing the volume of sounds in the frequency range where the player has a hearing defect, and may instead play the audio in another range where the player does not have a hearing defect, as a supplement or alternative to playing the audio in the original frequency range.

[0076] At block 615, parameters for dynamic music generation may be updated based on at least one user interaction. Updating the parameters may include modifying the parameters of the dynamic music generator during game play, including modifying the music generation parameters to include or remove features. For example, generated music that the user does not like, as indicated by a user interaction to stop the music, may be used to modify or remove the parameters for generating the music. Similarly, updating the parameters may include strengthening the dynamic music generation parameters based on at least one user interaction. Updating the parameters may include updating the dynamic music generation parameters to remove music output based on at least one user interaction.

[0077] Process 600 may include retraining the dynamic music generation at optional block 625 to avoid interfering with the output of the sound effects of the electronic game and to account for user interaction. At block 616, process 600 may optionally detect interference of the game music output with the audio output generated by the device. For example, if the game music includes tonal elements similar to the output frequency and sound level of the sound effects, the game music may be considered to be interfering. Therefore, updating the parameters of the game music may transform the tonal and / or frequency characteristics of the game music to avoid interference.

[0078] At box 620, process 600 includes outputting game music based on the updated parameters for music generation. Process 600 may optionally include retraining the game music generator at box 625. According to an embodiment, retraining the music generator may be performed during game play, before game play, and / or between game sessions based on the player's interaction with the game. For example, if the player turns down the music volume in some cases, it is a good indication that the music generated at those times is hindering the game play or the player does not like it. Retraining may take into account information about one or more users obtained between game sessions, such as viewing habits (e.g., movies, video clips, or TV shows that the player has watched) and music associated with viewing habits. The information obtained about the user may include sounds detected by the device or topics of interest from social media streams. User information may be used to find appropriate corresponding audio clips that can be used as examples for training AI game audio. This retraining of AI game audio between game sessions can keep the game audio fresh to the user and can help match the game audio to the emotions of each session of the player.

[0079] Retraining can be based on information obtained during game play (such as information obtained from a camera and / or microphone) and player-specific information (such as an indication that a friend has started playing a game or a specific song has started playing in the background). Retraining can be performed to provide audio in response to the current situation of the game being played. According to an embodiment, retraining during game play and / or between game sessions should take into account the players in the game and the audience in the living room. By taking into account all game players during the retraining process, the generated music can be beautiful and enjoyable for a group of friends playing multiplayer games. Similarly generated music can create a new community of players who like generated music with specific characteristics. In addition, the similarity of user music profiles can provide seeds for other users on the gaming platform. The device and / or system can be connected to or included to determine whether there are other people in the gaming environment, and even other factors about those people, such as whether they are watching the game or just passing by. When providing seed music for training, the seed can have a sign indicating whether the seed cannot be used verbatim (such as whether a license is required but not obtained). For seed material that cannot be used verbatim, filters can be used to ensure that the generated music can be distinguished from the seed material.

[0080] Figure 7A graphical representation of user interactions and game music training according to one or more embodiments is shown. According to an embodiment, generating game music and generating game audio can be based on interactions with a device such as a game console. The device can store interactions 700 for generating music and training the music generator. Interactions 700 can be detected during output of game music 710. According to an embodiment, actions 705 to control the output of game music 1 Can be detected as user interaction 715 1 Action 705 1 This may include lowering the volume, increasing the volume, muting or stopping the music output. User / Viewer Presence 705 2 Can be detected as user interaction 715 2 . Sound Interference 705 3 Sound parameters that can be detected as to be avoided 715 3 . Received and / or detected seed music 705 2 Can be used to update profile 715 n .

[0081] Figure 8 800 is a process for user-customized audio generation according to one or more embodiments. Process 800 can be configured to adjust audio output based on one or more of generated game music, user input, and sound effects for the game. Process 800 can be performed by a device such as Figure 1 The device 105 and / or controller 125 of the present invention is adopted to generate game audio based on at least one user preference and customize the audio spectrum generation of the game audio. Process 800 can also be performed to help generate accessible audio output and improve device operation. Process 800 can be initiated by a device (e.g., device 105) that detects at least one audio spectrum input at box 801. The audio spectrum input 801 can be based on a sound test performed by the device to output at least one frequency tone and receive at least one user feedback. The sound test may include displaying at least one graphical element for receiving audio spectrum feedback. At least one audio spectrum input range may be identified as a user cannot be here or preference is not here. According to an embodiment, an input for receiving a sound output generated by a gaming system can be detected. For example, process 800 may include receiving game music as input at box 802 and receiving sound effects as input at box 803.

[0082] At block 810, process 800 may include adjusting the audio output. The update at block 180 may be based on at least one audio spectrum input, such as a desired frequency range or game audio. Updating parameters for music generation in process 800 may include identifying at least one frequency range not included in generating game music. According to an embodiment, updating the parameters may include transforming the frequency output of the game music.

[0083] When generating audio for a game, the devices and systems herein can take into account the content that is present and the frequency range of the audio in use. For example, if there is important audio in the game that the player should hear, such as dialogue between non-player characters (NPCs) or monsters approaching, process 800 can determine one or more frequency ranges of that audio and ensure that there is not much content in these frequency ranges in the generated ambient sounds and music. This will ensure that other sounds do not mask the important audio and make it more difficult for the user to hear. This allows the game to have a musical accompaniment and feel more connected to the rest of the game play, thereby avoiding the noticeable difference that would occur if the music stopped or suddenly became quieter when the user had something important to listen to.

[0084] The audio spectrum input 801 can allow the device to detect frequency ranges that the user wants to avoid. These frequency ranges can include frequency ranges where the user has hearing loss, high-frequency ranges that disturb others in the house or wake others up, frequency ranges that hit resonances in the house and make things vibrate, or low-frequency ranges that pass through walls and disturb people in other rooms or apartments. The user can set the attenuation level for each frequency range. Similarly, the user can set a frequency range (such as a frequency range where they have partial hearing loss) to be enhanced, and therefore need the sounds in that range to be louder to be heard similar to sounds in other ranges.

[0085] The audio generation of a game may take into account different frequency ranges that are attenuated, boosted, or eliminated entirely. For example, a device may select a frequency range for a melody of music that avoids such a frequency range that makes the volume of the melody appear the same to the player and other listeners when its frequency changes, where a melody played in another frequency range may enter and leave the reduced frequency range and thus change volume significantly from one note to the next. As another example, a device may change the frequency of a sound effect in a game (such as the sound of a machine running, footsteps, or a waterfall) so that it avoids being within a frequency band that is attenuated or eliminated so that it can be heard.

[0086] The devices and system configurations herein may use different frequency ranges to indicate different game conditions. For example, an increase in the overall volume in one frequency range may indicate the presence of an enemy nearby, while an increase in the overall volume in another frequency range may indicate the presence of a trap or environmental hazard nearby, and an increase in the overall volume in a third frequency range may indicate the presence of a nearby drop that the player can pick up.

[0087] If there are multiple sounds from elements in the game environment, ambient background sounds, speech, or music in the same frequency range, the device can adjust the relative volume of the different sound sources so that important sounds are not hidden behind other sounds and become easily missed by the player. This volume adjustment can vary for different frequency ranges of a single sound source. The device can also shift a sound source up or down in pitch to put it in a different frequency range that does not compete with other sound sources. If the device notices that a particular frequency band has many sound sources in it, it can instruct the music generation to temporarily avoid music with a lot of content in that frequency band so that the music does not compete with other sound sources in the game and make the game elements more difficult to hear.

[0088] The device can also change the pitch of the sound based on user preferences or reactions to the sound. For example, if the device detects that a high-pitched baby cry is annoying the user, it can change the sound to a lower pitch to make it less annoying.

[0089] As the music in games becomes more dynamic and conveys more information about the state of the game to the user, there is a greater need to support users with hearing impairments. Traditional closed captions usually indicate that music is playing, but very little information about the music is given. There may be visual indicators that can be turned on in the game, which can indicate most of the information that the device uses to affect the real-time changes it makes when generating music, such as the proximity to game elements in a particular direction. If the graphics for such indicators are standardized in multiple games, it is easier for users to learn and use when playing new games. Graphics can also indicate when the music being played is generated while playing the game, rather than as a pre-synthesized fragment. Such instructions can warn users that changes in music may be triggered by changes in things in the game environment.

[0090] At block 815, process 800 may include outputting game music. Generating game music for the gaming system at block 815 may be based on updated parameters, wherein the game music is dynamically generated to include at least one music theme, the at least one theme being determined based on the parameters. Generating the game music may include controlling the volume to prevent interference from a user. Outputting the game music may include displaying graphical elements to identify the game elements as components of the generated music. The output of the game music may include output coordinated with the output of the game content. Process 800 may optionally include outputting sound effects at block 820 based on the adjustments at block 810.

[0091] Fig. 9AA graphical representation of audio spectrum customization of game music according to one or more embodiments is shown. According to an embodiment, a device (e.g., device 105) and a system (e.g., system 100) may be configured to present a user interface 900 on a display such as display 905. User interface 900 may include one or more elements, including frequency distribution selection 910 and sound / tone display element 915. Frequency distribution selection 910 may allow the device to receive feedback for a specific frequency range. Sound / tone display element 915 may allow the device to receive feedback of a sound or tone. User interface 900 may display a graphical element 916 showing a sound element (such as generated music or sound). Based on one or more interactions with frequency distribution selection 910 and sound / tone display element 915, the device may update the sound output. User interface 900 may display a graphical element 916 including an updated sound element. For example, graphical element 916 includes sound component 917, and graphical element 918 includes sound component 919 modified from a previous output.

[0092] The user interface 900 can be presented for configuration and player preferences, and the system (e.g., system 100) can perform a hearing test on the user to identify hearing difficulties in one or more different frequency bands. For example, the frequency band has limited hearing or no hearing due to exposure to loud noises in the work environment. Hearing can also be limited to sounds that are masked by tinnitus at certain frequencies. The user interface 900 can be used to identify other hearing limitations, such as the upper limit of the frequency that the player can hear, which often decreases with the age of the person. The user interface 900 can be used to perform a hearing test on the user by playing sounds at various frequencies and volumes and having the user respond when the sound is heard.

[0093] User interface 900 can be used to detect the difference between the user's left and right ears by performing a test using headphones, or by covering one ear for a portion of the test and the other ear for another portion of the test. The hearing difference between the ears can be used as a parameter for audio generation. For example, audio generation can avoid generating sounds in a frequency range where the user has a significant hearing difference between the left and right ears for an element, where part of the content being attempted to be conveyed is the direction to the element. When conveying the element of the game in the generated audio, audio generation can choose to use a different frequency range.

[0094] The user interface 900 can interoperate with elements of a system including a microphone to measure the frequency response of the speaker that will play the game audio and generate an audio playback profile. All speaker systems have a resonant frequency and a frequency that is less efficient than other speaker systems. Some speaker systems also have damage or other problems that negatively affect their sound generation. The audio playback profile can be generated by playing a tone at various frequencies, or by scanning the tone through the frequency range of each speaker individually and measuring the sound level picked up by the microphone. Other known techniques for measuring the performance of an audio reproduction system can be used, such as measuring the playback of white noise. In some cases, a dedicated calibration microphone can be used. In some cases, a standard microphone can be used, such as a microphone sold as part of a gaming system, such as a microphone for hearing voice commands or a microphone built into a game controller. In this case, a calibration curve can be used to adjust the frequency response of the microphone used. In some cases, an unknown microphone can be used, in which case the calibration information will be unknown and the collected data will be less accurate, but may still be useful, such as detecting distortion or missing frequency ranges. When generating an audio playback profile, the microphone should be close to where the player's ears are located.

[0095] Audio playback profiles can be used to adjust the volume of sounds produced at various frequencies to achieve a flatter frequency response, which will cause the audio generated in the air to more closely match the digital audio generated by the game and will create a more natural-sounding and immersive sound for the player. The system can automatically apply frequency-based volume adjustments to get the flattest sound possible from each speaker.

[0096] The audio playback profile may also identify any frequency ranges that the system cannot adequately compensate for by adjusting the volume of that frequency range. For example, a system with small speakers may not be able to reproduce certain low frequencies. As another example, a system may have a speaker cone that is cracked, damaged, or completely broken. The system may also identify when there is distortion in the sound produced, such as from a cracked or damaged speaker cone. In some cases, the system may identify that the playback of a certain frequency through a certain speaker reaches a resonant frequency that causes something to vibrate or rattle around. The system may provide information about any frequency ranges that the game audio playback cannot fully reproduce to the game audio generation, which may use this information because it will use the player's hearing profile to adjust the generated audio.

[0097] If a hearing test is performed through the speakers that will be used for sound reproduction during gameplay, the test will measure a combination of the player's hearing and the frequency response of the speakers through which the game audio will be played. If an audio playback profile has been generated, the audio playback profile can be used to determine which portion of the hearing test performed through the speakers is attributable to the player's hearing and which portion is attributable to the audio playback.

[0098] The system can make the player's hearing profile and system response profile available to games played on the system. In some cases, aspects of the system are implemented in individual game titles. A software library can provide code that supports including aspects of the system in individual game titles.

[0099] During game play, the audio generation can detect when the system or player cannot adequately reproduce or hear sounds at the frequencies normally used by the audio generation. The audio generation can respond by using a different frequency to communicate that it will use an invalid frequency to communicate the element.

[0100] In some cases, audio generation can indicate to the system or other parts of the game code that it cannot fully communicate the element to the player through audio. For example, the element to be communicated can be something that is usually communicated through low frequencies that are felt more than heard, but the system playing the game may not be able to produce many sounds in the lower frequency range, such as game audio played through the speakers of a cheap TV. Examples of such elements can include explosions, earthquakes, sudden impacts, or the rumble of a large rock moving on the ground. In this case, the element can also be communicated by other means, such as visually or through tactile feedback. For example, the intensity of the screen can flicker to convey the sound of a gunshot, or the display can shake to convey an earthquake.

[0101] In some cases, the gaming system may use a microphone to measure the volume of ambient sounds in different frequency ranges and provide this information to the AI ​​audio generation. The competing audio may come from different sources, some of which will only last for a short duration, while some will last longer. For example, a child in a room may start using a toy that makes a lot of noise in a specific frequency range. The AI ​​audio generation may then use this information to detect when there is competing audio in different frequency ranges in the gaming environment, and compensate for the fact that the audio it generates for the game in a frequency range that currently has a lot of competing audio will be masked by the competing audio and will be more difficult to hear. Audio generation according to an embodiment can compensate by increasing the volume of the sounds in this frequency range to compensate for them being more difficult to hear, or can compensate by transforming the sounds to a different frequency range, in which they will not compete with the ambient sounds.

[0102] Fig. 9BA graphical representation of frequency range customization for game music according to one or more embodiments is shown. According to an embodiment, customized audio generation can include frequency range customization for game music according to one or more embodiments. Frequency customization can include changing the frequency of musical accompaniment based on other sound / auditory outputs of the gaming system and preferences. Unlike changing the tones or musical compositions selected for gameplay, frequency modification can selectively identify frequency ranges or tones that interfere with gameplay and / or ambient sounds.

[0103] Process 920 may include a graphical element 925 that displays a frequency range of the audio output, the graphical element 925 including at least one frequency range, such as a frequency component 926. Process 920 may allow a user to specify a frequency range to be avoided, such as a frequency range where the user has hearing loss, causes vibration, or is disturbing / unpleasant. Process 920 may also allow the frequency of the game's sound effects to be transformed so that they are audible, the frequency of the output volume to increase to indicate the presence of a gaming opportunity or threat, the volume or pitch to be transformed based on a review of the musical accompaniment and sound effects, and / or the change in pitch or frequency may be based on user preferences and / or reactions to the sound. Process 920 also allows the use of a visual indicator that may be displayed to indicate information used by the AI ​​to generate music, such as proximity to a gaming element.

[0104] Based on one or more sound tests, the device may determine that frequency component 926 having frequency value 927 requires adjustment due to hearing loss or interference. Process 920 may include identifying frequency value 927 and generating game audio 930 having frequency component 926 transformed to modified frequency value 928. According to an embodiment, process 920 may include removing frequency value 927 and generating game audio 931 with the frequency component removed.

[0105] While the present disclosure has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the scope of the claimed embodiments.

Claims

1. A method for generating user preference customized game music, the method comprising: detecting, by a device, at least one user preference for game music; determining, by the device, a game state; generating, by the device, game music based on the at least one user preference and the game state, wherein the game music is dynamically generated to include at least one music theme, the at least one theme being selected based on the at least one user preference and the game state; The game music is output by the device.

2. The method of claim 1, wherein the at least one user preference for game music comprises at least one of a music genre, a music style, a song title, an identification of a music artist, a composer, user feedback, and music preferences.

3. The method of claim 1, wherein the at least one user preference for gaming music comprises at least one of country of origin, nationality, geographic region, religious beliefs, and language preference.

4. The method of claim 1 , wherein detecting the at least one user preference for game music comprises displaying a user interface comprising a feedback display element for at least one of input of a user preference and feedback on the game music generated by the device. 5 . The method of claim 1 , wherein detecting the at least one user preference for gaming music comprises determining the user preference from user profile data.

6. The method according to claim 1, wherein: The game state is the current game situation within the electronic game.

7. The method of claim 1, wherein generating game music comprises selecting a musical theme for a game element of the game state, wherein selection of the theme is guided by the at least one user preference.

8. The method of claim 1, wherein generating game music comprises selecting a music library based on the user preference, and generating the game music from one or more themes in the music library.

9. The method of claim 1, wherein generating game music comprises selecting a music style based on at least one of a cultural parameter and a language parameter.

10. The method of claim 1, wherein outputting the game music comprises outputting dynamic music coordinated with outputting game content and controlling prominence of the game music relative to outputting the game content.

11. A device configured for user preference customized generation of game music, the device comprising: a memory for storing executable instructions; as well as a controller coupled to the memory, wherein the controller is configured to: detecting at least one user preference for game music; Determine the game state; generating game music based on the at least one user preference and the game state, wherein the game music is dynamically generated to include at least one music theme, the at least one theme being selected based on the at least one user preference and the game state; and The game music is output.

12. The apparatus of claim 11, wherein the at least one user preference for game music comprises at least one of a music genre, a music style, a song title, an identification of a music artist, a composer, user feedback, and music preferences.

13. The device according to claim 11, wherein: The at least one user preference for gaming music includes at least one of country of origin, nationality, geographic region, religious beliefs, and language preference.

14. The device of claim 11, wherein detecting the at least one user preference for game music comprises displaying a user interface comprising a feedback display element for at least one of input of a user preference and feedback on game music generated by the device.

15. The apparatus of claim 11, wherein detecting the at least one user preference for gaming music comprises determining the user preference from user profile data.

16. The apparatus of claim 11, wherein the game state is a current game situation within an electronic game.

17. The apparatus of claim 11, wherein generating game music comprises selecting a musical theme for a game element of the game state, wherein selection of the theme is guided by the at least one user preference.

18. The device of claim 11, wherein generating game music comprises selecting a music library based on the user preference, and generating the game music from one or more themes in the music library.

19. The apparatus of claim 11, wherein generating game music comprises selecting a music style based on at least one of a cultural parameter and a language parameter.

20. The apparatus of claim 11, wherein outputting the game music comprises output of dynamic music coordinated with output of game content and controlling prominence of the game music relative to output of the game content.

Images