System and method for automatically customizing voice filtering

Abstract

Systems and methods for audio processing are described. An audio processing system receives audio content including speech samples. The audio processing system analyzes a speech sample to identify a sound type in the speech sample. The sound type corresponds to the pronunciation of at least one specified character in the voice sample. The audio processing system generates filtered speech samples at least in part by filtering the speech samples to modify a sound type. The audio processing system outputs the filtered speech samples.

Description

background Technical Field

[0001] The present technology relates to audio processing. More specifically, the present technology can generate a filtered speech sample from a speech sample by modifying the type of sounds present in the speech sample. Background Art

[0002] Different types of speech, such as speech from people of different ages, genders, accents, etc., can exhibit different audio characteristics. Certain audio characteristics, such as sibilance, can make certain types of speech unpleasant or difficult to understand. These types of sounds can correspond to the pronunciation of certain characters or character sequences in certain words, such as the "s," "sh," "c," "ch," "f," or "t" sounds that may exhibit sibilance in certain words. In some cases, different types of speech may be more susceptible to undesirable audio characteristics such as sibilance. In some cases, a microphone or other audio device may not be optimized equally for certain types of speech as for other types of speech. Therefore, in some cases, a microphone or other audio device may exacerbate or fail to address such audio characteristics. This may result in different users with different speech types experiencing different levels of sound quality. Summary of the invention

[0003] Aspects of the present technology include systems and methods for audio processing. In some examples, an audio processing system receives audio content including a speech sample. The audio processing system analyzes the speech sample to identify a sound type in the speech sample. The sound type corresponds to the pronunciation of at least one specified character in the speech sample. For example, the sound type may correspond to a hissing sound, a pronunciation from certain characters or character sequences, such as the "s", "sh", "c", "ch", "f" or "t" sounds in certain words. The audio processing system generates a filtered speech sample at least in part by filtering the speech sample to modify the sound type. The audio processing system may filter the speech sample using a filter that is personalized and / or customized according to the speech corresponding to the speech sample (e.g., using a trained machine learning model trained based on a previous speech sample in the same speech). The audio processing system outputs the filtered speech sample, for example, by playing the filtered speech sample using an audio output device and / or by transmitting the filtered speech sample to a receiver device.

[0004] In one example, a system for audio processing is provided. The system includes a memory and one or more processors coupled to the memory (e.g., implemented in a circuit system). The one or more processors are configured and can: receive audio content including a speech sample; analyze the speech sample to identify a sound type in the speech sample, wherein the sound type corresponds to the pronunciation of at least one specified character in the speech sample; generate a filtered speech sample at least in part by filtering the speech sample to modify the sound type; and output the filtered speech sample.

[0005] In another example, an audio processing method is provided. The method includes: receiving audio content including a speech sample; analyzing the speech sample to identify a sound type in the speech sample, wherein the sound type corresponds to a pronunciation of at least one specified character in the speech sample; generating a filtered speech sample at least in part by filtering the speech sample to modify the sound type; and outputting the filtered speech sample.

[0006] In another example, a non-transitory computer-readable medium is provided having instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to: receive audio content including a speech sample; analyze the speech sample to identify a sound type in the speech sample, wherein the sound type corresponds to the pronunciation of at least one specified character in the speech sample; generate a filtered speech sample at least in part by filtering the speech sample to modify the sound type; and output the filtered speech sample.

[0007] In another example, an apparatus for audio processing is provided. The apparatus includes: means for receiving audio content including a speech sample; means for analyzing the speech sample to identify a sound type in the speech sample, wherein the sound type corresponds to the pronunciation of at least one specified character in the speech sample; means for generating a filtered speech sample at least in part by filtering the speech sample to modify the sound type; and means for outputting the filtered speech sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Illustrative embodiments of the present application are described in detail below with reference to the following drawings:

[0009] Figure 1 shows a network environment in which a system for audio processing may be implemented according to some examples;

[0010] Figure 2AAn exemplary unified data system (UDS) that may be used to provide data to a system for audio processing according to some examples is shown;

[0011] Figure 2B is a conceptual diagram illustrating an example table of various objects and associated events according to some examples;

[0012] Figure 3 is a conceptual diagram illustrating an example of an audio processing system according to some examples;

[0013] Figure 4 is a conceptual diagram illustrating an example of generating filtered speech samples from speech samples using a customized speech filter according to some examples;

[0014] Figure 5 is a block diagram illustrating generating filtered speech samples from speech samples using one or more machine learning models of a machine learning engine according to some examples;

[0015] Figure 6 is a flowchart illustrating an audio processing process according to some examples; and

[0016] Figure 7 is a block diagram of an exemplary electronic entertainment system that may be used in embodiments of the present invention. DETAILED DESCRIPTION

[0017] Different types of speech, such as speech from people of different ages, genders, accents, dialects, speech impediments, etc., may exhibit different audio characteristics. Certain audio characteristics, such as hissing, may make certain types of speech unpleasant or difficult to understand. These types of sounds may correspond to the pronunciation of certain characters or character sequences in certain words, such as the "s," "sh," "c," "ch," "f," or "t" sounds that may exhibit hissing in certain words. In some cases, different types of speech may be more susceptible to undesirable audio characteristics such as hissing. In some cases, a microphone or other audio device may not be equally optimized for certain types of speech as it is for other types of speech. Therefore, in some cases, a microphone or other audio device may exacerbate or fail to address such audio characteristics. This may result in different users with different speech types obtaining different levels of sound quality.

[0018] In some examples, an audio processing system receives audio content including a speech sample. The audio processing system analyzes the speech sample to identify a sound type in the speech sample. The sound type corresponds to the pronunciation of at least one specified character in the speech sample. For example, the sound type may correspond to a hissing sound, a pronunciation from certain characters or character sequences, such as the "s", "sh", "c", "ch", "f" or "t" sounds in certain words. The audio processing system generates a filtered speech sample at least in part by filtering the speech sample to modify the sound type. The audio processing system may filter the speech sample using a filter that is personalized and / or customized according to the speech corresponding to the speech sample (e.g., using a trained machine learning model trained based on a previous speech sample in the same speech). The audio processing system outputs the filtered speech sample, for example, by playing the filtered speech sample using an audio output device and / or by transmitting the filtered speech sample to a receiver device.

[0019] The audio processing methods and systems described herein provide various technical solutions for various technical problems. For example, the audio processing methods and systems described herein can improve the audio quality of certain voice types, such as the voice types of female users, who may be more prone to audio characteristics such as hissing when using traditional audio systems. The audio processing methods and systems described herein can improve the audio quality of certain relative positions of the user relative to the microphone, because using traditional audio systems, certain relative positions of the user relative to the microphone may be more prone to audio characteristics such as hissing. The audio processing methods and systems described herein can improve the audio quality of each user by providing personalized and / or customized voice filters on a per-user basis or per-user category basis, which is impossible for traditional audio systems.

[0020] Figure 1An exemplary network environment 100 in which a system for providing a contextual layer for digital content can be implemented is shown. The network environment 100 may include: one or more content source servers 100, which provide digital content (e.g., games, other applications and services) for distribution; one or more content provider server application program interfaces (APIs) 120; a content distribution network server 130; an audio processing server 140; and one or more user devices 150. The servers described herein may include any type of server as known in the art, including standard hardware computing components, such as network and media interfaces, non-transitory computer-readable storage devices (memory), and a processor for executing instructions or accessing information that may be stored in the memory. The functionality of multiple servers may be integrated into a single server. Any of the above servers (or integrated servers) may have certain client-side, cache, or proxy server features. These features may depend on the specific network arrangement of the server or certain configurations of the server.

[0021] The content source server 110 may maintain and provide a variety of digital content available for distribution. The content source server 110 may be associated with any content provider that makes its content available for access through a communication network. Such content may include not only digital videos and games, but also other types of digital applications and services. Such applications and services may include any of a variety of different digital content and functionality that may be provided to the user device 150.

[0022] Content from the content source servers 110 may be provided via a content provider server API 120 that allows various types of content source servers 110 to communicate with other servers (e.g., user devices 150) in the network environment 100. The content provider server API 120 may be specific to the content source server 110 providing the content and the specific language, operating system, protocol, etc. of the user device 150. In a network environment 100 that includes a variety of different types of content source servers 110, there may likewise be a corresponding number of content provider server APIs 120 that allow various formatting, conversion, and other cross-device and cross-platform communication processes for providing content and other services to different user devices 150 that may use different operating systems, protocols, etc. to handle such content. Thus, applications and services in different formats may be made available for compatibility with a variety of different user devices 150.

[0023] The content provider server API 120 may also facilitate each of the user devices 150 to access content hosted or services provided by the content source server 110, either directly or via the content distribution network server 130. Additional information about the accessed content or services, such as metadata, may also be provided to the user devices 150 by the content provider server API 120. As described below, the additional information (i.e., metadata) may be used to provide details about the content or services provided to the user devices 150. In some embodiments, the services provided to the user devices 150 from the content source servers 110 via the content provider server API 120 may include supporting services associated with other content or services, such as chat services, ratings, and profiles associated with a particular game, team, community, etc. In such cases, the content source servers 110 may also communicate with each other via the content provider server API 120.

[0024] The content distribution network server 130 may include a server that provides resources, files, etc. (including various content and service configurations) related to content from the content source server 110 to the user device 150. The content distribution network server 130 may also be called by the user device 150 requesting access to specific content or services. The content distribution network server 130 may include a global management server, a game server, a streaming media server, a server that hosts downloadable content, and other content distribution servers known in the art.

[0025] The audio processing server 140 may include a data server capable of audio processing. In an exemplary implementation, the audio processing server 140 may receive audio content including a speech sample. The audio processing server 140 may analyze the speech sample to identify the type of sound in the speech sample. The sound type corresponds to the pronunciation of at least one specified character in the speech sample. For example, the sound type may correspond to a hissing sound, a pronunciation from certain characters or character sequences, such as the "s", "sh", "c", "ch", "f" or "t" sounds in certain words. The audio processing server 140 may generate a filtered speech sample at least in part by filtering the speech sample to modify the sound type. The audio processing server 140 may filter the speech sample using a filter personalized and / or customized according to the speech corresponding to the speech sample (e.g., using a trained machine learning model trained based on a previous speech sample in the same speech). The audio processing system outputs the filtered speech sample, for example, by playing the filtered speech sample using an audio output device and / or by transmitting the filtered speech sample to a receiver device.

[0026] The user device 150 may include a variety of different types of computing devices. The user device 150 may be a server that provides internal services (e.g., to other servers) in the network environment 100. In such a case, the user device 150 may correspond to one of the content servers 110 described herein. Alternatively, the user device 150 may be a client device, which may include any number of different game consoles, mobile devices, laptops, and desktop computers. Such user devices 150 may also be configured to access data from other storage media, such as, but not limited to, appropriate memory cards or disk drives in the case of downloaded services. Such user devices 150 may include standard hardware computing components, such as, but not limited to, network and media interfaces, non-temporary computer-readable storage devices (memory), and processors for executing instructions that may be stored in the memory. These user devices 150 may also run using a variety of different operating systems (e.g., iOS, Android), applications, or computing languages ​​(e.g., C++, JavaScript). In this article about Figure 7 Exemplary client devices 150 are described in detail. Each user device 150 may be associated with a participant or other type of viewer of a collection of digital content streams.

[0027] Figure 2A An exemplary universal or unified data system (UDS) that can be used to provide data to a system for audio processing is shown. Based on the data provided by the UDS, the audio processing server 140 can be informed of which in-game objects, entities, activities, and events the user has participated in, and thus support the analysis and coordination of in-game activities. Each user interaction can be associated with the metadata of the type used for in-game interaction, the position in the in-game environment, and the time point in the in-game timeline, as well as other players, objects, entities, etc. involved. Therefore, metadata can be tracked for any of the various user interactions that may occur during a game session, including associated activities, entities, settings, results, actions, effects, positions, and character statistics. Such data can also be aggregated, applied to a data model, and analyzed. This UDS data model can be used to assign context information to each information portion across games in a unified manner.

[0028] like Figure 2A, an exemplary console 228 (e.g., user device 130) and exemplary servers 218 (e.g., streaming server 220, activity feed server 224, user generated content (UGC) server 232, and object server 226) are shown. In one example, console 228 may be implemented on platform server 120, on a cloud server, or on any of servers 218. In the exemplary example, content recorder 202 may be implemented on platform server 120, on a cloud server, or on any of servers 218. Such content recorder 202 receives content (e.g., media) from interactive content title 230 and records the content on content ring buffer 208. Such ring buffer 208 may store multiple content segments (e.g., v1, v2, and v3), the start time of each segment (e.g., V1_START_TS, V2_START_TS, V3_START_TS), and the end time of each segment (e.g., V1_END_TS, V2_END_TS, V3_END_TS). Such segments may be stored by the console 228 as a media file 212 (e.g., MP4, WebM, etc.). Such media files 212 may be uploaded to the streaming server 220 for storage and subsequent streaming or use, but the media files 212 may be stored on any server, cloud server, any console 228, or any user device 130. Such start time and end time of each segment may be stored by the console 228 as a content timestamp file 214. Such content timestamp file 214 may also include a streaming ID that matches the streaming ID of the media file 212, thereby associating the content timestamp file 214 to the media file 212. Such content timestamp files 214 may be uploaded and stored to the activity feed server 224 and / or the UGC server 232, but the content timestamp files 214 may be stored on any server, cloud server, any console 228, or any user device 130.

[0029] Concurrently with the content recorder 202 receiving and recording content from the interactive content title 230, the object library 204 receives data from the interactive content title 230, and the object recorder 206 tracks the data to determine when an object starts and ends. The object library 204 and the object recorder 206 may be implemented on the platform server 120, on a cloud server, or on any of the servers 218. When the object recorder 206 detects the start of an object, the object recorder 206 receives object data (e.g., whether the object is an activity, user interaction with the activity, activity ID, activity start time, activity end time, activity results, activity type, etc.) from the object library 204 and records the activity data on the object ring buffer 210 (e.g., ActivityID1, START_TS; ActivityID2, START_TS; ActivityID3, START_TS). Such activity data recorded on the object ring buffer 210 may be stored in the object file 216. Such object files 216 may also include the activity start time, activity end time, activity ID, activity results, activity type (e.g., competition, quest, mission, etc.), user or peer data related to the activity. For example, object files 216 may store data about items used during the activity. Such object files 216 may be stored on object server 226, but object files 216 may be stored on any server, cloud server, any console 228, or any user device 130.

[0030] Such object data (e.g., object file 216) may be associated with content data (e.g., media file 212 and / or content timestamp file 214). In one example, UGC server 232 stores content timestamp file 214 and associates the content timestamp file with object file 216 based on a match between a streaming ID of content timestamp file 214 and a corresponding activity ID of object file 216. In another example, object server 226 may store object file 216 and may receive a query for object file 216 from UGC server 232. Such a query may be performed by searching for an activity ID of object file 216 that matches the streaming ID of content timestamp file 214 transmitted with the query. In another example, the query may be performed by matching the start time and end time of content timestamp file 214 with the start time and end time of the corresponding object file 216 transmitted with the query for the stored content timestamp file 214. Such object files 216 may also be associated with matching content timestamp files 214 by the UGC server 232, but the association may also be performed by any server, cloud server, any console 228, or any user device 130. In another example, the object files 216 and content timestamp files 214 may be associated by the console 228 during the creation of each file 216, 214.

[0031] Figure 2B is a conceptual diagram showing an example table of various objects and associated events. Figure 2B Such object data (e.g., object file 216) may be associated with event information regarding activity availability changes and may be related to other objects having associated object information, as shown in example table 250 of . Media-object binding may form telemetry between objects shown in at least a portion of the streaming media and the streaming media. For example, such object data may be activity data file 251, zone data file 252, actor data file 254, mechanic data file 256, game media data file 258, and other gameplay related data files.

[0032] Such activity data files 251 (e.g., object files 216) can be classified as push-up, open or competitive. Such activity data files 251 can include optional attributes, such as longer descriptions of the activity, images associated with the activity, whether the activity is available to the player before starting the game, whether the activity requires completion of the game, whether the activity can be repeatedly played in the game and whether there are nested tasks or associated sub-activities. Such activity data files 251 can include activity availability change events, which can indicate a list or array of activities currently available to the player. For example, this can be used to determine which activities are displayed in the game plan.

[0033] Such a zone data file 252 may indicate an area of ​​the associated game world with a single coordinate system, wherein the zone may have a 2-D map associated therewith and may be used to display a location on the zone. If the zone data file 252 is applicable, each zone may include a zone ID and a localizable short name for the zone. Such a zone data file 252 may be associated with a view projection matrix (4×4) to convert from a 3-D world coordinate to a 2-D map position. Such a zone data file 252 may be associated with a position change event indicating an update to the player's current in-game position. Such a position change event may be published periodically or whenever the player's in-game position changes significantly. The platform server 120 may store the latest value in the 'state'. Such a zone data file 252 may include the x, y, z position of the player's character in the zone and the a, b, c vector indicating the player's character orientation or direction. Such a zone data file 252 may be associated with an activity start event and / or an activity end event, and for an activity end event, the result of completion, failure or abandonment may be associated with the activity (e.g., activity ID).

[0034] Such actor data files 254 may be associated with entities that have behaviors in the game, and may be player controllers or game controlled, and may be dynamically changed during gameplay. Such actor data files 254 may include an actor ID for an actor, a localizable name for an actor, an image for an actor, and / or a brief description for an actor. Such actor data files 254 may be associated with an actor selection event indicating that the actor selected by the player has changed. The selected actor may represent the actor that the player is controlling in the game and may be displayed in the player's profile and other spaces via the platform server 120. Multiple actors may be selected at one time, and each game may replace its actor list when loading save data.

[0035] Such mechanics data files 256 may be associated with items, skills or effects (e.g., bows, arrows, stealth attacks, fire damage) that may be used by players or games to affect gameplay and may exclude items (e.g., collectibles) that do not affect gameplay. Such mechanics data files 256 may include a mechanics ID of the mechanic, a short name of the mechanic, an image of the mechanic and / or a short description of the mechanic. Such mechanics data files 256 may be associated with a mechanics availability change event indicating that the mechanics available to the player have changed. Available may mean that the mechanics are available to the player in the game world, but may require the player to complete some steps to obtain it in the inventory (e.g., purchase from a store, pick up from the world) before the mechanics can be used. Each game may replace its mechanics list when loading save data.

[0036] Such a mechanic data file 256 may be associated with a mechanic inventory change event indicating that the player's inventory has changed. An inventory may refer to a mechanic that a player can use immediately without having to take additional steps in the game before using it. Inventory information is used to estimate the player's readiness for various activities, which may be forwarded to the platform server 120. A game may replace its mechanic inventory list when loading save data. Mechanics on cooldowns may be considered part of the inventory. Mechanic counts (e.g., ammunition, healing points) with any non-zero value may be considered "in inventory." Inventory mechanics may be considered a subset of available mechanics.

[0037] Such mechanism data file 256 can be associated with the mechanism use event that the indication mechanism has been used by the player or for the player and can be used to be displayed as the mechanism use situation in the UGC context. Such mechanism data file 256 can include the list or array of the mechanism of whether (for example, flaming arrow, flame damage) or the initiator is the player, so that the mechanism is used by the player or for the player. Such mechanism data file 256 can include the current zone ID of the initiator actor ID, the initiator actor and / or the current x, y, z position of the initiator actor. Such mechanism data file 256 can be associated with the mechanism influence event that the indication mechanism has an impact on the gameplay (for example, arrows hit the enemy) and can be used to display the mechanism image in the UGC context. The mechanism use and mechanism image event may not be associated. Such mechanism data file 256 can include the initiator action ID, the current zone ID of the initiator actor, the current x, y, z position of the initiator actor, the target actor ID, the current zone ID of the target actor, the current x, y, z of the target actor, and the mitigation mechanism that can mitigate the initiator mechanism.

[0038] Such game media data files 258 may include a game media ID of the game media, a localizable name of the game media, a media format (e.g., images, audio, video, text, etc.), a category or type of media (cut scenes, audio logs, posters, developer comments, etc.), a URL or server-preset media file, and / or whether the game media is associated with a particular activity. Such game media data files 258 may be associated with a game media start event indicating that a particular game media segment has now started in the game and a game media end event indicating that a particular game media segment has ended.

[0039] In some examples, media files 212, activity files 216, activity summary 224, activity data files 251, zone data files 252, actor data files 254, mechanic data files 256, and / or game media data files 258 may be examples of, or may include, audio clips and / or voice samples. Examples of such audio clips and / or voice samples include voice samples 330, audio content 335, voice samples 405, voice samples 505, audio content of operation 605, voice samples of operation 605, or combinations thereof.

[0040] Figure 3 310. The audio processing system 300 includes a console 310. The console 310 may be an example of an audio processing server 140, a user device 150, a console 228, one or more servers 218, an ML engine 520, an entertainment system 700, or a combination thereof. In some examples, the console 310 may be a video game console. A user 305 is shown playing a video game using the console 310. The console 310 is outputting video content to be displayed using a display 315. The video game is shown as a racing video game depicted on the display 315, but may be any type of video game or other interactive content, such as video conferencing, a virtual reality (VR) interface, an augmented reality (AR) interface, a mixed reality (MR) interface, an extended reality (XR) interface, or a combination thereof. The user 305 is shown providing input using a controller 320 to control the gameplay of the video game, which transmits signals corresponding to the input to the console 310.

[0041] User 305 is also shown speaking a voice sample 330 into microphone 325. Voice sample 330 may include user 305 speaking one or more characters, and in some examples, one or more words. Microphone 325 records audio content 335 including voice sample 330. In some examples, voice sample 330 may be used to conduct a voice chat and / or video chat with other users (such as users of one or more receiver devices 375) in a network-based multiplayer video game.

[0042] The console 310 receives audio content 335 with voice samples 330 from the microphone 325. In some examples, the console 310 sends the audio content 335 with voice samples 330 to the audio processing server 140 remote from the console 310 over a network. In some examples, the console 310 performs at least some of the operations of the audio processing server 140 itself. The console 310 and / or the audio processing server 140 identifies a user profile 340 associated with the user 305 from a set of user profiles in a data storage device 380. The data storage device 380 may include one or more data structures (e.g., one or more databases, tables, lists, arrays, matrices, or a combination thereof) stored on the console 310, on the audio processing server 140, stored separately as part of a network attached storage system, or a combination thereof. Figure 3 In the example of , data storage device 380 is shown as storing three different user profiles corresponding to three different users, including user profile 340 corresponding to user 305, second user profile 350 corresponding to a second user, and third user profile 360 ​​corresponding to a third user.

[0043] Each user profile includes a corresponding customized voice filter. For example, user profile 340 includes customized voice filter 345, user profile 350 includes customized voice filter 355, and user profile 360 ​​includes customized voice filter 365. Each customized voice filter can be customized and / or personalized according to a specific voice type. In some examples, customized voice filter 345 can be customized and / or personalized according to the voice of user 305. For example, customized voice filter 345 may include one or more trained machine learning (ML) models (e.g., trained ML model 725), which have been trained based on previous voice samples spoken by user 305 and recorded via one or more microphones (e.g., microphone 325). In some examples, the customized speech filter 345 can be customized and / or personalized according to the speech type into which the speech of the user 305 can be classified, for example, a speech type corresponding to the gender of the user 305, the gender characteristics of the user 305, the age of the user 305, the ethnic background of the user 305, the dialect of the user 305 (e.g., a regional dialect), the detected accent of the user 305, the speech disfluencies of the user 305 (e.g., stuttering, stuttering, sound repetition, sound prolongation, sound blocking, sound avoidance, fluency interruption, hesitation), or a combination thereof. For example, the customized speech filter 345 may include one or more trained machine learning (ML) models (e.g., trained ML models 725) that have been trained based on previous speech samples spoken by the user 305 and / or other users belonging to the same category and recorded via one or more microphones (e.g., microphone 325).

[0044] The console 310 and / or the audio processing server 140 selects the custom speech filter 345 to filter the audio content 335 and / or the speech sample 330. The console 310 and / or the audio processing server 140 generates a filtered speech sample 370 using the custom speech filter 345 by filtering the audio content 335 and / or the speech sample 330 using the custom speech filter 345. In some examples, the console 310 and / or the audio processing server 140 may output the filtered speech sample 370 back to the console 310 and / or the display 315, for example, to be played by an audio output device (e.g., a speaker or headphone output) corresponding to the console 310 and / or the display 315. In some examples, console 310 and / or audio processing server 140 may output filtered voice samples 370 by transmitting filtered voice samples 370 to one or more receiver devices 375, such as other consoles of other users with whom user 305 is using console 310 to play a network-based multiplayer game, one or more multiplayer gaming platform servers for managing multiplayer gameplay of a network-based multiplayer game, one or more other computing systems, or a combination thereof.

[0045] Figure 4 4 is a conceptual diagram 400 showing an example of generating a filtered speech sample 410 from a speech sample 405 using a custom speech filter 450. The speech sample 405 and the filtered speech sample 410 are shown as waveforms showing amplitude and / or frequency along the vertical axis and time along the horizontal axis. Examples of the speech sample 405 include a speech sample 330, audio content 335, a speech sample 505, a media file 212, an activity file 216, an activity summary 224, an activity data file 251, a zone data file 252, an actor data file 254, a mechanic data file 256, a game media data file 258, or a combination thereof. Examples of filtered speech samples 410 include filtered speech samples 370, filtered speech samples 530, filtered speech samples 560, filtered speech samples of operation 615, media files 212, activity files 216, activity feeds 224, activity data files 251, zone data files 252, actor data files 254, mechanic data files 256, game media data files 258, or a combination thereof. Examples of custom speech filters 450 include custom speech filters 345, custom speech filters 355, custom speech filters 365, trained ML models 525, filters for filtering of operation 615, or a combination thereof.

[0046] The custom voice filter 450 may be used (e.g., by the console 310 and / or the audio processing server 140) to analyze the voice sample 405 to identify various sound types in the voice sample. The custom voice filter 450 may be used to generate the filtered voice sample 410 at least in part by filtering the voice sample 405 to modify the sound type. In some examples, the sound type identified using the custom voice filter 450 may include a sound type corresponding to the pronunciation of a specified character (e.g., a letter or number or other character) in the voice sample, a specified character sequence (e.g., a letter and / or number and / or other character) in the voice sample, a specified word in the voice sample, a specified word sequence in the voice sample, or a combination thereof. In some examples, the sound type identified using the custom voice filter 450 may include a sound type corresponding to a sibilant sound. For example, in some examples, the sound type identified using the custom voice filter 450 may include a sound type corresponding to the pronunciation of an "s", "sh", "c", "ch", "f", or "t" sound that produces a sibilant sound in the pronunciation of certain words. The custom speech filter 450 performs hiss reduction 415, as shown in two corresponding portions of the filtered speech sample 410 and the speech sample 405, which are highlighted using two corresponding boxes with dashed outlines. The hiss reduction 415 can be achieved using a compressor (e.g., a static compressor and / or a dynamic compressor) of the custom speech filter 450, which compresses, attenuates, and / or filters a specified frequency range corresponding to the type of voice, pronunciation of a specified character, and / or hiss.

[0047] Customized speech filter 450 can also modify other sound types present in speech sample 405 to generate filtered speech sample 410. For example, a type of sound can include background noise, and customized speech filter 450 can use noise reduction 425 to compress, attenuate and / or filter out the background noise. Customized speech filter 450 can also use standardization 420 to standardize the changes in amplitude (e.g., volume) and / or frequency (e.g., pitch) of speech sample 405. Customized speech filter 450 can use smoothing 435 to remove or attenuate abnormal sounds. Customized speech filter 450 can use frequency equalization 430 to actively modulate, attenuate and / or modify the various frequency bands of speech sample 405. Each of these effects is illustrated using a corresponding box with a dotted outline, which highlights the relevant parts of speech sample 405, which are modified by customized speech filter 450 using corresponding techniques to generate filtered speech sample 410.

[0048] In some examples, the custom voice filter 450 may include one or more passive filters, one or more active filters, or a combination thereof. In some examples, the custom voice filter 450 may include one or more of the following: a low-pass filter, a passive low-pass filter, an active low-pass filter, a high-pass filter, a passive high-pass filter, an active high-pass filter, a band-pass filter, a passive band-pass filter, an active band-pass filter, a band-stop filter, a passive band-stop filter, an active band-stop filter, a notch filter, a passive notch filter, an active notch filter, an all-pass filter, a passive all-pass filter, an active all-pass filter, an equalization filter, a passive equalization filter, an active equalization filter, or a combination thereof. The custom voice filter 450 may use some of these filters to achieve hiss reduction 415, standardization 420, noise reduction 425, frequency equalization 430, smoothing 435, compression, other audio processing techniques described herein, or a combination thereof.

[0049] In some examples, the custom speech filter 450 can modify (e.g., reduce or increase) the volume and / or amplitude and / or frequency of a portion (e.g., a portion corresponding to a type of sound) of the speech sample 405. Such modifications can be used for hiss reduction 415, normalization 420, noise reduction 425, frequency equalization 430, smoothing 435, compression, other audio processing techniques described herein, or a combination thereof.

[0050] Figure 5 is a block diagram illustrating the generation of a filtered speech sample 530 from a speech sample 505 using one or more machine learning models 525 of a machine learning engine 520. Examples of the ML engine 520 include the network environment 100, one or more interactive content servers 110, one or more content provider server APIs 120, a content distribution network server 130, an audio processing server 140, a user device 150, a network environment 200, a console 228, one or more servers 218, a console 310, a data storage device 380, an audio processing system performing an audio processing process 600, an entertainment system 700, or a combination thereof. Examples of the ML model 525 include a customized speech filter 345, a customized speech filter 355, a customized speech filter 365, a customized speech filter 345, a customized speech filter 450, a customized speech filter, or a combination thereof.

[0051] The ML engine 520 generates, trains, and uses an ML model 525 based on initial training 565 using training data 570. The ML engine 520 trains the ML model 525 to generate a filtered speech sample 530 when the speech sample 505 is input to the ML model 525. The speech sample 505 may include audio data containing the speech sample, such as the speech sample 330, the audio content 335, the speech sample 405, the media file 212, the activity file 216, the activity summary 224, the activity data file 251, the zone data file 252, the actor data file 254, the mechanic data file 256, the game media data file 258, or a combination thereof.

[0052] The ML model 525 and / or the ML engine 520 may include, for example, one or more neural networks (NNs), one or more convolutional neural networks (CNNs), one or more trained time-delay neural networks (TDNNs), one or more deep networks, one or more autoencoders, one or more deep belief networks (DBNs), one or more recurrent neural networks (RNNs), one or more generative adversarial networks (GANs), one or more conditional generative adversarial networks (cGANs), one or more other types of neural networks, one or more trained support vector machines (SVMs), one or more trained random forests (RFs), one or more transformers, one or more classifiers, one or more computer vision systems, one or more deep learning systems, or a combination thereof. The training of the ML model 525 by the ML engine 520 may include supervised training, unsupervised training, or a combination thereof.

[0053] The filtered speech sample 530 output by the ML model 525 is a filtered variation of the speech sample 505. Examples of the filtered speech sample 530 include the filtered speech sample 370, the filtered speech sample 410, the filtered speech sample 560, the filtered speech sample of operation 615, the media file 212, the activity file 216, the activity summary 224, the activity data file 251, the zone data file 252, the actor data file 254, the mechanic data file 256, the game media data file 258, or a combination thereof. To generate filtered speech samples 530 from speech samples 505, ML model 525 may modify at least a subset of speech samples 505 as discussed with respect to custom speech filter 345, custom speech filter 355, custom speech filter 365, custom speech filter 450, hiss reduction 415, normalization 420, noise reduction 425, frequency equalization 430, smoothing 435, compression, other audio processing techniques described herein, or combinations thereof.

[0054] exist Figure 5, a graph representing an ML model 525 shows a group of circles connected to each other. Each circle can represent a node, a neuron, a perceptron, a layer, a portion thereof, or a combination thereof. The circles are arranged in columns. The leftmost column of white circles represents an input layer that receives an input (e.g., a speech sample 505). The rightmost column of white circles represents an output layer that outputs an output (e.g., a filtered speech sample 530) based on the input from the input layer (e.g., a speech sample 505). The two columns of shaded circles between the leftmost column of white circles and the rightmost column of white circles represent hidden layers, respectively. Some hidden layers can correspond to various modifications made to the speech sample 505 in the process of generating the filtered speech sample 530, such as hiss reduction 415, standardization 420, noise reduction 425, frequency equalization 430, smoothing 435, compression, other audio processing techniques described herein, or a combination thereof.

[0055] The training data 570 used by the ML engine 520 to train the ML model 525 includes speech sample data (e.g., similar to the speech sample 505) and pre-filtered speech sample data corresponding to the speech sample data (e.g., similar to the filtered speech sample 530 corresponding to the speech sample 505, and / or similar to the filtered speech sample 560 corresponding to the speech sample 505). During the initial training 565, the ML model 525 develops hidden layers between the input layer and the output layer, and / or weights and / or connections between nodes of each layer, each of which is related to various aspects of the filtered speech sample 530, such as any of the aspects described herein (e.g., related to various sounds and sound types and modifications thereto).

[0056] In some examples, the ML engine 520 can continue to train and / or update the ML model 525 over time, e.g., based on validation 575 using the filtered speech samples 530 and the speech samples 505. In some examples, the filtered speech samples 560 of the speech samples 505 (separate from the filtered speech samples 530 generated by the ML model 525) can be provided to the ML engine 520 for use in performing validation 575. In some examples, the filtered speech samples 560 can be generated by a different entity than the ML model 525, e.g., a different set of ML models (not shown), and / or manually generated by one or more human operators of the audio processing software and / or hardware. If during validation 575 , the ML engine 520 determines that the filtered speech sample 530 generated by the ML model 525 matches the filtered speech sample 560 , the ML engine 520 may treat this as positive feedback and may further train 555 the ML model 535 based on the filtered speech sample 530 , the speech sample 505 , and / or the filtered speech sample 560 , e.g., to strengthen and / or enhance the weights in the ML model 525 associated with generating the filtered speech sample 530 , and / or to weaken or remove other weights in the ML model 525 other than the weights associated with generating the filtered speech sample 530 . If, during validation 575 , the ML engine 520 determines that the filtered speech sample 530 generated by the ML model 525 is different from the filtered speech sample 560 , the ML engine 520 can treat this as negative feedback and can further train 555 the ML model 535 based on the filtered speech sample 530 , the speech sample 505 , and / or the filtered speech sample 560 , for example to weaken or remove weights in the ML model 525 associated with generating the filtered speech sample 530 , and / or to strengthen and / or enhance other weights in the ML model 525 other than the weights associated with generating the filtered speech sample 530 .

[0057] In some examples, the ML engine 520 receives feedback 550 about the filtered speech sample 530. The feedback may include a reaction made by a user of a user device (e.g., the audio processing server 140, the user device 150, the console 228, the server 218, the console 310, the receiver device 375, and / or the electronic entertainment system 700) via a user interface, a user reaction determined based on sensor data from the user device, and / or a decision made by the user and / or the user device whether to use the filtered speech sample 530 for further application. Positive feedback may be used to strengthen and / or reinforce the weights in the ML model 525 associated with generating the filtered speech sample 530, and / or weaken or remove other weights in the ML model 525 other than the weights associated with generating the filtered speech sample 530. Negative feedback may be used to weaken or remove the weights in the ML model 525 associated with generating the filtered speech sample 530, and / or strengthen and / or reinforce other weights in the ML model 525 other than the weights associated with generating the filtered speech sample 530.

[0058] Figure 6 6 is a flow chart illustrating an audio processing process 600. At least a subset of the audio processing process 600 may be performed by and / or using an audio processing system. The audio processing system may include and / or use, for example, a network environment 100, one or more interactive content servers 110, one or more content provider server APIs 120, a content distribution network server 130, an audio processing server 140, a user device 150, a network environment 200, a console 228, one or more servers 218, a console 310, a receiver device 375, a data storage device 380, an ML engine 520, an entertainment system 700, a system, an apparatus, a digital signal processor (DSP), a non-transitory computer-readable storage medium having a program to be executed by a processor embodied thereon, or a combination thereof.

[0059] At operation 605, the audio processing system is configured to and can receive audio content including a voice sample. Examples of audio content include voice samples 330, audio content 335, voice samples 405, voice samples 505, media files 212, activity files 216, activity synopsis 224, activity data files 251, zone data files 252, actor data files 254, mechanic data files 256, game media data files 258, or a combination thereof. Examples of voice samples include voice samples 330, audio content 335, voice samples 405, voice samples 505, media files 212, activity files 216, activity synopsis 224, activity data files 251, zone data files 252, actor data files 254, mechanic data files 256, game media data files 258, or a combination thereof.

[0060] At operation 610, the audio processing system is configured and operable to analyze a speech sample to identify a sound type in the speech sample. The sound type corresponds to a pronunciation of at least one specified character in the speech sample.

[0061] In some examples, the sound types in the speech sample may include sound types corresponding to the pronunciation of specified characters in the speech sample (e.g., letters or numbers or other characters), specified character sequences in the speech sample (e.g., letters and / or numbers and / or or other characters), specified words in the speech sample, specified word sequences in the speech sample, or combinations thereof. In some examples, the sound types in the speech sample may include sound types corresponding to sibilant sounds. In some examples, the sound types in the speech sample may include sound types corresponding to certain frequencies or frequency ranges. For example, in some examples, the sound types in the speech sample may include sound types corresponding to the pronunciation of “s,” “sh,” “c,” “ch,” “f,” or “t” sounds that produce sibilant sounds in the pronunciation of certain words. In some examples,

[0062] In some examples, the voice type corresponds to the pronunciation of the at least one specified character according to the voice type. In some examples, the voice type corresponds to at least one of gender, gender characteristics, age, ethnic background, accent, dialect (e.g., regional dialect), speech dysfluency (e.g., stuttering, stammering, sound repetition, sound extension, sound blocking, sound avoidance, fluency interruption, hesitation), or a combination thereof.

[0063] In some examples, the sound type corresponds to the pronunciation of the at least one specified character according to the voice frequency. In some examples, the sound type corresponds to the pronunciation of the at least one specified character according to the relative position between the microphone (e.g., microphone 325) and the person (e.g., user 3050) during recording of the audio content (e.g., audio content 335 and / or voice sample 330) using the microphone, wherein the voice sample is spoken by the person.

[0064] At operation 615, the audio processing system is configured to and can generate a filtered voice sample at least in part by filtering the voice sample to modify the sound type. Examples of filtered voice samples include filtered voice sample 370, filtered voice sample 410, filtered voice sample 530, filtered voice sample 560, media file 212, activity file 216, activity summary 224, activity data file 251, zone data file 252, actor data file 254, mechanism data file 256, game media data file 258, or a combination thereof.

[0065] In some examples, filtering the speech sample includes applying a filter to the speech sample. Examples of filters include custom speech filter 345, custom speech filter 355, custom speech filter 365, custom speech filter 450, trained ML model 525, low pass filter, passive low pass filter, active low pass filter, high pass filter, passive high pass filter, active high pass filter, band pass filter, passive band pass filter, active band pass filter, band stop filter, passive band stop filter, active band stop filter, notch filter, passive notch filter, active notch filter, all pass filter, passive all pass filter, active all pass filter , equalization filters, passive equalization filters, active equalization filters, de-essers, compressors that filter a specified frequency range (e.g., where the audio content includes audio in a specified frequency range, and where the specified frequency range corresponds to a type of sound), hiss reduction 415, normalization 420, noise reduction 425, frequency equalization 430, smoothing 435, compression, volume modification of a portion of audio, amplitude modification of a portion of audio, frequency modification of a portion of audio, other audio processing filters described herein, other audio processing operations described herein, or combinations thereof.

[0066] In some examples, the filter is customized according to the voice type corresponding to the voice sample, such as in the customized voice filter 345, the customized voice filter 355, the customized voice filter 365, the customized voice filter 450 and / or the trained ML model 525. The voice type may correspond to one or more voices of one or more people. In some examples, the voice type may correspond to the voice of a single person, such as the voice of the user 305. In some examples, the voice type may correspond to a person category into which multiple people (e.g., including the user 305) can be classified, such as a category corresponding to gender, gender characteristics, age, ethnic background, accent, dialect (e.g., regional dialect), speech disfluency (e.g., stuttering, stuttering, sound repetition, sound extension, sound blocking, sound avoidance, fluency interruption, hesitation) or a combination thereof. In some examples, the filter includes a trained machine learning model (e.g., ML model 525) customized according to the voice type. The trained machine learning model may be one that has been trained using training data (e.g., training data 570, validation 575, feedback 550, and / or any other additional training data 555 for further training), the training data including one or more additional speech samples associated with the speech type. In some examples, the audio processing system is configured to and may use training data including the one or more additional speech samples associated with the speech type to train the trained machine learning model. In some examples, the audio processing system is configured to and may request at least one of the one or more additional speech samples from a user device (e.g., for use as part of the training data). The user device is associated with a speech type. In some examples, the audio processing system is configured to and may use additional training data to update the trained machine learning model. The additional training data includes at least a speech sample (e.g., speech sample 505) and a filtered speech sample (e.g., filtered speech sample 530), for example, as discussed with respect to validation 575, feedback 550, and / or any other additional training data for further training 555.

[0067] At operation 620, the audio processing system is configured to and can output the filtered speech sample. In some examples, outputting the filtered speech sample includes causing the filtered speech sample to be output using an audio output device (such as a set of one or more speakers or a set of one or more headphones). In some examples, outputting the filtered speech sample includes outputting the filtered speech sample using the audio output device. In some examples, the audio processing system includes an audio output device.

[0068] In some examples, outputting the filtered speech samples includes transmitting the filtered speech samples to a receiver device (e.g., receiver device 375) via a communication interface. In some examples, the audio processing system includes a communication interface. In some examples, the audio processing system includes a receiver device.

[0069] Figure 7 is a block diagram of an exemplary electronic entertainment system 700 . Figure 7 The entertainment system 700 includes a main memory 705, a central processing unit (CPU) 710, a vector unit 88, a graphics processing unit 720, an input / output (I / O) processor 725, an I / O processor memory 730, a controller interface 735, a memory card 740, a universal serial bus (USB) interface 745, and an IEEE interface 750. The entertainment system 700 also includes an operating system read-only memory (OS ROM) 755 connected to the I / O processor 725 via a bus 775, a sound processing unit 760, an optical disk control unit 770, and a hard disk drive 765.

[0070] The entertainment system 700 may be an electronic game console. Alternatively, the entertainment system 700 may be implemented as a general purpose computer, a set-top box, a handheld gaming device, a tablet computing device, or a mobile computing device or a phone. The entertainment system may include more or fewer operating components depending on a particular form factor, purpose, or design.

[0071] Figure 7 The CPU 710, vector unit 88, graphics processing unit 720 and I / O processor 725 communicate via the system bus 785. In addition, Figure 7 The CPU 710 communicates with the main memory 705 via a dedicated bus 780 , while the vector unit 88 and the graphics processing unit 720 may communicate via a dedicated bus 790 . Figure 7 The CPU 710 executes programs stored in the OS ROM 755 and the main memory 705 . Figure 7 The main memory 705 may contain pre-stored programs and programs transferred from a CD-ROM, DVD-ROM or other optical disk (not shown) via the I / O processor 725 using the optical disk control unit 770. Figure 7 The I / O processor 725 may also allow for the introduction of content transmitted via a wireless or other communication network (e.g., 5G, 4G, LTE, 3G, etc.). Figure 7 The I / O processor 725 mainly controls data exchange between various devices of the entertainment system 700 , including the CPU 710 , the vector unit 88 , the graphics processing unit 720 , and the controller interface 735 .

[0072] Figure 7The graphics processing unit 720 executes graphics instructions received from the CPU 710 and the vector unit 88 to generate images for display on a display device (not shown). For example, Figure 7 The vector unit 88 can transform the object from three-dimensional coordinates to two-dimensional coordinates and send the two-dimensional coordinates to the graphics processing unit 720. In addition, the sound processing unit 760 executes instructions to generate sound signals, which are output to an audio device such as a speaker (not shown). Other devices can be connected to the entertainment system 700 via the USB interface 745 and the IEEE 1394 interface 750 such as a wireless transceiver, which can also be embedded in the system 700 or as part of some other component such as a processor.

[0073] Figure 7 The user of the entertainment system 700 provides instructions to the CPU 710 via the controller interface 735. For example, the user may instruct the CPU 710 to store certain game information on the memory card 740 or other non-transitory computer-readable storage medium, or instruct a character in the game to perform a certain specified action.

[0074] The present invention may be implemented in applications that may be operable by a variety of end-user devices. For example, the end-user device may be a personal computer, a home entertainment system (e.g., a Sony PlayStation2® or a Sony PlayStation3® or a Sony PlayStation4®), a portable gaming device (e.g., a Sony PSP® or a Sony Vita®), or a home entertainment system from a different lesser manufacturer. It is fully contemplated that the present method described herein may be operable on a variety of devices. The present invention may also be implemented in the context of cross-title neutrality, where embodiments of the present system may be utilized across a variety of titles from a variety of publishers.

[0075] The present invention may be implemented in applications that may be operated using a variety of devices. Non-transitory computer-readable storage media refers to any one or more media that participate in providing instructions to a central processing unit (CPU) for execution. Such media may take many forms, including but not limited to non-volatile and volatile media, such as optical or magnetic disks and dynamic memory, respectively. Common forms of non-transitory computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROM disks, digital video disks (DVDs), any other optical media, RAM, PROM, EPROM, FLASHEPROM, and any other memory chip or cassette.

[0076] Various forms of transmission media may be involved in transmitting one or more sequences of one or more instructions to the CPU for execution. The bus transmits the data to the system RAM, from which the CPU retrieves and executes the instructions. The instructions received by the system RAM may optionally be stored on a fixed disk before or after execution by the CPU. Various forms of storage devices and the necessary network interfaces and network topologies may be similarly implemented to achieve the same purpose.

[0077] The foregoing detailed description of the technology has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the technology to the precise form disclosed. In view of the above teachings, many modifications and variations are possible. The described embodiments are selected in order to best explain the principles of the technology, its practical application, and to enable others skilled in the art to utilize the technology in various embodiments and make various modifications suitable for the intended specific use. It is intended that the scope of the technology be defined by the claims.

Claims

1. A device for audio processing, the device comprising: at least one memory storing instructions; as well as at least one processor that executes the instructions, wherein execution of the instructions by the at least one processor causes the at least one processor to: receiving audio content including speech samples; analyzing the speech sample to identify a sound type in the speech sample, wherein the sound type corresponds to a pronunciation of at least one specified character in the speech sample; generating a filtered speech sample at least in part by filtering the speech sample to modify the sound type; as well as The filtered speech samples are output.

2. The apparatus of claim 1, wherein the sound type comprises a hissing sound.

3. The apparatus of claim 1, wherein the sound type corresponds to a pronunciation of the at least one designated character according to a phonetic type.

4. The apparatus of claim 3, wherein the voice type corresponds to at least one of gender or gender characteristics.

5. The apparatus of claim 3, wherein the voice type corresponds to at least one of age, accent, dialect, or ethnic background.

6. The apparatus of claim 1, wherein the sound type corresponds to a pronunciation of the at least one designated character according to a voice frequency.

7. The apparatus of claim 1, wherein the sound type corresponds to a pronunciation of the at least one designated character according to a relative position between the microphone and a person during recording of the audio content using the microphone, wherein the speech sample is spoken by the person.

8. The apparatus of claim 1, wherein the sound type corresponds to a speech disfluency, wherein filtering the speech sample comprises correcting the speech disfluency.

9. The apparatus of claim 1, wherein filtering the speech samples comprises applying a filter to the speech samples, wherein the filter comprises a de-esser.

10. The apparatus of claim 1, wherein filtering the speech sample comprises applying a filter to the speech sample, wherein the filter comprises a compressor that filters a specified frequency range, wherein the audio content comprises audio in the specified frequency range, wherein the specified frequency range corresponds to the sound type.

11. The apparatus of claim 1, wherein filtering the speech sample comprises applying a filter to the speech sample, wherein the filter is customized based on a type of speech corresponding to the speech sample.

12. The apparatus of claim 11, wherein the filter comprises a trained machine learning model customized according to the speech type, the trained machine learning model having been trained using training data comprising one or more additional speech samples associated with the speech type.

13. The apparatus of claim 12, wherein execution of the instructions by the at least one processor causes the at least one processor to: At least one of the one or more additional speech samples is requested from a user device, wherein the user device is associated with the speech type.

14. The apparatus of claim 12, wherein execution of the instructions by the at least one processor causes the at least one processor to: The trained machine learning model is updated using additional training data, wherein the additional training data includes at least the speech sample and the filtered speech sample.

15. The apparatus of claim 1, wherein outputting the filtered speech sample comprises causing the filtered speech sample to be output using an audio output device.

16. The apparatus of claim 1, wherein outputting the filtered speech samples comprises transmitting the filtered speech samples to a receiver device through a communication interface.

17. The device of claim 1, wherein the device comprises a digital signal processor (DSP).

18. An audio processing method, the method comprising: receiving audio content including speech samples; analyzing the speech sample to identify a sound type in the speech sample, wherein the sound type corresponds to a pronunciation of at least one specified character in the speech sample; generating a filtered speech sample at least in part by filtering the speech sample to modify the sound type; as well as The filtered speech samples are output.

19. The method of claim 18, wherein filtering the speech sample comprises applying a filter to the speech sample, wherein the filter is customized based on a type of speech corresponding to the speech sample.

20. A non-transitory computer-readable storage medium having a program embodied thereon, wherein the program is executable by a processor to perform an audio processing method, the method comprising: receiving audio content including speech samples; analyzing the speech sample to identify a sound type in the speech sample, wherein the sound type corresponds to a pronunciation of at least one specified character in the speech sample; generating a filtered speech sample at least in part by filtering the speech sample to modify the sound type; as well as The filtered speech samples are output.

Images