Method, system, and medium for recommending and composing emotional music

Abstract

To provide a method and system for recommending and composing emotional music.SOLUTION: A method includes identifying a listener's current emotional state and a target emotional state, and generating an audio stream that is a music playlist to be presented to a user as auditory stimulation with an intention of producing a controlled trajectory of the listener's emotional state from the current state to the target state. The audio stream is generated by a machine learning system trained using data from the listener and / or other users indicating effectiveness of specific audio segments having specific features in producing a desired emotional trajectory, and is updated based on changes in an emotional state induced in the listener after exposure to the auditory stimulation. The machine learning system acquires, over time, a robust interpretation of a relationship between music and human emotion, and composes, masters, and / or adapts music configured to induce specific emotional responses in listeners.SELECTED DRAWING: Figure 17A

Description

[Technical Field]

[0001] At least some example embodiments relate to music recommendation and music composition systems, and in particular to systems for composing and recommending music intended to induce specific changes in the emotional state of a listener. [Background technology]

[0002] Affect is a concept used in psychology to describe the experience of emotion, mood, or feeling. Humans experience different emotional states under different conditions. External stimuli can affect a person's mood or emotion.

[0003] Many people exhibit particularly acute emotional responses to various types of musical stimuli. Music plays a significant role in human culture, primarily due to its effectiveness in inducing particular emotional states in listeners. Individual listeners typically select the music they listen to based on a desire to maintain or achieve a target emotional state, such as energized, relaxed, melancholic, nostalgic, happy, or active.

[0004] Music recommendation systems attempt to recommend music to listeners based on inferences about what the listener is likely to like. These inferences are typically based on data collected from the listener. This data may be collected through interactions with the listener at the time of the recommendation or during previous interactions that suggest overall listener preferences. The recommendation system may present the listener with a set of themes and recommend music based on the listener's selection. These themes may be organized around any of several principles, including genre (classical, country, rap), season or setting (Christmas music, beach music), or historical era (1960s, 1980s, contemporary). Some themes may be organized around mood or other emotional state information, such as music intended to evoke or match the listener's emotional state (sad, happy, relaxed, energized). Themes may also be organized around activities suggesting a particular emotional state (calm music for cooking or meditation, active music for working out, upbeat, rhythmic music for dancing).

[0005] Some music recommendation systems also infer listener preferences through data that associates listeners with particular musical compositions. The system may, for example, collect data about the listener's previous music listening choices or previous purchases of music recordings. These choices or purchases may be cross-referenced with other available musical compositions, for example, by identifying musical compositions with characteristics similar to those preferred by the listener. Some systems may use collaborative filtering to identify compositions preferred by other users with similar tastes or similar selection or purchase histories to the listener. Some systems may collect feedback from listeners regarding the system's recommendations and update their models of user preferences and the confidence of their inferences accordingly.

[0006] Systems that include emotion-related themes typically identify music compositions that fit a given theme based on global characteristics of the composition that are not specific to individual listeners. Emotion-related features of a music composition are typically identified with respect to the music composition as a whole. Experts trained in music classification are typically employed to listen to music and characterize the music according to a process called music information retrieval (MIR). MIR involves the extraction and representation of musical features. Standards for representing features in MIR have been promulgated by IRCAM (Institute for Research and Coordination in Acoustics / Music), and software exists to support MIR, such as the MIR toolbox software package for Matlab. Some existing collections of MIR data include emotion-related tags or features associated with specific music compositions, but this is based on expert listeners rating the emotion-related features of a music composition as a whole.

[0007] Musical compositions are usually not supported by sophisticated formal models of human emotion in terms of musical elements: music intended to induce particular emotional states in listeners is generally composed by human composers based on their own subjective criteria. Summary of the Invention

[0008] This disclosure describes example devices, methods, systems, and non-transitory media for recommending and composing emotional music. In some embodiments, a listener's current emotional state and a target emotional state are identified, and an audio stream (e.g., a music playlist, sound design, or algorithmically composed piece of music) is generated with the intent of producing a controlled trajectory of the listener's emotional state from the current state to the target state. The audio stream is generated by a machine learning model trained using data from listeners and / or other users that indicates the effectiveness of particular audio segments, i.e., audio segments with particular features, in producing the desired emotional trajectory. In some embodiments, a piece of music can be composed to achieve a particular target emotional state or a particular emotional trajectory based on a model of how a particular listener or group of listeners emotionally responds to particular musical elements.

[0009] An exemplary embodiment is directed to a method for generating an audio stream for inducing an emotional state change in a listener, the method including: identifying a current emotional state of the listener; identifying a target emotional state of the listener; identifying an emotional trajectory from the current emotional state to the target emotional state; using a trained segment identification machine learning model to identify a first audio segment that is likely to elicit a desired emotional response in the listener corresponding to at least an initial portion of the emotional trajectory when the first audio segment is presented to the listener as an auditory stimulus; generating an audio stream based on at least a portion of the first audio segment; and transmitting audio stream data based on the audio stream to a listener device.

[0010] A second embodiment is directed to a system for generating an audio stream for inducing a change in an emotional state in a listener. The system includes a processor system, a communication system, and a storage device having stored thereon an executable trained segment identification machine learning model and executable instructions. When executed by the processor system, the executable instructions cause the system to: identify a current emotional state of the listener based on listener state data received by the communication system; identify a target emotional state of the listener based on target emotional state data received by the communication system; identify an emotional trajectory from the current emotional state to the target emotional state; identify, using the trained segment identification machine learning model, a first audio segment that, when presented to the listener as an auditory stimulus, is likely to elicit a desired emotional response in the listener corresponding to at least an initial portion of the listener's emotional trajectory; generate an audio stream based on at least a portion of the first audio segment; and transmit audio stream data based on the audio stream to a listener device using the communication system.

[0011] According to a further aspect that can be combined with other embodiments disclosed herein, after identifying a first audio segment using the trained segment identification machine learning model, an emotional inference process is used to infer an inferred new emotional state based on a current emotional state and a set of audio feature values ​​of the first audio segment. An updated emotional trajectory to a target emotional state is identified from the inferred new emotional state data. The trained segment identification machine learning model is used to identify a subsequent audio segment that is likely to elicit a subsequent desired emotional response in a listener when presented as an auditory stimulus to the listener, the subsequent audio segment corresponding to at least an initial portion of the updated emotional trajectory. The audio stream is generated based at least in part on the first audio segment and the subsequent audio segment.

[0012] According to a further aspect, which can be combined with other embodiments disclosed herein, the trained segment identification machine learning model is trained using the reward data received from the emotional inference process, and the emotional inference process generates the reward data by inferring an inferred emotional response of a listener to a set of audio feature values ​​of the audio stream and generating the reward data based on comparing the inferred emotional response to a desired emotional response.

[0013] According to a further aspect, which can be combined with other embodiments disclosed herein, the emotional inference process comprises a trained emotional inference machine learning model, and the trained emotional inference machine learning model is trained using training data including training audio feature data corresponding to a plurality of training audio segments and emotional state data collected from one or more human subjects in conjunction with exposing each of the human subjects to a plurality of audio stimuli corresponding to the plurality of training audio segments.

[0014] According to a further aspect, which may be combined with other embodiments disclosed herein, the one or more human subjects include a listener.

[0015] According to a further aspect, which may be combined with other embodiments disclosed herein, the trained segment-discriminating machine learning model comprises a reinforcement learning model.

[0016] According to a further aspect, which may be combined with other embodiments disclosed herein, the trained segment-discriminating machine learning model comprises a deep learning neural network.

[0017] According to a further aspect, which can be combined with other embodiments disclosed herein, the audio stream data includes recommendation data recommending an audio stream.

[0018] According to a further aspect, which may be combined with other embodiments disclosed herein, the audio stream data includes an audio stream.

[0019] According to a further aspect that can be combined with other embodiments disclosed herein, after transmitting the audio stream data to a listener device, updated current emotional state data is received from the listener, and the trained emotion inference machine learning model uses run-time training data including audio feature data corresponding to a first audio segment and each of a plurality of subsequent audio segments and the updated current emotional state data.

[0020] According to a further aspect, which may be combined with other embodiments disclosed herein, identifying a target emotional state for the listener includes receiving target emotional state data from the listener via a listener device, and identifying the target emotional state for the listener based on the target emotional state data.

[0021] According to a further aspect, which may be combined with other embodiments disclosed herein, identifying the current emotional state of the listener includes receiving emotional self-assessment data from the listener via the listener device, and identifying the current emotional state of the listener based on the emotional self-assessment data.

[0022] According to a further aspect, which may be combined with other embodiments disclosed herein, identifying the current emotional state of the listener includes receiving physiological data associated with the listener or a further physiological state of the listener, and identifying the current emotional state of the listener based on the physiological data.

[0023] Further embodiments are directed to non-transitory processor-readable media comprising instructions for performing one or more of the above methods.

[0024] Further embodiments are directed to non-transitory storage media containing audio streams produced by one or more of the above methods.

[0025] Further embodiments are directed to a method of training a machine learning model to predict human emotional responses to musical features, the method including presenting music having a set of musical features to a listener, obtaining emotional response data from the listener indicative of the listener's emotional response to the presentation of the music, labeling the musical features of the music with the emotional response data to generate labeled musical feature data, and using the labeled musical feature data as training data for training a machine learning model to predict the emotional response data based on the musical feature data.

[0026] A further embodiment is directed to a system for predicting human responses to music, comprising a processor system and a memory device having stored therein a machine learning model trained according to one of the above methods.

[0027] According to a further aspect, which can be combined with other embodiments disclosed herein, the machine learning model is a generative model that receives desired emotional response data indicative of a desired emotional response, and generates music having musical features predicted by the machine learning model to elicit the desired emotional response in a listener.

[0028] According to a further aspect, which may be combined with other embodiments disclosed herein, generating the music includes generating musical feature data indicative of musical features using a generative model that processes the desired emotional response data, and generating the music based on the musical feature data.

[0029] According to a further aspect that can be combined with other embodiments disclosed herein, the musical feature data is Music Information Retrieval (MIR) data.

[0030] According to a further aspect, which can be combined with other embodiments disclosed herein, the MIR data is a music MIR blueprint.

[0031] According to a further aspect, which can be combined with other embodiments disclosed herein, the generative model is a generative adversarial network (GAN).

[0032] According to a further aspect, which can be combined with other embodiments disclosed herein, the GAN comprises a generator network, a probability network, and a control network.

[0033] According to a further aspect, which may be combined with other embodiments disclosed herein, the GAN comprises a conditional GAN.

[0034] According to a further aspect, which may be combined with other embodiments disclosed herein, the generator network comprises a generator neural network.

[0035] According to a further aspect, which may be combined with other embodiments disclosed herein, the probabilistic network comprises a discriminator neural network.

[0036] According to a further aspect, which may be combined with other embodiments disclosed herein, the generator neural network and the discriminator neural network each comprise a recurrent neural network (RNN) with a long short-term memory (LSTM).

[0037] According to a further aspect that can be combined with other embodiments disclosed herein, generating music based on the musical feature data includes generating a score based on the musical feature data, and generating music based on the score.

[0038] According to a further aspect, which may be combined with other embodiments disclosed herein, the score is a MIDI (Musical Instrument Digital Interface) score.

[0039] According to a further aspect, which may be combined with other embodiments disclosed herein, generating the score includes receiving composition intention information and generating the score based on the musical feature data and the composition intention information.

[0040] According to a further aspect, which can be combined with other embodiments disclosed herein, the composition intention information includes one or more of score type information, instrumentation information, and score length information.

[0041] According to a further aspect, which may be combined with other embodiments disclosed herein, the composition intention information is indicated by a composition intention user input received from a user.

[0042] According to a further aspect that can be combined with other embodiments disclosed in this specification, the memory further stores a score generation machine learning model, and generating the score includes processing the musical feature data using the score generation machine learning model to generate the score.

[0043] According to a further aspect that can be combined with other embodiments disclosed in this specification, the score-generating machine learning model is a score-generating generative adversarial network (GAN).

[0044] According to a further aspect that can be combined with other embodiments disclosed herein, the score-generating GAN comprises a generator network, a probability network, and a control network.

[0045] According to a further aspect, which may be combined with other embodiments disclosed herein, the score-generating GAN comprises a conditional GAN.

[0046] According to a further aspect, which may be combined with other embodiments disclosed herein, the generator network comprises a generator neural network.

[0047] According to a further aspect, which may be combined with other embodiments disclosed herein, the probabilistic network comprises a discriminator neural network.

[0048] According to a further aspect, which may be combined with other embodiments disclosed herein, the generator neural network and the discriminator neural network each comprise a recurrent neural network (RNN) with a long short-term memory (LSTM).

[0049] According to further aspects, which may be combined with other embodiments disclosed herein, generating music based on the score includes presenting the score to a user, receiving a rough mix user input from the user, generating a rough mix based on the rough mix user input, and generating music based on the rough mix.

[0050] According to a further aspect, which may be combined with other embodiments disclosed herein, generating music based on the score further includes generating a composition lead sheet based on the musical feature data and presenting the composition lead sheet to a user.

[0051] According to further aspects that may be combined with other embodiments disclosed herein, generating music based on the rough mix includes generating a production lead sheet based on the musical feature data and the rough mix, presenting the composition lead sheet to a user, receiving final mix user input from the user, generating a final mix based on the final mix user input, and generating music based on the final mix.

[0052] According to a further aspect that can be combined with other embodiments disclosed herein, the memory further stores a mastering machine learning model, and generating the final mix includes processing the musical feature data and the final mix using the mastering machine learning model to generate music.

[0053] According to a further aspect, which may be combined with other embodiments disclosed herein, the mastering machine learning model is further configured to receive an existing song and process the musical feature data and the existing song to generate music having the musical features.

[0054] According to a further aspect, which can be combined with other embodiments disclosed herein, the mastering machine learning model is further configured to generate music having musical features based on a plurality of existing music systems.

[0055] According to a further aspect, which may be combined with other embodiments disclosed herein, the mastering machine learning model comprises a recurrent deep Q-network (DQN).

[0056] According to a further aspect, which can be combined with other embodiments disclosed herein, the mastering machine learning model comprises a branching recurrent DQN.

[0057] According to a further aspect, which can be combined with other embodiments disclosed herein, the mastering machine learning model includes a long short-term memory (LSTM).

[0058] A further embodiment is directed to a non-transitory storage medium containing an audio stream generated by the system described above.

[0059] According to a further aspect, which may be combined with other embodiments disclosed herein, the machine learning model is further configured to receive a plurality of pre-existing songs, receive the desired emotional response data, and identify a pre-existing song from the plurality of pre-existing songs that is likely to elicit the desired emotional response in a listener. [Brief explanation of the drawings]

[0060] Embodiments will now be described, by way of example, with reference to the accompanying drawings, in which like reference numerals may be used to refer to like features, and in which:

[0061] [Figure 1] FIG. 1 is a block diagram of an example system for affective music recommendation, according to example embodiments described herein.

[0062] [Figure 2A] FIG. 2A is a system diagram of an example system for affective music recommendation operating in stream generation mode, according to example embodiments described herein.

[0063] [Figure 2B] FIG. 2B is a system diagram of an example system for affective music recommendation operating in training mode, according to example embodiments described herein.

[0064] [Figure 3] FIG. 3 is a flowchart of an example method for emotional music recommendation according to example embodiments described herein.

[0065] [Figure 4A] FIG. 4A is a schematic diagram illustrating a simplified neural network for emotional state inference according to an example embodiment described herein.

[0066] [Figure 4B] FIG. 4B is a schematic diagram illustrating a simplified recurrent neural network for emotional state inference according to an example embodiment described herein.

[0067] [Figure 5] FIG. 5 is a schematic diagram illustrating a neural network for identifying audio segments likely to achieve a target emotional response, according to an example embodiment described herein.

[0068] [Figure 6] FIG. 6 is a series of exemplary user interface screens of a listener device showing listener interactions before a stimulus, according to an exemplary embodiment described herein.

[0069] [Figure 7] FIG. 7 is a series of exemplary user interface screens of a listener device illustrating listener interactions after a stimulus, according to exemplary embodiments described herein.

[0070] [Figure 8] FIG. 8 is a series of exemplary user interface screens of a listener device illustrating listener interaction with personalization according to exemplary embodiments described herein.

[0071] [Figure 9] FIG. 9 is an exemplary user interface screen of a listener device that is presented to a user as part of a profile creation process according to exemplary embodiments described herein.

[0072] [Figure 10] FIG. 10 is a series of exemplary user interface screens of a listener device illustrating contextual information gathering interactions according to exemplary embodiments described herein.

[0073] [Figure 11] FIG. 11 is a block diagram of an example multi-model system for affective music recommendation, according to example embodiments described herein.

[0074] [Figure 12] FIG. 12 is an example user interface screen 1202 for rating musical enjoyment according to example embodiments described herein.

[0075] [Figure 13] FIG. 13 is an example MIR feature prediction system according to an example embodiment described herein.

[0076] [Figure 14] FIG. 14 is a block diagram illustrating steps and components of an exemplary process for creating a database of music models for a listener, according to an exemplary embodiment described herein.

[0077] [Figure 15] FIG. 15 is a schematic diagram illustrating the personalization factors used in the transfer learning process of the personalized model of FIG.

[0078] [Figure 16] FIG. 16 is a block diagram of an example system for emotional music composition, according to example embodiments described herein.

[0079] [Figure 17A] FIG. 17A is a block diagram illustrating the relationships between processes in a first portion of the exemplary emotional music composition system of FIG.

[0080] [Figure 17B] FIG. 17B is a block diagram illustrating the relationships between processes in a second portion of the exemplary emotional music composition system of FIG.

[0081] [Figure 17C] FIG. 17C is a block diagram illustrating the relationships between processes in the third portion of the exemplary emotional music composition system of FIG.

[0082] [Figure 17D] FIG. 17D is a block diagram illustrating the relationships between processes in the fourth portion of the exemplary emotional music composition system of FIG.

[0083] [Figure 18] FIG. 18 is a schematic diagram illustrating a simplified recurrent neural network for emotional state inference used in the emotional music composition system of FIG.

[0084] [Figure 19] FIG. 19 is a block diagram illustrating the MIR generator process used in the emotional music composition system of FIG.

[0085] [Figure 20] FIG. 20 is a block diagram illustrating a score generator process used in the emotional music composition system of FIG.

[0086] [Figure 21A] FIG. 21A is a block diagram illustrating a first portion of a mastering agent for use in the emotional music composition system of FIG.

[0087] [Figure 21B] FIG. 21B is a block diagram illustrating a second portion of a mastering agent for use in the emotional music composition system of FIG.

[0088] [Figure 22A]FIG. 22A is a block diagram illustrating a first portion of an adaptive musical agent for use in the emotional music composition system of FIG.

[0089] [Figure 22B] FIG. 22B is a block diagram illustrating a second portion of an adaptive musical agent for use in the emotional music composition system of FIG.

[0090] [Figure 22C] FIG. 22C is a block diagram illustrating a third portion of an adaptive musical agent for use in the emotional music composition system of FIG.

[0091] [Figure 23] FIG. 23 is a block diagram illustrating a composition lead sheet process for use in the emotional music composition system of FIG.

[0092] [Figure 24A] FIG. 24A is an exemplary emotional space indicator for a composition lead sheet generated by the composition lead sheet process of FIG.

[0093] [Figure 24B] FIG. 24B is an exemplary composition lead sheet generated by the composition lead sheet process of FIG.

[0094] [Figure 25] FIG. 25 is a block diagram illustrating a production lead sheet process for use in the emotional music composition system of FIG.

[0095] [Figure 26] FIG. 26 is an exemplary production lead sheet generated by the production lead sheet process of FIG.

[0096] [Figure 27] FIG. 27 is a block diagram of an exemplary use case of remixing an album using embodiments described herein.

[0097] [Figure 28] FIG. 28 is a block diagram of an exemplary use case of music remixing to target a wellness playlist using embodiments described herein.

[0098] [Figure 29] FIG. 29 is a block diagram of an exemplary use case of remixing a collection of music to generate multiple albums with distinct emotional goals using embodiments described herein.

[0099] [Figure 30] FIG. 30 is a block diagram of an exemplary use case of music adaptation to generate emotional music using a library of existing stems using embodiments described herein. DETAILED DESCRIPTION OF THE INVENTION

[0100] Example embodiments will be described with respect to methods, systems, and non-transitory media for emotional music recommendation and composition. The music recommendation system will be first described with reference to Figures 1-15. Components of these music recommendation systems will then be referenced in describing the music composition system with reference to Figures 16-30.

[0101] The described music recommendation system and method generates an audio stream for inducing a change in an emotional state in a listener. Some embodiments utilize two separate machine learning models to generate an audio stream, such as a music playlist, that is likely to induce a desired emotional response in a listener. One machine learning model is an emotional inference model that estimates an emotional response to a set of audio feature values ​​of audio segments, such as MIR feature values ​​of music segments. The other machine learning system is a reinforcement learning model with a deep learning neural network, also known as a deep Q-network (DQN), that is trained to estimate an emotional response to the audio segments using a set of audio segments (e.g., songs or epochs excerpted from songs) and feedback from the emotional inference model based on the audio feature values ​​(e.g., MIR feature values) of the audio segments.

[0102] A first exemplary embodiment of an emotional music recommendation system 100 for generating an audio stream for inducing an emotional state change in a listener will now be described with reference to FIG.

[0103] FIG. 1 shows an affective music recommendation system 100 that includes a processor system 102 that executes computer program instructions, a storage device 104 that stores executable instructions and data, and a communication system 106 that communicates data with other devices or components.

[0104] The emotional music recommendation system 100 may be implemented on one or more computer systems. It may be embodied by a single computer, multiple computers, a virtual machine, a distributed or cloud computing platform, or any other platform capable of performing the method steps described herein. In some embodiments, the emotional music recommendation system 100 may encompass one or more electronic devices used by listeners (listener devices 190), while in other embodiments, the emotional music recommendation system 100 communicates with such devices directly or indirectly (e.g., via communication network 170) using communication system 106.

[0105] The processor system 102 may be embodied as any processing resource capable of executing computer program instructions, such as one or more processors on a computer or computing platform. The storage device 104 may be embodied as any data storage resource, such as one or more disk drives, random access memory, or volatile or non-volatile memory on one or more computing platforms. The communication system 106 may be embodied as one or more communication links or interfaces, including wired or wireless communication interfaces, such as Ethernet, Wi-Fi, or Bluetooth interfaces. In some embodiments, one or more of the listener devices 190 may be implemented on the same platform as the emotional music recommendation system 100. In such embodiments, the communication system 106 may comprise an internal communication bus or other intra-platform data transfer system.

[0106] The storage device 104 may store several types of computer programs in the form of executable instructions. A set of executable instructions 110 for performing the method steps described herein may be stored therein. There may also be one or more machine learning models for identifying audio segments intended to elicit a particular emotional response in a listener, which are represented herein as a plurality of deep Q-networks (also referred to as deep learning neural networks), i.e., first deep Q-network 122, second deep Q-network 124, etc. through Nth deep Q-network 126. The storage device 104 may also have stored therein an emotion inference machine learning model, represented herein as emotion inference neural network 140, for inferring an emotional state induced by a listener's exposure to an audio segment having a particular set of audio feature values. These machine learning models may be trained and then deployed on the emotional music recommendation system 100, as described further below.

[0107] The storage device 104 may have several types of data 180 stored thereon. The data 180 may include data related to previous recordings of experiences with the affective recommendation system 100 (e.g., emotional data and segment selections). The data 180 may also include an audio library 184 including a plurality of audio segments 186 and audio feature data corresponding to each of the plurality of audio segments 186. The audio segments 186 may include digital audio data stored as separate audio clips, or the segments may be extracted from audio clips stored in the audio library 184, such as epochs of fixed duration extracted from songs of variable duration. The audio feature data is shown here as library MIR data 182. This may include MIR metadata associated with each audio segment 186 indicating MIR features of the audio segment 186 with corresponding values. The audio feature data may also include non-MIR data or metadata in some embodiments.

[0108] Listener device 190 may be an electronic device operated by a listener or end user of emotional music recommendation system 100, such as a computer or smartphone, that communicates with emotional music recommendation system 100 via communication network 170. Emotional music recommendation system 100 may support multiple types of listener devices 190. Some listener devices 190 include user interface components, such as a touchscreen 194 that displays visual data and receives user input, and an audio output 192, such as a wired or wireless interface to speakers and / or headphones. Communication with emotional music recommendation system 100 is performed by a communication system 196, which may communicate via communication network 170.

[0109] 2A shows a functional system diagram of the emotional music recommendation system 100. Various functional steps are performed by the emotional music recommendation system 100 by using a processor system 102 to execute executable instructions 110 stored in a memory device 104.

[0110] The emotional music recommendation system 100 executes instructions 110 to perform a method for generating an audio stream 234 that induces a change in an emotional state in a listener. To perform the method steps, the emotional music recommendation system 100 uses several functional blocks that are implemented by the execution of instructions 110, a segment discrimination machine learning model (e.g., deep Q-networks 122, 124-126), and an emotion inference machine learning model (e.g., emotion inference neural network 140). The emotional music recommendation system 100 operates in two different modes: an audio stream generation mode, which typically occurs during a user session, and a training mode, which may occur between user sessions. The operation of the emotional music recommendation system 100 in the audio stream generation mode will be first described with reference to FIG. 2A, followed by a description of the operation in the training mode with reference to FIG. 2B.

[0111] Listener state data associated with the listener is received via communication system 106 and used in current state identification process 202 to identify the listener's current emotional state 212. The listener state data, in various embodiments, may include emotional self-assessment data 206, physiological data 208, and / or other types of data potentially relevant to identifying the listener's emotional state. The listener state data may be received from one or more sources, such as from the listener device 190, from other devices, and / or from sources internal to the emotional music recommendation system 100. As described in further detail below, the emotional self-assessment data 206 may be generated by the listener device 190. The physiological data 208 may be received from the listener device 190 or another device configured to collect physiological sensor data from the listener, as described further below. Other listener state data used to determine a listener's emotional state may include camera data showing the listener's facial expressions or behavior, audio data showing the listener's intonation or speech content, or any other data that may be used to assist in identifying a listener's emotional state.

[0112] In the field of affective computing, there are many known techniques for identifying human emotional states using physiological, self-reported, and / or other data types, and for representing emotional states within data. One common model for representing emotional states is the two-dimensional model of emotion, sometimes called the circumplex model, in which a given emotional state is represented as a valence value (representing a degree of positive or negative emotion) and an arousal or activation value (representing a degree of emotional alertness or energy). In a two-dimensional valence-activation model of emotion, for example, sadness may be represented as negative valence and low activation, anger may be represented as negative valence and high activation, enthusiasm may be represented as positive valence and high activation, and relaxation may be represented as positive valence and low activation. The embodiments described herein will generally refer to the two-dimensional model of emotion with valence and activation values. However, in some embodiments, other emotional models may be used, including models that use more or less than two dimensions to characterize emotional states, models that use time-varying emotional values ​​to model emotional states, and models that use lists of discrete emotional states without using numerical values.

[0113] In some embodiments, the current state identification process 202 may receive listener state data that explicitly identifies the listener's current emotional state. In other embodiments, the current state identification process 202 may identify the listener's emotional state based on the listener state data using an emotional identification engine, such as an additional machine learning model trained to identify emotional states in specific listeners or humans generally. Additional biological markers inferred from physiological data may also be used as inputs to the current state identification process beyond the two-dimensional emotional valence and activity values, such as anxiety level, concentration level, agitation level, etc.

[0114] Some embodiments may receive listener state data at specific times during operation of the emotional music recommendation system 100, such as at the beginning and end of a user session. In other embodiments, listener state data may be received continuously or at times determined by user input. For example, some embodiments may receive a continuous stream of physiological data 208, while other embodiments may receive user-induced emotional self-assessment data 206 at listener-determined times.

[0115] The target state identification process 204 is used to identify a target emotional state 214 for the listener based on target emotional state data 210 received from a source, such as the listener device 190. In some embodiments, the target emotional state data 210 may be predetermined based on the nature of the intended use. For example, a relaxation use may always provide target emotional state data 210 indicating a low-activity, positive-valence state, while a concentration use may provide target emotional state data 210 indicating a high-activity, positive-to-neutral valence state. In other embodiments, the listener's target emotional state 214 may be identified based on listener preference data received from the listener device 190 before or during a user session.

[0116] In some embodiments, based on the current emotional state 212 and the target emotional state 214, an emotional trajectory process 216 identifies an emotional trajectory 218 from the current emotional state 212 to the target emotional state 214. In embodiments using a two-dimensional emotional model, the emotional trajectory 218 can be represented as a two-dimensional curve 250. The exemplary curve 250 is plotted in an exemplary emotional space defined, for example, by emotional valence (left=negative, right=positive) in the horizontal dimension and activity (active=up, passive=down) in the vertical dimension. The listener's current emotional state 212 is plotted as a starting point 252 of the curve 250. The target emotional state 214 is plotted as an end point 254 of the curve 250. One or more intermediate waypoints may be plotted along the curve 250, such as a first waypoint 256 and a second waypoint 258, to indicate intermediate emotional states along the emotional trajectory 218. An initial portion 260 of the curve 250 is defined by the start point 252 and the first waypoint 256. A subsequent second portion 262 of the curve 250 is defined by the first waypoint 256 and the second waypoint 258. A subsequent third and final portion 264 of the curve 250 is defined by the second waypoint 258 and the end point 254. Machine learning techniques may also be implemented to learn the best trajectories for individuals using the system, making these trajectories dynamic based on the user's previous success in achieving the user's target emotional state. In other embodiments, the trajectory may be omitted entirely, and the session may be driven solely by the user's current emotional state and the target emotional state. An embodiment using this emotional trajectory process is intended to implement controlled emotional state changes over time through a moving target for DQN.

[0117] An audio segment identification process 222 is used to select or identify audio segments that, when presented to a listener as auditory stimuli, are likely to elicit at least the initial portion 260 of the emotional trajectory 218 in the listener's emotional state. The audio segments 230 are identified using a trained segment identification machine learning model, denoted as DQN 120, which selects the audio segments 230 from a subset of the audio segments 220 stored in the audio library 184. The audio segments 230 are selected based on an assessment by DQN 120 that the audio segments 230 are more likely to elicit at least the initial portion 260 of the emotional trajectory 218 in the listener than other audio segments in the subset of audio segments 220, i.e., that the audio segments 230, when played to the listener as auditory stimuli, are more likely to elicit an emotional state in the listener that is closer to the state represented by the first waypoint 256 or one of the subsequent points 258, 254 on the emotional trajectory 218.

[0118] The audio segment identification process 222 can also use as input contextual cues 191 received from the listener device 190, such as the time of day, whether the listener is in a private environment, whether the listener is in a noisy environment, etc.

[0119] The DQN 120 used in the emotional music recommendation system 100 may be any of the various DQNs (first DQN 122 through Nth DQN 126) from FIG. 1 under various circumstances. In some embodiments, each of the multiple DQNs 122, 124, 126 is used to select from a different subset 220 of audio segments from the audio library 184. These subsets of audio segments may be generated based on various criteria. Examples of subsets of audio segments may include songs selected based on listener-indicated preference data, music genre, or other grouping criteria. One purpose of using a subset instead of the entire audio library 184 may be to simplify the calculations performed by the DQN 120 by limiting the number of audio segments included in the subset 220 of audio segments to a maximum set size, such as 100 or 120 audio segments. Multiple DQNs (eg, DQNs 122, 124-126) can be used in place of one another in different user sessions to select audio segments from different subsets of audio segments 220 depending on the context.

[0120] In some embodiments, DQN 120 may then identify one or more subsequent audio segments 230 that, when presented to a listener as an auditory stimulus, are likely to elicit one or more subsequent desired emotional responses in the listener that correspond to subsequent portions of emotional trajectory 218 (e.g., portions 262 and / or 264).

[0121] A single iteration of the process of identifying audio segments 230 may be referred to as a “step.” After each step (i.e., after identifying the first audio segment and after identifying each of multiple subsequent audio segments), a trained emotion inference machine learning model (shown herein as emotion inference neural network 140) may be used to generate inferred new emotional state data 226 and feed the inferred new emotional state data 226 back to DQN 120 to inform decisions made by DQN 120. This is performed as part of emotion inference process 224, in which audio segments 230 identified by audio segment identification process 222 and target emotion data 214 are received using emotion inference neural network 140 to infer an inferred new emotional state 226 of the listener that is likely to result from exposing the listener to an audio stimulus having audio features that match the audio features of audio segment 230 (e.g., segment MIR data 183).

[0122] In some embodiments, the emotional inference process 224 may use various techniques to generate the inferred new emotional state 226, such as various machine learning or artificial intelligence models, or predetermined sets of correlations or rules.

[0123] Thus, the emotion inference neural network 140 simulates the environment in which it operates by providing feedback and reinforcement to the DQN 120, i.e., the emotion inference neural network 140 simulates the listener's emotional response by using audio feature data (e.g., segment MIR data 183) and the listener's current emotional state(s) 212 to infer the user's likely emotional response, as determined by prior training of the emotion inference neural network 140. The training of an exemplary emotion inference neural network 140 is described in detail below.

[0124] In some embodiments, the emotional inference process 224 may be replaced, in whole or in part, by a stream of real-time emotional data measurements from actual users, in which the data stream is sufficiently reliable to provide the emotional state data necessary to inform the next "step" in the decision-making process made by the DQN 120.

[0125] The stream generation process 240 generates an audio stream 234 based on one or more audio segments 230 identified by the audio segment identification process 222. The audio stream may, in some embodiments, be a music playlist, which may be expressed as a series of identifiers and / or other metadata corresponding to one or more audio segments 230. The stream generation process 240 also generates audio stream data 236 for transmission to the listener device 190 via the communication system 106. In some embodiments, the audio stream data may include metadata corresponding to one or more audio segments 230 included in the audio stream 234. This metadata may be transmitted to the listener device so that the listener can review the suggested playlist and provide input via the touchscreen 194 to play or modify the playlist. In some embodiments, the audio stream data may include audio segment data corresponding to one or more audio segments 230. The listener device may be configured to present this audio data as auditory stimuli to the listener via the audio output 192. Communication between the emotional music recommendation system 100 and the listener device 190, including the transmission of different types of audio stream data 236 at different times and under different conditions, may, in various embodiments, comprise a typical interaction between a user and an online music recommendation, curation, or playlist service.

[0126] The curve 250 of the emotional trajectory 218 identified by the trajectory identification process 216 may have different characteristics in different embodiments and / or in different situations. Some embodiments may further include processes, such as additional machine learning models, to shape the curve 250 into a user-dependent or user-independent shape based on emotional feedback data collected over time. The curve 250 may be a simple linear trajectory (i.e., a two-dimensional sloping line) from a first state to a second state, or it may be curved according to principles or patterns extracted from scientific literature or emotional data analysis. In embodiments that do not use a numerically based emotional model, the emotional trajectory 218 may not be plotted in emotional space, but instead may pass through one or more intermediate emotional states identified as intermediate emotional states necessary to move from the current state to the goal state.

[0127] The embodiments described herein are configured to induce one or more intermediate emotional states (e.g., waypoints 256, 258) along the emotional trajectory 218 before inducing the final target emotional state 214. This approach to emotional state change using musical stimuli follows the iso principle used in music therapy for mood management. The iso principle states that music should initially match the listener's current mood and then gradually transition over time toward the desired target mood. However, some embodiments may take different approaches to eliciting an emotional response, such as embodiments that attempt to induce an emotional state change using a single audio segment or embodiments that use a nonlinear emotional trajectory curve 250.

[0128] The above description provides an overview of the operation of the various functional blocks and data used by the emotional music recommendation system 100 in the audio stream generation mode. In the training mode, the various functional blocks and data shown in Fig. 2A may be used for various purposes and may be supplemented with further functional blocks and data as described below with reference to Fig. 2B.

[0129] 2B shows the emotional music recommendation system 100 of FIG. 2A operating in training mode. For example, in training mode, the inferred new emotional state 226 is compared to the target emotional state 214 by the inference training process 228, and reward data 273 is generated based on this comparison. In some embodiments, the reward data 273 provides a positive reward to the DQN 120 when the inferred new emotional state 226 is similar to the target emotional state 214, but provides a negative reward when the inferred new emotional state 226 is different from the target emotional state 214.

[0130] In some embodiments, the reward function 272 can be used by the inference training process 228 to generate reward data 273. The reward function 272 can vary over the course of multiple sessions. In the direct training process 270, the reward function 272 receives emotional self-assessment data 206 and / or physiological data 208 from the listener device 190 at the end of the session (or after the user has finished listening to the audio stream 234), which indicates the listener's final emotional state. The reward function 272 generates reward data 273 that either rewards (i.e., a positive reward) or punishes (i.e., a negative reward) the decisions of the DQN 120 throughout the session based on a comparison between the listener's current emotional state 212 at the end of the session (i.e., the listener's final emotional state after listening to the audio segment 230) and the target emotional state 214.

[0131] In some embodiments, the final emotional state data includes all emotional state data collected from the listener throughout the session and current emotional state data 212 collected from the listener at the end of or after the session. Reward data 273 is used to retrain DQN 120 to make better decisions in future sessions, effectively personalizing the model. In other embodiments, intermediate and long-term emotional state goals may be set to reward actions taken at individual steps as well as complete sequences. Some embodiments may also use additional feedback data (not shown) collected from listener devices 190. Some embodiments may also discourage repeat selections of the same audio segment 230 by negatively rewarding certain repeat selections, or may negatively reward selections of a particular audio segment if the user “skips” that particular audio segment while listening to audio stream 234.

[0132] In some embodiments, the final emotional state data (i.e., current emotional state data 212 collected at the end of a session) correlated with segment MIR data 183 of audio segments played throughout the session may also be used to retrain the emotional inference neural network 140 to make better predictions in future sessions.

[0133] In some embodiments, the shape of the emotional trajectory 218 can be adapted for a particular user by implementing the emotional trajectory process 216 using a trajectory shape generation machine learning model. The last emotional state data (i.e., current emotional state data 212 collected at the end of a session) and a trajectory reward function 262 can be used by the trajectory shape generation feedback process 260 to shape-generate the emotional trajectory 218 based on successful results from previous sessions. In embodiments where the trajectory shape generation machine learning model is implemented to optimize the best emotional trajectory for a particular user, the last emotional state data and additional reward data 263 are used to train and optimize the model to personalize the trajectory based on the user.

[0134] In some embodiments, the emotional trajectory 218 can be skipped entirely, and the reward function 272 can generate reward data 273 by simply comparing the user's final emotional state with the target emotional state 214 and rewarding the complete sequence predicted by the DQN 120 accordingly.

[0135] In some embodiments, DQN 120 may be replaced with any machine learning algorithm that learns through trial and error in the real world during user experience (at runtime) and through simulated environments (off-the-shelf). This can be achieved with any model-based or model-free reinforcement learning algorithm. All instances of DQN 120 functionality described herein may, in some embodiments, be replaced with a different model-free or model-based reinforcement learning agent, including, but not limited to, approaches such as MBAC (Model-Based Actor Critic), Advantaged A3C (Advantaged Model-Free Actor Critic), Q-learning, deep Q-learning, and TDM (Temporal Difference Model).

[0136] 3 shows a flowchart of an example method 300 for generating an audio stream that induces an emotional state change in a listener. In step 302, the listener's current emotional state 212 is identified, as described above. In step 304, the listener's target emotional state 214 is identified, as described above. In step 306, the emotional trajectory 218 is identified, as described above. In step 308, a trained segment identification machine learning model (e.g., DQN120) is used to identify a first audio segment (e.g., audio segment 230) that is likely to induce a desired emotional response in the listener corresponding to at least an initial portion (e.g., initial portion 256) of the emotional trajectory 218 when the first audio segment is presented to the listener as an auditory stimulus, as described above.

[0137] In step 310, the emotional inference process 224 uses the trained emotional inference machine learning model 140 to predict how the audio segment selected by the deep learning neural network 120 in step 308 will affect the user. New emotional state data 226 resulting from this inference is generated by the emotional inference machine learning model 140 in step 310 and sent to the DQN 120 as state data input in step 312.

[0138] In step 316, the emotional music recommendation system 100 may determine whether the method 300 has reached the last audio segment identification step of multiple such steps, or whether one or more subsequent audio segments remain to be identified and added to the audio stream 234. In some embodiments, the audio stream may have a fixed length and / or require a fixed number of audio segment identification steps, i.e., the audio stream 234 may always be of a first fixed duration (e.g., 240 seconds in duration) and each audio segment may be of a second fixed duration (e.g., an 80-second epoch excerpted from a song), thereby requiring three audio segment identification steps to generate the audio stream 234. If the method 300 determines in step 316 that the final step has not been reached, subsequent audio segments likely to elicit subsequent portions of the emotional trajectory are identified as described above with reference to FIG. 2A . The method returns to step 308 to identify subsequent audio segments and then to step 316 until the final step is completed. Next, the deep neural network, in step 308, uses the inferred emotional state data 226 from the emotional inference model 140 in step 310 as the "latest" emotional state 212 for the next audio segment prediction.

[0139] Once the final step is complete, audio stream 234 is generated in step 318, as described above. In step 320, audio stream data 236 is generated and transmitted to listener device 190, as described above. This completes the process of generating an audio stream and transmitting it to the listener for possible presentation as an auditory stimulus.

[0140] In some embodiments, the listener may be prompted or presented with the option to provide updated current emotional state data after listening to all or a portion of audio stream 234. This updated current emotional state data may be used to train an emotional inference machine learning model and improve its inferences regarding likely listener emotional responses to audio features of the audio segments that make up audio stream 234. This data may also be used to generate reward data 273 using reward function 272 to reinforce the selections made by DQN 120 in step 308 of the method outlined in FIG.

[0141] In step 322, updated current emotional state data is received from the listener device 190, for example, via the communication system 106. In step 324, an emotion inference machine learning model (e.g., the emotion inference neural network 140) is trained using the updated current emotional state data, as described in more detail below. This step typically ends the user session.

[0142] In the embodiments described herein, the segment discrimination machine learning model (e.g., DQN 120) is also trained using one or both of an inference training process 224 and / or a direct training process 270. This training may occur at the end of a user session or during idle time when the listener is not using the system 100. In step 326, the segment discrimination machine learning model (e.g., DQN 120) is retrained using an inference training process 228. Reward data 273 is generated by a reward function 272 based on the inferred new emotional state 226 and the target emotional state 214. This training step may repeat the initial training of the DQN 120, as described in more detail below. This training can be performed asynchronously and / or offline.

[0143] In step 328, the segment discrimination machine learning model (e.g., DQN 120) is retrained using a direct training process 270. Reward data 273 is generated by a reward function 272 using updated current emotional state data 212 received from the listener through the listener device 190 at the end of the session. This training can also be performed asynchronously and / or offline.

[0144] In some embodiments, steps 318 and 320 may be performed immediately after step 308 to generate an audio stream after the first prediction made in 308. In these embodiments, a continuous stream of reliable current emotional state data 212 is received from the user, eliminating the need for inferred emotional state data 226. In these embodiments, step 318 occurs after step 308, followed by steps 320 and 322, and then returning to step 308 to obtain new emotional state data 226 via direct user feedback. This process may be repeated multiple times until an audio experience of the desired length is completed. In these embodiments, the training process for DQN 120 is performed using only direct user feedback, eliminating the need for steps 310, 312, 324, and 326.

[0145] Examples of the emotion inference machine learning model and the segment identification machine learning model will now be described in more detail, including the process for training each model, with reference to Figures 4A-B and 5, respectively.

[0146] 4A shows a schematic diagram of an exemplary embodiment of an emotion inference machine learning model as an emotion inference neural network 140. The emotion inference neural network 140 includes at least an input layer 402 of neurons 408, one or more hidden layers 404, and an output layer 406. The input layer 402 receives data inputs (shown herein as a first MIR feature value 412 through an Mth MIR feature value 414 and one or more non-MIR data values ​​416), transforms these inputs using known techniques, and provides outputs 410 to the neurons in the first hidden layer 404. Each neuron 408 in the hidden layer 404 weights each of the outputs 410 received from the previous layer and transforms the weighted output values ​​to generate a further set of outputs 410 for the next layer. Neurons 408 in output layer 406 similarly weight their received outputs 410 and transform the weighted outputs to generate output values ​​(shown herein as valence 420 and activity 422). By adjusting the weights applied to the inputs of each neuron 408 during training, the emotion inference neural network 140 can be trained to infer a likely listener's emotional response (defined herein as an inferred valence value 420 and activity value 422) to a set of audio features, such as MIR feature values ​​412-414. Additional non-MIR or non-audio feature values ​​may also be used to train and run the emotion inference neural network 140. That is, for example, in some embodiments, variables such as time of day or listener environment may be used to assist the emotion inference neural network 140 in making inferences about a likely user's emotional response.

[0147] In other embodiments, different machine learning models can be used in place of the neural networks shown in Figures 4A-B and 5. Emotion Inference Neural Network 140 can be replaced with various forms of supervised and unsupervised machine learning systems while maintaining the same core inputs and outputs necessary to infer a user's emotional state based on music and / or audio selections. Similarly, DQN 120 can be replaced with various forms of supervised and unsupervised machine learning systems while maintaining the same core inputs and outputs necessary to select audio segments based on their inferred effectiveness in inducing a desired emotional trajectory in a listener.

[0148] In some embodiments, a recurrent neural network may be used for the emotion inference neural network 140, allowing it to use a time series of MIR features as a set of inputs and provide an output of inferred states based on a sequence of musical features instead of an average representation of the time-based MIR features for an entire segment of music. This may allow the emotion inference neural network 140 to make its predictions more granular and time-series, providing a more realistic representation of the musical experience (as human cognition perceives music over time rather than a track as a cohesive whole).

[0149] 4B shows an example of a recurrent emotional inference neural network (AINN) 430. The MIR extraction process 225 is used to extract segment MIR features 183 from the audio segment 230 selected by the audio segment identification process 222, referred to herein as an N-length array 433 of global MIR features for the audio segment 230, denoted as first global MIR feature 435 through Nth global MIR feature 436. The global MIR features may represent musical characteristics of the entire audio segment 230, such as tempo. A series of short subsegments (also called epochs) (e.g., 30-second subsegments) of the audio segment 230 are also analyzed for MIR features by the MIR extraction process 225 to generate multiple time-series MIR feature arrays 434, each corresponding to a subsegment and including a first time-series MIR feature 438 through an Nth time-series MIR feature 440 for the current subsegment. Each of the time-series MIR features 438-440 in the time-series MIR feature array 434 represents a feature for the current epoch of the audio segment 230, e.g., a Mel-Frequency Cepstrum (MFC) spectrogram value for a particular temporal epoch of the audio segment 230.

[0150] At each time step, each global MIR feature in the global MIR feature array 433 and each MIR feature in the time-series MIR feature array 434 for the current epoch (e.g., starting with the first 30-second subsegment) are provided as inputs to the recurrent emotion inference neural network 430, along with other input data, such as the current emotional state 212, and other data, such as context information 432. The context information 432 may include stored values ​​representing, for example, a user's profile (e.g., personality, age, gender, etc.), a taste profile (e.g., music preferences), time of day, weather, etc. At each subsequent time step, the time-series MIR feature array 434 for the subsequent epoch (e.g., the second 30-second subsegment) is provided as inputs 438...440, along with other inputs 212, 432, 435...436, and the outputs of the time-series neurons are provided as a further set of feedback inputs, thereby providing time-based iterations. Based on these inputs, the recurrent emotional inference neural network 430 predicts an inferred emotional state 450 (e.g., valence and activation value) that will be induced in the listener by listening to the audio segment 230.

[0151] The emotion inference neural network 140 may first undergo an initial training process before being deployed as part of the emotional music recommendation system 100. In some embodiments, the initial training can be performed using techniques and / or datasets similar to those described in the following publication: Vempala, Naresh & Russo, Frank. (2012). Predicting emotion from music audio features using neural networks. Proceedings of the 9th International Symposium on Computer Music Modeling and Retrieval (CMMR) (hereinafter Vempala), which is incorporated herein by reference in its entirety. Vempala describes using a selected subset of MIR features of a music collection to train a neural network to predict the emotional response of human subjects listening to this music. Specifically, Vempala uses 13 low- and medium-level MIR features related to dynamics, rhythm, timbre, pitch, and tonality: rms (RMS), low energy, event density, tempo, pulse clarity, zero crossing, centroid, spread, rolloff, brightness, irregularity, inharmonicity, and mode. These MIR features of music are used as inputs to a feedforward neural network, which generates output values ​​of valence and arousal (i.e., activity). After listening to music, self-reported emotional valence and arousal information is collected from the subject. Backpropagation is then used to train the neural network using a loss function that compares the predicted valence / arousal outputs with the user's self-reported data.

[0152] In an exemplary embodiment, the initial training of the emotion inference neural network 140 may be performed using a set of audio features different from the 13 features used in Vempala.

[0153] Vempala normalizes the values ​​of various MIR features of interest on a scale of 0 to 1. Similarly, exemplary embodiments may normalize MIR data and / or other audio or non-audio data used in emotion inference neural network 140.

[0154] Once initial training is complete, the emotion inference neural network 140 may be deemed effective at predicting listeners' independent emotional responses to audio features. However, the emotion inference neural network 140 may be trained to predict a specific listener's emotional response using emotional feedback from that listener. The expanded emotion inference neural network 140 may also use more audio and non-audio features as inputs to identify the influence of less universal factors on the listener's emotional response. By comparing the emotion inference made in step 310 with the updated current emotional state data received from the listener in step 322, the emotion inference neural network 140 can use backpropagation or other training techniques to update the weights used by its neurons 408 to improve modeling of the listener's emotional response to a set of audio feature values.

[0155] In some embodiments, an inference training process may be provided that allows listeners to train their own personalized emotional inference neural network 140 on-demand by listening to audio segments and providing contextual emotional state data. This data may be used to train the emotional inference neural network 140, as described above.

[0156] During use or training, the emotion inference neural network 140 may occasionally encounter outliers, i.e., audio stimuli that elicit emotional responses in listeners that are very different from the emotion inference neural network 140's predictions. To avoid over-fitting, feedback data from these outlier stimuli can be separated from the feedback data used for training. This separated data can be used by a separate analysis process to extract listener-specific data that can help identify patterns in listener-specific responses to stimuli. For example, a listener may have an extreme emotional response while listening to an audio segment due to other events in the listener's environment unrelated to the audio segment, or a listener may have an emotional association with a particular song that is unrelated to the song's audio features. These non-audio feature-related associations need not be used to train the emotion inference neural network 140, but instead may be used to identify non-audio data that could potentially be used as input to the emotion inference neural network 140, for example, to aid in the accuracy of future predictions.

[0157] FIG. 5 shows a schematic diagram of an exemplary embodiment of an audio segment identification machine learning model as a deep learning neural network or deep Q-network (DQN) 120. The DQN 120 comprises at least an input layer 502 of neurons 508, one or more hidden layers 504, and an output layer 506. As in the emotion inference neural network 140 of FIG. 4A, the input layer 502 receives data input values ​​and transforms these inputs using known techniques to provide outputs 510 to the neurons of the first hidden layer 504. Each neuron 508 in the hidden layer 504 weights each of the outputs 510 received from the previous layer and transforms the weighted output values ​​to generate a further set of outputs 510 for the next layer. The neurons 508 in the output layer 506 similarly weight their received outputs 510 and transform the weighted outputs to generate output values.

[0158] The DQN 120 effectively matches a standard deep learning neural network, with its significant derivatives utilized as the core predictive component of a reinforcement learning (RL) agent 530 in navigating a reinforcement learning (RL) environment 531. This type of deep learning neural network also uses training techniques such as backpropagation, but the DQN model uses reward data 273 to drive the training process, as opposed to the correlated data sets found in the training process of standard deep learning neural networks used in the emotional inference neural network 140. This system is modeled after Q-learning systems, replacing standard Q-tables with deep Q-networks, thereby enabling significantly faster training of the reinforcement learning (RL) agent 530. A standard Q-table tabulates the Q-values ​​for all possible actions based on the current state of the environment, based on previous sessions during training. This requires multiple iterations of each state-action relationship before the RL agent 530 can optimize its performance. With a large number of states and actions, this training becomes expensive and inefficient. By replacing Q-tables with deep learning neural networks, i.e., DQN120, the network's input represents the relationships between all possible states, and its output represents the Q-values ​​of all possible actions. These networks enable relational training, which approximates the values ​​of all actions at each iteration, significantly reducing the time it takes to optimize the performance of an RL agent. See, e.g., LeCun, Yann & Bengio, Y. & Hinton, Geoffrey. (2015). Deep Learning. Nature. 521.436-44.10.1038 / nature14539, and Mnih et al., 2013, Playing Atari with Deep Reinforcement Learning, Nature. 518.529-533.10.1038 / nature14536, each of which is incorporated herein by reference in its entirety.

[0159] 5, the Q-values ​​520-522 are provided to a reinforcement learning agent 530, which takes action 532 by selecting an audio segment 530 and providing it to one of two alternative environments: a real environment representing the user's actual emotional state, or a simulated environment representing the emotional inference process 224. In the former case, the stream generation process 240 generates an audio stream and transmits it to the listener device 190, which then provides updated current emotional state data 212 after the listener is exposed to the audio stream. In the latter case, the audio segment 234 is provided to the emotional inference process 224, which generates an inferred new emotional state 226.

[0160] 5 (first DQN input 512, goal state value 514, and current state value 516) is a partial and simplified list of inputs used in exemplary embodiments. In some embodiments, the DQN input (e.g., first DQN input 512) may include context clues 191 described above with reference to FIGS. 2A-2B. The exemplary embodiment may use the following list of inputs at runtime: These include the mode (indicating a target emotional state, such as “calm,” “focused,” or “energetic”), the step count (indicating the total number of audio segments 230 in the sequence of audio stream 234), the current step (indicating which of the sequence of audio segments 230 is currently identified), the initial state valence (the valence value of the listener's current emotional state 212 identified at the start of the session), the initial state activity (the activity value of the listener's current emotional state 212 identified at the start of the session), the updated state valence (the valence value of the inferred new emotional state 226 or updated current emotional state 212 before the current step), and the updated state activity (the activity value of the inferred new emotional state 226 or updated current emotional state 212 before the current step).

[0161] Depending on how the user is feeling and what their current musical preferences are, a user's various reactions to music can vary greatly from day to day. Thus, in some embodiments, system 100 can implement further personalization methods by providing additional inputs to deep Q-network 120 and affective inference neural network 140 to allow other factors to influence the prediction process. In some embodiments, the desired musical aesthetic the user is seeking at that moment may be identified and provided as an additional input. For example, if a user is feeling sad and wants to listen to loud and rebellious music (e.g., rock music), they will likely respond differently to a combination of MIR features, a library of tracks, or a selection of audio segments than if they are feeling sad and want to listen to reflective and complex music (e.g., classical music). This difference between the current emotional state (e.g., sadness) and the emotional content of the current desired music (e.g., loud and rebellious versus thoughtful and complex) can be represented using data values ​​and provided as further inputs to neural networks 120 and 140 of Figures 4A-B and / or 5 (e.g., "current music preference" input to input layer 402 or 502).

[0162] User profile data may also be used as potential inputs to neural networks 120 and 140 depicted in FIGS. 4A-B and 5 (e.g., input of one or more “user profile data” to input layer 402 or 502). User profile data may include data fields such as a user's demographic profile, baseline music preference profile, baseline mood profile, and personality profile. All of this and other user profile data may potentially be valuable in predicting how a user will respond to a music track or MIR features, given their current state. Some of this data may be collected from the user using the user interface screens described below with reference to FIGS. 8 and 9. Some of this data may also be inferred by system 100 based on user input provided by the user via the user interface screens of FIGS. 6-7. For example, a user's music genre preferences may be inferred from the user's genre selections within UI screen 608 of FIG. 6 across multiple sessions.

[0163] Contextual information, such as the time of day and the user's surroundings (e.g., whether the user is in a noisy or quiet location), may be used by system 100 as a source of relevant data and may be provided as one or more additional data inputs to the input layers 402 and / or 502 of the neural networks of Figures 4A-B and / or 5. The user's current environment and context may have a significant impact on how music is cognitively perceived by the user and may provide valuable input data to system 100. Contextual data may be collected by system 100 through user interactions, for example, as shown in screens 604 and 610 of Figure 6 and screen 1002 of Figure 10.

[0164] The user's current state value 516 can also be provided as an input to the input neuron 402 of FIG. 4A. By using the user's current state value 516 to notify the prediction of the emotion inference neural network 140, the system 100 can take into account the user's current emotional state when predicting what emotional results will be brought about by exposure to a combination of MIR features. For example, a segment of music can evoke two very different emotions depending on the user's initial emotional state.

[0165] During the inference training mode, the input layer neuron 502 will receive output data from the emotion inference neural network 140 to set the initial weights for the DQN 120. The target state value 514, as well as all other variable parameters (e.g., the number of steps), are randomly generated, and the output audio segment Q values 520 - 522 are then used by the RL agent 530 to select an audio segment 230. This audio segment 230 is converted into segment MIR data 183 by the MIR extraction process 225, and the emotion inference neural network 140, which essentially operates as an RL environment 531, predicts what the next emotional state will be based on the actions taken by the RL agent 530. Next, this new state (i.e., the newly inferred emotional state 226) is used as the current state value 516 for the next step in the prediction sequence, and this cycle continues until all steps are completed. This complete sequence is executed in thousands of iterations where new goals and new numbers of steps are randomly generated each time. During initial training, this process is generally done in 200,000 - 500,000 iterations to fully optimize the DQN 120.

[0166] In some embodiments, the MIR extraction process 225 extracts segment MIR features 183 from the audio segment 230 in real - time or refers to a table of previously extracted features. The extraction of MIR features from an audio segment may be performed using known techniques.

[0167] By adjusting the weights applied to the input of each neuron 508 during training, the DQN 120 can be trained to select or identify audio segments 230 with optimal Q-values ​​520-522 relative to other candidate audio segments that are likely to elicit a target emotional state over a series of steps, or in the short term per step if using an emotional trajectory 218. Training can be performed in two modes: initial training and retraining or updating during idle time.

[0168] Initial training of each DQN 120 is performed before deployment. Each user of the emotional music recommendation system 100 has multiple DQNs (e.g., DQNs 122, 124, 126) associated with their personal account, as well as a personal emotional inference neural network 140. The emotional inference neural network 140 is trained to model the user's (listener's) emotional responses to various combinations of audio features and other non-audio variables, as described above. The emotional inference neural network 140 then operates to model the environment in which each DQN 120 operates, i.e., it serves as a proxy for the user's actual emotional responses. The emotional inference neural network 140 provides all of the inferred new emotional state 226 data, which serves as the current state data 516 used to train the DQNs 120. During initial training, the reward function 272 is also fed inferred new emotional state 226 data from the emotion inference neural network 140, eliminating the need for the user to provide final emotional state data (i.e., current emotional state data 212 provided at the end of the session). During initial training, each DQN 120 goes through a training regimen involving different combinations of input variables. The training regimen may be randomly generated, for example, by generating random values ​​for each input variable and repeating various sets of random inputs for a predetermined number of reinforcement iterations. In other embodiments, the training regimen may include, for example, a systematic exploration of different input values ​​covering the entire emotional space, where the valence and activation values ​​of both the initial and target emotional state inputs are changed by a predetermined amount after each reinforcement iteration. In some embodiments, the inputs to DQN 120 used during training may differ from those used during runtime, as described above; for example, rather than tracking the number of steps and progress toward a final target emotional state 214, DQN 120 may simply evaluate the accuracy of each audio segment selection step independently.

[0169] In some embodiments, retraining or updating of DQNs 120 during idle time can also occur using a newly updated emotion inference neural network 140. The emotion inference neural network 140 is updated after a user session using actual, updated current emotional state feedback from the listener, as described above. Once the emotion inference neural network 140 is updated, each DQN 120 can use system idle time when the user is not engaging with the system to retrain itself using the updated emotion inference neural network 140. The initial training regimen is reapplied to each DQN 120 to retrain each DQN 120 using the updated emotion inference neural network 140 as its environment simulation.

[0170] In embodiments using a different reinforcement learning approach in place of DQN 120, DQN 120 shown in FIG. 5 may be a different reinforcement learning algorithm representation, and all data provided as inputs 512, ... 514, ... 516 to input layer neurons 502 (e.g., step count values ​​or goal state values) may be treated as a vector of input data representing the state of the environment used by the selected reinforcement learning algorithm. This input vector would still be treated as data enabling the inference behavior of the algorithm replacing DQN 120. In some embodiments, the functions of DQN 120 and reinforcement learning agent 230 may be performed by a single model or algorithm that uses input vectors 512, ... 514, ... 516 to predict agent behavior 532, resulting in audio segment 230 as described above. The algorithm replacing DQN 120 and / or RL agent 230 may be a different DQN, a simple Q-table, an actor / critic model, or an environment model / planner implementation within a model-based RL algorithm. Different approaches may offer various advantages and disadvantages, but they can all theoretically provide equivalent results.

[0171] The use of physiological data 208 to determine the user's current emotional state 212 may employ sensors and techniques known in the field of affective computing, such as wearable sensors, cameras, and / or other sensors. Galvanic skin response (GSR), electroencephalography (EEG) signals, breathing patterns, heart rate, pupil dilation, subcutaneous blood flow, muscle tone, and other biological markers may correlate with various human emotional phenomena. These biological markers may be detected using biosensors, such as GSR sensors, breathing sensors, electrocardiogram or electroencephalogram electrodes, active or passive visual sensors using visible and / or invisible light spectrums, eye tracking systems, and electromyogram electrodes. One or more such sensors may be used with system 100 to collect physiological data 208, which may be processed by emotional music recommendation system 100 or by a process external to system 100 to identify the listener's emotional state. In some cases, the physiological data 208 is collected and / or processed by the listener device 190. The collection and / or processing of the physiological data 208 can be performed using techniques such as those described in Shu L, Xie J, Yang M, et al. A Review of Emotion Recognition Using Physiological Signals. Sensors (Basel). 2018;18(7):2074. Published 2018 Jun 28. doi:10.3390 / s18072074, which is incorporated by reference in its entirety. In other embodiments, such physiological data can be used as a separate input in addition to the emotional state data when other biological markers are linked to the desired outcome from use of the system. For example, in an embodiment in which the desired outcome includes a target emotional state and a target physiological outcome (i.e., a 10% heart rate reduction), the physiological data can also be used as a component representing the user's current state 516 and target state 514.

[0172] Other data collected from the listener can also be used to infer an emotional state. Camera data can be used to analyze facial expressions or other behavioral patterns that correlate with an emotional state or emotional response. Speech recordings or recordings can reveal patterns of prosody, intonation, or speech content that correlate with an emotional state or emotional response. In some embodiments, the listener device 190 or another process internal or external to the system 100 may be used to collect and / or process camera, speech, or other user data to assist in identifying the listener's current emotional state 212. Camera data showing the user's face can be analyzed for emotional state information according to techniques such as those described in Samara, A., Galway, L., Bond, R. et al. Affective state detection via facial expression analysis within a human-computer interaction context. J Ambient Intell Human Comput 10, 2175-2184 (2019) doi:10.1007 / s12652-017-0636-8, which is incorporated herein by reference in its entirety.

[0173] 6-10 illustrate exemplary user interface (UI) screens. These screens can be presented to the user on the touchscreen 194 of the listener device 190 and can serve to provide information to the user as well as collect user input. Referring to FIG. 6, a session initiation sequence 600 of UI screens is shown that is presented to the user to initiate a session of interaction with the emotional music recommendation system 100. A first screen 602 presents the user with a set of mode choices, each corresponding to a target emotional state 214. The modes shown in this screen 602 are "focused" (e.g., corresponding to high activity) and "calm" (e.g., corresponding to low activity). A "Train Your AI" input area is also presented that, once selected, can allow the user to enter the inference training process of the emotion inference neural network 140 as described above.

[0174] Once the user selects a target emotional mode, system 100 may use this input to determine the target emotional state 214 to be used during the user session. Next, the user is presented with a second screen 604, which allows the user to identify the nature of their current situation or environment, such as whether the environment is quiet and private, noisy and private, quiet and public, noisy and public, or changing over time due to movement. A third screen 606 allows the user to choose between a playlist mode and an immersive mode. In this screen 606, for example, the user may be presented with stream data 236 indicating a playlist (in playlist mode), or the stream data 236 may simply include the audio stream 234 itself and be able to transmit it directly to the listener device 190 for auditory presentation without further user interaction. In some embodiments, this mode selection screen 606 may allow the user to specify which subset of audio segments 220 from the music library 184 to use for the user session (and therefore which DQNs to select from DQNs 122, 124-126).

[0175] A fourth screen 608 allows the user to select a genre of music to use for the user session, thereby selecting or further narrowing the possible subset of audio segments 220 to use for the user session. A fifth screen 610 allows the user to select a duration for the audio stream 234. The user-selected duration may be used to determine the number of audio segments 230 included in the audio stream 234, which in turn determines how many segment identification steps DQN 120 will perform. In some embodiments, the user may also use this screen 610 to select an environmental track to determine the length of the session (i.e., the number of steps DQN 120 will take) or other audio characteristics of the audio stream 234.

[0176] The sixth screen 612 allows the user to identify their current emotional state 212. The illustrated embodiment uses a two-dimensional representation of an emotional space, with activity on the vertical axis and valence on the horizontal axis. The two-dimensional space is further formed with a number of regions corresponding to named emotional states, e.g., the green region in the upper right quadrant of the space is labeled "Energetic," the red region in the upper left quadrant is labeled "Tense," the white region in the lower left quadrant is labeled "Sad," the blue region in the lower right quadrant is labeled "Calm," the dark gray or black region in the center left region is labeled "Fear," and the gray region around the center is labeled "Neutral." The user can move a state indicator cursor, shown here as a white circle near the center of the space, to a location within the space corresponding to their self-rated current emotional state or mood. Some embodiments may also include additional inputs, such as an anxiety meter slider shown at the bottom of screen 612, to show a third dimension or additional data that complements the two-dimensional self-reported display of emotional state.

[0177] A seventh screen 614 allows the user to provide facial expression data to supplement, or as an alternative to, the self-reported data from the sixth screen 612. If the user chooses to take a photo, an eighth screen 616 is presented using the camera control interface of the listener device 190. Once the user has taken a photo of their face in the eighth screen 616, a ninth screen 618 shows the image annotated by the system's emotional analysis based, for example, on the user's facial expressions.

[0178] In some embodiments, self-reported data collected from user input, such as in sixth screen 612, can be used to train or calibrate a facial emotion analysis process or other process related to emotions associated with user data (e.g., physiological signals, speech data, etc., as described above).

[0179] Once the user's current emotional state 212 is identified based on input from the sixth screen 612 and / or the ninth screen 618, the system 100 generates the audio stream 234 and transmits the audio stream data 236 to the listener device 190 as described above. The user may be presented with a tenth screen 620 showing metadata corresponding to the first audio segment within a music control display at the bottom of the screen 620, here shown as the song title and duration of the song "Feeling Good," along with music playback controls. At the top and center of the screen 620, the user is provided with an updated current emotional state input area, as described above, to initiate updates of the user's current emotional state during the user session.

[0180] FIG. 7 illustrates an end-of-session sequence 700 of UI screens presented to a user to end a session of interaction with the emotional music recommendation system 100. The first screen 702 presents an emotional state input display for reporting the user's updated current emotional state at the end of the user session after listening to the audio stream 234. This updated current emotional state data is used to update the emotional inference neural network 140, as described above. The second screen 704 presents an option to supplement or replace the self-reported data from the first screen 702 with facial image data, which is collected and analyzed according to the same process as the seventh screen 614 through the ninth screen 618 before the session, resulting in the third screen 706. The fourth screen 708 may then present an emotional response result to the user based on the user's initially reported current emotional state 212 from screens 612 and 618 and the user's subsequently updated current emotional state from screens 620, 702, and 706. A graph at the top of screen 708 may show the user's emotional trajectory over the session and may include the identity and / or magnitude of one or more named emotional states over time. A series of bar graphs at the bottom of screen 708 may break down the user's emotional response by activity and emotional valence, and / or by self-report and facial analysis.

[0181] When a user session ends, the system 100 may use the idle time to retrain the DQN 120 as described above.

[0182] 8 shows a user customization sequence 800 of UI screens presented to a new user or to a user who chooses to update or augment their personal profile stored by the system. A first screen 802 prompts the user to provide personal data, such as date of birth and gender identification. A second screen 804 prompts the user to identify the frequency with which they experience each of a number of named emotional states. A third screen 806 prompts the user for the frequency of an additional set of named emotional states and also the frequency with which the user has trouble sleeping. A fourth screen 808 prompts the user to identify the frequency with which they experience stress or anxiety. The system may calibrate or customize the training and / or operation of various portions of the system 100 based on user input provided through sliders and other user input elements in this sequence 800. For example, a user who frequently feels sad but rarely feels energized may have the user's emotion inference neural network 140 calibrated to normalize the weights given to these states based on a baseline or average set of emotional state values ​​specific to the user. The system can also use this user-input data to advise the user on how to use the system to achieve the user's goals, such as mental health or mood management goals. Figure 9 shows an exemplary user profile data collection UI screen 902 presented to the user as part of the profile creation process. A progress bar at the top of the UI screen 902 indicates how far the user has progressed through the user profile creation process. The bottom of the UI screen 902 shows an input area for the user to indicate how upset the user typically feels.

[0183] FIG. 10 shows an exemplary context information collection UI screen 1002. Context information used as input to the neural network or reinforcement learning algorithms of FIGS. 4A-B and 5 can be collected using the context information collection interactions of this screen 1002. The top of screen 1002 shows a current emotional state input area as a two-dimensional emotional state indicator. The bottom of screen 1002 shows a current environment input area as a binary toggle indicating whether the current environment is quiet or noisy, and whether the current environment is private or public. Context information may be collected in association with a timestamp, which can be used by system 1000 to determine time of day.

[0184] In some embodiments, multiple systems 100, or multiple versions of various components of system 100 (e.g., DQN 120 or Affective Inference Network 140, etc.) may be used to interact with different users. Various systems 100 or components may be personalized by the user, and a storage mechanism may be used to store different systems or different models (e.g., 120 or 140) indexed by a user identifier.

[0185] FIG. 11 illustrates an exemplary multi-model system 1100. Each user of such a multi-model system 1100 can have their own unique model that can be trained to learn the unique dynamics of how the individual user responds to music. In the multi-model system 1100, the listener device 190 provides a user identifier 1104 (e.g., a user ID number) to a database 1102 that stores personalized machine learning models for individual users, such as personalized DQNs 120 and individually trained emotion inference algorithms 140. In some embodiments, users of such a system 1100 can similarly create their own unique collections (i.e., subsets 220) of audio segments 186, which can also be stored or identified by the database 1102. The DQNs 122, 124,...126 can be composed of behaviors (or subsets 220 of audio segments 186) specific to that user, or can be selections from that user's music library.

[0186] In some embodiments of the multi-model system 1100, the DQNs 120 and / or subset 220 of audio segments 186 can be selected from the database 1102 by collecting information about the user's baseline music preferences (e.g., as part of the user profile creation process described above with reference to FIGS. 8-9 ) and matching the user with the DQNs 120 and / or subset 220 of audio segments 186 that contain music that matches those preferences. Thus, as shown in FIG. 11 , a given multi-model system 1100 can be specific to a particular user and the particular subset 220 of audio segments 186 selected from the audio library 184. The subset 220 of audio segments 186 can be selected either based on the user's preferences (e.g., the user's music library) or based on an algorithm that determines which set is best for the user in the current context (e.g., based on the user's current emotional state 212, situation, and / or environment).

[0187] In some embodiments, individual personalization of the multi-model system 1100 occurs in several stages. A user can start with a model trained either based on all users or based on users with similar profiles (e.g., sharing their music tastes, baseline mood profiles, personality profiles, and / or demographics). A gated process may be used to have users start with these more general models and gradually train their own individual models. Once these personalized models reach a performance threshold, they will be replaced by the more general models for use at runtime.

[0188] In embodiments where personalized models are used, a music enjoyment meter can also be added to provide important input data in the DQN training process. If a user dislikes an audio segment, they can annotate the track representing that audio segment. A reward function 272 in the direct training process 270 can factor in either a reward or a penalty depending on the rating. This allows the system to quickly learn whether a user enjoys a given audio segment, which can have a significant impact on the effectiveness of the audio segment in eliciting a certain emotional state in the user.

[0189] 12 shows an exemplary user interface screen 1202 for rating music enjoyment that allows a user to provide input regarding the enjoyment of the current audio segment being played. The top of screen 1202 shows an emotional state update input area. The bottom of screen 1202 shows playback controls. The center of screen 1202 shows a music enjoyment input area where the user can move a slider to indicate the enjoyment level of the current audio segment being played.

[0190] The components of the multi-model system 1100 can enable deep and highly specific personalization over time, starting with contextual personalization, progressing to profile personalization (once enough users of each profile are active on the network), and then to unique personalization (once each individual user has logged enough sessions). Contextual personalization can include data such as initial emotional state, current music preferences, and time of day. Profile personalization can include data such as personality, demographics, and baseline music taste profiles.

[0191] In some embodiments, the output of DQN 120 and reinforcement learning agent 230 is a selection of MIR features from library MIR data 182 instead of a selection of audio segments 230 from audio library 184.

[0192] FIG. 13 illustrates an exemplary MIR feature prediction system 1300. In some such systems 1300, the output neurons 506 of the DQN 120 will include one neuron per MIR feature, which will be used to determine the level or value of each MIR feature. The set of DQN 120 outputs can represent MIR vectors 1302 that can then be matched with audio segments 230 (e.g., audio segments that best fit the MIR features in a given library, such as a user-selected library 1306) as part of the agent behavior 532 by sending a query 1304 containing the MIR vectors 1302 to a library 1306. These MIR features (e.g., MIR vectors 1302) may then be provided to the emotion inference process 224 and / or the MIR features may be directly annotated by the user during the direct training process 270. In such an embodiment, the user may have only one DQN (because the DQN is currently determining the combination of MIR features), but instead may have one or more libraries of audio segments 186 created by the user based on the requirements of the user's current musical taste, a baseline profile, or a library (such as library 1306) from which the user can select audio segments 230 based on their compatibility with the MIR features.

[0193] 14 shows the steps and components of an exemplary model database generation process 1400 for creating a model database 1402 of emotion inference models (e.g., AINN 140) and / or deep Q-networks 122, 124, 126 specific to a given listener. The model database 1402 can be used, for example, to generate and train models for each user of the multi-model system 1100.

[0194] The model database generation process 1400 includes two stages: a generalist model training process 1450 and a personalized model training process 1460. The model database 1402 is formed of a generalist model set 1432 and a personalized model set 1434. Each set of models 1432, 1434 may include, for example, an AINN 140 and multiple DQNs 122, 124, 126. The generalist model 1432 may be used to recommend music in inference mode until the personalized model 1434 achieves a performance threshold, at which point the system (e.g., the multi-model system 1100) can switch to using the personalized model 1434 to recommend music to the user.

[0195] The generalist model training process 1450 begins by extracting a subset of music tracks from the music catalog 1420 based on the curation process 1422 using the music tracks' MIR data. The music catalog 1420, in some embodiments, may be an audio library 184 of music tracks. The music tracks, in some embodiments, may be audio segments 230. The MIR data associated with each music track may, in some embodiments, be stored as library MIR data 182 within the music catalog 1420. In other embodiments, the MIR data associated with the music tracks may be extracted from the music tracks using, for example, the MIR extraction process 225.

[0196] Thus, in some embodiments, the curation process 1422 requires a labeled dataset (i.e., the audio library 184) that describes thematic MIR metadata associated with each track. For example, the music catalog 1420 might include a set of jazz music from the 1920s that is related to love, and / or a set of rock music from the 1960s that is generally gentle and energetic. The curation process 1422 can be performed algorithmically using any of several known computational methods, such as, for example, a supervised clustering algorithm.

[0197] The subset of music tracks generated by the curation process 1422 is used in groups by the DQN process 1424 to generate groups of music tracks associated with each generalist deep Q-network to be trained. The relationship between audio segments 230 with particular MIR features and a given DQN is described above with reference to FIG. 2A.

[0198] The generalist model training process 1450 also uses existing user data 1426 from the entire user population collected from listening devices 190, as described with reference to FIGS. 2A-2B above. The existing user data 1426 may include user profile data, records of users listening to music that correlates with the users' emotional responses, etc. The user grouping process 1428 may cluster or segment the existing user data 1426 to identify types of users or groups of user types. An example of identifying user groups or types is described in Feiyun Zhu, Jun Guo, Zheng Xu, Peng Liao, Junzhou Huang, “Group-driven Reinforcement Learning for Personalized mHealth Intervention,” 2017, arXiv:1708.04001, https: / / arxiv.org / abs / 1708.04001, which is incorporated herein by reference in its entirety.

[0199] The generalist model transfer learning process 1408 is used to train one or more sets of generalist models 1432. The generalist model transfer learning process 1408 may include an affective inference neural network (AINN) generalist model training process 1430 that trains the generalist AINN 140, and a deep Q-network (DQN) generalist model training process 1431 that trains the set of generalist DQNs 122, 124, 126. Various transfer learning techniques are known in the field of machine learning. An example of a transfer learning process is described in Kieran Woodward and Eiman Kanjo and David J. Brown and T.M. McGinnity: “On-Device Transfer Learning for Personalizing Psychological Stress Modeling using a Convolutional Neural Network”, 2020, arXiv:2004.01603, https: / / arxiv.org / abs / 2004.01603, which is incorporated herein by reference in its entirety.

[0200] The AINN generalist model training process 1430 may use the AINN 140 for users in various user groups (i.e., the user groups output by the user grouping process 1428) to train generalist AINNs 140 for a set of generalist models 1432 using transfer learning. The DQN generalist model training process 1431 may use the generalist AINNs 140 for the generalist models 1432 and the DQN-grouped music tracks output by the group by DQN process 1424 to train generalist DQNs 122, 124, 126 for the set of generalist models 1432 using transfer learning. For example, a particular user group identified in the user grouping process 1428 may be associated with a particular generalist AINN 140, and existing user data 1426 from those users may be used to train a given generalist AINN 140, while a particular subset of music tracks associated with a user group may be associated with a particular generalist DQN, and the generalist AINN 140 for that user group may be used to train a given DQN.

[0201] The personalized model training process 1460 begins with the addition of a new user 1404 to the system 1100. The new user 1404 may be presented with a baseline creation process 1406 that initializes a user profile, for example, using the user profile generation process described above with reference to Figure 11. The baseline creation process 1406 may also prompt the user for input to help identify the new user's 1404 musical preferences; for example, the new user 1404 may be prompted to fill out a baseline profile and / or the system 1100 may have the new user 1404 listen to music clips in rapid succession to understand the new user's 1404 musical preferences and / or behavioral responses.

[0202] A matching process 1410 is performed based on the music preferences, personality, user profile, and / or other information collected during the baseline creation process 1406. The matching process 1410 may, in some embodiments, be performed algorithmically using machine learning techniques or other ranking or matching algorithms. A set of generalist models (e.g., AINN 140 and DQN 122, 124, 126) is selected from the database 1102 that are good matches for the new user 1404, for example, based on the output of the baseline creation process 1406. A labeling process 1436 is used to facilitate the matching process by labeling the generalist models 1432 based on the user group (from the user grouping process 1428) and the music track subset (from the group by the DQN process 1424). The label data used by the labeling process 1436 can be any type of discriminant vector. A matching process 1410 can match a new user 1404 to a particular user group based on data (e.g., user profile data) output by the baseline creation process 1406, and a set of generalist models 1432 can be selected based on the matching and used to initially form the model database 1402.

[0203] A set of personalized models 1430 is then generated for the new user 1404. Initially, the personalized models 1430 may be copies of the generalist models 1432 selected for the new user 1404 based on the matching process 1410. However, the personalized models 1430 are trained using data from the new user 1404, thereby personalizing them and improving their accuracy until they reach a performance threshold and are used by the system 110 for inference as described above.

[0204] In the music selection process 1412, the new user 1404 manually creates a subset of music that they want curated by the system 1100. The music selection process 1412 creates a user music catalog 1414, for example, by allowing the new user 1404 to identify music tracks from a local or remote catalog or library of music tracks.

[0205] The grouping by DQN process 1416 of the personalized model training process 1460 operates similarly to the grouping by DQN process 1424 of the generalist model training process 1450 described above. A subset of music tracks from the user music catalog 1414 is identified and used to group the music tracks in association with each of the deep Q-networks of the personalized model 1430 to be trained.

[0206] Data collected from new users 1404 is used to train personalized models 1430, particularly the AINN 140 of the personalized models 1430, after each user experience using a personalized model transfer learning process 1418, similar to the generalist model transfer learning process 1408 described above. The personalized model transfer learning process 1418 may also use data collected from the baseline creation process 1406. Factors used by the personalized model transfer learning process 1418 are described below with reference to FIG. 15.

[0207] The DQN personalized model training process 1417 operates similarly to the DQN personalized model training process 1417 described above to train the DQNs 122, 124, 126 of the personalized model 1430 using the personalized AINN 140 of the personalized model 1430.

[0208] Figure 15 shows personalization factors 1500 used in the personalization model transfer learning process 1418 of Figure 14. The goal of the personalization model transfer learning process 1418 is to work from the top layer of the pyramid 1502, which corresponds to contextual personalization 1504, through the middle layers of the pyramid 1502, which corresponds to profile personalization 1506, until it reaches the bottom layer of the pyramid 1502, which corresponds to an individualized model 1508 for each user (e.g., new user 1404).

[0209] Contextual personalization 1504 may include personalization of models (e.g., AINN 140 and DQN 122, 124, 126) based on an initial state (e.g., the user's emotional state), the user's currently expressed music preferences, or other environmental variables (e.g., the time of day). Thus, contextual personalization 1504 may correspond to factors taken into account in traditional music recommendation systems. Profile personalization 1506 may include personalization of models based on the user's personality, demographic membership, baseline music taste profile, and other general user characteristics. In some embodiments, profile personalization 1506 may be based on the user's membership in user groups determined in user grouping process 1428. Individual model 1508 is an individual, personalized model trained using a particular user's emotional responses to music to accurately infer that particular user's emotional responses, for example, as described above with reference to FIGS. 2A-2B .

[0210] Thus, a hierarchy of personalization models 1430 can be selected and trained using the model database generation process 1400, ranging from slightly personalized using contextual personalization 1504, to more detailed personalized using profile personalization 1506, to very detailed personalized using individual models 1508.

[0211] Individual components of the systems described herein may be used for applications beyond emotional music recommendation. The emotional inference neural network 140 may be used in the fields of music analysis, marketing, or insights, allowing novel content to be evaluated based on its predicted emotional impact. For example, the inputs and outputs of the emotional inference neural network 140 may be switched to provide a set of MIR features likely to induce a target mood state given a particular user profile. Data generated by the various systems described herein may be useful for various purposes within the music industry, such as providing new insights into how the human mind responds to music, which may be further categorized by demographic groups, user profile groups, and various preference profiles. With sufficient data, the emotional inference capabilities of the systems described herein may be able to create new metadata about musical compositions, thereby increasing reliability for functional applications (e.g., movie synchronization, music therapy, etc.).

[0212] Examples of methods, systems, and non-transitory media for the composition of emotional music will now be described with reference to Figures 16-30. The described music composition systems can be used at one or more stages of the music composition process to generate MIR blueprints, scores, composition lead sheets, production lead sheets, mixes, and / or masters of songs intended to induce changes in emotional states in listeners. Various components described above in the context of emotional music recommendation, such as emotional inference models (e.g., AINN) and MIR extraction processes, may be used in various embodiments of the music composition systems described herein.

[0213] Figure 16 shows an exemplary emotional music composition system 1600. The emotional music composition system 1600 may be implemented using a computing platform or system with hardware and software components very similar to the emotional music recommendation system 100 of Figure 1. The example shown includes a storage device 1638 in communication with a processor system 1640, which in turn is in communication with a user interface 1642 used to interact with a user.

[0214] The storage device 1638 stores the following software and data used to implement the methods and processes for emotional music composition described herein: software instructions 1610 executed by a processor system 1640 to implement the techniques described herein, including training and operating various models using machine learning techniques; multiple MIR generator generative adversarial networks (GANs) used to generate MIR data for various listener profiles, genres, and / or styles, shown as a first MIR generator GAN 1612 through an Nth MIR generator GAN 1614; and multiple score generator GANs shown as a first score generator GAN 1616 through an Nth score generator GAN 1618. The set of models includes a plurality of score generator GANs used to generate scores for various genres and / or styles, shown as first emotional inference neural network 1620 through Nth emotional inference neural network 1622, a plurality of emotional inference models used to predict emotional responses for various listener profiles, and a plurality of adaptive musical agent models used to adapt an existing track or audio stem library 1628 to achieve target emotional states or trajectories for various listener profiles, genres, and / or styles, shown as first adaptive musical agent model 1624 through Nth adaptive musical agent model 1626. Of course, while each set of models is shown as including N models, the value of N may vary from set to set.

[0215] The storage device 1638 also stores data including an audio stem library 1628 , one or more MIR generator training databases 1630 , a score generator training database 1632 , a mastering agent training database 1634 , and an adaptive music agent training database 1636 .

[0216] The relationship of the various software and data components of the emotional music composition system 1600, and their interaction with a user via the user interface 1642, will now be described with reference to Figures 17A-D. The operation of the various subsystems shown in Figures 17A-D will then be described with reference to Figures 18-26.

[0217] FIG. 17A illustrates the relationships among processes in the first portion 1700a of the exemplary emotional music composition system 1600 of FIG. 16, including an MIR generator process 1900. The MIR generator process 1900 is used to generate an MIR blueprint for an audio segment (e.g., a song) intended to elicit a particular emotional response in a listener. The MIR blueprint generated by the MIR generator process 1900 typically identifies MIR features for the song as a whole, as well as MIR features for each of multiple epochs (i.e., temporal subsegments) of the audio segment, that will elicit the desired emotional response. A user 1702 (who may be an artist, composer, or other user involved in composing or producing music) interacts with the system 1600 via a user interface 1642. The user 1702 can interact with a composition intention setting process 1704, which informs the various subsystems of the system 1600, described below, about the desired characteristics of the musical data to be generated during the composition process. The MIR generator process 1900 shown in FIG. 17A can be managed by providing information such as, for example: Does the user 1702 want to create a complete audio segment 230 (e.g., a musical track) or simply a stem (i.e., a grouping of similar instruments, such as the string component or lead vocal component of a song)? How long does the user 1702 want the musical track or stem to be? The composition intention setting process 1704 determines any conditions 1706 used to optionally constrain the MIR generation, such as a specified key, tempo, and / or song structure. The composition intention setting process 1704 also determines the segment length 1708 (e.g., the length of the musical track or stem).

[0218] The user 1702 may also interact with an emotional intention setting process 1710, which informs the various subsystems of the system 1600, described below, about the desired emotional state or trajectory to be evoked in a listener of the musical data to be produced during the composition process. The emotional intention setting process 1710 may generate target emotional state data 1712 that indicates how the user 1702 wants a listener to feel after listening to the music produced during the composition process.

[0219] The optional conditions 1706, segment length 1708, and target emotional state data 1712 are used as inputs to an MIR generation process 1900, which is described in more detail below with reference to FIG.

[0220] The MIR generator process 1900 includes an MIR generator GAN, shown here as the first MIR generator GAN 1612. In some embodiments, the MIR generator process 1900 is implemented as a conditional GAN ​​with a control network, where any conditions 1706, segment length 1708, and target emotional state data 1712 are used as conditional inputs to the MIR generator GAN 1612. The target emotional state data 1712 may also be used as an input to the control network, as described in more detail below.

[0221] The MIR generator GAN 1612 is trained by a MIR generator GAN direct training process 1714 using a database 1716 of MIR data labeled with emotional state data and optionally other conditions as training data. The database 1716 of MIR data may be formed or otherwise generated using various data sources, such as existing user data 1426 collected from listening devices 190 of a population of users (described above with reference to FIG. 14 ), and / or existing data records 1718 that associate MIR data with emotional responses and / or other conditional data (e.g., any conditions 1706, such as key, tempo, and / or song structure). In some embodiments, the existing user data 1426 may be collected using the affective music recommendation system 100 as described above.

[0222] The MIR generator GAN 1612 is also trained by the MIR generator GAN inference training process 1720 using the composition AINN 1800, as described below with reference to FIG. 18.

[0223] The MIR generator process 1900 generates an MIR blueprint 1730, which in some embodiments may include a Mel-Frequency Cepstrum (MFC) spectrogram, potentially along with other MIR features. The MIR blueprint 1730 may be used by other components of the affective music composition system 1600 to assist in the generation of musical data (e.g., scores, tracks, or stems).

[0224] 18 shows a simplified recurrent neural network representing a composition AINN 1800 used by the emotional music composition system 1600. The composition AINN 1800 operates very similarly to the AINN 140 described above with reference to FIG. 4B. However, instead of using an individual user's current emotional state 212 as input, the composition AINN 1800 is trained using training data from a database of MIR data 1716, including MIR data labeled with emotional response data as well as contextual information 432 associated with the MIR data. Thus, the composition AINN 1800 operates in inference mode to predict a generic user's emotional response to a set of MIR features (including global MIR features 433 as well as epoch-specific MIR features of a time series 434) regardless of the user's current emotional state.

[0225] 19 illustrates an exemplary MIR generator process 1900 for use in the emotional music composition system 1600. The MIR generator process 1900 may be configured as an MIR generator GAN (consisting of a generator network 1902 and a probabilistic network 1904) with a control network 1906, as described in Cong Jin, Yun Tie, Yong Bai, Xin Lv, Shouxun Liu, “A Style-Specific Music Composition Neural Network,” 9 June 2020, Neural Processing Letters (2020) 52:1893-1912, https: / / doi.org / 10.1007 / s11063-020-10241-8 (hereinafter “Jin”), which is incorporated herein by reference in its entirety. The MIR generator GAN itself, and in particular the generator network 1902, may be configured as a conditional GAN, as described in Yi Yu, Simon Canales, “Conditional LSTM-GAN for Melody Generation from Lyrics,” 2019, arXiv:1908.05551, https: / / arxiv.org / abs / 1908.05551, which is incorporated herein by reference in its entirety. The generator network 1902 comprises a generator neural network 1907, and the probability network 1904 comprises a discriminator neural network 1908, where each neural network 1907, 1908 may be a recurrent neural network (RNN) with long short-term memory (LSTM), a convolutional neural network, a standard multilayer perceptron neural network, or some other type of neural network or machine learning model. The functionality of the MIR generation process 1900 can also be achieved by other generative deep learning modalities, such as variational autoencoders (VAEs) or simply recurrent neural networks (RNNs) themselves. GAN models have been evaluated as an effective means of performing the required functionality, but additional similar algorithms may also be effective, especially with advances in machine learning.

[0226] The MIR generator process 1900 operates in a training mode, or MIR blueprint generation mode, which has two training phases. In the first training phase of the training mode, the classifier neural network 1908 of the probabilistic network 1904 is trained to recognize MIR blueprints using actual stored MIR blueprints 1910 from the database of MIR data 1716 (i.e., MIR blueprints corresponding to actual songs composed by human artists).

[0227] In the second training phase, the generative network 1902 receives as inputs (i.e., conditions) from a database of MIR data 1716: arbitrary conditions 1706, segment lengths 1708, and target emotional state data 1712. The generative network 1902 includes a generative neural network 1907 configured to generate MIR blueprints (shown as generated MIR blueprints 1912). At the start of the training process, each generated MIR blueprint 1912 is essentially random data. However, a probabilistic network 1904 is used to provide feedback to the generative network 1902, mediated by a control network 1906, to train the generative network 1902 to generate increasingly plausible MIR blueprints. A classifier neural network 1908 compares each generated MIR blueprint 1912 to actual stored MIR blueprints 1910 from the database of MIR data 1716. The result of this comparison is a probability value 1916 (e.g., a value between 0 and 1) that indicates the likelihood of the classifier neural network's 1908 inference as to whether the generated MIR blueprint 1912 is an MIR blueprint for an actual song.

[0228] The control network 1906 comprises an emotional inference model (denoted as the composition AINN 1800) and a critic 1914 (as described in Jin). The critic 1914 operates to constrain the generated MIR blueprints 1912 to those that meet target emotional criteria. Thus, in the second training phase, the critic 1914 receives three inputs: a probability value 1916 generated by the probabilistic network 1904 based on the generated MIR blueprint 1912; a predicted emotion 1920 (i.e., the listener's predicted emotional response to music that matches the MIR blueprint) generated by the composition AINN 1800 based on the generated MIR blueprint 1912; and the target emotional state data 1712 received from the database of MIR data 1716. The critic 1914 applies a reward function to the probability values ​​1916, the predicted emotions 1920, and the target emotional state data 1712 to generate a reward 1918 based on how plausible the generated MIR blueprint 1912 is and how likely it is to satisfy the target emotional state data 1712. The reward 1918 is used as feedback to train the generator neural network 1907, thereby improving its ability to generate plausibly generated MIR blueprints 1912 that satisfy the target emotional state data 1712 specified by the reward function. The reward function of the critic 1914 is driven by an emotional inference model, such as the composition AINN 1800.

[0229] The generative network 1902 and the probabilistic network 1904, aided by the control network 1906, thus jointly construct an MIR generator GAN (such as the first MIR generator GAN 1612). In the MIR blueprint generation mode, the arbitrary conditions 1706, segment lengths 1708, and target emotional state data 1712 are provided by the user 1702 via the composition intention setting process 1704 and the emotional intention setting process 1710, instead of being provided by a database of MIR data 1716. The generator network 1902 is used to generate a generated MIR blueprint 1912 (e.g., the MIR blueprint 1730 of FIG. 17A ) that matches the target emotional state data 1712.

[0230] Figure 17B illustrates the relationships among the processes of the second portion 1700b of the exemplary emotional music composition system 1600, including the score generator process 2000. Composing music refers to writing a musical piece, and the ultimate result of composition is the production of a score—a document that specifies which notes should be played at which times, on which instruments, and in which manner to create the audio characteristics of a musical recording. As shown in Figure 17A, a user 1702 can interact with a composition intention setting process 1704 to manage the score generator process 2000 by providing, for example, the following information: Does the user 1702 want a harmonic progression, melody, rhythm, or full score? What instruments does the user 1702 want? How long does the user 1702 want the score to be? Thus, the composition intention setting process 1704 determines the score type 1732, instruments 1734, and score length 1736. The score type 1732 may specify rhythm, harmonic progression, and / or melody.

[0231] The score generation process 2000 receives as conditional inputs a score type 1732, an instrument 1734, and a score length 1736. The score generation process 2000 also receives a set of MIR data 1738 required for the score to be generated, which is used as a conditional input and / or input to the control network of the score generation process 2000. The set of required MIR data 1738, in some embodiments, can be extracted from the MIR blueprint 1730 generated by the MIR generation process 1900, or can be extracted from another MIR blueprint or source of MIR data. In some embodiments, the set of required MIR data 1738 used by the score generation process 2000 may be only a portion of the MIR data contained in the MIR blueprint 1730, while in other embodiments, it may include all of the MIR data contained in the MIR blueprint 1730.

[0232] The score generation process 2000 includes a score generator GAN, which is denoted herein as first score generator GAN 1616. The score generation process 2000 operates on its input to generate as output a score 1742, such as a Musical Instrument Digital Interface (MIDI) score. The operation of the score generation process 2000 on its input to generate its output is described in more detail below with reference to FIG. 20.

[0233] The score generation process 2000 is trained in training mode by the score generator training process 1744 using labeled score data provided by a database of labeled score data 1746. The labeled score data may include human-composed scores labeled with MIR blueprints and / or other conditional inputs to be used in training the score generator process 200 and its score generator neural network (e.g., first score generator neural network 1616).

[0234] FIG. 20 illustrates an exemplary score generator process 2000, as described with reference to FIG. 17B above. The score generator process 2000 can be configured similarly to the MIR generator process 1900, i.e., a conditional GAN ​​comprising a generator network 2002 and a probabilistic network 2004, with a control network 2006. The generator network 2002 comprises a generator neural network 2007, and the probabilistic network 2004 comprises a discriminator neural network 2008, each of which may be a recurrent neural network (RNN) with LSTM (long short-term memory), a convolutional neural network, a standard multilayer perceptron neural network, or some other type of neural network or machine learning model. The functionality of the score generation process 2000 can also be achieved by other generative deep learning modalities, such as a variational autoencoder (VAE) or simply the recurrent neural network (RNN) itself. GAN models have been evaluated as an effective means of performing the required functionality, but additional similar algorithms may also be effective, especially as machine learning advances.

[0235] The score generator process 2000 operates in a training, or score generation, mode, which has two training phases. In the first training phase of the training mode, the classifier neural network 2008 of the probabilistic network 2004 is trained to recognize scores using actual stored scores 2010 (i.e., scores composed by human artists) from a database of labeled score data 1746.

[0236] In the second training phase, the generative network 2002 receives as input (i.e., conditions) any required MIR data 1738, score length 1736, instrument 1734, and score type 1732 from the database of labeled score data 1746. The generative network 2002 includes a generative neural network 2007 configured to generate scores (shown as generated scores 2012). At the start of the training process, each generated score 2012 is essentially random data. However, a probability network 2004 is used to provide feedback to the generative network 2002, mediated by a control network 2006, to train the generative network 2002 to generate increasingly plausible scores. A classifier neural network 2008 compares each generated score 2012 to actual stored scores 2010 from the database of labeled score data 1746. The result of this comparison is a probability value 2016 (eg, a value between 0 and 1) that indicates the likelihood of the classifier neural network's 2008 inference as to whether the generated score 2012 is the actual score.

[0237] The control network 2006 includes a critic 2014. The critic 2014 operates to restrict the generated scores 2012 to those that satisfy a set of music theory rules 2020, which may be stored, for example, as a database of music theory rules. Thus, in the second training phase, the critic 2014 receives three inputs: probability values ​​2016 generated by the probabilistic network 2004 based on the generated scores 2012, the music theory rules 2020, and the required MIR data 1738 provided by the database of labeled score data 1746. The critic 2014 applies a reward function to the probability values ​​2016, the music theory rules 2020, and the required MIR data 1738 to generate a reward 2018 based on how plausible the generated score 2012 is, how likely it is to satisfy the music theory rules 2020, and how close the generated score 2012 is to matching the set of MIR data (i.e., the required MIR data 1738 provided by the database of labeled score data 1746). The reward 2018 is used as feedback to train the generator neural network 2007, thereby improving its ability to generate plausibly generated scores 2012 that satisfy the music theory rules 2020 and match the required MIR data 1738.

[0238] The generative network 2002 and the probabilistic network 2004, aided by the control network 2006, thus jointly constitute a score generator GAN (such as the first score generator GAN 1616). In score generation mode, the score length 1736, the instruments 1734, and the score type 1732 are provided by the user 1702 via the composition intention setting process 1704, and optionally the required MIR data 1738 is provided by the MIR blueprint 1730 via the required MIR data extraction process 1740 instead of being provided by a database of MIR data 1716. The generator network 2002 is used to generate a plausibly generated score 2012 (e.g., score 1742 in FIG. 17A ) that matches the MIR blueprint 1730 and follows music theory rules 2020.

[0239] Returning to FIG. 17B , the score 1742 generated by the score generator process 2000 may be used as a human artist's score for a subsequent music creation process. However, the score 1742 may also be used by subsequent processes in the emotional music composition system 1600 to automate one or more of the subsequent steps of music creation. In some embodiments, a user (e.g., an artist or producer interacting with the system 1600 or another user that generates the score 1742) can interact with the music composition process 1748 to generate a rough mix 1750 of a musical track or stem based on the score 1742 and / or a composition lead sheet 2400. The composition lead sheet 2400 may be generated by a composition lead sheet process 2300 based on the MIR blueprint 1730. The composition lead sheet process 2300 is described in more detail below with reference to FIG. 23 , and an exemplary composition lead sheet 2400 is described with reference to FIGS. 24A-B.

[0240] A user 1702 can interact with the music composition process 1748 via a user interface 1642, such as a digital audio workstation (DAW) or other audio workstation.

[0241] Figure 23 shows a composition lead sheet process 2300 used in the emotional music composition system 1600 of Figure 16 and shown in Figure 17B. The composition lead sheet process 2300 performs data transformations 2302 on the MIR blueprint 1730 to generate a composition lead sheet 2400.

[0242] Figure 24A is an exemplary emotional space indicator 2420 for a composition lead sheet generated by the composition lead sheet process of Figure 23. The emotional space indicator 2420 is similar to the two-dimensional visual indicator of emotional state used by the UI screens of Figures 6, 7, 10, and 12, i.e., the indicator identifies the emotional state 2424 of the composition lead sheet (based on the MIR blueprint 1730) located within a two-dimensional emotional space where the horizontal axis 2430 is valence (positive to the right) and the vertical axis 2428 is activity (higher toward the top). Multiple moods are shown as areas such as tenderness 2422 and calmness 2426. In this example, the emotional state 2424 of the composition lead sheet is located between tenderness 2422 and calmness 2426, indicating very positive valence and slightly lower activity.

[0243] Figure 24B shows an exemplary composition lead sheet 2400 generated by the composition lead sheet process 2300 of Figure 23. The composition lead sheet 2400 indicates various types of information related to the composition process to be performed to generate a musical segment (e.g., a musical track or stem).

[0244] Various fields indicate characteristics of the musical segment as a whole. Mix quality field 2402 provides information related to mix quality, shown here as the text "Instrumentation choices and performance should provide a wide spectral range (EQ) with a strong low-frequency presence." Tempo field 2412 provides information regarding the musical segment length, shown here as the text "111." Meter field 2414 provides information regarding the musical segment length, shown here as the text "Simple." Composition field 2416 provides information regarding the musical segment length, shown here as the text "5 sections." Key field 2418 provides information regarding the musical segment length, shown here as the text "A major." Key clarity field 2420 provides information regarding the musical segment length, shown here as the text "Low." Harmonic complexity field 2422 provides information regarding the musical segment length, shown here as the text "High." The harmonic balance field 2424 provides information regarding the length of the musical segment, shown here as the text "Minor bias." The dissonance field 2426 provides information regarding the length of the musical segment, shown here as the text "Medium."

[0245] Several time series of musical segment data show various characteristics of the musical segment at epochs 2442, 2444, 2446, 2448, 2450, 2452, 2454, or sub-segments, at different points within the musical segment, from the first epoch 2442 through the seventh epoch 2454. Rhythmic complexity time series 2434 plots the rhythmic complexity of each epoch from complex (high) to simple (low). Pitch center time series 2436 plots the pitch center of each epoch from high (high) to low (low). Intensity time series 2438 plots the intensity of each epoch from high (high) to low (low). Density time series 2440 plots the density of each epoch from thick (high) to thin (low).

[0246] The composition lead sheet 2400 may also include an emotional space indicator 2420 or other emotional state data, as described with reference to FIG. 24A.

[0247] 17B, ​​a user 1702 interacting with a music composition process 1748 can rely on the information provided in the composition lead sheet 2400 to guide the creation of a rough mix 1750, with or without the aid of a score 1742. Alternatively, the user 1702 can rely on the score 1742 itself in creating the rough mix 1750.

[0248] The rough mix 1750 may be further refined by subsequent operation of the system 1600, as described with reference to the third portion 1700c and fourth portion 1700d of the system 1600 shown in Figures 17C-D.

[0249] 17C illustrates the relationships among the processes of the third portion 1700c of the exemplary emotional music composition system 1600, including the automatic mastering process 2100. Mastering refers to the setting of sound levels and other sound equalization settings for the various instrument tracks (i.e., stems) to produce a master recording (also called a master), i.e., the final, official recording of the song ready to be presented to an audience. The master is used as the source from which all subsequent copies of the song are made. A user 1702—who may be the same user or a different user as the user 1702 interacting with the music composition process 1748, the composition intention setting process 1704, and / or the emotional intention setting process 1710—interacts with the music production process 1758 to produce a rough mix 1750 or a final mix 1760. As with the music composition process 1748, the user 1702 may interact with the music production process 1758 via a user interface 1642, such as a digital audio workstation (DAW) or other audio workstation, and the user 1702 may be assisted by a production lead sheet 2600 (described below with reference to FIG. 26 ) and / or an existing rough mix 1750 (such as the rough mix 1750 generated by the music composition process 1748).

[0250] The production lead sheet 2600 may be generated by the production lead sheet process 2500 (described below with reference to FIG. 25) based on an MIR blueprint 1730, such as an MIR blueprint 1730 generated in an earlier portion 1700a, 1700b of the system 1600.

[0251] The automatic mastering process 2100 can be used to automate the mastering stage of music production. The automatic mastering process 2100 includes a mastering agent 2102, which may be implemented using a reinforcement learning model, as described in more detail below with reference to FIGS. 21A-B. The automatic mastering process 2100 may perform automated mastering on either a final mix 1760 generated by the music production process 1758, an existing track in stereo format 1762 (for remastering an existing track), or an adapted final mix 1790 (described below with reference to FIG. 17D). Based on its inputs, the automatic mastering process 2100 can generate a final master 1770, which may be saved as a final master music track (i.e., audio segment) in the audio library 184. In some implementations, the automatic mastering process 2100 may be used mid-production. That is, the user 1702 can interact with the music production process 1758 to cause the automated mastering process 2100 to perform one or more iterations of the mastering process on a human-composed piece of music that has reached the mastering stage.

[0252] The mastering agent 2102 may be trained using a mastering agent training process 1754, which uses a database of mastering criteria and / or rules 1752 to provide training data. The operation of the mastering agent training process 1754, and the automatic mastering process 2100, is described in more detail with reference to Figures 21A-B below.

[0253] As in the second portion 1700b of the system 1600, a required MIR data extraction process 1756 can be used to extract from the MIR blueprint 1730 those portions of the MIR data, required MIR data 1757, required by the automated mastering process 2100, and provide the required MIR data 1757 therein.

[0254] Figure 21A is a block diagram illustrating a first portion 2100a of an automatic mastering process 2100 for use in the emotional music composition system of Figure 16. The automatic mastering process 2100 operates over multiple iterations, also referred to herein as episodes. The original audio segment 2104 received as input (i.e., the final mix 1760, the existing track 1762 in stereo format, or the adapted final mix 1790) is decomposed into n epochs in an epoch division process 2106. Each original (pre-mastered) epoch 2108 is sent to an epoch mastering process 2152 for one or more mastering passes (called "passes" or "iterations"). In each pass, the epoch mastering process 2152 applies mastering behaviors (described below) to the epoch 2108 until the epoch 2108 meets the MIR goal for that epoch. Once the first epoch 2108 meets its respective MIR target, mastering of the epoch 2108 is considered complete, and the next epoch 2108 undergoes the epoch mastering process 2152 one or more times until it meets its respective MIR target. This cycle continues until each epoch 2108 of the segment 2104 has been mastered.

[0255] MIR features are extracted from each original audio epoch 2108 by an MIR extraction process 2110, providing a reference for the MIR features at the start of that epoch (i.e., pre-mastering). The extracted MIR features are referred to as the current MIR epoch 2112. The current MIR epoch 2112 may also take into account additional MIR information generated by an MIR extraction, prediction, or reference process 2124 performed on the mastered audio epoch (described in more detail below with reference to FIG. 21B). The MIR extraction, prediction, or reference process 2124 may extract MIR data via a tool such as an MIR toolbox, may reference MIR data from a table of previously extracted MIR features, or may predict MIR data using a model that receives raw audio as input and outputs MIR data predictions.

[0256] The MIR features required by the automatic mastering process 2100 are received as required MIR data 1757. The required MIR data 1757 is divided into epochs (i.e., periods) of MIR data corresponding to the MIR features required per epoch of the mastered audio segment by an MIR epoch division process 2114. These epochs of MIR data are called target MIR epochs 2116, which indicate the MIR feature targets of the mastering process for a given epoch. To maintain the same timeline throughout the mastering process, the epoch size is synchronized between the epoch division process 2106 and the MIR epoch division process 2114.

[0257] A short-term reward network 2120 is used to generate a short-term reward 2122 for training the mastering agent 2102. The short-term reward 2122 is used in the reinforcement learning process of training the mastering agent 2102. The short-term reward network 2120 includes a critic 2118 that applies a reward function to three inputs: a target MIR epoch 2116 that indicates the desired MIR features in the current epoch, a current MIR epoch 2112 that indicates the pre-mastered MIR features for the current epoch, and mastering criteria and / or rules from the mastering criteria and rules database 1752. The reward function applied by the critic 2118 generates a short-term reward 2122 that is constrained by the mastering rules and / or criteria and is based on whether the current MIR epoch 2112 satisfies the MIR features of the target MIR epoch 2116.

[0258] The short-term reward 2118 is provided as input to the mastering agent 2102, which may be implemented as a reinforcement learning agent, such as a branching recurrent deep Q-network (DQN). The branching recurrent DQN may include an LSTM to implement a memory of mastering actions executed for each epoch. An example of a branching Q-learning network is described in Arash Tavakoli, Fabio Pardo, and Petar Kormushev, “Action Branching Architectures for Deep Reinforcement Learning”, 2018, arXiv:1711.08946, https: / / arxiv.org / abs / 1711.08946, which is incorporated herein by reference in its entirety. An example of a recurrent deep Q-learning network is described in Matthew Hausknecht and Peter Stone, “Deep Recurrent Q-Learning for Partially Observable MDPs,” 2017, arXiv:1507.06527, https: / / arxiv.org / abs / 1507.06527, which is incorporated herein by reference in its entirety. Further reinforcement learning algorithms that can be used in the automated mastering process 2100 are the model-based actor-critic algorithm, the A3C algorithm, or the contextual multi-armed bandit algorithm. Combining a branching deep Q-network with recurrent deep Q-learning provides an effective algorithmic solution to this functionality.

[0259] 21B is a block diagram illustrating a second portion 2100b of the automatic mastering process 2100. The mastering agent 2102 is included in a state-behavior network 2130. The mastering agent 2102 is trained using short-term rewards 2122 and long-term rewards 2144, described below. The mastering agent 2102 also receives state data in the form of a target MIR epoch 2116 and a current MIR epoch 2112. By comparing the target MIR epoch 2116 with the current MIR epoch 2112, the mastering agent 2102 makes a decision on the appropriate parameter values ​​to use for each of multiple mastering tools to apply to the current original audio epoch 2108 in the current mastering pass. Decisions regarding parameter values ​​for each mastering tool are made by respective behavioral branches of the mastering agent 2102, denoted as first behavioral branch 2132, second behavioral branch 2134, third behavioral branch 2136, etc. through nth behavioral branch 2138, where n may be any positive integer. The mastering behaviors applied by the mastering tools may include mastering behaviors such as limiting and equalizing, for example.

[0260] An epoch mastering process 2152 applies the mastering tool parameter values ​​from each behavioral branch 2132-2138 to the original audio epoch 2108 to be mastered in order to perform a mastering pass, i.e., an audio mastering pass. After a mastering pass is performed, the resulting audio epoch is checked for mastering completion. If the audio epoch is now deemed fully mastered, the mastered epoch is stored for future reference and used as an input to the critic 2142 of the long-term reward network 2140. The mastered epoch is also stored for concatenation with other mastered epochs for final concatenation by the audio concatenation process 2162 to generate the final master 2170, i.e., the final mastered song composed by concatenating all mastered epochs together. The mastered epoch is also provided to the MIR extraction, prediction, or reference process 2124 for use in generating the current MIR epoch 2112, as described above.

[0261] The long-term reward network 2140 uses its critic 2142 to apply a reward function to data from one or more stored mastered epochs and mastering standards and / or rules from the mastering standards and rules database 1752 to generate a long-term reward 2144, which indicates how well multiple epochs over time meet the mastering standards and / or rules.

[0262] Once the final master 2170 is generated, the final master 2170 may be used by other processes in the system 1600, for example, by storing it in the audio library 184 as the final master 1770.

[0263] 25 shows an example of a production leadsheet process 2500 used by the emotional music composition system 1600 to generate a production leadsheet 2600 for use by a music creation process 1758. MIR data is extracted from a current mix 2504 (e.g., rough mix 1750) received as input by an MIR extraction process 2506 to generate current mix MIR data 2508. The MIR blueprint 1730 is compared with the current mix MIR data 2508 and differences therebetween are calculated in a difference calculation process 2510 to generate a set of changes 2512 required for each MIR feature of the current mix MIR data 2508 to match the MIR blueprint 1730. A data transformation process 2514 is performed on the set of changes required to generate the production leadsheet 2600. The production leadsheet 2600 can be used, for example, by the music creation process 1758 to guide the creator (i.e., user 1702) in their production decisions. This process 2500 performed by the MIR accuracy module 2502 may be repeated for multiple iterations corresponding to the multiple iterations of the automatic mastering process 2100 .

[0264] 26 shows an exemplary production leadsheet 2600 generated by the production leadsheet process 2500. The production leadsheet 2600 includes several general fields indicating the set of changes 2512 required in the overall MIR data for the audio segment, here shown as a percentage of the required change in a spectral variation field 2602, a spectral balance field 2604, a spectral flatness field 2606, and a silence ratio field 2608. The production leadsheet 2600 also includes several time series graphs indicating, for each epoch of the audio segment, the set of changes 2512 required in the MIR data per epoch, here shown as a brightness time series graph 2612, a density time series graph 2614 (indicating roughness), a loudness time series graph 2616 (measured as root-mean-squared loudness difference), and an intensity time series graph 2618 (indicating event density). Each graph 2612, 2614, 2616, 2618 shows the current MIR data for the epoch as a solid line and the MIR target (from the MIR blueprint 1730) as a dashed line, with the epochs shown from left to right over time. While the illustrated example shows the same values ​​for each graph for simplicity, it will be understood that an actual production leadsheet 2600 will likely have different time series values ​​for each graph.

[0265] The production leadsheet 2600 may also include an emotional space indicator 2420 or other emotional state data, as described with reference to FIG. 24A.

[0266] FIG. 17D illustrates the relationships among the processes of the fourth section 1700d of the exemplary emotional music composition system 1600, including the adaptation process 2200. Adaptation refers to the process by which a musical composition is arranged for performance using instruments or voices different from those originally specified. Thus, adaptation may take an existing mix, recording, or musical composition as input and produce as output a new mix that replaces one or more of the elements specified in the input with new elements. In a first example, referred to as the "remix" example, the input may be an existing track 1774 in multi-track format, i.e., an existing musical recording (such as the final master 1770) formatted so that each instrument track is separate from the other tracks (as opposed to a track in stereo format, where individual instrument tracks are combined together in the left and right channels). In a second example, described below, referred to as the "new track" example, a library of stems (i.e., instrument tracks or temporal sub-segments of instrument tracks) is used as material for constructing an adaptation of an existing score.

[0267] In the remix example, an adaptation process 2200 receives as input an existing track 1774 in multi-track form, decomposes the existing track 1774 into its constituent stems, and replaces different stems to arrange the music of the existing track 1774 to meet particular emotional response criteria identified by the MIR data received as input (shown here as MIR data required by the adaptation process 1783). The required MIR data 1783 can be extracted from the MIR blueprint 1730 by an MIR data extraction process 1782. In some embodiments, like the MIR data extraction processes 1756, 1740, respectively, the MIR data extraction process 1782 may be omitted, and the required MIR data 1783 may simply be the entire MIR blueprint 1730.

[0268] The aforementioned database of music theory rules 2020 is used by adaptive musical agent training process 1772 to train adaptive musical agents 2202 of adaptation process 2200, as described in more detail below with reference to Figures 22A-B.

[0269] The adaptation process 2200 utilizes a library of audio stems 1776 and a database of instrument timbre profiles 1778 to identify and select stems to swap to replace the original stems of the existing track 1774. Once the stems have been swapped to adapt the existing track 1774 to meet the MIR features of the required MIR data 1783, the adaptation process 2200 outputs the new mix as an adapted rough mix 1780 and / or a mix template 1786 that instructs the artist or producer on how to manually adapt the existing track 1774 to achieve the specified MIR features, including indicators of the desired or required MIR features 1783 themselves.

[0270] The user 1702 then interacts with the music production process 1758 to generate an adapted final mix 1790. As with the music composition process 1748, the user 1702 may interact with the music production process 1758 through a user interface 1642, such as a digital audio workstation (DAW) or other audio workstation. The user 1702 may be aided by a mix template 1786 and / or may begin interacting with the adapted rough mix 1780 as input.

[0271] In some embodiments, the adaptation process 2200 may be used during production, i.e., the user 1702 may interact with the music production process 1758 to cause the adaptation process 2200 to perform one or more iterations of the application process on a human-composed piece of music that has reached the adaptation stage.

[0272] 22A is a block diagram illustrating a first portion 2200a of the adaptation process 2200. The adaptation process 2200 can be used to generate either a music track (referred to as a "track" embodiment) or an MIR template (referred to as a "template" embodiment) that guides a user or another system in generating a track. In either of these embodiments, the generated track or template may be a remix of an existing track (referred to as a "remix" implementation) or an entirely new track (referred to as a "new track" implementation). In a "remix" embodiment, the adaptation process 2200 begins by dividing the existing track 1774 into its component stems (shown as a first stem 2204, a second stem 2206, and a third stem 2208). These constituent stems 2204, 2206, 2208 have their respective MIR data extracted or predicted using the MIR extraction or prediction techniques described above by an MIR extraction or prediction process 2210, thereby generating a corresponding set of MIR data for each stem: an MIR for the first stem 2214, an MIR for the second stem 2216, and an MIR for the third stem 2218. The MIR data for each stem of the existing track is then ranked by a ranking existing stems process 2220, which is described below with reference to the second portion 2200b. Constituent stems from the existing track 1774 may also be added to a library of audio stems 1776 (not shown).

[0273] 22B is a block diagram illustrating a second portion 2200b of the adaptation process 2200. The existing stem ranking process 2220 uses the MIR data of each stem 2214, 2216, 2218 to identify which stems in the existing track 1774 closely match the required MIR data 1783 and ranks the stems 2204, 2206, 2208 for their similarity to the required MIR data 1783. The ranking process may use k-nearest neighbor vector similarity calculations, as described in Madison Schott, “K-Nearest Neighbors (KNN) Algorithm for Machine Learning,” at https: / / medium.com / capital-one-tech / k-nearest-neighbors-knn-algorithm-for-machine-learning-e883219c8f26, which is incorporated herein by reference in its entirety. The stem rejection process 2222 determines which stems to reject based on the rankings generated by the existing stem ranking process 2220 and a set of music theory rules. The stem rejection process 2222 can be implemented algorithmically in some embodiments (e.g., by using a database of music theory rules 2020 (not shown)), but in other embodiments, it may be implemented by a human composer. The use of music theory rules is important to the stem rejection process 2222 because it may make sense to keep some stems that are essential to the composition of the song, even if they contradict the MIR blueprint 1730 (or required MIR data 1783).

[0274] The stems of the existing track 1774 that are not rejected (shown here as second stem 2206 and third stem 2208) are fed forward to a concatenation process 2236, which concatenates the retained stems (i.e., the stems that were not rejected) into a stereo mix. They are also fed forward for concatenation into the rough mix 1780 of the final remixed track in the "track" embodiment, as described below with reference to third portion 2200c. A further MIR extraction or prediction process 2238 is used to extract or predict MIR features of the stereo mix generated by concatenation process 2236, as described above, thereby generating track MIR data 2239. The track MIR data 2239 extracted from the retained stems is fed forward for concatenation into the mix template 1786 of the final remixed track in the "template" embodiment, as described below with reference to third portion 2200c. The epoch separation process 2244 performs a separation of the track MIR data 2239 into track MIR epochs 2250, the duration of each epoch being dictated by the duration of a section of the original track 1774 (dictated by rules such as music theory rules).

[0275] In addition to providing input to the existing stem ranking process 2220, the required MIR data 1783 can be used to generate MIR epochs for either a remix of an existing track 1774 or a new track adapting a score using a library of stems. In a remix example, the required MIR data 1783 is divided into epochs based on the sections of the track to be remixed (i.e., epochs of the existing track 1774), i.e., the required MIR data 1783 is divided into epochs by the remix MIR division process 2246, but the duration of each epoch is defined by the duration of a section of the original track 1774. In a "new track" example, the required MIR data 1783 is decomposed by the new track MIR division process 2248 into epochs having durations based on sections of the MIR blueprint 1730 (or required MIR data 1783), instead of sections of the existing track 1774 (as there is no existing track 1774 in this example).

[0276] The next phase proceeds one epoch at a time. Each track MIR epoch 2250 and each corresponding target MIR epoch 2252 are provided, one epoch at a time, as state data to the adaptive musical agent 2202, described below with reference to the third portion 2200c. Similar to the short-term reward network 2120 of FIG. 21A, the short-term reward network 2260 uses a critic 2262 to apply a reward function to three inputs: the output of the MIR combination process 2251 (described below) from the previous epoch, the target MIR epoch 2252 from the previous epoch, and music theory rules from the database of music theory rules 2020. The reward function of the critic 2262 generates a short-term reward 2264 based on how closely the music theory rules 2020 are followed and how well the target MIR epoch 2252 is matched by the track MIR epoch 2250. Therefore, the short-term reward 2264 generated as input in generating epoch n is based on the MIR combine process 2251 and the target MIR epoch 2252 for epoch (n-1).

[0277] 22C is a block diagram illustrating a third portion 2200c of the adaptation process 2200. The state-behavior network 2270 comprises an adaptive musical agent 2202 configured to select a stem for each of a plurality of behavioral branches (denoted as a first behavioral branch, a second behavioral branch, a third behavioral branch, etc. through an Nth behavioral branch, where N can be any positive integer) in response to each epoch of input data (i.e., each track MIR epoch 2250, each corresponding target MIR epoch 2252, and each corresponding short-term reward 2264). Each behavioral branch corresponds to a layer in the mix, i.e., typically an instrumental layer, e.g., drums, bass, guitar, etc. A branch can be omitted if the corresponding instrument is already present in the remaining stem from the existing track 1774 and no further layers are desired. The stem for each behavioral branch is selected from available stems provided by a library of audio stems 1776.

[0278] In some embodiments, the adaptive musical agent 2202 can be implemented as a branching recurrent deep Q-learning network with an LSTM for memory of actions taken (i.e., selected stems) for all previous epochs. The LSTM records track MIR features from one or more previous time steps (e.g., epochs), denoted herein as n time steps, namely, n previous target MIR epochs 2252 and n previous track MIR epochs 2250, and provides a state vector of these previous MIR features to the adaptive musical agent's 2202 policy. The adaptive musical agent's 2202 planner receives this state vector and short-term rewards 2264, along with previous adaptive actions taken (i.e., selected stems for each branch, as described below), and the planner updates the policy accordingly. The planner is trained using data from previous adaptation sessions. The adaptive musical agent 2202 is trained using reinforcement learning, with a library of audio stems 1776 defining the action space, and short-term rewards 2264 and long-term rewards 2282 (described below) providing reward feedback. The adaptive musical agent's 2202 time steps may correspond to an epoch, such as four bars of music. Similar to the MIR generator GAN 1612, score generator GAN 1616, and mastering agent 2102, the adaptive musical agent 2202 uses actor-critic behavior for reinforcement learning and deep learning in the described embodiment. In other embodiments, the adaptive musical agent 2202 may be implemented using other models, such as a model-based actor-critic model, an A3C model, or any other suitable machine learning model.

[0279] In a "track" embodiment, the stem concatenation process 2272 concatenates selected stems provided by the library of audio stems 1776 to generate a generated audio epoch 2274 (i.e., a mix of stems for the current epoch) consisting of new stems selected by the adaptive musical agent 2202 to be layered against the retained stems of an existing track (in a "remix" embodiment), or new stems selected to build an entire song (in a "new track" embodiment). Alternatively, in a "template" embodiment, the stem concatenation process 2272 uses a database of instrument timbre profiles 1778, which include MIR data associated with various instrument choices for creating a new track or remix template, to generate an MIR profile for creating a generated epoch template 2276 for the generated epoch. The generated epoch template 2276 includes the MIR data for the epoch.

[0280] Once the generated audio epoch 2274 or generated epoch template 2276 is generated, the length of the generated audio or template is compared to the desired total length of the track or template (based on sections of the existing track 1774 and / or required MIR data 1783) in step 2282. If the desired length has been reached, the generated epoch templates 2276 are combined with each other and with the MIR data of the retained stems of the original track (shown as track MIR data 2239 in second portion 2200b) to generate a mix template 1786, or the generated audio epochs 2274 are combined with each other and with the retained stems of the original track (shown as second stem 2206 and third stem 2208 in second portion 2200b) to generate an adapted rough mix 1780. The mix template 1786 or adapted rough mix 1780 is also provided to a long-term reward network 2280 (described below) to train the adaptive musical agent 2202. When an adapted rough mix 1780 is provided, its MIR data is first extracted, predicted, or retrieved by an MIR extraction process 2284. As mentioned above, the MIR data can either be extracted, referenced (i.e., looked up) from a table of previously extracted MIR features via a tool such as the MIR toolbox, or predicted using a model that inputs raw audio and outputs an MIR prediction. The extracted (or predicted, or retrieved) MIR data is then provided to the long-term reward network 2280. The adaptation process then ends. In some embodiments, while the current track or template is being generated, the entire track or template generated so far (both the original stem and the generated stem) can be provided to the long-term reward network 2280 to train the adaptive musical agent 2202 before the adaptation process is complete.

[0281] If, in step 2282, the desired total length of the track or template has not been reached, the current generated audio epoch 2274 or generated epoch template 2276 is provided as feedback to the MIR combining process 2251. In a "track" embodiment, the current generated audio epoch 2274 first passes through the MIR extraction process 2286, which extracts, predicts, or retrieves MIR data for the current generated audio epoch 2274, as described above, to generate MIR data corresponding to the current generated audio epoch 2274. In a "remix" embodiment, the MIR combining process 2251 combines the MIR extracted from the retained stem of the original track (i.e., track MIR data 2239) with the feedback MIR data (from the MIR extraction process 2286 or generated epoch template 2276) to generate combined MIR data for that epoch, including both the retained stem and the new stem. This combined MIR data is provided to the critic 2262 to generate a short-term reward 2264 for the subsequent epoch. That is, the combined MIR data for epoch (n-1) is provided to the critic 2262 along with the target MIR epoch 2252 for epoch (n-1), which generates a short-term reward 2264 for use by the adaptive agent 2202 in selecting a stem for epoch (n). Of course, in the "new track" example, there is no stem retained, since there is no original track to be remixed. Thus, the output of the MIR combination process 2251 is simply the feedback MIR data received from the MIR extraction process 2286 or the generated epoch template 2276.

[0282] The long-term reward network 2280 comprises a critic 2278 that applies a reward function to three inputs: required MIR data 1783, music theory rules from a database of music theory rules 2020, and either MIR data extracted from the rough mix 1780 adapted by an MIR extraction block 2284 or a mix template 1786. The reward function generates a long-term reward 2282 based on the required MIR data 1783 applied to the music theory rules and the MIR data of the entire track (or entire template) that was adapted.

[0283] The emotional music recommendation system 100 and / or the emotional music composition system 1600, and / or aspects and components thereof, can be recombined in various configurations to address specific use cases related to selecting or creating music to elicit particular emotional responses in one or more listeners. These use cases may involve therapeutic, entertainment, or lifestyle applications. Some exemplary use cases of the exemplary embodiments described herein will now be described with reference to Figures 27-30.

[0284] FIG. 27 illustrates an example use case 2700 for an album remix, using embodiments described herein. A client 2732, who is an artist with an existing album of songs, works with a producer 2734 to establish a set of wellness goals 2736 for the album songs, utilizing an onboarding document 2738 that describes the emotional operation process and an emotional framework (e.g., the GEMS / Russell Circumplex Model of Emotion). The producer 2734 receives the current album mix 2702 (i.e., a mix of songs from the album) from the client 2732 and provides them to an MIR extraction process 2704 to convert them into MIR data. A trained emotional inference model, such as the composition AINN 1800, is used to generate predicted emotional dynamics 2706 for the track. The predicted emotional dynamics 2706 are converted into a visualization 2708 of the song's emotional dynamics. Client 2732 reviews visualization 2708 and selects a first track to optimize with the desired emotional trajectory guided by the track selected in step 2710, thereby providing track 2712 and emotional target 2714 (e.g., desired emotional trajectory) as inputs to subsequent processes in use case 2700. Track MIR data 2716 for track 2712 is provided as a first input to emotional accuracy model 2720 (e.g., MIR generation process 1900). An MIR blueprint 1730 corresponding to emotional target 2714 is used as a second input to emotional accuracy model 2720. The emotional accuracy model 2720 is used by an engineer 2724 performing the mixing and mastering process (e.g., via the music production process 1758 of the emotional music composition system 1600) to generate a production lead sheet (such as the production lead sheet 2600 created by the emotional music composition system 1600) and an MIR blueprint that identifies the MIR features that need to be modified, shown together as document 2722, to produce a final master 2730 of the selected tracks configured to achieve the desired emotional goal.In this example, mastering techniques are applied to segments of the track (e.g., to correct the timbre of the piano), an ambient track is added to the mix to better meet the MIR targets, and binaural entrainment (2Hz) is added to the track.

[0285] 28 illustrates an exemplary music composition use case 2800 targeting a wellness playlist using embodiments described herein. The same onboarding process is used in use case 2700, but in this example, the goal setting step 2836 involves selecting a playlist (e.g., a playlist from an existing online music service) to target wellness goals for the client's 2732 albums, selecting the emotional trajectory of the tracks, and identifying the track length and number of tracks required. For example, a nostalgic track and a peaceful track are selected to target a "chill" playlist, while two "focus" tracks and one melancholic track are selected to target a "sad beats" playlist, all of which are 3 minutes ± 20 seconds in duration.

[0286] Composition of each selected track begins with generating an MIR blueprint 1730 for the track using the MIR generation process 1900 of the composition system 1600. A score (e.g., a MIDI score 1742) and a composition lead sheet 2400 are generated (e.g., using the composition system 1600), and the composer 2734 and client 2732 may refine the score 1742 and the composition lead sheet 2400 over one or more additional iterations of the process by modifying various parameters in accordance with the MIR blueprint 1730. For example, the MIDI score 1742 may be adjusted to provide track-level chord progressions and melodies that are in line with the goals 2836.

[0287] The producer 2734 and client 2732 collaborate to generate rough mixes 1750 of the various tracks. The emotional accuracy model 2720 uses the rough mixes 1750 and the MIR blueprint 1730 to generate a data set of musical features needed to modify and create the production lead sheet 2600. The producer 2734 then applies mixing techniques (e.g., using a music production process 1758) to the tracks to better meet the MIR goals set by the production lead sheet, to generate a set of final mixes 1760. The mastering agent 2102 can then be used to perform automatic mastering of the final mixes 1760, generating the final master 1770.

[0288] 29 shows an example use case 2900 of adapting a music collection to generate multiple albums with distinct emotional goals using embodiments described herein. The client 2732 is an artist with an existing collection of songs who wants to release the songs as three albums, each with a specific set of emotional goals. The artist 2732 works with the producer 2734 to set goals 2836 as in use case 2800. A collection of tracks 2902 is received from the client 2732 and provided to the MIR extraction process as in use case 2700. Use case 2900 proceeds as in use case 2700 on a track-by-track basis, except that the client 1732 selects multiple tracks 2712 for optimization in step 2910 by selecting tracks that generally approximate the desired emotional goals.

[0289] The emotional targets 2714 selected by the client 1732 during onboarding 2836 are used to generate an MIR blueprint 1730 for each track. The track MIR data 2716 for each track 2712, along with each track's MIR blueprint 1730, are provided to an emotional accuracy model 2720 to generate a production lead sheet 2600 for each track, which identifies musical features that need to be modified. The producer 2734 uses the production lead sheet 2600 to perform mixing 2912, the output of which (e.g., rough mix 1760) is provided to the mastering agent 2102, which is curated by a curation agent 2914 to generate the final master 1770. The curation agent 2914 may be a deep Q-network from the emotional music recommendation system 100 that has been trained on the composition AINN 1800.

[0290] FIG. 30 illustrates an exemplary music adaptation use case 3000 for generating emotional music using a library of existing stems using embodiments described herein. A client 1732 provides a library of unused stems 1776 to be used to generate new songs for an album. The MIR generation process 1900 is used to generate an MIR blueprint 1730 based on the emotional target 2714. The adaptive music agent 2202 utilizes the stem library 1776 to generate a track 3002 intended to match the MIR blueprint 1730. At this stage, the track 3002 can be shared with the client 2732 for approval, providing an opportunity to add additional elements. The track 3002 is then converted to MIR data at 2704 to generate track MIR data 2716. The MIR blueprint 1730 and track MIR data 2716 are used by an emotional accuracy model 2720 to generate a production lead sheet 2600 for the track. The producer 2734 performs the mixing 2912 and then uses the mastering agent 2102 and the curation agent 2914 to generate the final master 1770.

[0291] In some embodiments, the described systems and methods may use non-auditory stimuli and non-audio data instead of, or in addition to, the audio data and auditory stimuli described above. In some embodiments, tactile or visual data may be used to gather and predict a user's emotional response to tactile or visual stimuli using techniques similar to those described above.

[0292] In some embodiments, an audio segment may include monaural or binaural beat data, by itself or integrated with other auditory data. Monaural and binaural beats have been shown to have the ability to elicit specific emotional responses in humans under some conditions. See, for example, comparable literature work by Chaieb et al., "Auditory Beat Stimulation and Its Effects on Cognition and Mood States," Frontiers in Psychiatry, Vol. 6, 2015, https: / / www.frontiersin.org / article / 10.3389 / fpsyt.2015.00070, which is incorporated herein by reference in its entirety.

[0293] While the present disclosure may be described, at least in part, with respect to methods and apparatuses, those skilled in the art will understand that the present disclosure is also directed to various components for implementing at least some of the aspects and features of the described methods through hardware components, software, or any combination of the two. Accordingly, the technical solutions of the present disclosure may be embodied in the form of a software product. A suitable software product may be stored on a pre-recorded storage device or other similar non-volatile or non-transitory computer-readable or processor-readable medium, including, for example, a DVD, a CD-ROM, a USB flash disk, a removable hard disk, or other storage medium. A software product includes tangible instructions stored thereon that enable a processing device (e.g., a personal computer, a server, or a network device) to execute an embodiment of a method or system disclosed herein.

[0294] Those skilled in the art will also understand that the output of the above-described method and device, i.e., the audio stream 234 including the audio segments 230 themselves, may be stored as music data (e.g., an audio file) on a storage medium, such as a non-volatile or non-transitory computer-readable or processor-readable medium, including a DVD, CD-ROM, USB flash disk, removable hard disk, or other storage medium. The music may also be stored on other digital or analog storage media suitable for use in audio applications or audio playback or broadcast devices, such as cassette tapes, vinyl records, or any other storage medium for digital or analog music data. In one embodiment, an audio stream may be identified as likely to elicit a particular emotional trajectory, either user-specific or user-independent, and the audio stream may be stored for later listening by the user.

[0295] In the described methods or block diagrams, each box may represent an event, step, function, process, module, message, and / or state-based operation, etc. While some of the above examples are described as occurring in a particular order, those skilled in the art will understand that some of the steps or processes may be performed in a different order, provided that the results of changing the order of any given step do not prevent or impair the occurrence of subsequent steps. Furthermore, some of the messages or steps described above may be eliminated or combined in other embodiments, and some of the messages or steps described above may be divided into multiple sub-messages or sub-steps in other embodiments. Furthermore, some or all of the steps may be repeated as necessary. Elements described as methods or steps equally apply to systems or sub-components, and vice versa. References to the terms "sending" or "receiving" are interchangeable depending on the perspective of a particular device.

[0296] The above-described embodiments are to be considered as illustrative and not restrictive. Exemplary embodiments described as methods will equally apply to systems and vice versa.

[0297] Modifications may be made to some exemplary embodiments, including any combinations and subcombinations of the above. The various embodiments described above are merely examples and are not intended to limit the scope of the present disclosure. Variations of the invention described herein will be apparent to those skilled in the art, and such variations are within the intended scope of the present disclosure. In particular, features from one or more of the above-described embodiments may be selected to create alternative embodiments consisting of subcombinations of features not expressly described above. In particular, features from one or more of the above-described embodiments may be selected and combined to create alternative embodiments consisting of combinations of features not expressly described above. Features suitable for such combinations and subcombinations will be readily apparent to those skilled in the art upon review of the present disclosure in its entirety. The subject matter described herein is intended to cover and encompass all appropriate technical modifications.

Claims

1. 1. A method of generating music for eliciting a desired emotional response in a listener, said method comprising: receiving desired emotional response data indicative of the desired emotional response; generating the music having musical features predicted by a trained generative machine learning model to elicit the desired emotional response in the listener; processing the desired emotional response data using the trained generative machine learning model to generate musical feature data indicative of the musical features; generating the music based on the musical feature data; A method comprising:

2. training the generative machine learning model, presenting music having a set of musical features to a listener; obtaining emotional response data from the listener indicative of the listener's emotional response to the presentation of the music; labeling the musical features of the music using the emotional response data to generate labeled musical feature data; training the generative machine learning model to predict the emotional response data based on the labeled musical feature data using the labeled musical feature data as training data; The method of claim 1 further comprising:

3. generating the music based on the musical feature data, generating a score based on the musical feature data; generating the music based on the score; The method of claim 1 , comprising:

4. The method of claim 3 , wherein the score is a Musical Instrument Digital Interface (MIDI) score.

5. generating the score receiving composition intent information; generating the score based on the musical feature data and the composition intention information; The method of claim 3, comprising:

6. The method of claim 5 , wherein the composition intent information includes one or more of score type information, instrument information, and score length information.

7. The method of claim 5 , wherein the composition intent information is indicated by composition intent user input received from a user.

8. The method of claim 3 , wherein generating the score comprises processing the musical feature data to generate the score using a trained score generation machine learning model.

9. The method according to any one of claims 1 to 8, wherein the musical feature data is Music Information Retrieval (MIR) data.

10. The method of claim 9 , wherein the MIR data is an MIR blueprint of the music.

11. The method of any of claims 1 to 8, wherein the trained generative machine learning model is a generative adversarial network (GAN).

12. The method of claim 11 , wherein the GAN includes a generator network, a probability network, and a control network.

13. The method of claim 12 , wherein the GAN comprises a conditional GAN.

14. The method of claim 13 , wherein the generator network comprises a generator neural network.

15. The method of claim 14 , wherein the probabilistic network comprises a classifier neural network.

16. 16. The method of claim 15, wherein the generator network and the classifier neural network each comprise a recurrent neural network (RNN) with long short-term memory (LSTM).

17. generating the music based on the score, presenting the score to a user; receiving a rough mix user input from the user; generating a rough mix based on said rough mix user input; generating the music based on the rough mix; The method of claim 3, comprising:

18. generating the music based on the musical feature data, generating a composition lead sheet based on the musical feature data; presenting said composition lead sheet to a user; receiving a rough mix user input from the user; generating a rough mix based on said rough mix user input; generating the music based on the rough mix; The method of claim 1 , comprising:

19. generating the music based on the rough mix, generating a production lead sheet based on the musical feature data and the rough mix; presenting the production lead sheet to the user; receiving final mix user input from said user; generating a final mix based on said final mix user inputs; generating the music based on the final mix; 19. The method of claim 17 or 18, comprising:

20. 1. A system for generating music for eliciting a desired emotional response in a listener, the system comprising: a processor system; a storage device, The storage device includes:

1. A generative machine learning model, comprising: presenting music having a set of musical features to a listener; obtaining emotional response data from the listener indicative of the listener's emotional response to the presentation of the music; generating labeled musical feature data by labeling musical features of the music using the emotional response data; using the labeled musical feature data as training data to train the generative machine learning model to predict the emotional response data based on the musical feature data; a generative machine learning model trained by instructions executable by the processor system to cause the system to perform an operation, the operation comprising: receiving desired emotional response data indicative of the desired emotional response; generating music having musical features predicted by the trained generative machine learning model to elicit the desired emotional response in the listener; processing the desired emotional response data using the trained generative machine learning model to generate musical feature data indicative of the musical features; generating the music based on the musical feature data; an instruction, The system contains:

Images