Information processing device, and its control method

Abstract

To provide an information processing device that enables even beginners to easily adjust tone to be output, and its control method.SOLUTION: An information processing system 100 includes: an operation unit 105 and a display unit 108 integrally configured as a touch panel display where natural language including adjectives is input by a user; and an estimation unit 203 that outputs tone data based on user-input natural language by using a learned model that outputs tone data from the adjectives, which is a function executed by a GPU 102.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing apparatus that adjusts a tone color to be output based on tone color data, and a control method thereof. [Background technology]

[0002] 2. Description of the Related Art Conventionally, synthesizers capable of outputting a tone color adjusted using tone color data consisting of waveform data and effect parameters have been known.

[0003] For example, Patent Document 1 discloses a music performance device that, when an input means is brought into contact with a display unit that displays pitch and timbre on two axes, outputs sound at a pitch and timbre that corresponds to the coordinate position of the input means that is brought into contact with the display unit.

[0004] Furthermore, for example, Patent Document 2 discloses a tone color setting system that can automatically set a tone color that matches the user's psychological state, such as mood and emotion, based on the user's actual performance. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-156109 [Patent Document 2] Japanese Patent Application Laid-Open No. 2006-30414 Summary of the Invention [Problem to be solved by the invention]

[0006] However, even with the technologies of Patent Documents 1 and 2, it was difficult for beginners to find waveform data for the type of instrument they wanted to play or adjust the tone color with effect parameters by operating the numerous buttons and knobs provided on conventional synthesizers.

[0007] In view of the above circumstances, an object of the present invention is to provide an information processing device and a control method thereof that allow even a beginner to easily adjust the tone color to be output. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, an information processing device according to one aspect of the present invention includes an input module into which natural language including adjectives is input by a user, and a timbre estimation module that outputs timbre data based on the natural language input by the user using a trained model that outputs timbre data from adjectives. [Effects of the Invention]

[0009] According to the present invention, even a beginner can easily adjust the tone color to be output. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a hardware configuration of an information processing apparatus according to an embodiment of the present invention; [Figure 2] FIG. 2 is a block diagram showing a software configuration of the information processing device. [Figure 3] FIG. 10 is a diagram showing the state in which each effect parameter included in the collected training data is mapped onto a latent space. [Figure 4] 1 is a flowchart illustrating a training process for a learning model in an embodiment of the present invention. [Figure 5] 4 is a flowchart showing a process for estimating timbre data according to an embodiment of the present invention. [Figure 6] 1. FIG. 3 is a diagram showing an example of a UI of the input unit and output unit in FIG. 2, which is displayed on the display unit in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Each embodiment described below is merely an example of a configuration that can realize the present invention. Each of the following embodiments can be modified or changed as appropriate depending on the configuration of the device to which the present invention is applied and various conditions. Furthermore, not all combinations of elements included in each of the following embodiments are necessarily essential for realizing the present invention, and some elements can be omitted as appropriate. Therefore, the scope of the present invention is not limited to the configurations described in each of the following embodiments. Furthermore, configurations that combine multiple configurations described in the embodiments can also be adopted as long as they are not mutually contradictory.

[0012] The information processing device 100 according to this embodiment is realized by a synthesizer, but is not limited to this. For example, the information processing device 100 may be an information processing device (computer) such as a personal computer or a server that transmits timbre data to be set to an external synthesizer.

[0013] Here, the tone color data in this embodiment is data including at least one of waveform data of various musical instruments such as piano, organ, guitar, etc., and effect parameters such as chorus, reverb, distortion, etc.

[0014] In general terms, the information processing device 100 of this embodiment sets candidates for timbre data to be used for timbre adjustment based on natural language input by the user when adjusting the timbre for performance on the information processing device 100, and displays each candidate in a list with sample timbre playback available. After that, when the user selects one of the candidates displayed in the list, the sample timbre that is played back is the timbre that the user wants to use for performance, and the information processing device 100 performs timbre adjustment so that the sample timbre becomes the timbre when played on the information processing device 100.

[0015] FIG. 1 is a block diagram showing the hardware configuration of an information processing device 100 according to an embodiment of the present invention.

[0016] 1, the information processing device 100 of this embodiment includes a CPU 101, a GPU 102, a ROM 103, a RAM 104, an operation unit 105, a microphone 106, a speaker 107, a display unit 108, and an HDD 109, which are connected to one another via a bus 110. Although not shown in FIG. 1, the information processing device 100 also includes a keyboard that can be played by the user.

[0017] The CPU 101 is one or more processors that control each unit of the information processing device 100 in accordance with a program stored in the ROM 103, for example, and uses the RAM 104 as a work memory.

[0018] The GPU 102 can perform efficient calculations through parallel processing of data, and therefore, the processing for learning using a learning model, as will be described later, is performed by the GPU 102.

[0019] The RAM 104 is a volatile memory and is used as a temporary storage area such as the main memory and work area of ​​the CPU 101.

[0020] The microphone 106 converts the collected voice into an electrical signal (voice data) and supplies it to the CPU 101. For example, the microphone 106 collects voice in a natural language spoken by a user toward the microphone 106, converts this voice, and supplies the converted voice data to the CPU 101.

[0021] The speaker 107 produces a tone with adjusted tone when playing using the information processing device 100, when step S402 in FIG. 4 (described later) is executed, and when step S509 in FIG. 5 (described later) is executed.

[0022] The HDD 109 is a non-volatile memory, and stores in predetermined areas tone color data and other data, as well as various programs for operating the CPU 101. The HDD 109 may be any non-volatile memory capable of storing the above data and programs, and may be, for example, a flash memory or other memory.

[0023] The operation unit 105 and the display unit 108 are integrally configured as a touch panel display that accepts user operations on the information processing device 100 and displays various information. However, the operation unit 105 and the display unit 108 may each be independent user interfaces, for example, the operation unit 105 may be configured as a keyboard and a mouse, and the display unit 108 may be configured as a display.

[0024] The bus 110 is a signal transmission path that interconnects the hardware elements of the information processing device 100 described above.

[0025] FIG. 2 is a block diagram showing the functional configuration of the information processing device 100. As shown in FIG.

[0026] 2, the information processing device 100 includes a learning unit 201, an input unit 202, an estimation unit 203, and an output unit 204.

[0027] The input unit 202 (input module) is a function executed by the CPU 101 to output an adjective input by a user to the estimation unit 203.

[0028] Specifically, the input unit 202 displays an I / F 601 (FIG. 6(a)) on the display unit 108, and acquires a natural language input by a user to the I / F 601 using the operation unit 105. Thereafter, the input unit 202 performs a morphological analysis of the acquired natural language, extracts adjectives input by the user, and outputs the extracted adjectives to the estimation unit 203.

[0029] The input unit 202 is not limited to the present embodiment as long as it can acquire adjectives input by the user. For example, adjectives input by the user may be acquired based on the natural language spoken by the user collected by the microphone 106, or an I / F 602 (FIG. 6(b)) including tags of multiple adjectives may be displayed on the display unit 108, and an adjective of a tag selected by the user using the operation unit 105 may be acquired as the adjective input by the user.

[0030] The details of the processing by the input unit 202 will be described later with reference to FIG.

[0031] The learning unit 201 is a function implemented by a learning model configured by a CVAE (conditional variational autoencoder), which is a type of neural network, and executed by the GPU 102. The GPU 102 trains the learning model that configures the learning unit 201 by supervised learning using training data consisting of effect parameters and adjectives tagged thereto, and outputs decoder parameters (described later) of the generated trained model to the estimation unit 203.

[0032] The learning model constituting the learning unit 201 has an encoder and a decoder. Here, the encoder is a neural network that, when effect parameters (input data x) tagged with an adjective (label y) are input as training data, extracts latent variables z tagged with the adjective (label y) in a latent space from the training data. The decoder is a neural network that, when latent variables z tagged with an adjective (label y) are input, reconstructs effect parameters (output data x~) tagged with the adjective (label y). The GPU 102 compares the input data x with the output data x~ to adjust the parameters of the encoder and decoder constituting the learning unit 201. Furthermore, the GPU 102 adjusts the encoder parameters so that a cluster is formed for each label y by the latent variables z in the latent space shown in FIG. 3. The GPU 102 repeats this process and optimizes the parameters of the learning model constituting the learning unit 201, thereby training the learning model and generating a trained model. Details of the training process of the learning model by the GPU 102 will be described later with reference to FIG. 4.

[0033] The estimation unit 203 (timbre estimation module) is a neural network (hereinafter simply referred to as a decoder) that is the same as the decoder of the trained model generated by the training unit 201, and is a function executed by the GPU .

[0034] When the parameters are output from the learning unit 201 to the estimation unit 203, the GPU 102 updates the parameters of the decoder that constitutes the estimation unit 203 with the parameters.

[0035] Furthermore, when an adjective input by the user is output from the input unit 202 to the estimation unit 203, the GPU 102 acquires the latent variable z tagged with the adjective from among the latent variables z in the latent space shown in Fig. 3 and inputs this to a decoder constituting the estimation unit 203, thereby reconstructing (estimating) effect parameters (timbre data) tagged with the adjective. Thereafter, the GPU 102 outputs the reconstructed effect parameters to the output unit 204. Details of the timbre data estimation process by the GPU 102 will be described later with reference to Fig. 5.

[0036] The neural network used in the learning unit 201 and the estimation unit 203 is not particularly limited, but examples thereof include DNN, RNN / LSTM, recurrent neural network, and CNN (convolutional neural network).In place of a neural network, other models, such as HMM (hidden Markov model) and SVM (support vector machine), may be used.

[0037] Furthermore, although the learning unit 201 is configured with only a CVAE to perform supervised learning, it may also be configured to include a variational autoencoder (VAE) or a generative adversarial network (GAN). In this case, the learning unit 201 performs semi-supervised learning, which combines unsupervised learning using a VAE or GAN, i.e., learning using clustering in which effect parameters not tagged with adjectives are used as training data, with supervised learning using a CVAE.

[0038] Furthermore, the learning unit 201 and the estimation unit 203 may be one device (system).

[0039] Furthermore, in this embodiment, the learning unit 201 and the estimation unit 203 are executed by the GPU 102, which is a single processor, but the GPU 102 may be configured with multiple processors to perform distributed processing. Also, the learning unit 201 and the estimation unit 203 may be functions executed not only by the GPU 102 but also in cooperation with the CPU 101.

[0040] The output unit 204 (presentation module) is a function executed by the CPU 101 that displays (presents) a list of multiple effect parameters output from the estimation unit 203 as candidates for effect parameters to be used for adjusting the tone when the user plays using the information processing device 100.

[0041] Specifically, the output unit 204 displays, on the display unit 108, an I / F 603 (FIG. 6(c)) including a plurality of tabs linked to each candidate effect parameter. As shown in FIG. 6(c), each tab of the I / F 603 is provided with a play button linked to a sample sound when the timbre is adjusted using each effect parameter. When the user then presses one of the play buttons on the I / F 603, the output unit 204 places the tab on which the play button is provided in a user-selected state and plays the sample timbre linked to that play button. The user presses each play button displayed on the I / F 603, and when the sample timbre desired by the user is played, the user presses the enter button 604. When the enter button 604 is pressed, the output unit 204 determines that the effect parameters linked to the tab currently selected by the user will be used for timbre adjustment of the information processing device 100.

[0042] The details of the processing by the output unit 204 will be described later with reference to FIG.

[0043] FIG. 3 is a diagram showing the state in which each effect parameter included in the collected training data is mapped onto the latent space.

[0044] When the GPU 102 generates a trained model in the training unit 201, the effect parameters (input data x) are mapped as latent variables z in the latent space. Many of these latent variables z are included in one of the clusters formed for each label y. In this embodiment, as shown in FIG. 3 , the latent space includes a cluster 301 for the adjective "beautiful," which is one of the labels y tagged to the input data x, and a cluster 302 for the adjective "gorgeous," which is also one of the labels y.

[0045] In this embodiment, the input data x to the learning unit 201 is only effect parameters, but this is not limited to timbre data. For example, the input data x to the learning unit 201 may be timbre data consisting of waveform data only, a combination of waveform data and effect parameters, or a timbre data set including multiple timbre data.

[0046] FIG. 4 is a flowchart showing the training process of the learning model in this embodiment.

[0047] In this process, the CPU 101 reads out a program stored in the ROM 103 and executes it using the RAM 104 as a working memory.

[0048] First, in step S401, the CPU 101 acquires effect parameters from the HDD 109. Note that the effect parameters may also be acquired from an external source via a communication unit (not shown in FIG. 1).

[0049] In step S402, the CPU 101 acquires adjectives to be tagged for each of the effect parameters collected in step S401.

[0050] Here, the adjectives to be tagged are specifically acquired as follows.

[0051] First, the CPU 101 uses the collected effect parameters to adjust the tone of the default waveform data, which is piano waveform data, and causes the speaker 107 to produce the tone, while also displaying the I / F 601 (Figure 6(a)) on the display unit 108.

[0052] Thereafter, when CPU 101 detects that the operator has input an adjective that is associated with the tone sounded from speaker 107 into I / F 601 using operation unit 105, CPU 101 acquires the input adjective as an adjective to be tagged. The acquired adjective may be singular or plural.

[0053] Furthermore, since the adjectives to be tagged are obtained using the above method, a correlation between the tone data contained in the training data and the adjectives tagged thereto can be inferred in light of the common technical knowledge at the time of filing.

[0054] In step S403, CPU 101 tags the adjectives acquired in step S402 with the effect parameters acquired in step S401, and generates training data. Note that a data set consisting of such effect parameters and the adjectives tagged therewith may be obtained by crowdsourcing.

[0055] In step S404, the CPU 101 inputs the training data generated in step S403 to the learning unit 201, causing the GPU 102 to learn the learning model that constitutes the learning unit 201, and generates a trained model. Thereafter, the GPU 102 outputs the decoder parameters of the trained model from the learning unit 201 to the estimation unit 203, updates the decoder parameters that constitute the estimation unit 203, and then terminates this processing.

[0056] In this embodiment, the tone produced by the speaker 107 in step S402 is obtained by adjusting the tone of the waveform data of a piano. However, tone adjustment may be performed on waveform data of multiple musical instruments. In this case, adjectives to be tagged for each musical instrument type for the same effect parameter are acquired in step S402. Furthermore, a trained model is generated for each musical instrument type in step S404.

[0057] Next, the timbre data estimation process according to this embodiment, which is executed after the process of FIG. 4, will be described with reference to FIG.

[0058] FIG. 5 is a flowchart showing the timbre data estimation process in this embodiment.

[0059] In this process, the CPU 101 reads out a program stored in the ROM 103 and executes it using the RAM 104 as a working memory.

[0060] First, in step S501, CPU 101 displays I / F 601 on display unit 108 and acquires natural language input by the user to I / F 601 using operation unit 105. After that, any morphological analysis is performed on the acquired natural language to extract adjectives input by the user.

[0061] For example, if the natural language phrase "beautiful piano sound" is entered into I / F 601, the three words "beautiful," "piano," and "sound" are obtained through morphological analysis of the entered natural language, and the word "beautiful" is extracted from these as the adjective entered by the user.

[0062] Furthermore, when the natural language phrase "a brilliant and beautiful piano sound" is input to the I / F 601, the two words "brilliant" and "beautiful" are extracted as adjectives input by the user.

[0063] Note that the method of step S501 is not limited to the method of this embodiment as long as the adjective input by the user can be acquired. For example, instead of displaying I / F 601, an I / F 602 may be displayed that displays the multiple adjectives acquired in the processing of step S402 as user-selectable tags, and the adjective displayed in the user-selected tag may be acquired as the adjective input by the user. Also, instead of displaying I / F 601, voice data including a natural language spoken by the user through microphone 106 may be converted into text data using any voice recognition technology, and any morphological analysis may be performed on the text data to extract the adjective input by the user.

[0064] Next, in step S502, the CPU 101 acquires latent variables tagged with the adjectives extracted in step S501 from the latent space, and inputs the latent variables tagged with the adjectives to the decoder constituting the estimation unit 203. This causes the GPU 102 to output effect parameters tagged with the adjectives from the decoder constituting the estimation unit 203. Note that if multiple adjectives are extracted in step S501, all of the adjectives are input to the decoder constituting the estimation unit 203.

[0065] For example, when the adjective “beautiful” is extracted in step S501, the estimation unit 203 outputs an effect parameter tagged with the adjective “beautiful” that is reconstructed by the latent variable z tagged with the adjective “beautiful” in the latent space, such as the latent variable z that forms the cluster 301 shown in FIG. 3 .

[0066] For example, when the adjectives “beautiful” and “gorgeous” are extracted in step S501, the estimation unit 203 outputs effect parameters tagged with these two adjectives, which are reconstructed using latent variables z to which these two adjectives are tagged in the latent space, such as the latent variables z forming the cluster 301 shown in FIG. 3.

[0067] In addition, if a trained model is generated for each instrument type in step S404, and if not only adjectives but also instrument types are extracted in step S501, the adjectives extracted in step S501 are input to the decoder of the extracted instrument type in the estimation unit 203.

[0068] In step S503, CPU 101 sets candidate effect parameters to be used by the user for tone adjustment from among the plurality of effect parameters output in step S502. In this embodiment, a randomly selected effect parameter from among the plurality of effect parameters output in step S502 is set as the candidate effect parameter to be used by the user for tone adjustment. Note that, of the plurality of effect parameters output in step S502, a parameter whose likelihood is equal to or greater than a threshold may be set as the candidate effect parameter to be used by the user for tone adjustment.

[0069] In step S504, CPU 101 determines whether or not a user input of an instrument type has been made. Specifically, if an instrument type is found among the words acquired by the arbitrary morphological analysis in step S501, it is determined that a user input of an instrument type has been made.

[0070] For example, if the natural language phrase "beautiful piano sound" is input to the I / F 601 in step S501, the CPU 101 determines in step S504 that the user has input the musical instrument type "piano."

[0071] If the user has input a type of musical instrument (YES in step S504), the process proceeds to step S505, where CPU 101 acquires waveform data of the type of musical instrument input by the user from HDD 109, and the process proceeds to step S507.

[0072] In this case, CPU 101 further narrows down (selects) the candidates set in step S503 according to the instrument type input by the user. For example, if the instrument type input by the user is "piano," "distortion" is not usually used for tone adjustment, so if "distortion" is included in the set candidates, it is removed from the candidates.

[0073] On the other hand, if the user has not input an instrument type (NO in step S504), the process proceeds to step S506, where CPU 101 acquires waveform data for the default instrument type "piano" from HDD 109, and then proceeds to step S507. Note that the waveform data for the default instrument type is not limited to that of this embodiment, and may be waveform data for other instrument types, such as organ or guitar. Also, in step S506, CPU 101 may display multiple tags, each describing a multiple instrument type, on display unit 108, and acquire waveform data for the instrument type displayed in the tag selected by the user from HDD 109.

[0074] In step S507, CPU 101 displays a list of the effect parameter candidates set in step S503 on display unit 108. Specifically, as shown in I / F 603 in Fig. 6, the effect parameter candidates set in step S503 are displayed as user-selectable tabs such as a "Tone 1" tab, a "Tone 2" tab, etc. Each tab is also provided with a play button.

[0075] In step S508, CPU 101 determines whether an instruction to play one of the effect parameter candidates set in step S503 has been issued. Specifically, CPU 101 determines whether any of the play buttons provided on each tab of I / F 603 has been pressed. If an instruction to play one of the candidates has been issued (YES in step S508), the process proceeds to step S509.

[0076] In step S509, CPU 101 inverts the color of the tab (or the play button portion) on display unit 108 for which the play button was pressed, notifying the user that the tab has been selected by the user, and adjusts the tone using the effect parameters of the candidate for which playback was requested and the waveform data obtained in either step S505 or S506, and produces (plays) the sample tone from speaker 107.

[0077] In step S510, CPU 101 determines whether the candidate for which playback has been instructed has been selected by the user as an effect parameter to be used for tone adjustment. Specifically, if, after the sample tone is produced by speaker 107 in step S508, enter button 604 is pressed in I / F 603 without pressing any other playback button, CPU 101 determines that the candidate for which playback has been instructed has been selected by the user as an effect parameter to be used for tone adjustment.

[0078] That is, if an instruction to play one of the other candidates is given without pressing the decision button 604 (NO in step S510, YES in step S508), the processing from step S509 onwards is repeated. On the other hand, if the decision button 604 is pressed without giving an instruction to play one of the other candidates (YES in step S510), the CPU 101 adjusts the timbre of the played sample timbre so that it matches the timbre to be played on the information processing device 100, and then proceeds to step S511.

[0079] In step S511, CPU 101 causes GPU 102 to perform additional training of the trained model generated by training unit 201, based on the adjectives extracted in step S501 and the effect parameters used for timbre adjustment selected by the user in step S510. After that, the parameters of the decoder constituting estimation unit 203 are updated with the parameters of the decoder part of the trained model after the additional training, and then this processing ends. As a result, the more the user adjusts timbre using the processing in FIG. 5 when performing on information processing device 100, the more customized effect parameter candidates will be displayed in a list on I / F 603.

[0080] According to this embodiment, when a user inputs a natural language character representing a tone color that the user wants to use in a performance on the information processing device 100 into the I / F 601 on the display unit 108, the CPU 101 sets effect parameter candidates that the user will use to adjust the tone color based on the input natural language character, and displays a play button on the I / F 603 for playing sample tone colors for each candidate. The user presses the play button displayed on the I / F 603 to play the sample tone color, and when the user confirms that the sample tone color is the tone color that the user wants to use in a performance on the information processing device 100, the user can adjust the tone color when performing on the information processing device 100 simply by pressing the enter button 604. In other words, even if the user is a beginner and finds it difficult to adjust the effect parameters that the user wants to use in a performance on the information processing device 100 by operating the many buttons and knobs provided on a conventional synthesizer, the user can easily adjust the tone color when performing on the information processing device 100.

[0081] Furthermore, waveform data for each type of musical instrument to be played on the information processing device 100 can be easily set without operating the many buttons and knobs that are provided on conventional synthesizers.

[0082] The method of additional learning performed in step S511 is not particularly limited. For example, the training data generated in step S403 may be updated based on the content selected by the user using I / F 603 in the process of Fig. 5, or reinforcement learning may be performed in step S510 to reward the user for making a selection.

[0083] In this embodiment, the information processing device 100 performs all of the processes shown in FIGS. 4 and 5, but the present invention is not limited to such a configuration. For example, the information processing device 100 may be connected to a mobile terminal (not shown) such as a tablet or smartphone, or a server (cloud) (not shown), and may operate cooperatively with these devices, i.e., the processing may be shared among the devices, allowing the processing to be performed anywhere. For example, a trained model may be generated in the cloud, and the I / F 601 in FIG. 6 may be displayed on a mobile terminal.

[0084] Any machine learning method can be used to train the learning model and additionally learn the learned model in the learning unit 201. For example, methods such as Gaussian process regression (Bayesian optimization), a policy gradient method which is a type of policy iteration method, and a genetic algorithm which is a method that imitates the process of biological evolution can be adopted.

[0085] The same effect may be achieved by reading a storage medium storing each control program represented by software for achieving the present invention into each device. In this case, the program code itself read from the storage medium realizes the novel functions of the present invention, and the non-transitory computer-readable recording medium storing the program code constitutes the present invention. The program code may also be supplied via a transmission medium, in which case the program code itself constitutes the present invention. In these cases, storage media may include, in addition to ROM, floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, magnetic tape, and non-volatile memory cards. The term "non-transitory computer-readable recording medium" also includes volatile memory (e.g., DRAM (Dynamic Random Access Memory)) within a computer system that acts as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and that retains the program for a certain period of time. [Explanation of symbols]

[0086] 100 Information processing device, 101 CPU, 102 GPU, 105 Operation unit, 107 Speaker, 108 Display unit, 109 HDD, 201 Learning unit, 202 Input unit, 203 Estimation unit, 204 Output unit

Claims

1. An information processing device characterized by adjusting tone using a data structure in which multiple audio data are mapped to form clusters in a specific space as groups of tone data associated with multiple adjectives.

2. receiving a natural language input from a user that includes one of the plurality of adjectives; Identifying clusters corresponding to adjectives contained in the input natural language from the data structure; The information processing apparatus according to claim 1 , wherein at least one piece of timbre data is output from the specified cluster.

3. An information processing device as described in Claim 2, which extracts multiple different tone data from the identified cluster and presents them as candidates selectable by the user as the at least one audio data to be output.

4. An information processing device as described in Claim 3, characterized in that if the input natural language further includes an instrument type, the presented candidates are further narrowed down according to the instrument type.

5. An information processing device as described in Claim 4, characterized in that when a user gives an instruction to play one of the presented candidates, audio adjusted using the candidate instructed to be played and waveform data corresponding to the instrument type is played.

6. An information processing device as described in Claim 3, characterized in that if the input natural language does not include an instrument type, when the user instructs to play one of the presented candidates, audio adjusted using the candidate instructed to be played and waveform data corresponding to the instrument type set by default is played.

7. An information processing device described in any one of claims 3 to 6, further performing a process of updating a data structure based on tone data selected by a user from the presented candidates and the input natural language.

8. A computer-implemented control method for adjusting tone using a data structure in which multiple audio data are mapped to form clusters in a specific space as groups of tone data associated with multiple adjectives.

Images