Information processing system, information processing method, and information processing program

Abstract

An information processing system according to the present disclosure comprises: an acquisition unit that acquires music data to be analyzed; an identification unit that identifies music-related information relating to reproduction of the music data by analyzing the music data; and an output unit that outputs the music-related information.

Description

Information processing system, information processing method, and information processing program

[0001] This disclosure relates to an information processing system, an information processing method, and an information processing program.

[0002] Technologies for automatically generating audio such as music are available. For example, a method for searching for instruments using reference music as a query is provided (see, for example, Non-Patent Document 1).

[0003] Kyungsu Kim, Minju Park, Haesun Joung, Yunkee Chae, Yeongbeom Hong, Seonghyeon Go, Kyogu Lee, "Show Me the Instruments: Musical Instrument Retrieval from Mixture Audio", [online] [Retrieved April 4, 2025], Internet <URL: https: / / arxiv.org / abs / 2211.07951> Josh Gardner, Ian Simon, Ethan Manilow, Curtis Hawthorne, Jesse Engel, "MT3: Multi-Task Multitrack Music Transcription", [online] [Retrieved April 4, 2025], Internet <URL: https: / / arxiv.org / abs / 2111.03017> Gakusei Sato, Taketo Akama, "Annotation-free Automatic Music Transcription with Scalable Synthetic Data and Adversarial Domain "Confusion", [online] [Retrieved April 4, 2025], Internet <URL: https: / / arxiv.org / abs / 2312.10402>

[0004] However, the conventional art has room for improvement. For example, in the conventional art, although it is possible to identify the types of musical instruments such as guitars and pianos used in music data such as music through a search using a query, the type of musical instrument is not sufficient to reproduce the music data, making it difficult to appropriately reproduce the music data. Therefore, it is desired to appropriately identify information related to the reproduction of music data.

[0005] Accordingly, the present disclosure proposes an information processing system, an information processing method, and an information processing program that can appropriately identify information related to the reproduction of music data.

[0006] To solve the above problem, an information processing system according to an aspect of the present disclosure includes an acquisition unit that acquires music data to be analyzed, an identification unit that identifies music-related information related to reproduction of the music data by analyzing the music data, and an output unit that outputs the music-related information.

[0007] It is a diagram showing an example of information processing according to an embodiment of the present disclosure. It is a diagram showing an example of processing executed by an information processing apparatus. It is a diagram showing a configuration example of an information processing system according to an embodiment of the present disclosure. It is a flowchart showing the procedure of information processing according to an embodiment of the present disclosure. It is a diagram showing another example of information processing executed by the information processing system. It is a diagram showing an example of a service provided by the information processing system. It is a schematic diagram showing an example of learning of a model used for information processing. It is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing apparatus.

[0008] Hereinafter, embodiments of the present disclosure will be described in detail based on the drawings. Note that the information processing system, information processing method, and information processing program according to the present application are not limited by this embodiment. Further, in the following respective embodiments, the same reference numerals are assigned to the same parts, thereby omitting duplicate descriptions.

[0009] This disclosure will be described in the following order of items: 1. Embodiments 1-1. Overview of information processing according to the embodiment of this disclosure 1-2. Example of an information processing system according to the embodiment 1-3. Configuration of the information processing device according to the embodiment 1-4. Procedure for information processing according to the embodiment 1-5. Other information processing examples (musical note sequence) 2. Other embodiments 2-1. Other system configuration examples, etc. 2-2. Others 3. Hardware configuration

[0010] <1. Embodiments> <1-1. Overview of Information Processing According to the Embodiments of the Disclosure> First, an overview of the information processing according to the embodiments of the Disclosure will be described using Figures 1 and 2. Figure 1 is a diagram showing an example of information processing according to the embodiment of the Disclosure. Figure 2 is a diagram showing an example of processing performed by the information processing device. Specifically, in Figures 1 and 2, an overview of the information processing will be described as an example in which instrument information is identified as information related to the reproduction of the target music data (also called "music-related information") for the music data to be analyzed (also called "target music data"). Note that the music-related information is not limited to instrument information but may be any information that can reproduce music data such as song data.

[0011] The information processing according to the embodiment of this disclosure is realized by an information processing system 1 (see Figure 3) having an information processing device 100. Details of the configuration will be described later, but the information processing device 100 is an example of a computer (inference device) that performs specific processing to identify music-related information relating to the reproduction of music data. In Figure 1, the information processing device 100 uses a music analysis model M1, referred to as a "deep learning model," to take target music data as input and output output information relating to music-related information of the target music data. For example, the music analysis model M1 is an AI (Artificial Intelligence) model (also simply called a "model") generated by machine learning methods such as deep learning.

[0012] Figure 1 shows an example of how the information processing system 1 processes audio a, which is music data, as the target music data. In Figure 1, the content that presents information to the user and accepts information input from the user is shown as Content CT11 to CT14 depending on the display content, but when not distinguishing between them, it may be referred to as Content CT. For example, Content CT is content that provides the user with an AI-assisted music production service. For example, Content CT is a display screen for the user to perform music production using the information processing system 1.

[0013] Information processing system 1 receives input from the user via content CT. Information processing system 1 receives an instruction from the user via content CT to set audio a as the target music data. For example, information processing device 100 sets audio a as the target music data in response to the user's operation. Content CT 11 indicates that audio a has been set as the target music data, and the waveform to the right of the string "audio a" shows the audio waveform of audio a (for example, corresponding to audio waveform OW11 in Figure 2). Information processing device 100 displays content CT 11.

[0014] The information processing system 1 performs a specific process to identify music-related information concerning the reproduction of the target music data (step S11). In Figure 1, the information processing system 1 uses audio a as the target music data and performs a specific process to identify instrument information concerning the reproduction of audio a. For example, the information processing system 1 performs the specific process using a music analysis model M1 as shown in Figure 2.

[0015] The music analysis model M1 outputs music-related information corresponding to the music data when music data is input. For example, when music data is input, the music analysis model M1 outputs output information that includes at least one of the following: characteristic information of the music data, information on the constituent elements of the music data, or parameters of a tool that reproduces the music data. For example, the tool could be a synthesizer, sampler, or neural synthesizer. For example, a synthesizer is a tool (software) that synthesizes (generates) musical sounds etc. using electronic engineering methods. For example, a sampler is a tool (software) used when sampling existing sounds. For example, a neural synthesizer is a tool (software) that synthesizes (generates) musical sounds etc. using machine learning techniques.

[0016] Furthermore, the above is merely an example, and the music analysis model M1 may output various types of information. For example, the music analysis model M1 may output sound data used to reproduce music data such as one-shot data, and information about the effects applied to that sound data. For example, one-shot data is audio sample data in the MIDI (Musical Instruments Digital Interface) standard. One-shot data is audio data in which sounds are pre-registered, and is audio data of sounds divided into a certain unit (in this case, pitch), such as the note "C" or the note "D". For example, one-shot data is audio data of individual notes, such as the note "C" or the note "D". Note that one-shot data is not limited to pitch, and may be audio data of sounds divided into any unit. Also, one-shot data may be audio data of actual sounds pre-recorded in real space.

[0017] The music analysis model M1 may be trained using any learning method, as long as it can output the desired output information. Here, an example of training the music analysis model M1 will be explained using Figure 7. Figure 7 is a schematic diagram showing an example of training a model used in information processing. First, let's explain each element shown in Figure 7. Audio data OD1 to OD4 are audio data generated by a tool TL1, such as a synthesizer or sampler.

[0018] Note that when explaining audio data OD1 to OD4 without distinguishing between them, they may be referred to simply as audio data OD. For example, audio data OD is an element (constituent information) that makes up music data such as audio a, and is audio data used as part of music data. Note that audio data OD is not limited to audio data generated by tool TL1, but may also be, for example, one-shot data. MIDI data MD1 is data in a format based on the MIDI standard (MIDI data). For example, MIDI data is data (a sequence of notes) that digitizes musical performance information so that it can be played back on a computer (electronic instrument, PC, etc.). A detailed explanation of MIDI data is omitted, but MIDI data is data like a digital musical score, where the horizontal axis represents time and the vertical axis represents pitch. Pitch corresponds to the keys on a piano, for example.

[0019] The audio waveform OW21 represents the audio waveform of music data generated using audio data OD1 to OD4. For example, the audio waveform OW21 corresponds to music data created by pasting and synthesizing audio data OD1 to OD4 along MIDI data MD1. In Figure 7, the audio waveform OW21 is generated from audio data OD1 to OD4 and MIDI data MD1 by a waveform concatenation type speech synthesis process PS1, which is labeled "Concatenative Synthesis".

[0020] Furthermore, the audio waveform OW22 corresponds to music data to which effects have been applied to the audio waveform OW21. The audio waveform OW22 is generated by the effect processing PS2, which is labeled as "effector," applied to the audio waveform OW21. Note that if no effects are applied to the music data, the effect processing PS2 does not need to be performed.

[0021] For training the music analysis model M1, data is used in which information related to the generation of audio waveforms OW22 (music data) is associated with audio waveforms OW22. For example, the music analysis model M1 is trained using training data in which information related to effects added during the production process of audio waveforms OW22 (music data) (also called "effect-related information") and information related to the timbre contained in audio waveforms OW22 (music data) (also called "timbre-related information") are associated with audio waveforms OW22 (music data) as correct answer information. In this way, for training the music analysis model M1, music data is used in which correct answer information, including effect-related information and timbre-related information that shows the production process of the music data, is associated with each piece of music data such as audio waveforms OW22.

[0022] For example, effect-related information includes parameters of effects in the PS2 effect processing during the production process. Also, for example, timbre-related information includes audio data OD or its embedding (vector) contained in the audio waveform OW22, and parameters of tool TL1. The parameters of tool TL1 are parameters set in tool TL1 to reproduce the timbre of the audio data OD. For example, the parameters of tool TL1 are parameters set in tool TL1 when generating (synthesizing) data to reproduce the timbre of the audio data OD in tool TL1. For example, the embedding of audio data OD is information that indicates the characteristics of the audio data OD (characteristic information). Note that the above is just an example, and various types of information may be included in the effect-related information and timbre-related information. For example, timbre-related information may include information that indicates tool TL1 itself.

[0023] For example, when an audio waveform OW22 is input, the music analysis model M1 is trained to output effect-related information and timbre-related information of the audio waveform OW22, as shown in the output information OT21.

[0024] For example, in the training of a music analysis model M1, the training process is carried out using methods such as backpropagation so that the output information produced by the music analysis model M1, which receives music data as input, approaches the correct information associated with that music data. For example, in the training of a music analysis model M1, the values ​​of the weights (i.e., connection coefficients) that are considered when values ​​are transmitted between nodes during the training process are adjusted. In this way, the music analysis model M1 is trained by processes such as backpropagation that correct the parameters (connection coefficients) so that the error between the output of the music analysis model M1 and the correct information corresponding to the input is reduced. For example, the music analysis model M1 is trained (generated) by performing processes such as backpropagation to minimize a predetermined loss function.

[0025] The above description regarding the training of the music analysis model M1 is illustrative, and the training of the music analysis model M1 may be carried out using a training method appropriately selected according to the desired output information, obtainable information, etc. In other words, the training of the music analysis model M1 may be carried out using any method that enables it to be trained to output information corresponding to the correct answer information when input information contained in the training data is input.

[0026] For example, the music analysis model M1 can employ any model, such as an LLM (Large-Scale Language Model), as long as it can produce the desired output in response to the input. In this case, the music analysis model M1 may be a model that takes music data and instruction information, such as prompts that indicate output information, as input information, and outputs the information indicated by the instruction information for the input music data. For example, the music analysis model M1 may be a model that takes music data and instruction information such as "Please output a ranking of suitable tools and their parameter combinations for reproducing this instrument data" as input information, and outputs a ranking of each tool and its parameter combination as output information for the input music data.

[0027] As described above, Figure 7 shows how to synthesize (generate) musical audio (music data) from MIDI and one-shot (single-note) audio using concatenative synthesis and effect application. For example, in the training process, a deep learning model is trained that takes musical data as input and outputs MIDI and parameters such as one-shot embeddings, audio, and effects for each note. At that time, by learning the embeddings by distance, it becomes possible to search for sound sources such as one-shots, samplers, synthesizers, and neural synthesizers. Alternatively, the parameters of samplers, synthesizers, and neural synthesizers can be directly estimated instead of embeddings and audio.

[0028] Let's return to Figure 2 and continue explaining the specific processing. Below, we will explain, as an example, the case in which the music analysis model M1 outputs information on multiple instruments of the music data as output information when music data is input. In Figure 2, when music data is input, the music analysis model M1 outputs information showing the ranking of multiple instrument information of the music data as output information. For example, the instrument information includes information indicating the tool and its parameters.

[0029] The information processing system 1 inputs the audio waveform OW11 of the target music data, audio a, into the music analysis model M1, and causes the music analysis model M1 to output output information OT11, which includes information indicating the ranking of multiple instrument information that reproduce audio a. For example, by using the music analysis model M1, the information processing system 1 identifies instrument a1, instrument a2, and instrument a3 as multiple instrument information that reproduce audio a.

[0030] In Figure 2, the music analysis model M1, upon receiving audio a as input, outputs output information OT11 that includes information indicating the ranking of instruments a1, a2, and a3 in that order. Each of instruments a1 to a3 includes information indicating the tools and parameters used to reproduce audio a. As a result, the information processing system 1 identifies multiple instrument information that reproduce audio a.

[0031] Furthermore, if the music analysis model M1 outputs effect information and timbre information of the music data as output information, the information processing system 1 may use the output information to identify instrument information. For example, the information processing system 1 may identify effect information and timbre information similar to the output information output by the music analysis model M1 as instrument information by searching a data set of candidate music-related information that are candidates for instrument information in which effect information and timbre information have been registered. Each candidate music-related information includes information on the tool and its parameters corresponding to that embedding in its embedding.

[0032] For example, the information processing device 100 may extract instrument information similar to the output information of the music analysis model M1 from the data group of candidate music-related information stored in the storage unit 14 (see Figure 3). In this case, the information processing device 100 uses the output information output by the music analysis model M1, which has audio a as input, to search the data group of candidate music-related information, and extracts instrument information similar to the output information of the music analysis model M1 from the data group of candidate music-related information.

[0033] For example, the information processing device 100 extracts instrument information from music data by searching a data set of candidate music-related information using characteristic information such as the embedding of the music data. The information processing device 100 extracts instrument information from the data set of candidate music-related information based on the similarity between the embedding of the target music data output by the music analysis model M1, which has the target music data as input, and the embedding of each candidate music-related information. For example, the information processing device 100 extracts a predetermined number of candidate music-related information as instrument information, in order of decreasing similarity to the embedding of the target music data. The information processing device 100 determines a ranking of the predetermined number of extracted instrument information, in order of decreasing similarity to the embedding of the target music data.

[0034] The information processing device 100 identifies the extracted instrument information as the instrument information of audio a. Note that the above is merely an example; the information processing system 1 may identify the desired instrument information for the target music data by any processing method if it is possible to do so. As described above, by identifying the instrument information for the music data, the information processing system 1 can appropriately identify information related to the reproduction of the music data.

[0035] The information processing system 1 then displays a ranking IL 11 of instrument information that reproduces audio a, as shown in the content CT 12 of Figure 1. The content CT 12 includes a ranking IL 11 arranged vertically in the order of instrument a1, instrument a2, and instrument a3. In this way, the content CT 12 shows a state in which multiple instrument information identified for the target music data, audio a, is displayed in ranking order. The information processing device 100 displays the content CT 12.

[0036] When one of several instrument information entries is selected, the content CT transitions to a display mode that shows detailed information for that single instrument. An example of this is explained below.

[0037] For example, when the user positions the mouse cursor over the display area of ​​instrument a1 in ranking IL 11, the information processing system 1 displays detailed information IS 13 of instrument a1, as shown in content CT 13. The detailed information IS 13 includes information indicating the tools of instrument a1 that reproduce audio a and the parameters set for those tools. For example, when the user positions the mouse cursor outside the display area of ​​instrument a1 in ranking IL 11, the information processing system 1 terminates the display mode of content CT 13 and transitions the display to another display mode of content CT. For example, when the user positions the mouse cursor outside the display area of ​​ranking IL 11, the information processing system 1 transitions the display to the display mode of content CT 12.

[0038] For example, when the user positions the mouse cursor over the display area of ​​instrument a3 in the ranking IL 11, the information processing system 1 displays detailed information IS 14 of instrument a3, as shown in content CT 14. The detailed information IS 14 includes information indicating the tools of instrument a3 that reproduce audio a and the parameters set for those tools. For example, when the user positions the mouse cursor outside the display area of ​​instrument a3 in the ranking IL 11, the information processing system 1 terminates the display mode of content CT 14 and transitions the display to another display mode of content CT. For example, when the user positions the mouse cursor outside the display area of ​​ranking IL 11, the information processing system 1 transitions the display to the display mode of content CT 12. Note that the display of instrument a2 is the same as that of instrument a1 and instrument a3 described above, so a detailed explanation is omitted.

[0039] For example, there are conventional systems that, when a music audio file (music data) is input, display information about the MIDI (note sequences) or instruments contained within that music audio file. For instance, music includes not only mixed music, but also STEM, parameter data, loops, and other data broken down into individual instruments or short phrase-level data.

[0040] For example, in conventional systems that take a music audio file as input and convert it to MIDI, instrument information cannot be displayed, and effects cannot be estimated. Also, in conventional systems that take a music audio file as input and search for the timbres contained within it, effects cannot be estimated, the MIDI notes and timbres are not linked, it is not possible to respond to individual timbre changes, and editing and reuse, including phrases, is not possible.

[0041] Furthermore, conventional systems that present MIDI data and instrument names using GM (General MIDI) program numbers when a music track is entered only support a limited number of instrument types, up to 128, according to the GM program numbers. They do not support synthesizers, samplers, neural synthesizers, or one-shots, and do not estimate effects. Thus, conventional systems have difficulty appropriately identifying information related to the reproduction of music data.

[0042] Therefore, by performing the processing described above, the information processing system 1 can appropriately identify information related to the reproduction of music data. For example, the information processing system 1 analyzes which instrument can be used to create (reproduce) the sound of music data, such as song data, and presents it in a ranking format. For example, the information processing system 1 can show which sampler, synthesizer, neural synthesizer, or one-shot instrument, and with what parameters, can be used to create a sound similar to that music data.

[0043] Note that the information processing system 1 may perform various processes. For example, the information processing system 1 may narrow down the candidates (such as music-related information candidates) to only those within the synthesizer owned by the user. The information processing system 1 may also present effect parameters. Furthermore, the information processing system 1 may generate information indicating variations of instrument information (tools, parameters, effects, etc.) and present the information to the user. Thereby, the information processing system 1 can encourage the user to generate editing ideas. As such, the information processing system 1 is a system useful for music-related production, performance, education, marketing, and the like.

[0044] For example, the information processing system 1 is a system useful in the field of music production. For example, the information processing system 1 is useful for pre-production. For example, the information processing system 1 is useful for generating ideas such as timbre composition, soundscape, and timbre development from a reference musical piece (music).

[0045] For example, the information processing system 1 is useful in the field of arrangement. For example, the information processing system 1 is useful for arranging into similar but different timbres. For example, the information processing system 1 is useful for remixing. For example, the information processing system 1 is useful for borrowing timbres from other musical pieces.

[0046] For example, the information processing system 1 is useful in the field of music performance. For example, the information processing system 1 is useful for sessions and jamming. For example, the information processing system 1 is useful for capturing the timbre of another person's performance result and creating a performance based thereon. For example, the information processing system 1 is useful in the field of DJ (Disk Jockey). For example, the information processing system 1 is useful for selecting a musical piece with a similar timbre as the next music piece.

[0047] For example, the information processing system 1 is useful in the field of live looping. For example, the information processing system 1 is useful for successively layering partially similar timbres one over another.

[0048] For example, the information processing system 1 is useful in the field of performance review and integration with creation. For example, the information processing system 1 is useful for loading the timbre of performance results (timbre that also reflects playing techniques) into a DAW (Digital Audio Workstation), reviewing the performance, and transitioning to creation.

[0049] For example, the information processing system 1 is useful in the field of using products such as AI as inputs. For example, the information processing system 1 is useful for creating timbres from generated music and sounds.

[0050] For example, the information processing system 1 is useful in the field of music education. For example, the information processing system 1 is useful for musical composition (music) timbre configuration analysis. For example, the information processing system 1 is useful in the field of music marketing. For example, the information processing system 1 is useful for hit song analysis where timbre is a key factor.

[0051] <1-2. Example of Information Processing System According to Embodiment> The information processing outlined in FIG. 1 and FIG. 2 is implemented by the information processing system 1 shown in FIG. 3. FIG. 3 is a diagram illustrating a configuration example of the information processing system according to the embodiment of the present disclosure. As shown in FIG. 3, the information processing system 1 includes at least one information processing apparatus 100. The information processing apparatus 100 is communicably connected to other information processing apparatuses via a predetermined network (such as communication network N) in a wired or wireless manner.

[0052] Note that the configuration of the information processing system 1 shown in FIG. 3 is an example, and any apparatus configuration can be adopted as long as the information processing system 1 includes at least one information processing apparatus 100. For example, the information processing system 1 may include a plurality of information processing apparatuses 100. In addition, although FIG. 3 shows a case where the information processing apparatus 100 is a terminal apparatus used by a user as an example, the information processing apparatus 100 may be a server apparatus such as a cloud server. In this case, the information processing system 1 may include the information processing apparatus 100 as a server apparatus and a terminal apparatus used by a user, which will be described later.

[0053] <1-3. Configuration of an Information Processing Device According to an Embodiment> Next, the configuration of an information processing device 100, which is an example of an information processing device that performs information processing according to an embodiment, will be described. For example, the information processing device 100 shown in Figure 3 is an example of an information processing device. The information processing device 100 is a computer that realizes the functions of an information processing device, which will be described later.

[0054] As shown in Figure 3, the information processing device 100 includes a communication unit 11, an input unit 12, an audio output unit 13, a storage unit 14, a control unit 15, and a display unit 16. In the example in Figure 3, the information processing device 100 includes an input unit 12 (for example, a keyboard or mouse) that receives various operations from a user of the information processing device 100, and a display unit 16 (for example, a liquid crystal display) that displays various information.

[0055] The communication unit 11 is implemented, for example, by a NIC (Network Interface Card) or a communication circuit. The communication unit 11 is connected to a communication network N (such as the Internet) by wire or wireless connection, and transmits and receives information with other devices such as user terminals, which are other terminal devices used by the user, via the communication network N.

[0056] The input unit 12 accepts various inputs. The input unit 12 accepts user inputs. The input unit 12 accepts various user operations as inputs. The input unit 12 accepts music data to be used for processing. The input unit 12 may also accept user designation (selection) of music data to be used for processing from among the music data stored in the music data storage unit 141 of the storage unit 14.

[0057] The input unit 12 may receive various operations from the user via a keyboard, mouse, or touch panel provided on the information processing device 100. The input unit 12 also accepts detection by sensors as input. The input unit 12 may also accept user speech as input via a sound sensor (such as a microphone) that has the function of detecting sound.

[0058] The audio output unit 13 outputs sound. The audio output unit 13 is implemented by a speaker that outputs sound. The audio output unit 13 outputs the sound of the played music data. For example, the audio output unit 13 outputs the sound of music data that has been played by the user.

[0059] The storage unit 14 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs. The storage unit 14 includes a music data storage unit 141, a model information storage unit 142, a specific result information storage unit 143, and a generation information storage unit 144.

[0060] Furthermore, the storage unit 14 stores various types of information, not limited to the information stored in the music data storage unit 141, the model information storage unit 142, the specific result information storage unit 143, and the generation information storage unit 144. The storage unit 14 stores information about music production programs (music production programs). For example, the storage unit 14 stores music production software that generates music as a music production program. For example, the storage unit 14 stores information about instrument production applications (programs).

[0061] The music data storage unit 141 stores information related to music data. For example, the music data storage unit 141 stores music data to be analyzed. For example, the music data storage unit 141 stores music data entered by the user. For example, the music data storage unit 141 stores music data specified by the user.

[0062] The music data storage unit 141 is not limited to the above and may store various types of information depending on the purpose. For example, the music data storage unit 141 may store a file path name indicating the location where the music data is stored, a URL (Uniform Resource Locator), and the like.

[0063] The model information storage unit 142 stores information about the machine learning model. For example, the model information storage unit 142 stores information (model data) that shows the structure of the model (network). For example, the model information storage unit 142 includes items such as "model ID," "purpose," and "model data."

[0064] The "Model ID" field stores identification information to identify the model. The "Purpose" field stores information indicating the purpose of the corresponding model. The "Model Data" field stores information indicating the model's data. For example, the data stored in "Model Data" includes various pieces of information that make up the model, such as information about the network included in the model and functions.

[0065] For example, the model information storage unit 142 stores information indicating that its purpose is "music data analysis" associated with the model ID "M1" that identifies the deep learning model (music analysis model M1). The music analysis model M1 is a model that, for example, takes music data as input, analyzes the input music data, and outputs information based on the analysis of the music data (for example, music-related information).

[0066] The model information storage unit 142 is not limited to the above and may store various types of information depending on the purpose. For example, the model information storage unit 142 may store the training data used to train each model.

[0067] The specific result information storage unit 143 stores various types of information obtained through specific processing. The specific result information storage unit 143 stores the identified music-related information. For example, the specific result information storage unit 143 stores the identified instrument information. The specific result information storage unit 143 stores music-related information in association with the music data (target music data) that was the subject of the specific processing. For example, the specific result information storage unit 143 stores the identified instrument information in association with the music data (target music data) that was the subject of the specific processing.

[0068] Furthermore, the specific result information storage unit 143 is not limited to the above and may store various types of information depending on the purpose. The specific result information storage unit 143 may also store date and time information of when the specific process was executed. In this case, for example, the specific result information storage unit 143 stores the date and time information of when the specific process was executed in association with the music data (target music data) that was the subject of the specific process.

[0069] The generated information storage unit 144 stores various types of information obtained through the generation process. The generated information storage unit 144 stores generated (produced) music data. The generated information storage unit 144 stores music data generated (produced) using music-related information. The generated information storage unit 144 stores data of generated content. The generated information storage unit 144 stores content generated using music-related information.

[0070] The generated information storage unit 144 is not limited to the above and may store various types of information depending on the purpose.

[0071] The control unit 15 is implemented, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing a program (for example, the information processing program according to this disclosure) stored inside the information processing device 100 using RAM (Random Access Memory) or the like as a working area. The control unit 15 is also a controller and may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0072] As shown in Figure 3, the control unit 15 includes an acquisition unit 151, a specification unit 152, a generation unit 153, and a transmission unit 154, and realizes or executes the information processing functions and operations described below. Note that the control unit 15 is not limited to the configuration shown in Figure 3, and may have other configurations as long as it performs information processing. For example, if the information processing device 100 performs a learning process to learn a machine learning model such as a music analysis model M1, the control unit 15 may have a learning unit that performs the learning process.

[0073] The acquisition unit 151 acquires various types of information. The acquisition unit 151 acquires various types of information from other devices such as user terminals. The acquisition unit 151 acquires various types of information from the storage unit 14. The acquisition unit 151 acquires information received by the input unit 12.

[0074] The acquisition unit 151 acquires music data to be used for processing from the music data storage unit 141. The acquisition unit 151 acquires information to be used for estimation processing from the model information storage unit 142. The acquisition unit 151 acquires information to be used for generation processing from the specific result information storage unit 143. The acquisition unit 151 acquires content to be output from the generation information storage unit 144.

[0075] The acquisition unit 151 acquires the information identified by the identification unit 152. The acquisition unit 151 acquires the information generated by the generation unit 153. The acquisition unit 151 receives information from other devices such as user terminals via the communication unit 11. The acquisition unit 151 acquires the music data to be analyzed.

[0076] The identification unit 152 identifies various types of information. The identification unit 152 performs identification processing. For example, the identification unit 152 identifies various types of information based on information acquired by the acquisition unit 151. For example, the identification unit 152 identifies various types of information based on information stored in the storage unit 14. The identification unit 152 identifies various types of information based on information stored in the music data storage unit 141 and the model information storage unit 142.

[0077] The identification unit 152 identifies music-related information concerning the reproduction of music data by analyzing the music data. The identification unit 152 identifies parameters related to the tools used to reproduce the music data as music-related information. The identification unit 152 identifies parameters related to tools that include at least one of a sampler, a synthesizer, or a neural synthesizer. In this way, the information processing system 1 can appropriately identify information related to the reproduction of music data by identifying parameters related to the tools used to reproduce the music data.

[0078] The identification unit 152 identifies, as music-related information, sound data used to reproduce music data and information about effects applied to the sound data. The identification unit 152 identifies sound data that is one-shot data and information about effects applied to the sound data. In this way, the information processing system 1 can appropriately identify information related to the reproduction of music data by identifying the sound data and information about effects used to reproduce music data.

[0079] The identification unit 152 identifies note information corresponding to the music data by analyzing the music data. The identification unit 152 identifies multiple pieces of music-related information corresponding to the music data by analyzing the music data.

[0080] The identification unit 152 determines the ranking of multiple music-related pieces of information corresponding to the music data. The identification unit 152 determines the ranking based on the degree of similarity between the information about the music data and the information about the music reproduced by each of the multiple music-related pieces of information.

[0081] The identification unit 152 identifies music-related information of music data using a model that takes music data as input and outputs output information related to the input music data. The identification unit 152 identifies music-related information of music data using a model that outputs music-related information corresponding to music data as output information when music data is input.

[0082] The identification unit 152 identifies the output information output by the model that has received music data as music-related information of the music data. The identification unit 152 uses a model that outputs multiple pieces of music-related information of the music data as output information when music data is input. The identification unit 152 identifies at least one of the multiple pieces of music-related information included in the output information output by the model that has received music data as music-related information of the music data.

[0083] The identification unit 152 uses a model that, when music data is input, outputs multiple music-related information and information indicating the ranking of the multiple music-related information as output information. Based on the information indicating the ranking of the multiple music-related information output by the model to which the music data was input, the identification unit 152 determines the ranking of the multiple music-related information.

[0084] The identification unit 152 uses a model that, when music data is input, outputs output information that includes at least one of the following: feature information of the music data, component information that constitutes the music data, and parameters of a tool that reproduces the music data. The identification unit 152 identifies music-related information of the music data using at least one of the feature information, component information, and parameters output by the model to which the music data was input. The identification unit 152 identifies music-related information of the music data by searching for a data set containing candidate music-related information using the feature information output by the model to which the music data was input.

[0085] The generation unit 153 generates various types of information. The generation unit 153 executes generation processing. For example, the generation unit 153 generates various types of information based on information acquired by the acquisition unit 151. For example, the generation unit 153 generates various types of information based on information stored in the storage unit 14. The generation unit 153 generates various types of information based on information identified by the identification unit 152. For example, the generation unit 153 generates various types of information based on information stored in the identified result information storage unit 143.

[0086] The generation unit 153 generates various information according to the user's instructions. The generation unit 153 generates music data. The generation unit 153 generates music data using music-related information. The generation unit 153 generates content. The generation unit 153 generates content using music-related information.

[0087] The generation unit 153 generates various information, such as content to be output, using various technologies as appropriate. The generation unit 153 generates content to be displayed (output) on the display unit 16. For example, the generation unit 153 generates content to be displayed (output) on the display unit 16 based on information stored in the storage unit 14.

[0088] The generation unit 153 may generate content such as content CT by any processing method, as long as it is capable of generating content. For example, the generation unit 153 may use various technologies related to image generation and image processing as appropriate to generate content to be displayed (output) on the display unit 16. For example, the generation unit 153 may use various technologies such as Java® as appropriate to generate content to be displayed (output) on the display unit 16. The generation unit 153 may also generate content to be displayed (output) on the display unit 16 based on formats such as CSS, JavaScript®, or HTML (HyperText Markup Language). Furthermore, for example, the generation unit 153 may generate content in various formats such as JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), or PNG (Portable Network Graphics). In addition, the generation unit 153 may generate content to be displayed (output) on the display unit 16.

[0089] The transmitting unit 154 transmits various types of information. The transmitting unit 154 functions as an output unit that outputs information by transmitting it. The transmitting unit 154 provides various types of information. The transmitting unit 154 transmits various types of information to other devices such as user terminals. The transmitting unit 154 transmits information stored in the storage unit 14. The transmitting unit 154 transmits information stored in the music data storage unit 141, the model information storage unit 142, the specific result information storage unit 143, and the generated information storage unit 144.

[0090] The transmitting unit 154 transmits the identification result from the identification unit 152. The transmitting unit 154 transmits music-related information. The transmitting unit 154 transmits musical note information. The transmitting unit 154 transmits musical note information along with music-related information. The transmitting unit 154 transmits at least one of several pieces of music-related information. The transmitting unit 154 transmits multiple pieces of music-related information in ranking order.

[0091] The transmitting unit 154 transmits the information generated by the generating unit 153. The transmitting unit 154 transmits the content generated by the generating unit 153 to other devices such as user terminals. The transmitting unit 154 transmits content in which multiple music-related pieces of information are displayed in ranking order. When one of the multiple music-related pieces of information is specified, the transmitting unit 154 transmits content that displays detailed information about that one piece of music-related information. In this way, the information processing system 1 can appropriately output information related to the reproduction of music data by transmitting information based on information that identifies the reproduction of music data.

[0092] The display unit 16 displays various information. The display unit 16 functions as an output unit that outputs information by displaying it. The display unit 16 is a display device (display unit) such as a display and displays various information. The display unit 16 displays information of the identification result by the identification unit 152. The display unit 16 displays information generated by the generation unit 153. The display unit 16 displays content generated by the generation unit 153. The display unit 16 displays content whose display mode has been updated according to user operation.

[0093] The display unit 16 displays music-related information. The display unit 16 displays musical note information. The display unit 16 displays musical note information together with music-related information. The display unit 16 displays at least one of several pieces of music-related information. The display unit 16 displays multiple pieces of music-related information in ranking order.

[0094] The display unit 16 displays information generated by the generation unit 153. The display unit 16 displays content in which multiple music-related pieces of information are arranged in ranking order. When one of the multiple music-related pieces of information is specified, the display unit 16 displays content that shows detailed information about that one piece of music-related information. In this way, the information processing system 1 can appropriately output information related to the reproduction of music data by displaying information based on information identified regarding the reproduction of music data.

[0095] <1-4. Information Processing Procedure According to the Embodiment> Next, the information processing procedure according to the embodiment will be described using Figure 4. Figure 4 is a flowchart of the information processing procedure according to the embodiment of this disclosure.

[0096] As shown in Figure 4, the information processing device 100 acquires the music data to be analyzed (step S101). For example, the information processing device 100 acquires music data input by the user as the music data to be analyzed (target music data).

[0097] Then, the information processing device 100 identifies music-related information, which is information related to the reproduction of the music data, by analyzing the music data (step S102). For example, the information processing device 100 inputs the target music data entered by the user into a music analysis model M1, which is a deep learning model, and uses the output information output by the music analysis model M1 to identify instrument information, which is music-related information related to the reproduction of the target music data.

[0098] Then, the information processing device 100 outputs music-related information (step S103). For example, the information processing device 100 outputs instrument information related to the reproduction of the target music data by displaying instrument information, which is music-related information related to the reproduction of the target music data.

[0099] <1-5. Other Information Processing Examples (Note Sequences)> Note that the information processing system 1 may perform various processes, not limited to those described above. The information processing system 1 may present various types of information in addition to instrument information. For example, in addition to instrument information, the information processing system 1 may present note sequences such as MIDI. This point will be explained using Figure 5. Figure 5 is a diagram showing another example of information processing performed by the information processing system. Note that explanations of points similar to those described above in Figure 1, etc., will be omitted as appropriate.

[0100] Figure 5 shows an example of how the information processing system 1 processes audio a, which is music data, as the target music data. In Figure 5, content that presents information to the user and accepts information input from the user is shown as Content CT21 to CT24 depending on the display content, but in the explanation without distinction, it may be referred to as Content CT20. For example, Content CT20 is content that provides the user with an AI-assisted music production service, similar to Content CT. For example, Content CT20 is a display screen for the user to perform music production using the information processing system 1.

[0101] Information processing system 1 receives input from the user via content CT 20. Information processing system 1 receives an instruction from the user via content CT 20 to select audio a as the target music data. Content CT 21, like content CT 11, indicates that audio a has been set as the target music data. Information processing device 100 displays content CT 21.

[0102] The information processing system 1 performs a specific process to identify music-related information concerning the reproduction of the target music data (step S21). In Figure 5, the information processing system 1 uses audio a as the target music data and performs a specific process to identify instrument information concerning the reproduction of audio a. For example, the information processing system 1 performs the specific process using the music analysis model M1, similar to the process shown in Figure 1. In Figure 5, the information processing system 1 identifies multiple pieces of instrument information that reproduce audio a, similar to the process shown in Figure 1.

[0103] The information processing system 1 then displays a ranking IL21 of instrument information that reproduces audio a, and note sequence information SN21 corresponding to each instrument information, as shown in the content CT22 of Figure 5. The ranking IL21 in Figure 5 is the same as the ranking IL11 in Figure 1, so a detailed explanation is omitted. The note sequence information SN21 includes note sequences (MIDI data, etc.) corresponding to each of instrument a1, instrument a2, and instrument a3. For example, the note sequence information SN21 may be note information (data) based on the MIDI standard. For example, the note information may be various information related to notes, such as pitch, onset, offset, velocity, and pitch bend.

[0104] Each note sequence in the note sequence information SN21 is generated from audio a and the corresponding instrument information. For example, the information processing system 1 generates a note sequence corresponding to instrument a1 based on audio a and instrument a1. For example, the information processing system 1 generates a note sequence corresponding to instrument a2 based on audio a and instrument a2. For example, the information processing system 1 generates a note sequence corresponding to instrument a3 based on audio a and instrument a3. Each note sequence in the note sequence information SN21 may also be included in the output information output by the music analysis model M1. In this case, the information processing system 1 uses the output information output by the music analysis model M1 to identify the note sequence corresponding to the music data. The information processing system 1 uses the output information output by the music analysis model M1 to identify the note sequence corresponding to each of the multiple instrument information.

[0105] Furthermore, similar to Content CT, Content CT20 transitions to a display mode that displays detailed information of a single instrument when one instrument information is selected from among multiple instrument information. For example, when the user positions the mouse cursor over the display area of ​​instrument a1 in Ranking IL21, Information Processing System 1 displays the detailed information IS23 of instrument a1, as shown in Content CT23. The detailed information IS23 is the same information as the detailed information IS13 in Figure 1. For example, when the user positions the mouse cursor over the display area of ​​instrument a3 in Ranking IL21, Information Processing System 1 displays the detailed information IS24 of instrument a3, as shown in Content CT24. The detailed information IS24 is the same information as the detailed information IS14 in Figure 1. The display transitions of Content CT20 are the same as those described in Figure 1, so a detailed explanation is omitted.

[0106] By performing the processing described above, the information processing system 1 can appropriately identify information related to the reproduction of music data. For example, the information processing system 1 analyzes which instruments and note sequences can be used to create (reproduce) the sound of music data, such as song data, and presents this information in a ranking format. For example, the information processing system 1 can present appropriate information to the user by ranking and presenting instrument information associated with note sequences (such as MIDI note sets).

[0107] Furthermore, the information processing system 1 may perform various processes. For example, the information processing system 1 may assign instrument information to each note. The information processing system 1 may also identify the automation parameters (which change over time) of tools (instruments) and effectors. In addition, the information processing system 1 may output performance technique information such as legato, staccato, tremolo, and vibrato.

[0108] Furthermore, the information processing system 1 may output refined music data (music audio files) from audio synthesized using MIDI (note sequences) and instrument information. The information processing system 1 may also generate variations of MIDI and tools (instruments), as well as in-painting effects. This allows the information processing system 1 to facilitate the generation of editing ideas by the user. Thus, the information processing system 1 is a useful system for music production, performance, education, marketing, and other related activities.

[0109] For example, Information Processing System 1 is a useful system in the field of music production. For example, Information Processing System 1 is useful for pre-production. For example, Information Processing System 1 is useful for generating ideas for song structure, timbre, soundscape, and timbre development from reference songs (music). For example, Information Processing System 1 is useful for listener focus analysis. For example, Information Processing System 1 is useful for analyzing the volume, the abundance of harmonics in the timbre, the pitch, the magnitude of momentum, and the type of progression for each instrument.

[0110] For example, Information Processing System 1 is useful in the field of arrangement. For example, Information Processing System 1 is useful in the field of music arrangement and music conversion. For example, Information Processing System 1 is useful in the field of remixing.

[0111] For example, information processing system 1 is useful in the field of musical performance. For example, information processing system 1 is useful in sessions and jams. For example, information processing system 1 is useful in incorporating the timbre of other people's performance results and creating a performance based on it. For example, information processing system 1 is useful in the field of DJing. For example, information processing system 1 is useful in performing music editing to create natural music transitions.

[0112] For example, information processing system 1 is useful in the field of live looping. For example, information processing system 1 is useful in the fact that even poor performances are refined in the second and third loops.

[0113] For example, information processing system 1 is useful in areas such as reviewing performances and integrating them with production. For example, information processing system 1 is useful for loading the timbre of a performance (timbre that also reflects the playing technique) into a DAW, reviewing it, and transitioning to production.

[0114] For example, information processing system 1 is useful in fields that use AI-generated music and sound as input. For example, information processing system 1 is useful in editing, sound quality enhancement, etc.

[0115] For example, Information Processing System 1 is useful in the field of music education and appreciation. For example, Information Processing System 1 is useful in the analysis of musical pieces (music). For example, Information Processing System 1 is useful in the analysis of instrument playing techniques, superposition methods, and ensemble patterns. For example, Information Processing System 1 is useful in the creation of practice assignments, performance support (for sound sources without sheet music, etc.), and ear training.

[0116] For example, the information processing system 1 can provide users with useful information in the field of music education by presenting information as shown in Figure 6. Figure 6 is a diagram illustrating an example of a service provided by the information processing system. The information shown in Figure 6 is information presented to the user regarding the transcription of an ensemble performance. By presenting the information shown in Figure 6, the information processing system 1 can visualize the overlay of note sequences from multiple parts using a DAW or similar software, allowing the user to use it to study how instruments are layered. Furthermore, by presenting the information shown in Figure 6, the information processing system 1 can enable research even when the user does not have the sheet music or when the user is unfamiliar with sheet music.

[0117] Furthermore, for example, information processing system 1 is useful in the field of music marketing. For example, information processing system 1 is useful for hit song analysis and effectiveness measurement data analysis.

[0118] <2. Other Embodiments> The processes according to each embodiment described above may be carried out in various different forms (modifications) other than the embodiments and modifications described above.

[0119] <2-1. Other System Configuration Examples> Furthermore, the processes related to each embodiment and modification described above may be implemented in various different forms (modifications) other than those described above. For example, the information processing device 100 is not limited to a terminal device used by a user, but may also be a server device. In this case, the information processing system 1 includes the information processing device 100 and a terminal device used by a user. That is, a device that accepts input from a user and presents information to the user, such as displaying information (user terminal), and a device that provides services related to music production, such as specific processing to identify music-related information related to the reproduction of music data (service provision device), may be separate entities. In this case, the information processing system 1 may include a service provision device (e.g., information processing device 100), which is an example of an information processing device that identifies music-related information, and a user terminal (e.g., a terminal device) used by a user. Note that the above is just an example, and the information processing system 1 may be implemented in various configurations.

[0120] The following description will explain, as an example, a case in which the information processing device 100 performs a specific process to identify music-related information concerning the reproduction of music data, and the user terminal, which is a terminal device used by the user, presents the identified music-related information. In this case, the information processing system 1 includes a terminal device which is a user terminal and the information processing device 100. The information processing device 100 has at least the configuration for performing the specific process to identify music-related information concerning the reproduction of music data, as described above. For example, the information processing device 100 does not have the configuration for directly presenting information to the user, such as an audio output unit 13 or a display unit 16.

[0121] A user terminal is an information processing device used by a user. A user terminal can be implemented as, for example, a notebook PC (Personal Computer), a desktop PC, a smartphone, a tablet device, a mobile phone, or a PDA (Personal Digital Assistant). However, the user terminal can be any terminal device capable of presenting (displaying, audio outputting, etc.) the information transmitted (outputting) by the information processing device 100. A user terminal is a client terminal.

[0122] For example, the information processing device 100 is connected to the user terminal via a communication network N, either by wire or wirelessly. In this case, the information processing device 100 may communicate with the user terminal via the communication network N and receive various input information from the user terminal through user operations, etc. The information processing device 100 then uses the input information received from the user terminal to perform a specific process to identify music-related information relating to the reproduction of music data. The information processing device 100 transmits the identified music-related information to the user terminal. The user terminal, having received the music-related information from the information processing device 100, presents the received music-related information to the user.

[0123] For example, the user terminal transmits information indicating music data received through user operation, etc., as input information to the information processing device 100. Upon receiving the information indicating music data from the user terminal, the information processing device 100 performs specific processing on the music data received from the user terminal. The information processing device 100 transmits the music-related information identified from the music data received from the user terminal to the user terminal. Upon receiving the music-related information from the information processing device 100, the user terminal presents the received music-related information to the user by displaying it.

[0124] <2-2. Others> Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0125] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0126] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0127] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0128] <3. Hardware Configuration> The information devices such as the information processing device 100 according to each embodiment described above are realized by a computer 1000 having a configuration such as that shown in Figure 8. The information processing device 100 will be used as an example for explanation. Figure 8 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device. The computer 1000 has a processing circuitry 1100, RAM 1200, ROM 1300, secondary storage device 1400, communication interface 1500, input / output interface 1600, display unit 1700, camera unit 1800, microphone 1900, and speaker 2000. The various parts of the computer 1000 are connected by a bus 1050.

[0129] The processing circuit 1100 operates based on a program stored in the ROM 1300 or secondary storage device 1400, and controls each part. For example, the processing circuit 1100 loads the program stored in the ROM 1300 or secondary storage device 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0130] ROM 1300 stores boot programs such as the BIOS (Basic Input Output System) that are executed by the processing circuit 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0131] The secondary storage device 1400 is a computer-readable recording medium that non-temporarily stores programs executed by the processing circuit 1100 and data used by such programs. Specifically, the secondary storage device 1400 is a recording medium that stores programs for each process of the information processing device 100 according to the embodiment, which is an example of program data 1450.

[0132] The communication interface 1500 is an interface for the computer 1000 to connect to the external network 1550. The communication interface 1500 corresponds to the communication unit 11 of the information processing device 100. For example, the processing circuit 1100 receives data from other devices or transmits data generated by the processing circuit 1100 to other devices via the communication interface 1500.

[0133] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the processing circuit 1100 receives data from input devices such as a microphone 1900 or a touch panel via the input / output interface 1600. The processing circuit 1100 also transmits data to output devices such as a display unit 1700 or a speaker 2000 via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium (media). Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

[0134] The display unit 1700 is an interface for displaying information processed by the computer 1000. The display unit 1700 is, for example, a liquid crystal display or an organic electroluminescent display (Organic Electro Luminescence Display). Alternatively, the display unit 1700 may be a touch panel display device or an image projection device.

[0135] The camera unit 1800 is an interface for the computer 1000 to capture images. The microphone 1900 is an interface for the computer 1000 to capture sound. The speaker 2000 is an interface for the computer 1000 to output processed sound. The various parts of the computer 1000 are connected by the bus 1050. Each interface does not necessarily have to be located inside the computer 1000, but may be located outside the computer 1000 via a network or the like. Furthermore, each part of the computer 1000 may be controlled by a circuit different from the processing circuit 1100. For example, the display unit 1700 may be controlled not by the processing circuit 1100, but by a circuit dedicated to display processing provided within the display unit 1700.

[0136] For example, when the computer 1000 functions as an information processing device 100 according to the embodiment, the processing circuit 1100 of the computer 1000 functions as a control unit 15 by executing a program loaded onto the RAM 1200. The secondary storage device 1400 stores the information processing program according to this disclosure and various data stored by the storage unit 14. The processing circuit 1100 reads and executes the program data 1450 from the secondary storage device 1400, but as another example, these programs may be obtained from other devices via an external network 1550. In other words, the secondary storage device 1400 is not limited to being inside the computer 1000, but may be located outside the computer 1000. The processing circuit 1100 is an example of an integrated circuit, and CPU, MPU, GPU, APU, ASIC, and FPGA can all be considered integrated circuits.

[0137] Furthermore, this technology can also take the following configurations: (1) An information processing system comprising: an acquisition unit that acquires music data to be analyzed; an identification unit that identifies music-related information relating to the reproduction of the music data by analyzing the music data; and an output unit that outputs the music-related information. (2) The information processing system according to (1), wherein the identification unit identifies parameters relating to a tool used to reproduce the music data as the music-related information. (3) The information processing system according to (2), wherein the tool is at least one of a sampler, a synthesizer, or a neural synthesizer. (4) The information processing system according to any one of (1) to (3), wherein the identification unit identifies sound data used to reproduce the music data and information relating to effects applied to the sound data as the music-related information. (5) The information processing system according to (4), wherein the sound data is one-shot data. (6) The information processing system according to any one of (1) to (5), wherein the identification unit identifies note information corresponding to the music data by analyzing the music data, and the output unit outputs the note information. (7) The information processing system according to (6), wherein the output unit outputs the note information together with the music-related information. (8) The information processing system according to any one of (1) to (7), wherein the identification unit identifies a plurality of music-related pieces of information corresponding to the music data by analyzing the music data, and the output unit outputs at least one of the plurality of music-related pieces of information. (9) The information processing system according to (8), wherein the identification unit determines a ranking of the plurality of music-related pieces of information corresponding to the music data, and the output unit outputs the plurality of music-related pieces of information in the ranking order. (10) The information processing system according to (9), wherein the identification unit determines the ranking based on the similarity between the information relating to the music data and the information relating to the music reproduced by each of the plurality of music-related pieces of information.(11) The information processing system according to (9) or (10), wherein the output unit outputs content in which the plurality of music-related information is arranged in the ranking order. (12) The information processing system according to (11), wherein the output unit displays content in which detailed information of the one music-related information is displayed when one of the plurality of music-related information is specified. (13) The information processing system according to any one of (1) to (12), wherein the identification unit identifies the music-related information of the music data using a model that takes the music data as input and outputs output information relating to the input music data. (14) The information processing system according to (13), wherein when the music data is input, the model outputs the music-related information corresponding to the music data as output information, and the identification unit identifies the output information output by the model that has received the music data as the music-related information of the music data. (15) The information processing system according to (14), wherein when music data is input, the model outputs a plurality of music-related pieces of information of the music data as output information, and the identification unit identifies at least one of the plurality of music-related pieces of information included in the output information output by the model that has received the music data as the music-related piece of information of the music data. (16) The information processing system according to (15), wherein when music data is input, the model outputs a plurality of music-related pieces of information of the music data and information indicating the ranking of the plurality of music-related pieces of information as output information, and the identification unit determines the ranking of the plurality of music-related pieces of information based on the information indicating the ranking of the plurality of music-related pieces of information output by the model that has received the music data as input.(17) The information processing system according to any one of (13) to (16), wherein the model outputs output information including at least one of the following when music data is input: feature information of the music data, component information constituting the music data, and parameters of a tool for reproducing the music data, and the identification unit identifies the music-related information of the music data using at least one of the feature information, component information, and parameters output by the model to which the music data has been input. (18) The information processing system according to (17), wherein the identification unit identifies the music-related information of the music data by searching for a data group containing candidate music-related information using the feature information output by the model to which the music data has been input. (19) An information processing method comprising: acquiring music data to be analyzed; identifying music-related information relating to the reproduction of the music data by analyzing the music data; and outputting the music-related information. (20) An information processing program that causes a computer to perform the following actions: to acquire music data to be analyzed; to identify music-related information relating to the reproduction of the music data by analyzing the music data; and to output the music-related information.

[0138] 1 Information Processing System 100 Information Processing Device 11 Communication Unit 12 Input Unit 13 Audio Output Unit 14 Storage Unit 141 Music Data Storage Unit 142 Model Information Storage Unit 143 Specific Result Information Storage Unit 144 Generated Information Storage Unit 15 Control Unit 151 Acquisition Unit 152 Identification Unit 153 Generation Unit 154 Transmission Unit (Output Unit) 16 Display Unit (Output Unit)

Claims

1. An information processing system comprising: an acquisition unit that acquires music data to be analyzed; an identification unit that identifies music-related information relating to the reproduction of the music data by analyzing the music data; and an output unit that outputs the music-related information.

2. The information processing system according to claim 1, wherein the specified unit specifies parameters relating to a tool used to reproduce the music data as music-related information.

3. The information processing system according to claim 2, wherein the tool is at least one of a sampler, a synthesizer, or a neural synthesizer.

4. The information processing system according to claim 1, wherein the identifying unit identifies, as music-related information, sound data used to reproduce the music data and information relating to effects applied to the sound data.

5. The information processing system according to claim 4, wherein the sound data is one-shot data.

6. The information processing system according to claim 1, wherein the identification unit identifies note information corresponding to the music data by analyzing the music data, and the output unit outputs the note information.

7. The information processing system according to claim 6, wherein the output unit outputs the musical note information together with the music-related information.

8. The information processing system according to claim 1, wherein the identification unit identifies a plurality of music-related pieces of information corresponding to the music data by analyzing the music data, and the output unit outputs at least one of the plurality of music-related pieces of information.

9. The information processing system according to claim 8, wherein the identifying unit determines a ranking of a plurality of music-related pieces of information corresponding to the music data, and the output unit outputs the plurality of music-related pieces of information in the ranking order.

10. The information processing system according to claim 9, wherein the specific unit determines the ranking based on the degree of similarity between the information relating to the music data and the information relating to the music reproduced by each of the plurality of music-related pieces of information.

11. The information processing system according to claim 9, wherein the output unit outputs content in which the plurality of music-related information is arranged in the ranking order.

12. The information processing system according to claim 11, wherein the output unit displays the content that displays detailed information of the music-related information when one of the plurality of music-related information is specified.

13. The information processing system according to claim 1, wherein the identifying unit identifies the music-related information of the music data using a model that takes the music data as input and outputs output information relating to the input music data.

14. The information processing system according to claim 13, wherein the model outputs music-related information corresponding to the music data as output information when music data is input, and the identification unit identifies the output information output by the model when music data is input as music-related information of the music data.

15. The information processing system according to claim 14, wherein the model outputs a plurality of music-related pieces of information of the music data as output information when the music data is input, and the identification unit identifies at least one of the plurality of music-related pieces of information included in the output information output by the model that has received the music data as the music-related piece of information of the music data.

16. The information processing system according to claim 15, wherein the model outputs, when music data is input, a plurality of music-related pieces of information of the music data and information indicating a ranking of the plurality of music-related pieces of information as output information, and the specific unit determines the ranking of the plurality of music-related pieces of information based on the information indicating a ranking of the plurality of music-related pieces of information output by the model when music data is input.

17. The information processing system according to claim 13, wherein the model outputs output information including at least one of the following when music data is input: characteristic information of the music data, component information constituting the music data, and parameters of a tool for reproducing the music data, and the identification unit identifies the music-related information of the music data using at least one of the following output by the model when music data is input: characteristic information of the music data, component information, and parameters.

18. The information processing system according to claim 17, wherein the identifying unit identifies the music-related information of the music data by searching for a data set containing candidate music-related information using the feature information output by the model into which the music data is input.

19. An information processing method comprising: acquiring music data to be analyzed; identifying music-related information relating to the reproduction of the music data by analyzing the music data; and outputting the music-related information.

20. An information processing program that causes a computer to perform the following actions: acquire music data to be analyzed; identify music-related information relating to the reproduction of the music data by analyzing the music data; and output the music-related information.

Images