Information processing device, information processing method, and information processing program

Abstract

To propose an information processing device, an information processing method, and an information processing program that can improve convenience of an automatic composition function with AI.SOLUTION: An information processing device according to the present disclosure includes a selection unit that selects setting information for controlling a composition function based on machine learning, and a transmission / reception unit that transmits the setting information to an external server that executes the composition function based on the machine learning and receives music data composed by the external server through a network. The selection unit selects, as the setting information, designation information for designating material music to be learning data for the machine learning, the designation information including feature information indicating a feature of the designation information and multiple pieces of material music associated with the feature information. The transmission / reception unit transmits the designation information selected by the selection unit to the external server.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program, and more particularly, to the use of music data composed based on machine learning. [Background technology]

[0002] With the advancement of AI (Artificial Intelligence), the use of computers in the field of art is increasing.

[0003] For example, a technique is known in which existing music pieces are used as learning data to perform machine learning to generate a learning model for music generation, and then a computer is made to compose new music pieces (for example, Patent Document 1). With this technique, it is possible to use a Markov model to imitate the characteristics of existing music pieces and generate more natural melodies. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Pat. No. 9,110,817 Summary of the Invention [Problem to be solved by the invention]

[0005] According to conventional technology, users can use music data proposed (generated) by AI in composing, allowing them to compose music based on a wider variety of perspectives.

[0006] However, the above conventional technology does not necessarily improve the convenience of the AI-based automatic composition function. For example, currently, many users use DAWs (Digital Audio Workstations) to compose, arrange, record, and perform other tasks. However, when a user uses the above conventional technology in combination with a DAW, the user must work while switching between different work environments, which may reduce work efficiency. In addition, since the AI-based automatic composition function generally places a heavy load on information processing, if it is executed simultaneously with a DAW on a terminal device, it may not be able to fully function or may cause delays in processing on the DAW side.

[0007] Therefore, this disclosure proposes an information processing device, an information processing method, and an information processing program that can improve the convenience of the AI-based automatic composition function. [Means for solving the problem]

[0008] In order to solve the above problems, an information processing device according to one embodiment of the present disclosure includes a selection unit that selects setting information for controlling a composition function based on machine learning, and a transmission / reception unit that transmits the setting information via a network to an external server that executes the composition function based on machine learning and receives music data composed by the external server. The selection unit selects, as the setting information, designation information for designating material music pieces that serve as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information. The transmission / reception unit transmits the designation information selected by the selection unit to the external server. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a conceptual diagram showing a flow of information processing according to the embodiment. [Diagram 2] FIG. 1 is a diagram showing an example of a user interface according to an embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of a user interface according to the embodiment. [Figure 4] FIG. 13 is a diagram illustrating an example of a style palette according to the embodiment. [Diagram 5] 2 is a block diagram showing a flow of information processing according to the embodiment; [Figure 6] FIG. 1 is a diagram illustrating an information processing system according to an embodiment of the present disclosure. [Figure 7] FIG. 2 is a diagram illustrating a configuration example of a user terminal according to the embodiment. [Figure 8] FIG. 4 is a diagram showing an example of composition setting information according to the embodiment. [Figure 9] FIG. 2 is a diagram showing an example of composed music information according to the embodiment. [Figure 10] FIG. 4 is a diagram illustrating an example of history information according to the embodiment. [Figure 11] FIG. 4 is a diagram showing an example of played instrument information according to the embodiment. [Figure 12] FIG. 2 is a diagram illustrating an example of a configuration of a processing server according to the embodiment. [Figure 13] FIG. 4 is a diagram illustrating an example of user information according to the embodiment. [Figure 14] FIG. 4 is a diagram showing an example of music information according to the embodiment. [Figure 15] FIG. 11 is a diagram illustrating an example of style palette information according to the embodiment. [Figure 16] 11 is a diagram showing an example of style palette sequence information according to the embodiment. FIG. [Figure 17] FIG. 4 is a diagram showing an example of user composition information according to the embodiment. [Figure 18] FIG. 4 is a diagram illustrating an example of history information according to the embodiment. [Figure 19] 4 is a flowchart illustrating a procedure of information processing according to the embodiment. [Figure 20] FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and duplicated descriptions will be omitted.

[0011] The present disclosure will be described in the following order. 1. Embodiment 1-1. Example of information processing according to the embodiment 1-2. Configuration of information processing system according to embodiment 1-3. Configuration of information processing device (user terminal) according to embodiment 1-4. Configuration of external server (processing server) according to embodiment 1-5. Information Processing Procedure According to the Embodiment 2. Variations 2-1. Format of music data 2-2. Host app 2-3. Control via DAW 2-4. Information Processing System 3. Other embodiments 4. Effects of the information processing device according to the present disclosure 5. Hardware Configuration

[0012] (1. EMBODIMENT) [1-1. Example of information processing according to the embodiment] First, an example of information processing according to the present disclosure will be described with reference to Fig. 1. Fig. 1 is a conceptual diagram showing a flow of information processing according to the embodiment. The information processing according to the embodiment is executed by a user terminal 10, which is an example of an information processing device according to the present disclosure, and a processing server 100, which is an example of an external server according to the present disclosure. The user terminal 10 and the processing server 100 communicate with each other using a wired or wireless network N (e.g., the Internet) shown in Fig. 1. Note that the number of user terminals 10 and processing servers 100 is not limited to those illustrated in Fig. 1.

[0013] A user terminal 10 shown in Fig. 1 is an example of an information processing device according to the present disclosure. For example, the user terminal 10 is an information processing terminal such as a personal computer (PC), a tablet terminal, or a smartphone. The user terminal 10 is provided with (installed with) various program applications (hereinafter simply referred to as "apps"). The user terminal 10 launches and executes the various apps to execute various information processes.

[0014] In the embodiment, the user terminal 10 includes an application (so-called DAW) that realizes a comprehensive music production environment. In the following description, the application (DAW) is referred to as a first application or a host application. The first application according to the embodiment is capable of incorporating (inserting) other applications for expanding functions. In other words, the first application is capable of using other applications for expanding functions, so-called plug-ins. In this case, the first application functions as a host application for the incorporated plug-in.

[0015] In the embodiment, the user terminal 10 includes an application having an automatic composition function using AI. In the following description, the application (the application having the automatic composition function using AI) is referred to as a second application or a plug-in. The second application according to the embodiment is incorporated as a plug-in of the first application. The plug-in can take the form of, for example, VST (Steinberg's Virtual Studio Technology) (registered trademark), AudioUnits, AAX (Avid Audio eXtension), etc.

[0016] The processing server 100 shown in FIG. 1 is a server device that executes information processing related to a second application included in the user terminal 10. For example, the processing server 100 is a so-called cloud server, and executes a predetermined information processing based on information instructed from the user terminal 10 via the network N. Specifically, the processing server 100 generates music data based on a predetermined learning process or data output from a learned model based on information transmitted from the user terminal 10. In other words, the processing server 100 executes an automatic composition function using AI based on an instruction from the user terminal 10. For example, the processing server 100 provides the user terminal 10 with music data that has been automatically composed using a Markov model or the like, as shown in the above-mentioned prior art documents and the like.

[0017] As described above, the user terminal 10 can use the second application as a plug-in to drag and drop music data provided from the processing server 100 on the second application onto the first application, or perform editing work on the first application. Conventionally, the automatic composition function has depended on the processing performance (CPU power, etc.) of the terminal on which the processing is executed, but as shown in FIG. 1, the user terminal 10 does not perform learning processing, etc. on its own device, but has the processing server 100 execute the processing. This allows the user terminal 10 to execute the automatic composition function, which is a relatively high processing load, while saving the resources of its own device. Therefore, the user terminal 10 can solve the conventional problem of DAW processing delay (occurrence of latency, etc.). As a result, the user terminal 10 according to the present disclosure can improve the convenience of the automatic composition function by AI. Below, an overview of the information processing according to the present disclosure will be described along the flow with reference to FIG. 1.

[0018] 1, the user terminal 10 launches a host application (first application) 20 (step S1). The user terminal 10 also launches a plug-in (second application) 22, which is an example of a plug-in that runs on the host application 20 (step S2).

[0019] The user terminal 10, in accordance with the user's operation, selects setting information for the music to be automatically composed in the plug-in 22. As will be described in detail later, the user terminal 10, in accordance with the user's operation, selects setting information such as the chord progression of the music to be automatically composed, the subjective image of the music (dark, bright, etc.), and the structure of the music. Then, the user terminal 10 transmits the selected setting information to the processing server 100 (step S3).

[0020] The processing server 100 performs a predetermined learning process based on the setting information transmitted from the user terminal 10, and performs a composition process based on the learning result (step S4). For example, the process described in the above-mentioned prior art document may be used for the composition process. Then, the processing server 100 generates the composed music data.

[0021] Next, the processing server 100 transmits the generated music data to the user terminal 10 (step S5). The user terminal 10 receives the music data transmitted from the processing server 100 in the plug-in 22. For example, the music data includes information such as chord progressions, melodies, and bass note progressions generated by the processing server 100. The music data may be standard data such as MIDI (Musical Instrument Digital Interface) data, waveform data, or DAW-specific data. The user may edit the received music data on the plug-in 22, or may copy the music data to the host application 20 and use it on the host application 20.

[0022] In this way, the user terminal 10 controls the host application 20 and the plug-in 22 that functions as a plug-in that extends the function of the host application 20. The host application 20 also controls the operation of the plug-in 22 in the host application 20. The plug-in 22 also selects setting information for controlling a composition function based on machine learning, transmits the setting information to the processing server 100 via the network N, and receives music data composed by the processing server 100.

[0023] That is, the user terminal 10 uses the automatic composition function as a plug-in for the DAW. Therefore, the user can receive the support of the automatic composition function while remaining in the normal working environment of the DAW. In addition, the user can avoid processing delays in the DAW by offloading the processing load of the automatic composition function to the processing server 100. As a result, the user terminal 10 can improve the convenience of the AI-based automatic composition function.

[0024] Next, the automatic composition function by the plug-in 22 will be described in detail with reference to FIGS.

[0025] 2 is a diagram (1) showing an example of a user interface according to the embodiment. In FIG. 2, an example of a user interface when the plug-in 22 is displayed on the screen of the user terminal 10 is shown.

[0026] In the example shown in Fig. 2, the user interface 30 displays music data received by the plug-in 22. Note that, as will be described in detail later, the music data in the plug-in 22 is made up of three different types of data: melody, chords, and bass notes. Of the three different types of data, the user interface 30 shown in Fig. 2 displays data related to the melody.

[0027] The setting information 31 displays information related to a style palette, which is an example of setting information in the automatic composition function. The style palette is specification information for specifying material music pieces that serve as learning data for machine learning.

[0028] The setting information 32 displays information related to harmony, which is an example of setting information in the automatic composition function. The information related to harmony is, for example, information for determining the probability that the constituent tones included in a chord will appear in the melody in the music data composed by the processing server 100. For example, if the user sets the information related to harmony to "strict", the probability that the constituent tones included in a chord will appear in the melody in the automatically composed music data increases. On the other hand, if the user sets the information related to harmony to "loose", the probability that the constituent tones included in a chord will appear in the melody in the automatically composed music data decreases. The example in FIG. 2 indicates that the user has applied the information related to harmony to "strict".

[0029] The setting information 33 displays note length information, which is an example of setting information in the automatic composition function. Note length information is, for example, information for determining the length of notes in music data composed by the processing server 100. For example, if the user sets the note length information to "long", the automatically composed music data is more likely to contain notes with relatively long sound lengths (e.g., whole notes and half notes). On the other hand, if the user sets the note length information to "short", the automatically composed music data is more likely to contain notes with relatively short sound lengths (e.g., eighth notes and sixteenth notes).

[0030] The setting information 34 displays information for determining the type and amount of material music pieces other than those included in the designated information (the style palette designated by the user), which is an example of setting information in the automatic composition function. Such information is, for example, information for determining whether or not learning is to be strictly based on music pieces included in the style palette designated by the user in the music data composed by the processing server 100. For example, if the user sets such information to "never," there is a low tendency for music pieces other than those included in the style palette to be used in learning with the automatic composition. On the other hand, if the user sets such information to "only," there is a high tendency for music pieces other than those included in the style palette to be used in learning with the automatic composition.

[0031] The music data 35 displays specific music data transmitted from the processing server 100. In the example of FIG. 2, the music data 35 includes information indicating a chord progression such as Cm, information indicating the pitch and note length in a measure, transitions in note pitch (in other words, melody), and the like. As shown in FIG. 2, the music data 35 may include, for example, four different types of content. That is, the processing server 100 may transmit multiple pieces of music data instead of transmitting only one type of automatically composed music data. This allows the user to select the music data he or she likes from the multiple candidates of music data generated, or to compose a favorite piece of music by combining multiple pieces of music data.

[0032] Note that the user interface 30 shown in Fig. 2 displays data related to the melody among the three different types of data, melody, chords, and bass notes, contained in the music data, but other data is displayed in other user interfaces. This point will be explained with reference to Fig. 3. Fig. 3 is a diagram (2) showing an example of a user interface according to the embodiment.

[0033] As shown in Fig. 3, the user terminal 10 may display on the screen a user interface 38 that displays data related to chords and a user interface 39 that displays data related to bass notes, in addition to a user interface 30 that displays data related to a melody. Although not shown in Fig. 3, the user interface 38 and the user interface 39 display note information different from that of the music data 35 in the user interface 30. Specifically, the user interface 38 displays note information related to chords corresponding to the melody of the music data (e.g., constituent notes of the chord Cm). Furthermore, the user interface 39 displays note information related to bass notes corresponding to the melody and chords of the music data (e.g., "C" note for the chord Cm).

[0034] The user can select information to be copied to the host application 20 from the displayed user interface 30, user interface 38, and user interface 39, and can perform operations such as editing part of the bass sound, for example.

[0035] Next, a style palette, which is an example of the above-mentioned setting information, will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a style palette according to the embodiment.

[0036] Window 40 shown in Fig. 4 is an example of a user interface displayed in plug-in 22. The user refers to the feature information of the style palettes displayed in window 40, for example, and selects an image that matches the music piece that the user wishes to automatically compose. For example, the user selects style palette 41 in which feature information such as "bright" is described, style palette 42 in which feature information such as "dark" is described, etc. The user also selects style palette 43 in which the genre or type of music piece, "American," is described as feature information, or style palette 44 in which the structure of the music piece, "A melody -> B melody -> chorus," is described as feature information.

[0037] As described above, a style palette is information for specifying a piece of music used by the processing server 100 for learning. In other words, each style palette contains information for identifying an existing piece of music that has been composed in advance. For example, it is assumed that a constituent song list 50 is associated with the style palette 41. The constituent song list 50 contains a plurality of existing pieces of music. It is also assumed that a constituent song list 51 is associated with the style palette 42. The constituent song list 51 contains a plurality of existing pieces of music that are different from the pieces of music contained in the constituent song list 50.

[0038] Therefore, a learning model generated by machine learning based on the style palette 41 is different from a learning model generated by machine learning based on the style palette 42. This is because the learning data in machine learning changes depending on the style palette selected by the user. In other words, the style palette can be said to be specification information for specifying learning data in automatic composition.

[0039] The music that constitutes the style palette is registered in advance by, for example, an administrator or provider of the plug-in 22. For example, the administrator of the plug-in 22 extracts a plurality of music pieces that are subjectively felt to be "bright" to generate a constituent music piece list 50, and associates the constituent music piece list 50 with the style palette 41. The style palette and the music pieces that correspond to the style palette may be arbitrarily edited by a user who uses the plug-in 22. For example, the user may select music pieces from a web service such as a music distribution service or a social networking service (SNS), and combine the selected music pieces to generate a desired style palette. Specifically, the user may arbitrarily extract music pieces included in a playlist automatically generated by a specific music app or a playlist provided to users who use the music app, and change the constituent music pieces of the style palette that he or she has generated, or generate a new style palette. This allows the user to flexibly generate a style palette that he or she likes.

[0040] In addition, when selecting a style palette as setting information, the user may select multiple style palettes. For example, the user may select style palette 41 as setting information for composing a part of a song (e.g., the first 8 bars), and select style palette 42 as setting information for composing a different part of the song (e.g., the middle 8 bars). Such information including multiple style palettes is hereinafter referred to as a style palette sequence. In other words, a style palette sequence can be said to be combination designation information that combines designation information that designates a song called a style palette. By setting a style palette sequence and performing composition, the user can easily create diverse song data that has multiple characteristics in one song.

[0041] Next, the relationship between the host application 20 and the plug-in 22 will be conceptually shown with reference to Fig. 5. Fig. 5 is a block diagram showing the flow of information processing according to the embodiment.

[0042] A processing block 60 shown in Fig. 5 indicates a flow of processing executed by the DAW (host application 20) and the plug-in 22. Usually, a performer records the sound of playing an instrument and creates data such as MIDI. In the information processing according to the embodiment, music data corresponding to a melody, chords, and bass notes is generated by the plug-in 22 on behalf of the performer. For example, a melody 61, chords 62, and bass notes 63 are generated by the performer and the plug-in 22.

[0043] After that, the user records the melody 61, chords 62, and bass notes 63 to be used in the song in a recorder associated with the DAW, and creates tracks corresponding to the melody, chords, and bass notes. For example, the user sets instrument information indicating an instrument to be used for playing the melody 61 generated by the plug-in 22. Specifically, the user sets instrument information such as playing the melody 61 on a guitar registered in the DAW. Then, the user records the sound played on a virtual guitar in the recorder, and creates a track corresponding to the guitar. Note that the DAW is capable of creating multiple tracks, so a track based on the sound played by the performer and a track based on the song data created by the plug-in 22 may coexist.

[0044] After that, the user performs mixing on the DAW and creates music data through trackdown, etc. The user also performs mastering on the DAW and adjusts the audio signal level, etc., to create a music file 65 that can be played on a playback device, etc.

[0045] In this way, according to the information processing of the embodiment, the user can create a piece of music on the DAW by using data automatically composed by the plug-in 22 in accordance with the performance data of a performer's performance and the MIDI data that the user has created. For example, the user can create a piece of music on the DAW by mixing a melody automatically composed by AI with the performance data of a performer's performance and by incorporating a chord progression suggested by AI.

[0046] The overall flow of information processing according to the present disclosure has been outlined above. The configuration of the information processing system 1 including the user terminal 10 will be described with reference to Fig. 6 and subsequent figures, and various types of processing will be described in detail in sequence. [1-2. Configuration of the information processing system according to the first embodiment] 6 is a diagram illustrating an example of an information processing system 1 according to the first embodiment of the present disclosure. As illustrated in FIG. 1, the information processing system 1 includes a user terminal 10, a processing server 100, and a management server 200.

[0047] The user terminal 10 is an example of an information processing device according to the present disclosure, and controls the operations of the host application 20 and the plug-in 22.

[0048] The processing server 100 is an example of an external server according to the present disclosure, and cooperates with the plug-in 22 to perform automatic composition processing.

[0049] The management server 200 is a server managed by, for example, a company that provides the plug-ins 22 .

[0050] For example, the management server 200 manages the user authority of a user who uses the plug-in 22, and manages information on style palettes that can be used by the plug-in 22. For example, the management server 200 determines whether a user has the authority to use the plug-in 22 based on a user ID that uniquely identifies the user. The management server 200 also creates a style palette and edits the music that constitutes the style palette, and transmits information on the style palette to the user terminal 10 and the processing server 100. The management server 200 may be configured integrally with the processing server 100.

[0051] [1-3. Configuration of information processing device (user terminal) according to embodiment] Next, a configuration of a user terminal 10, which is an example of an information processing device according to the present disclosure, will be described with reference to Fig. 7. Fig. 7 is a diagram showing a configuration example of a user terminal 10 according to an embodiment of the present disclosure. As shown in Fig. 7, the user terminal 10 has a communication unit 11, an input unit 12, a display unit 13, a storage unit 15, and a control unit 16.

[0052] The communication unit 11 is realized by, for example, a NIC (Network Interface Card) etc. The communication unit 11 is connected to a network N (such as the Internet) by wire or wirelessly, and transmits and receives information to and from the processing server 100, the management server 200, etc. via the network N.

[0053] The input unit 12 is an input device that accepts various operations from a user. For example, the input unit 12 is realized by operation keys or the like provided on the user terminal 10. The display unit 13 is a display device for displaying various information. For example, the display unit 13 is realized by a liquid crystal display or the like. Note that, when a touch panel is adopted in the user terminal 10, a part of the input unit 12 and the display unit 13 are integrated together.

[0054] The storage unit 15 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 15 stores various data used in information processing.

[0055] 7, the storage unit 15 stores various types of information such as composition setting information 151, composed music information 152, history information 153, and instrument information 154. Each piece of information will be described below.

[0056] The composition setting information 151 is information used when the plug-in 22 (second application) executes automatic composition. Fig. 8 shows an example of the composition setting information 151 according to the embodiment. Fig. 8 is a diagram showing an example of the composition setting information 151 according to the embodiment.

[0057] 8, the composition setting information 151 includes composition corpus information and performance style information. The composition corpus information stores music data used as learning data for automatic composition or a location where the music data is saved (e.g., a data server address, etc.). The composition corpus information also includes information such as the average note duration and key changes of each piece of music.

[0058] The performance style information stores information on the performance style of the music piece used as learning data for automatic composition. The performance style includes, for example, the overall shuffle ratio, the division of chords and bass notes, the overall balance, and so on.

[0059] The composed music information 152 is information on music used when the plug-in 22 executes automatic composition. Fig. 9 shows an example of the composed music information 152 according to the embodiment. Fig. 9 shows an example of the composed music information 152 according to the embodiment.

[0060] As shown in Fig. 9, the composed music information 152 includes a music ID, a style palette ID, and a style palette sequence ID. The music ID indicates identification information for uniquely identifying an existing music piece used as learning data. The style palette ID indicates identification information for identifying a style palette composed of multiple existing music pieces. The style palette sequence ID indicates identification information for identifying a style palette sequence composed of multiple style palettes.

[0061] The history information 153 indicates a history of operations performed by the user in the host application 20 and the plug-in 22, and a history of music created by the user. Fig. 10 shows an example of the history information 153 according to the embodiment. Fig. 10 is a diagram showing an example of the history information 153 according to the embodiment.

[0062] 10, the history information 153 includes composed music information. The composed music information includes music data transmitted from the processing server 100, a plurality of candidate data included in the music data (for example, as shown in FIG. 2, music data 35 including four types of beat divisions generated based on certain setting information), music data edited by the user, and the like.

[0063] The assigned instrument information 154 indicates instrument information set for music data transmitted from the processing server 100 and for multiple candidate data included in the music data. Fig. 11 shows an example of the assigned instrument information 154 according to the embodiment. Fig. 11 is a diagram showing an example of the assigned instrument information 154 according to the embodiment.

[0064] 11, the assigned instrument information 154 includes assigned instrument information. The assigned instrument information indicates information for identifying an instrument to be set for virtually playing the music data transmitted from the processing server 100 and multiple candidate data included in the music data, the name of an instrument set in the music data, etc. For example, the assigned instrument information can be set to different information for each of the melody, chords, and bass notes.

[0065] Returning to Fig. 7, the explanation will be continued. The control unit 16 is realized, for example, by a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc., executing a program (for example, an information processing program according to the present disclosure) stored inside the user terminal 10 using a random access memory (RAM) or the like as a working area. The control unit 16 is a controller, and may be realized, for example, by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0066] 7, the control unit 16 has a host application control unit 161 and a plug-in application control unit 165, and realizes or executes functions and actions of information processing described below. The host application control unit 161 includes a plug-in control unit 162, a playback unit 163, and a display control unit 164. The plug-in application control unit 165 includes a selection unit 166, a transmission / reception unit 167, a playback unit 168, and a display control unit 169. Note that the internal configuration of the control unit 16 is not limited to the configuration shown in FIG. 7, and may be other configurations as long as they perform information processing described later.

[0067] The host application control unit 161 controls the host application 20 (DAW, which is a first application).

[0068] The plug-in control unit 162 controls the operation of various plug-ins in the host application 20. For example, the plug-in control unit 162 controls operations such as calling a plug-in in the host application 20, starting a plug-in on the host application 20, and copying data in a plug-in to the host application 20.

[0069] For example, the plug-in control unit 162 individually sets instrument information for specifying a tone when the plug-in plays back a chord, melody, or bass sound included in the music data received from the processing server 100. For example, the plug-in control unit 162 reads out information on virtual instruments registered on the DAW, and sets the virtual instrument information for each of the chords, melody, or bass sound included in the music data of the plug-in in order to play each of them.

[0070] The playback unit 163 controls playback processing in the host application 20. The playback unit 163 has a synchronous playback function in the host application 20, a playback information sending function, a sound synthesis playback function, a playback style arrangement function, and the like.

[0071] For example, the playback unit 163 cooperates with a synchronous playback function of the playback unit 168 of the plug-in to play music data held by the plug-in. For example, the playback unit 163 can obtain and play the melody, chords, and bass sound at the part corresponding to the playback position by passing time information indicating the position being played by the host application to the plug-in.

[0072] Furthermore, when a performance style or the like is set in the plug-in, the playback unit 163 may process the playback data in accordance with the performance style and play back the processed data.

[0073] The display control unit 164 controls the display control process in the host application 20. For example, the display control unit 164 has a performance information display function for displaying information about each track on the screen (display unit 13), a composed music information pasting function for copying information about music data, etc., to a track, and the like.

[0074] The display control unit 164 also controls the plug-in to individually display windows that display information about the chords, melody, or bass notes included in the music data received from the processing server 100. For example, the display control unit 164 displays user interfaces corresponding to each of the chords, melody, and bass notes on the screen of the DAW, as shown in Fig. 3 .

[0075] Furthermore, the display control unit 164 controls, in response to user operations, the transmission and reception of information between each window displaying information about chords, melodies, or bass notes and a window displaying information about the host application, allowing the user to quickly perform processes such as copying automatically composed music data to any track or editing on the DAW.

[0076] The display control unit 164 may control not only the exchange of information between the host application and the plug-in, but also the exchange of information between the windows of the displayed plug-in. In other words, the display control unit 164 may control the transmission and reception of information between the windows that display information about chords, melodies, or bass notes in accordance with the user's operation.

[0077] The plug-in application control unit 165 controls the operation of a plug-in that runs on the host application 20. For example, the plug-in application control unit 165 starts up a plug-in on the host application 20 in accordance with a user operation.

[0078] The selection unit 166 selects setting information for controlling the composition function based on machine learning. For example, the selection unit 166 selects designation information for designating material music pieces that become learning data for machine learning as the setting information. Specifically, the designation information corresponds to the style palette shown in FIG. 4 and the like.

[0079] For example, the selection unit 166 selects, in response to a user operation, designation information stored in advance in the storage unit 15, which designation information includes feature information indicating the features of the designation information and a plurality of material musical pieces associated with the feature information. For example, the user refers to feature information (such as "bright", "dark") of a style palette via a window 40 shown in FIG. 4 or the like. Then, based on the feature information, the user selects a style palette including the feature information desired for the musical piece that the user wants the AI ​​to compose.

[0080] The selection unit 166 may select combination designation information that combines first designation information corresponding to some of the measures in the music data composed by the external server with second designation information corresponding to some of the other measures. As described above, the combination designation information corresponds to a style palette sequence. Moreover, the first designation information corresponds to a style palette, which is setting information for composing some of the measures. Moreover, the second designation information corresponds to a style palette, which is setting information for composing some of the other measures.

[0081] In addition to the style palette, the selection section 166 may select detailed setting information related to the music data to be composed.

[0082] For example, the selection unit 166 may select, as the setting information, note length information included in music data composed by the processing server 100 based on the style palette. For example, the selection unit 166 accepts the selection of note length information from the user via a display such as a slider of the setting information 33 included in the user interface 30 shown in FIG.

[0083] Furthermore, the selection unit 166 may select, as the setting information, information for determining the probability that constituent tones included in a chord appear in a melody in music data composed by the processing server 100 based on the style palette. For example, the selection unit 166 accepts a selection of information for determining the probability that constituent tones included in a chord appear in a melody from the user via a display such as a slider of the setting information 32 included in the user interface 30 shown in FIG.

[0084] The selection unit 166 may also select, as the setting information, information for determining the types and amounts of material pieces of music other than the material pieces of music included in the style palette, in music data composed by the processing server 100 based on the style palette. For example, the selection unit 166 accepts a selection of information for determining the types and amounts of material pieces of music other than the material pieces of music included in the style palette, from the user via a display such as a slider of the setting information 34 included in the user interface 30 shown in FIG. 2.

[0085] The selection unit 166 may also select information other than the style palette as the setting information for automatic composition. As an example, the selection unit 166 may select, as the setting information, a chord progression in the music to be composed based on a user operation. In this case, the processing server 100 automatically generates music data based on the chord progression selected by the user.

[0086] The transmission / reception unit 167 transmits the setting information selected by the selection unit 166 via the network N to the processing server 100 that executes a composition function based on machine learning, and receives music data composed by the processing server 100.

[0087] For example, the transmitting / receiving unit 167 transmits the style palette selected by the selecting unit 166 to the processing server 100. Then, the transmitting / receiving unit 167 receives music data generated by the processing server 100 based on the style palette.

[0088] The transmitting / receiving unit 167 receives, as music data, for example, a chord in a bar of a specified length, a melody in the bar, and a bass note in the bar. Such information may be data such as MIDI or MusicXML, information unique to the DAW, or waveform data (WAV file, etc.).

[0089] Furthermore, the transmitting / receiving unit 167 may transmit the style palette sequence selected by the selecting unit 166 to the processing server 100. In this case, the transmitting / receiving unit 167 receives music data generated by the processing server 100 based on the style palette sequence.

[0090] When the transmitting / receiving unit 167 receives music data composed by the processing server 100 based on the style palette sequence, the transmitting / receiving unit 167 may store the music data in association with the style palette sequence in the storage unit 15. This allows the user to refer to a history of what music data was created using what style palette sequence, and thus the user can utilize such information in the composition work.

[0091] Furthermore, the transmitting / receiving unit 167 may transmit various setting information other than the style palette and the style palette sequence to the processing server 100. For example, the transmitting / receiving unit 167 transmits to the processing server 100 information on the length of notes set by the user, information for determining the probability that constituent notes included in a chord appear in a melody, information for determining the type and amount of material music pieces other than the material music pieces included in the style palette, and the like.

[0092] Furthermore, when a user performs a playback or editing operation on the music data after receiving the music data composed by the processing server 100, the transmitting / receiving unit 167 may transmit information on the playback or editing operation to the processing server 100. This allows the processing server 100 to obtain information on how the composed music data was used or how much it was used. In this case, the processing server 100 may adjust the learning method or the music data to be generated based on such information. For example, the processing server 100 may analyze past music data used by more users and generate music data having such characteristics preferentially.

[0093] The playback unit 168 controls playback processing in the plug-in. For example, the playback unit 168 plays music data received by the transmission / reception unit 167. Specifically, the playback unit 168 sets arbitrary instrument information for each of the melody, chords, and bass sounds included in the music data, and plays each piece of data. Note that the playback unit 168 may play back a combination of the melody, chords, and bass sounds.

[0094] The display control unit 169 controls display processing in the plug-in. For example, the display control unit 169 displays a window such as a user interface showing information about the plug-in on the screen.

[0095] The display control unit 169 acquires four types of four-bar music data, for example, as shown in Fig. 2, and displays four candidates side by side in the user interface. The user can select a final candidate by joining together good parts from the four types of music data candidates along the time axis (i.e., by comping). For example, the user can delete some of the notes in the melody, join them, change the length of the notes, or change the pitch of the notes.

[0096] Furthermore, the display control unit 169 may control to call up the history of past music data composed by the processing server 100 from the storage unit 15 in accordance with a user's operation and display the history of the past music data. This allows the user to proceed with the composition work while referring to data previously composed by the processing server 100. For example, the user can determine the final candidate while comparing the latest music created by editing with the history of music edited in the past.

[0097] The display control unit 169 may also control to call up from the storage unit 15 a history of editing operations performed on past music data composed by the processing server 100, and to display the editing operations performed on the past music data. This allows the user to refer to past editing operations and music data generated by the editing operations, thereby enabling efficient composition work.

[0098] Note that, although Figures 2 and 3 show examples in which music data is displayed on the user interface in a so-called piano roll format, which indicates the pitch and note duration, the display control unit 169 may also display music data in musical staff notation or in a format unique to the DAW.

[0099] [1-4. Configuration of external server (processing server) according to embodiment] Next, a description will be given of a configuration of the processing server 100, which is an example of an external server according to the present disclosure. Fig. 12 is a diagram showing an example of the configuration of the processing server 100 according to the embodiment.

[0100] 12, the processing server 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The processing server 100 may include an input unit (e.g., a keyboard, a mouse, etc.) that receives various operations from an administrator who manages the processing server 100, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

[0101] The communication unit 110 is realized by, for example, a NIC etc. The communication unit 210 is connected to a network N by wire or wirelessly, and transmits and receives information to and from the user terminal 10, the management server 20, etc. via the network N.

[0102] The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 stores various data used in information processing.

[0103] 12, the storage unit 120 stores various types of information such as user information 121, music information 122, style palette information 123, style palette sequence information 124, user composition information 125, and history information 126. Each piece of information will be described below.

[0104] The user information 121 indicates information about a user who uses the plug-in 22 (second application). Fig. 13 shows an example of the user information 121 according to the embodiment. Fig. 13 is a diagram showing an example of the user information 121 according to the embodiment.

[0105] As shown in Fig. 13, user information 121 includes a user ID, user meta information, and authority information. The user ID indicates identification information for uniquely identifying a user. The user meta information is additional information of the user, such as the user's name and address. The authority information is identification information, such as whether the user using the plug-in is an administrator, a general user, or a special user.

[0106] The music information 122 indicates information about music used in the automatic composition process. An example of the music information 122 according to the embodiment is shown in Fig. 14. Fig. 14 is a diagram showing an example of the music information 122 according to the embodiment.

[0107] As shown in Fig. 14, the music information 122 includes a music ID, music meta information, melody information, chord progression information, and bass note progression information. The music ID indicates identification information for uniquely identifying a music piece. The music meta information is, for example, information such as the title, composer, era, and genre of the music piece. The melody information is, for example, scale information expressing the vocal part. The chord progression information is, for example, time series information expressing the chord transitions of the music piece. The bass note progression information is time series information indicating the root note of the chord progression information.

[0108] The style palette information 123 indicates information related to a style palette used in the automatic composition process. Fig. 15 shows an example of the style palette information 123 according to the embodiment. Fig. 15 is a diagram showing an example of the style palette information 123 according to the embodiment.

[0109] As shown in FIG. 15, the style palette information 123 includes a style palette ID, style palette meta information, and a song ID. The style palette ID indicates identification information for uniquely identifying a style palette. The style palette meta information is, for example, the name of the style palette, subjective characteristic information such as bright or dark, fast or slow, the structure of the song such as an A melody, a B melody, and a chorus, and information such as the characteristics of the chord progression. The name of the style palette meta information may be given with information such as the chord progression. This allows the user to intuitively grasp the characteristics of the style palette. Also, as shown in FIG. 15, a style palette is registered in association with a plurality of song IDs. The style palette information 123 is registered by, for example, the management server 200.

[0110] The style palette sequence information 124 indicates information related to a style palette sequence used in the automatic composition process. Fig. 16 shows an example of the style palette sequence information 124 according to the embodiment. Fig. 16 shows an example of the style palette sequence information 124 according to the embodiment.

[0111] As shown in FIG. 16, the style palette sequence information 124 includes a style palette sequence ID, style palette sequence meta information, style palette ID, and blending information. The style palette sequence ID indicates identification information for uniquely identifying a style palette sequence. The style palette sequence meta information is, for example, the name of the style palette sequence, and subjective characteristic information such as bright or dark, fast or slow. Also, as shown in FIG. 16, a style palette sequence is registered in association with multiple style palette IDs. Also, the blending information is information regarding the arrangement of style palettes in the style palette sequence.

[0112] The user's composition information 125 indicates information related to composition received from the user terminal 10. Fig. 17 shows an example of the user's composition information 125 according to the embodiment. Fig. 17 is a diagram showing an example of the user's composition information 125 according to the embodiment.

[0113] As shown in FIG. 17, the user composition information 125 includes a user ID, a music piece ID, a style palette ID, and a style palette sequence ID. The user ID indicates identification information for uniquely identifying a user. The music piece ID indicates identification information for identifying a music piece generated for a user identified by the user ID. The style palette ID indicates identification information for identifying a style palette transmitted from a user identified by the user ID. The style palette sequence ID indicates identification information for identifying a style palette sequence transmitted from a user identified by the user ID.

[0114] The history information 126 is various types of history related to information processing of the processing server 100. Fig. 18 shows an example of the history information 126 according to the embodiment. Fig. 18 is a diagram showing an example of the history information 126 according to the embodiment.

[0115] As shown in Fig. 18, the history information 126 includes composition history information and operation history information. The composition history information is a history of music pieces generated by the processing server 100. The operation history information is information such as a history of editing operations performed by the user on the user terminal 10. The operation history information is, for example, information on recompositions made by the user, information on selection of composed music data, information on editing of music data, the number of times it has been played, the number of times playback has been skipped, and the like. These pieces of information may be used as learning data for the composition unit 134 described later.

[0116] Continuing the explanation by returning to Fig. 12. The control unit 130 is realized by, for example, a CPU, an MPU, a GPU, or the like executing a program stored inside the processing server 100 using a RAM or the like as a working area. The control unit 130 is also a controller, and may be realized by, for example, an integrated circuit such as an ASIC or an FPGA.

[0117] 12, the control unit 130 has a reception unit 131, a management unit 132, an acquisition unit 133, a composition unit 134, and a transmission unit 135, and realizes or executes the functions and actions of information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 12, and may be other configurations as long as they perform the information processing described below.

[0118] The reception unit 131 receives various information transmitted from the management server 200. For example, the reception unit 131 receives information on users who use plug-ins, information on style palettes, information on musical material used in automatic composition, and the like. For example, when a user purchases a product (plug-in, DAW, etc.) and activates it, the reception unit 131 executes a process of issuing a user ID to the user and receiving information on the user. In addition, the reception unit 131 receives registration of musical pieces to be linked to style palettes, editing of style palettes, and the like, in accordance with operations and commands from the management server 200.

[0119] The management unit 132 manages the various types of information received by the reception unit 131. For example, the management unit 132 stores the various types of information in the storage unit 120, and updates the stored information as appropriate.

[0120] For example, when the management unit 132 completes the style palette registration process, the user can obtain and view a list of style palette information.

[0121] The acquisition unit 133 acquires a request for automatic composition transmitted from the user terminal 10. The acquisition unit 133 also acquires setting information transmitted together with the request. For example, the acquisition unit 133 acquires a style palette desired by the user as the setting information.

[0122] The composing unit 134 composes music based on the setting information acquired by the acquiring unit 133. The composing unit 134 may compose music using various existing music generation algorithms. For example, the composing unit 134 may use a music generation algorithm using a Markov chain, or may use a music generation algorithm using deep learning. As described above, the composing unit 134 generates multiple music data for one piece of setting information transmitted by the user. This allows the user to receive multiple suggestions from the composing unit 134, and therefore allows the user to proceed with the composition work using more diverse information.

[0123] When the composition unit 134 generates music data by the composition process, the composition unit 134 associates the generated music data with the user ID of the user who transmitted the style palette, and stores the data in the storage unit 120 as history information.

[0124] The transmitting unit 135 transmits the music data generated by the composition unit 134 to the user terminal 10.

[0125] [1-5. Information processing procedure according to the embodiment] Next, the procedure of information processing according to the embodiment will be described with reference to Fig. 19. Fig. 19 is a flowchart showing the procedure of information processing according to the embodiment.

[0126] As shown in FIG. 19, the user terminal 10 starts up an automatic composition function (plug-in) on the host application in accordance with a user's operation (S101).

[0127] Next, the user terminal 10 determines whether or not a selection of a style palette, etc. has been accepted from the user (step S102). If a selection of a style palette, etc. has not been accepted from the user (step S102; No), the user terminal 10 waits until a selection is accepted.

[0128] On the other hand, if the selection of a style palette or the like is received from the user (step S102; Yes), the user terminal 10 selects a style palette in accordance with the user's operation (step S103). Note that the user terminal 10 may receive various setting information other than the style palette in step S102.

[0129] Thereafter, the user terminal 10 judges whether or not a composition request has been received from the user (step S104). If a composition request has not been received from the user (step S104; No), the user terminal 10 waits until a request is received.

[0130] On the other hand, when a composition request is received from the user (step S104; Yes), the user terminal 10 transmits the received setting information together with the composition request to the processing server 100 (step S105). After that, the user terminal 10 receives the music data composed (generated) by the processing server 100 (step S106).

[0131] Next, the user terminal 10 determines whether or not an editing process or the like has been performed by the user on the user terminal 10 (step S107). If an editing process or the like has not been performed (step S107; No), the user terminal 10 waits until an editing process or the like is accepted (step S107).

[0132] On the other hand, if an editing process or the like has been performed (step S107; Yes), the user terminal 10 reflects the editing and transmits information regarding the editing operation to the processing server 100 (step S108).

[0133] Thereafter, the user terminal 10 judges whether or not a composition request has been received from the user again (step S109). If a composition request has been received from the user (step S109; Yes), the user terminal 10 receives new setting information from the user.

[0134] On the other hand, if a composition request is not received from the user (step S109; No), the user terminal 10 determines whether or not a request to terminate the host application has been received (step S110). If a request to terminate the host application has not been received (step S110; No), the user terminal 10 continues editing the music data currently being received. On the other hand, if a request to terminate the host application has been received (step S110; Yes), the user terminal 10 terminates the host application and the plug-in, and ends the process.

[0135] (2. Modifications) The above-described information processing system 1 may be implemented in various different forms other than the above-described embodiment. Therefore, modifications of the embodiment will be described below.

[0136] [2-1. Song data format] In the above embodiment, the types of information set in the assigned instrument information 154 of the music data in the plug-in are assumed to be melody, chords, and bass notes, but are not limited to this. For example, the assigned instrument information 154 can be applied not only to melody, chords, and bass notes, but also to the performance parts of each instrument in a full orchestra.

[0137] [2-2. Host app] In the above embodiment, the host application is assumed to be a DAW, but is not limited to this. For example, the host application may be a video editing application instead of a music editing application.

[0138] [2-3. Control via DAW] In the above embodiment, an example has been shown in which the user terminal 10 selects setting information on a plug-in and transmits the selected information to the processing server 100. However, the setting information, etc. may be selected by the host application. That is, the user terminal 10 may transmit setting information (e.g., chord progression) selected in the host application to the processing server 100 and cause the processing server 100 to execute the automatic composition process. In this case, the host application may provide the plug-in with an API (Application Programming Interface) or the like for the plug-in to use the information of the host application, cause the plug-in to obtain information for generating a style palette from the host application, and control the transmission and reception process with the processing server 100, etc.

[0139] For example, the user terminal 10 generates an arbitrary chord progression using a chord generation function of a DAW, which is a host application. The user terminal 10 may then execute automatic composition based on the chord progression generated by the DAW. For example, the user terminal 10 inputs the chord progression generated by the DAW into a plug-in, and transmits the chord progression to the processing server 100 via the plug-in.

[0140] That is, the host application controls to send information about the chord progression generated in the host application to the plug-in. Then, the plug-in selects the information about the chord progression generated in the host application as setting information. Furthermore, the plug-in transmits the information about the chord progression generated in the host application to the processing server 100 and receives music data composed based on the information about the chord progression.

[0141] The user terminal 10 may also automatically select a style palette to be transmitted to the processing server 100 based on the chord progression generated by the DAW. For example, the user terminal 10 may select a style palette having similar characteristics to the chord progression generated by the DAW and transmit it to the processing server 100. The user terminal 10 may also sequentially select style palettes that match the progression based on the chord progression generated by the DAW, generate a style palette sequence, and transmit the generated style palette sequence to the processing server 100.

[0142] The user terminal 10 may also be configured to allow the plug-in to access information about the bass track on the host application. For example, the user may configure the bass track of the DAW to follow automatically composed music data. In this case, the bass track is automatically completed according to, for example, music data generated by the processing server 100 or a chord progression generated by the DAW.

[0143] The user terminal 10 may also be configured to allow the plug-in to access information on the melody track on the host application. For example, the user may configure the melody track of the DAW so that the track follows the automatically composed music data. In this case, when the user selects a bar and requests automatic composition, the generated melody is automatically inserted into the track. In addition, when the user sets the DAW to a mode in which multiple music data are combined and edited (referred to as a Comping mode, etc.), the user can select a desired part of multiple tracks that appear on the screen to complete the melody.

[0144] The user terminal 10 may also be configured to allow the plug-in to access information on the melody track and MIDI input on the host application. In this case, the user can use both the automatic composition function and the MIDI input to compose music. For example, the user inputs an arbitrary chord progression in four bars and loops it in the DAW. The user then inputs on the MIDI keyboard in time with the loop. The user terminal 10 uploads the chord progression and melody information to the processing server 100, so that the processing server 100 can automatically create a personal style palette. For example, the user can start and stop the creation of these personal style palettes, save, name, delete, etc., in a newly added style palette menu on the DAW. Such personal style palettes may be made publicly available via the style palette menu.

[0145] The user terminal 10 may also be configured to allow the plug-in to access information on audio tracks on the host application. An audio track is, for example, a track on which an instrument performance is recorded, such as a chord performance by a piano, a bass performance by a bass guitar, or a melody by a lead instrument. The plug-in accesses the audio track, analyzes audio data such as the melody, chords, and bass sounds of each track by signal processing, and obtains NIDI information of the melody, chord progression information, and the like. The plug-in may use, for example, 12-tone analysis technology for the analysis. In this case, the user terminal 10 may transmit the analyzed information to the processing server 100, and automatically infer an optimal chord progression by machine learning or the like. The processing server 100 then defines a style palette sequence based on this chord progression information. This allows the user to perform an assisted composition task based on the style palette sequence generated by the processing server 100, making it possible to, for example, recompose the entire piece, or recompose and replace a part of the piece.

[0146] The user terminal 10 may also be configured to allow the plug-in to access information on an existing master track on the host application. The master track is, for example, a track that has been mixed in the DAW and then tracked down to two stereo channels. The plug-in accesses the master track, analyzes the audio data by signal processing, and obtains chord progression information and the like. The user terminal 10 may transmit the analyzed information to the processing server 100, and automatically infer an optimal chord progression by machine learning or the like. The processing server 100 then defines a style palette sequence based on the chord progression information. This allows the user to perform an assisted composition task based on the style palette sequence generated by the processing server 100, making it possible to, for example, recompose the entire piece, or recompose and replace a part of the piece.

[0147] As described above, when the host application has various functions, the user terminal 10 may apply such functions to plug-ins and use them for information processing according to the present disclosure. For example, as described above, the user terminal 10 can stimulate composition processing among users by generating a style palette sequence based on a chord progression generated by a DAW and publishing the style palette sequence on a network.

[0148] [2-4. Information Processing System] In the embodiment, the processing server 100 is assumed to be installed on a cloud network, but this is not limited to the example, and the processing server 100 and the management server 200 may be installed on a network such as a LAN (Local Area Network) as long as they are capable of communicating with the user terminal 10.

[0149] In the embodiment, an example has been shown in which the first application and the second application are installed in the user terminal 10, but the first application and the second application may be applications installed in different devices. For example, the user terminal 10 may hold only the function of the first application and control the second application installed in a different device such as a tablet terminal or a smartphone to play back a sound source.

[0150] (3. Other embodiments) The processing according to each of the above-described embodiments may be implemented in various different forms other than the above-described embodiments.

[0151] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0152] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0153] Furthermore, the above-described embodiments and modifications can be appropriately combined as long as the processing contents are not contradictory.

[0154] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0155] (4. Effects of the information processing device according to the present disclosure) As described above, the information processing device (user terminal 10 in the embodiment) according to the present disclosure controls a first application (host application 20 in the embodiment) and a second application (plug-in 22 in the embodiment) that functions as a plug-in that extends the function of the first application. The first application includes a control unit (host application control unit 161 in the embodiment) that controls the operation of the second application in the first application. The second application includes a selection unit (selection unit 166 in the embodiment) that selects setting information for controlling a composition function based on machine learning, and a transmission / reception unit (transmission / reception unit 167 in the embodiment) that transmits the setting information via a network to an external server (processing server 100 in the embodiment) that executes the composition function based on machine learning and receives music data composed by the external server.

[0156] In this way, the information processing device according to the present disclosure handles the second application having the automatic composition function as a plug-in, and causes an external server to execute the actual composition process. This allows the information processing device to provide the user with an environment with good work efficiency while suppressing the processing load. In other words, the information processing device can improve the convenience of the automatic composition function by AI.

[0157] The transmitting / receiving unit receives, as music data, a chord in a bar of a specified length, a melody in the bar, and a bass note in the bar. This allows the information processing device to refer to and edit the music data individually, improving user convenience.

[0158] The control unit individually sets instrument information for specifying a tone color when playing back chords, melodies, or bass sounds included in the music data, thereby enabling the information processing device to provide a variety of playback environments.

[0159] The control unit controls the display of windows displaying information about the chords, melody, or bass notes included in the music data, respectively, so that the information processing device can improve the convenience of the user's editing operations.

[0160] The control unit controls, in accordance with a user's operation, transmission and reception of information between each window displaying information related to chords, melodies, or bass notes and a window displaying information related to the first application, thereby enabling the information processing device to exchange information between the first application and the second application by an operation such as drag and drop, thereby improving the convenience of the user's editing operation.

[0161] The control unit controls, in accordance with a user's operation, transmission and reception of information between each window that displays information related to chords, melodies, or bass notes, thereby enabling the information processing device to improve the convenience of the user's editing operation.

[0162] The selection unit selects, as the setting information, designation information (in this embodiment, a style palette) for designating material music pieces that will become learning data for machine learning. The transmission / reception unit transmits the designation information selected by the selection unit to an external server. This enables the information processing device to execute automatic composition by designating various features desired by the user.

[0163] The selection unit selects, in response to a user's operation, designation information that is stored in advance in a storage unit (storage unit 15 in this embodiment) and that includes characteristic information indicating characteristics of the designation information and a plurality of material musical pieces associated with the characteristic information, thereby enabling the information processing device to improve the convenience for the user when selecting designation information.

[0164] The selection unit selects combination designation information (in this embodiment, a style palette sequence) that combines first designation information corresponding to some of the bars in the music data composed by the external server with second designation information corresponding to other bars, thereby enabling the information processing device to automatically generate a variety of music pieces.

[0165] When the transmitting / receiving unit receives music data composed by an external server based on the combination designation information, the transmitting / receiving unit stores the combination designation information and the music data in the storage unit in association with each other. This allows the information processing device to improve the convenience for the user when referring to the combination designation information, etc., on which the music data created in the past is based.

[0166] The selection unit selects, as the setting information, note length information included in the music data composed by the external server based on the designation information. The transmission / reception unit transmits the designation information and the note length information to the external server. This enables the information processing device to generate music data having characteristics desired by the user.

[0167] The selection unit selects, as the setting information, information for determining the probability that constituent tones included in a chord will appear in a melody in music data composed by an external server based on the designation information. The transmission / reception unit transmits the designation information and the information for determining the probability that constituent tones included in a chord will appear in a melody to the external server. This enables the information processing device to generate music data having characteristics desired by the user.

[0168] The selection unit selects, as the setting information, information for determining the type and amount of material pieces of music other than the material pieces of music included in the designation information in the music data composed by the external server based on the designation information. The transmission / reception unit transmits the designation information and the information for determining the type and amount of material pieces of music other than the material pieces of music included in the designation information to the external server. This enables the information processing device to generate music data having characteristics desired by the user.

[0169] The second application further includes a display control unit (display control unit 169 in the embodiment) that controls to call up the history of past music data composed by an external server from the storage unit in accordance with a user's operation and to display the history of the past music data. This enables the information processing device to improve the convenience for the user when referring to the past operation history, etc.

[0170] The display control unit controls to call up from the storage unit a history of editing operations performed on past music data composed by an external server, and to display the editing operations performed on the past music data, thereby enabling the information processing device to improve the convenience for a user when referring to a past operation history, etc.

[0171] When the user performs a playback or editing operation on the music data after receiving the music data composed by the external server, the transmitting / receiving unit transmits information on the playback or editing operation to the external server, so that the information processing device can cause the processing server 100 to perform further learning based on the editing or the like performed by the user.

[0172] The selection unit selects a chord progression for the music piece to be composed as setting information based on a user operation. The transmission / reception unit transmits the chord progression selected by the selection unit to an external server. This allows the information processing device to provide music piece data desired by the user without relying on designation information.

[0173] The control unit controls to send information about the chord progression generated in the first application to the second application. The selection unit selects the information about the chord progression generated in the first application as the setting information. The transmission / reception unit transmits the information about the chord progression generated in the first application to an external server, and receives music data composed based on the information about the chord progression. This allows the information processing device to execute composition processing that utilizes the functions of the first application, such as a DAW.

[0174] (5. Hardware Configuration) Information devices such as the user terminal 10, processing server 100, and management server 200 according to each of the above-described embodiments are realized by a computer 1000 having a configuration as shown in FIG. 20, for example. The following description will be given taking the user terminal 10 according to the embodiment as an example. FIG. 20 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the user terminal 10. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The various components of the computer 1000 are connected by a bus 1050.

[0175] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.

[0176] The ROM 1300 stores boot programs such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000.

[0177] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0178] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0179] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading a program or the like recorded on a predetermined recording medium. The medium may be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0180] For example, when the computer 1000 functions as the user terminal 10 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to realize functions of the control unit 16, etc. Also, the information processing program according to the present disclosure and data in the storage unit 15 are stored in the HDD 1400. Note that the CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be obtained from other devices via an external network 1550.

[0181] The present technology can also be configured as follows. (1) An information processing device that controls a first application and a second application that functions as a plug-in that extends a function of the first application, The first application a control unit for controlling an operation of the second application in the first application, The second app is a selection unit for selecting setting information for controlling a machine learning based composition function; a transceiver unit that transmits the setting information via a network to an external server that executes the machine learning-based composition function, and receives music data composed by the external server; An information processing device comprising: (2) The transmitting / receiving unit is As the music data, a chord in a bar of a specified length, a melody in the bar, and a bass note in the bar are received. The information processing device according to (1). (3) The control unit is Instrument information for specifying a tone color when playing chords, melodies or bass sounds included in the music data is individually set. The information processing device according to (2). (4) The control unit is A control is performed so that windows displaying information about chords, melodies, or bass notes included in the music data are individually displayed. The information processing device according to (3). (5) The control unit is Controlling transmission and reception of information between each window displaying information about the chord, melody, or bass note and a window displaying information about the first application in accordance with a user's operation. The information processing device according to (4). (6) The control unit is Controlling transmission and reception of information between each window that displays information about the chord, melody or bass note according to a user's operation The information processing device according to (4) or (5). (7) The selection unit is selecting, as the setting information, designation information for designating material music pieces that will be used as learning data for the machine learning; The transmitting / receiving unit is The designated information selected by the selection unit is transmitted to the external server. The information processing device according to any one of (1) to (6). (8) The selection unit is Selecting designation information stored in advance in a storage unit in accordance with a user's operation, the designation information including characteristic information indicating characteristics of the designation information and a plurality of material musical pieces associated with the characteristic information The information processing device according to (7). (9) The selection unit is Selecting combination designation information that combines first designation information corresponding to some of the measures in the music data composed by the external server with second designation information corresponding to other measures The information processing device according to (7) or (8). (10) The transmitting / receiving unit is When receiving music data composed by the external server based on the combination designation information, the combination designation information and the music data are stored in a storage unit in association with each other. The information processing device according to (9). (11) The selection unit is selecting, as the setting information, note duration information included in music data composed by the external server based on the designation information; The transmitting / receiving unit is The designation information and the note length information are transmitted to the external server. The information processing device according to any one of (7) to (10). (12) The selection unit is selecting, as the setting information, information for determining the probability that a constituent note included in a chord will appear in a melody in the music data composed by the external server based on the designation information; The transmitting / receiving unit is The designation information and information for determining the probability that a constituent note included in the chord will appear in the melody are transmitted to the external server. The information processing device according to any one of (7) to (11). (13) The selection unit is selecting, as the setting information, information for determining the types and amounts of material music pieces other than the material music pieces included in the designation information, in the music piece data composed by the external server based on the designation information; The transmitting / receiving unit is The designation information and information for determining the type and amount of material music pieces other than the material music pieces included in the designation information are transmitted to the external server. The information processing device according to any one of (7) to (12). (14) The second app is The apparatus further includes a display control unit that controls to call up a history of past music data composed by the external server from the storage unit in accordance with a user's operation and to display the history of the past music data. The information processing device according to any one of (1) to (13). (15) The display control unit is A history of editing operations performed on past music data composed by the external server is retrieved from a storage unit, and the editing operations performed on the past music data are displayed. The information processing device according to (14). (16) The transmitting / receiving unit is After receiving the composed music data from the external server, when a user performs a playback or editing operation on the music data, information on the playback or editing operation is transmitted to the external server. The information processing device according to any one of (1) to (15). (17) The selection unit is As the setting information, a chord progression in the music piece to be composed is selected based on a user operation; The transmitting / receiving unit is The chord progression selected by the selection unit is transmitted to the external server. The information processing device according to any one of (1) to (16). (18) The control unit is Controlling the transmission of information regarding the chord progression generated in the first application to the second application; The selection unit is selecting, as the setting information, information regarding a chord progression generated in the first application; The transmitting / receiving unit is Transmitting information about the chord progression generated in the first application to the external server, and receiving music data composed based on the information about the chord progression The information processing device according to any one of (1) to (17). (19) An information processing method executed by an information processing device that controls a first application and a second application that functions as a plug-in that extends a function of the first application, The first application Controlling an operation of the second application in the first application; The second app is Select the settings to control the machine learning-based composition function, Transmitting the setting information to an external server that executes the machine learning-based composition function via a network, and receiving music data composed by the external server. Information processing methods. (20) An information processing device that controls a first application and a second application that functions as a plug-in that extends a function of the first application, The first application Controlling an operation of the second application in the first application; The second app is Select the settings to control the machine learning-based composition function, Transmitting the setting information to an external server that executes the machine learning-based composition function via a network, and receiving music data composed by the external server. An information processing program that causes the information processing device to function in this manner.

[0182] The present technology can also be configured as follows. (1) a selection unit for selecting setting information for controlling a machine learning based composition function; a transceiver unit that transmits the setting information via a network to an external server that executes the machine learning-based composition function, and receives music data composed by the external server; Equipped with The selection unit is selecting, as the setting information, designation information for designating material music pieces to be used as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information; The transmitting / receiving unit is The designated information selected by the selection unit is transmitted to the external server. Information processing device. (2) The transmitting / receiving unit is As the music data, a chord in a bar of a specified length, a melody in the bar, and a bass note in the bar are received. The information processing device according to (1). (3) The selection unit is Selecting combination designation information that combines first designation information corresponding to some of the measures in the music data composed by the external server with second designation information corresponding to other measures The information processing device according to (1) or (2). (4) The transmitting / receiving unit is When receiving music data composed by the external server based on the combination designation information, the combination designation information and the music data are stored in a storage unit in association with each other. The information processing device according to (3). (5) The selection unit is selecting, as the setting information, note duration information included in music data composed by the external server based on the designation information; The transmitting / receiving unit is The designation information and the note length information are transmitted to the external server. The information processing device according to any one of (1) to (4). (6) The selection unit is selecting, as the setting information, information for determining the probability that a constituent note included in a chord will appear in a melody in the music data composed by the external server based on the designation information; The transmitting / receiving unit is The designation information and information for determining the probability that a constituent note included in the chord will appear in the melody are transmitted to the external server. The information processing device according to any one of (1) to (5). (7) The selection unit is selecting, as the setting information, information for determining the types and amounts of material music pieces other than the material music pieces included in the designation information, in the music piece data composed by the external server based on the designation information; The transmitting / receiving unit is The designation information and information for determining the type and amount of material music pieces other than the material music pieces included in the designation information are transmitted to the external server. The information processing device according to any one of (1) to (6). (8) The apparatus further includes a display control unit that controls to call up a history of past music data composed by the external server from the storage unit in accordance with a user's operation and to display the history of the past music data. The information processing device according to any one of (1) to (7). (9) The display control unit is A history of editing operations performed on past music data composed by the external server is retrieved from a storage unit, and the editing operations performed on the past music data are displayed. The information processing device according to (8). (10) The transmitting / receiving unit is After receiving the composed music data from the external server, when a user performs a playback or editing operation on the music data, information on the playback or editing operation is transmitted to the external server. The information processing device according to any one of (1) to (9). (11) The selection unit is As the setting information, a chord progression in the music piece to be composed is selected based on a user operation; The transmitting / receiving unit is The chord progression selected by the selection unit is transmitted to the external server. The information processing device according to any one of (1) to (10). (12) the plurality of material musical pieces associated with the characteristic information are musical pieces pre-registered in the external server; The information processing device according to any one of (1) to (11). (13) the plurality of material musical pieces associated with the characteristic information are musical pieces combined by a user from musical pieces preregistered in the external server; The information processing device according to any one of (1) to (12). (14) the plurality of material songs associated with the characteristic information are songs selected from a music distribution service; The information processing device according to any one of (1) to (13). (15) The plurality of material music pieces associated with the feature information is a playlist automatically generated by a predetermined music application. The information processing device according to (14). (16) The plurality of material music pieces associated with the characteristic information are composed of music pieces arbitrarily combined by a user in a music distribution service. The information processing device according to any one of (1) to (15). (17) The selection unit is Selecting a plurality of pieces of the specified information as the setting information. The information processing device according to any one of (1) to (17). (18) The information processing device includes: An information processing device that controls a first application and a second application that functions as a plug-in that extends a function of the first application, The first application a control unit for controlling an operation of the second application in the first application, The second app is The selecting unit and the transmitting / receiving unit are provided. The information processing device according to any one of (1) to (17). (19) The control unit is Instrument information for specifying a tone color when playing chords, melodies or bass sounds included in the music data is individually set. The information processing device according to (18). (20) The control unit is A control is performed so that windows displaying information about chords, melodies, or bass notes included in the music data are individually displayed. The information processing device according to (19). (twenty one) The control unit is Controlling transmission and reception of information between each window displaying information about the chord, melody, or bass note and a window displaying information about the first application in accordance with a user's operation. The information processing device according to (20). (twenty two) The control unit is Controlling transmission and reception of information between each window that displays information about the chord, melody or bass note according to a user's operation The information processing device according to (20) or (21). (twenty three) The control unit is Controlling the transmission of information regarding the chord progression generated in the first application to the second application; The selection unit is selecting, as the setting information, information regarding a chord progression generated in the first application; The transmitting / receiving unit is Transmitting information about the chord progression generated in the first application to the external server, and receiving music data composed based on the information about the chord progression The information processing device according to any one of (18) to (22). (twenty four) The first application is a DAW (Digital Audio Workstation). The information processing device according to any one of (18) to (23). (twenty five) The control unit is Displaying information regarding the specified information; The information processing device according to any one of (18) to (24). (26) The control unit is Display information about harmony, The information processing device according to any one of (18) to (25). (27) The information about the harmony is information for determining a probability that a constituent note included in a chord appears in a melody in the music data composed by the external server. The information processing device according to (26). (28) the information about harmony is a slide bar for setting the probability that a constituent note included in a chord will appear in a melody in the music data composed by the external server; The information processing device according to (27). (29) The control unit is Display note duration information, The information processing device according to any one of (18) to (28). (30) The note duration information is information for determining the duration of notes in music data composed by the external server. 30. The information processing device according to claim 29. (31) The note duration information is a slide bar for determining the duration of the note. The information processing device according to (30). (32) The control unit is displaying information for determining the type and amount of material music pieces other than the material music pieces included in the designation information; The information processing device according to any one of (18) to (31). (33) The control unit is displaying a slide bar for determining the type and amount of material music pieces other than the material music pieces included in the designation information; The information processing device according to (32). (34) The control unit is displaying at least two of the following information in a row: information on harmony, information on note duration, and information for determining the type and amount of material music other than the material music included in the designated information; The information processing device according to any one of (18) to (33). (35) The control unit is displaying a plurality of pieces of music data composed by the external server; The information processing device according to any one of (18) to (34). (36) The computer Select the settings to control the machine learning-based composition function, transmitting the setting information to an external server that executes the machine learning based composition function via a network, and receiving music data composed by the external server; Including, Further, the selecting step comprises: selecting, as the setting information, designation information for designating material music pieces to be used as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information; The transmitting and receiving includes: transmitting the selected specification information to the external server; An information processing method comprising: (37) Computer, a selection unit for selecting setting information for controlling a machine learning based composition function; a transceiver unit that transmits the setting information via a network to an external server that executes the machine learning-based composition function, and receives music data composed by the external server; An information processing program that causes an information processing device to function as an information processing device comprising: The selection unit is selecting, as the setting information, designation information for designating material music pieces to be used as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information; The transmitting / receiving unit is transmitting the designated information selected by the selection unit to the external server; Information processing program. [Explanation of symbols]

[0183] 1. Information Processing Systems 10 User terminal 11 Communications Department 12 Input section 13 Display section 15 Storage section 151 Composition Settings Information 152 Composition information 153 History Information 154 Instrument Information 16 Control section 161 Host application control section 162 Plug-in control section 163 Playback Department 164 Display control section 165 Plug-in application control section 166 Selection Section 167 Transmitter / Receiver 168 Playback Department 169 Display control section 100 Processing Servers 200 Management Server

Claims

1. a selection unit for selecting setting information for controlling a machine learning based composition function; a transceiver unit that transmits the setting information via a network to an external server that executes the machine learning-based composition function, and receives music data composed by the external server; Equipped with The selection unit is selecting, as the setting information, designation information for designating material music pieces to be used as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information based on a user's input for a plurality of pieces of feature information previously presented on a user interface; The transmitting / receiving unit is Transmitting the designated information selected by the selection unit to the external server; The plurality of material music pieces associated with the feature information is a playlist automatically generated by a predetermined music application. Information processing device.

2. a selection unit for selecting setting information for controlling a machine learning based composition function; a transceiver unit that transmits the setting information via a network to an external server that executes the machine learning-based composition function, and receives music data composed by the external server; Equipped with The selection unit is selecting, as the setting information, designation information for designating material music pieces to be used as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information; The transmitting / receiving unit is Transmitting the designated information selected by the selection unit to the external server; the plurality of material songs associated with the characteristic information are songs selected from a music distribution service; Information processing device.

3. The transmitting / receiving unit is As the music data, a chord in a bar of a specified length, a melody in the bar, and a bass note in the bar are received.

3. The information processing device according to claim 1 or 2.

4. The selection unit is Selecting combination designation information that combines first designation information corresponding to some of the measures in the music data composed by the external server with second designation information corresponding to other measures 3. The information processing device according to claim 1 or 2.

5. The transmitting / receiving unit is When receiving music data composed by the external server based on the combination designation information, the combination designation information and the music data are stored in a storage unit in association with each other. The information processing device according to claim 4.

6. The selection unit is selecting, as the setting information, note duration information included in music data composed by the external server based on the designation information; The transmitting / receiving unit is The designation information and the note length information are transmitted to the external server.

3. The information processing device according to claim 1 or 2.

7. The selection unit is selecting, as the setting information, information for determining a probability that a constituent note included in a chord will appear in a melody in the music data composed by the external server based on the designation information; The transmitting / receiving unit is The designation information and information for determining the probability that a constituent note included in the chord will appear in the melody are transmitted to the external server.

3. The information processing device according to claim 1 or 2.

8. The selection unit is selecting, as the setting information, information for determining the types and amounts of material music pieces other than the material music pieces included in the designation information, in the music piece data composed by the external server based on the designation information; The transmitting / receiving unit is The designation information and information for determining the type and amount of material music pieces other than the material music pieces included in the designation information are transmitted to the external server.

3. The information processing device according to claim 1 or 2.

9. The apparatus further includes a display control unit that controls to call up a history of past music data composed by the external server from the storage unit in accordance with a user's operation and to display the history of the past music data.

3. The information processing device according to claim 1 or 2.

10. The display control unit is A history of editing operations performed on past music data composed by the external server is retrieved from a storage unit, and the editing operations performed on the past music data are displayed. The information processing device according to claim 9.

11. The transmitting / receiving unit is After receiving the composed music data by the external server, when a user performs a playback or editing operation on the music data, information on the playback or editing operation is transmitted to the external server.

3. The information processing device according to claim 1 or 2.

12. The selection unit is As the setting information, a chord progression in the music piece to be composed is selected based on a user operation; The transmitting / receiving unit is The chord progression selected by the selection unit is transmitted to the external server.

3. The information processing device according to claim 1 or 2.

13. The plurality of material music pieces associated with the characteristic information are composed of music pieces arbitrarily combined by a user in a music distribution service. The information processing device according to claim 2 .

14. The selection unit is Selecting a plurality of pieces of the specified information as the setting information.

3. The information processing device according to claim 1 or 2.

15. The information processing device includes: An information processing device that controls a first application and a second application that functions as a plug-in that extends a function of the first application, The first application a control unit for controlling an operation of the second application in the first application, The second app is The selecting unit and the transmitting / receiving unit are provided.

3. The information processing device according to claim 1 or 2.

16. The control unit is Instrument information for specifying a tone color when playing chords, melodies or bass sounds included in the music data is individually set. The information processing device according to claim 15.

17. The control unit is A control is performed so that windows displaying information about chords, melodies, or bass notes included in the music data are individually displayed. The information processing device according to claim 16.

18. The control unit is Controlling transmission and reception of information between each window displaying information about the chord, melody or bass note and a window displaying information about the first application in accordance with a user's operation. The information processing device according to claim 17.

19. The control unit is Controlling transmission and reception of information between each window that displays information about the chord, melody or bass note according to a user's operation The information processing device according to claim 17.

20. The control unit is Controlling the transmission of information regarding the chord progression generated in the first application to the second application; The selection unit is selecting, as the setting information, information regarding a chord progression generated in the first application; The transmitting / receiving unit is Transmitting information about the chord progression generated in the first application to the external server, and receiving music data composed based on the information about the chord progression The information processing device according to claim 15.

21. The first application is a DAW (Digital Audio Workstation). The information processing device according to claim 15.

22. The control unit is Displaying information regarding the specified information; The information processing device according to claim 15.

23. The control unit is Display information about harmony, The information processing device according to claim 15.

24. The information about the harmony is information for determining a probability that a constituent note included in a chord appears in a melody in the music data composed by the external server. The information processing device according to claim 23.

25. the information about harmony is a slide bar for setting the probability that a constituent note included in a chord will appear in a melody in the music data composed by the external server; The information processing device according to claim 24.

26. The control unit is Display note duration information, The information processing device according to claim 15.

27. The note duration information is information for determining the duration of notes in music data composed by the external server.

27. The information processing device according to claim 26.

28. The note duration information is a slide bar for determining the duration of the note.

28. The information processing device according to claim 27.

29. The control unit is displaying information for determining the type and amount of material music pieces other than the material music pieces included in the designation information; The information processing device according to claim 15.

30. The control unit is displaying a slide bar for determining the type and amount of material music pieces other than the material music pieces included in the designation information; 30. The information processing device according to claim 29.

31. The control unit is displaying at least two of the information on harmony, the information on note duration, and the information for determining the type and amount of material music pieces other than the material music pieces included in the designation information side by side; The information processing device according to claim 15.

32. The control unit is displaying a plurality of pieces of music data composed by the external server; The information processing device according to claim 15.

33. The computer Select the settings to control the machine learning-based composition function, transmitting the setting information to an external server that executes the machine learning-based composition function via a network, and receiving music data composed by the external server; Including, Further, the selecting step comprises: selecting, as the setting information, designation information for designating material music pieces to be used as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information based on a user's input for a plurality of pieces of feature information previously presented on a user interface; The transmitting step includes: transmitting the selected designation information to the external server; The plurality of material music pieces associated with the feature information is a playlist automatically generated by a predetermined music application. Information processing methods.

34. Computer, a selection unit for selecting setting information for controlling a machine learning based composition function; a transceiver unit that transmits the setting information via a network to an external server that executes the machine learning-based composition function, and receives music data composed by the external server; An information processing program that causes an information processing device to function as an information processing device comprising: The selection unit is selecting, as the setting information, designation information for designating material music pieces to be used as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information based on a user's input for a plurality of pieces of feature information previously presented on a user interface; The transmitting / receiving unit is Transmitting the designated information selected by the selection unit to the external server; The plurality of material music pieces associated with the feature information is a playlist automatically generated by a predetermined music application. Information processing program.

35. The computer Select the settings to control the machine learning-based composition function, transmitting the setting information to an external server that executes the machine learning-based composition function via a network, and receiving music data composed by the external server; Including, Further, the selecting step comprises: selecting, as the setting information, designation information for designating material music pieces to be used as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information; The transmitting step includes: and transmitting the selected designation information to the external server. the plurality of material songs associated with the characteristic information are songs selected from a music distribution service; Information processing methods.

36. Computer, a selection unit for selecting setting information for controlling a machine learning based composition function; a transceiver unit that transmits the setting information via a network to an external server that executes the machine learning-based composition function, and receives music data composed by the external server; An information processing program that causes an information processing device to function as an information processing device comprising: The selection unit is selecting, as the setting information, designation information for designating material music pieces to be used as learning data for the machine learning, the designation information including feature information indicating features of the designation information and a plurality of material music pieces associated with the feature information; The transmitting / receiving unit is Transmitting the designated information selected by the selection unit to the external server; the plurality of material songs associated with the characteristic information are songs selected from a music distribution service; Information processing program.

Images