Information Processing Apparatus, Information Processing Method, Information Processing Program, and Information Processing System

Abstract

An information processing device according to the present disclosure comprises: an acquisition unit which acquires musical piece information; an extraction unit which extracts a feature amount for each of a plurality of types from the musical piece information acquired by the acquisition unit; and a generation unit which generates, as musical piece feature information to be used as learning data in a musical composition process using machine learning, information in which prescribed identification information is associated with the feature amount for each of the plurality of types, which is extracted by the extraction unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method , feeling and an information processing program and information processing system thereof.

Background Art

[0002] With the progress of AI (Artificial Intelligence), the use of computers in the field of art has been promoted. For example, there is known a technique of performing machine learning using existing music as learning data to generate a model for music generation and having a computer compose a new music (for example, Patent Document 1). In such a technique, it is possible to imitate the characteristics of existing music or generate a more natural melody using a Markov model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] According to the prior art, since music information proposed (generated) by AI can be used in the composition work, a user can compose based on more diverse viewpoints.

[0005] The automatic composition function by AI is set for general users, and general users can receive automatically created music information only by setting an image such as bright or dark. On the other hand, since producers who create music often specifically set features of a song such as chord progressions and bass progressions in the process of creating a song, there has been a demand from producers to receive the provision of music information that matches the features of the song rather than an image.

[0006] Therefore, in the present disclosure, an information processing apparatus, an information processing method capable of improving the convenience of a music composition function by a user , feeling information processing program and information processing system is proposed.

Means for Solving the Problem

[0007] In order to solve the above problems, an information processing apparatus according to an aspect of the present disclosure includes an acquisition unit that acquires music information, an extraction unit that extracts a plurality of types of feature amounts from the music information acquired by the acquisition unit, and the plurality of types of feature amounts extracted by the extraction unit and a predetermined identification information Music features information to generate constituting a generation unit, A transmission unit that transmits presentation information of the music feature information according to instruction information received from a terminal device in which an application related to music creation is installed; when receiving a selection of the music feature information from the terminal device, based on the selected music feature information, A composition unit that composes music information using machine learning and transmits the composed music information to a terminal device and has.

Brief Description of the Drawings

[0008] [Figure 1] It is a conceptual diagram showing the flow of information processing according to an embodiment. [Figure 2] It is a diagram showing an example of the data configuration of style information according to an embodiment. [Figure 3] It is a diagram showing an example of the display screen of a user terminal according to an embodiment. [Figure 4] It is a diagram showing an example of the display screen of a user terminal according to an embodiment. [Figure 5] It is a diagram showing an example of the display screen of a user terminal according to an embodiment. [Figure 6] It is a diagram showing an example of the display screen of a user terminal according to an embodiment. [Figure 7] It is a diagram showing an example of the display screen of a user terminal according to an embodiment. [Figure 8] It is a diagram showing the flow of information processing according to an embodiment. [Figure 9] It is a diagram showing an example of an information processing system according to an embodiment. [Figure 10] The configuration of the work management device according to the embodiment will be described. [Figure 11] It is a diagram showing an example of the copyrighted music information storage unit according to an embodiment. [Figure 12] This is a diagram showing an example of the music memory unit according to the embodiment. [Figure 13] This is a diagram showing an example of the feature information memory unit according to the embodiment. [Figure 14] This is a diagram for explaining the configuration of the overall information according to the embodiment. [Figure 15] This is a diagram showing an example of the time series information constituting the music feature information according to the embodiment. [Figure 16] This is a diagram showing a configuration example of the information processing apparatus according to the embodiment. [Figure 17] This is a diagram showing an example of the user information memory unit according to the embodiment. [Figure 18] This is a diagram showing an example of the style information memory unit according to the embodiment. [Figure 19] This is a diagram showing an example of the ownership information memory unit according to the embodiment. [Figure 20] This is a diagram showing an example of the production information memory unit according to the embodiment. [Figure 21] This is a diagram showing a configuration example of the user terminal according to the embodiment. [Figure 22] This is a sequence diagram showing the procedure of information processing according to the embodiment. [Figure 23] This is a sequence diagram showing the procedure of information processing according to the embodiment. [Figure 24] This is a sequence diagram showing the procedure of information processing according to the embodiment. [Figure 25] This is a diagram showing an example of the conceptual diagram of the configuration of the information processing system. [Figure 26] This is a diagram showing an example of the user interface according to the embodiment. [Figure 27] This is a diagram showing an example of the user interface according to the embodiment. [Figure 28] This is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing apparatus, the work management apparatus, and the user terminal.

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited by this embodiment. Also, in each of the following embodiments, the same parts are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] The present disclosure will be described in accordance with the item order shown below. 1. Embodiment 1-1. An example of information processing according to the embodiment 1-2. Configuration of the information processing system according to the embodiment 1-3. Configuration of the work management device according to the embodiment 1-4. Configuration of the information processing apparatus according to the embodiment 1-5. Configuration of the user terminal according to the embodiment 1-6. Procedure of information processing according to the embodiment 1-6-1. Generation process of style information of copyrighted music information 1-6-2. Generation process of style information of music information created by a producer 1-6-3. Update process of style information 1-7. Conceptual diagram of the configuration of the information processing system 1-7-1. Regarding the overall configuration 1-7-2. Regarding the copyrighted music management server device 1-7-3. Regarding the server device 1-7-4. Regarding the application part for music producers 2. Effects according to the present embodiment 3. Other embodiments 3-1. Other configuration examples 3-2. Others 4. Hardware configuration

[0011] [1. Embodiment] [1-1. An example of information processing according to the embodiment] First, an example of information processing according to the present disclosure will be described with reference to FIG. 1. FIG. 1 is a conceptual diagram showing the flow of information processing according to an embodiment. The information processing according to the embodiment is executed by a copyrighted work management device 100, an information processing device 200, and a user terminal 300.

[0012] In the present embodiment, an example is shown in which the information processing device 200 is an information processing device that provides a service (also simply referred to as "service") related to the creation of content (information) as a copyrighted work. Hereinafter, although a music piece (music content) is shown as an example of the content, the content may be various contents such as video content such as a movie or text content such as a book (novel, etc.), not limited to music pieces. Also, the music piece here is a concept that includes not only a completed single piece (whole), but also various music information such as a part of the sound source that constitutes a single piece (music piece) and a short sound used for sampling.

[0013] The information processing device 200 communicates with the copyrighted work management device 100 that manages copyrighted music information via a private network N2 (see FIG. 9). The private network N2 is, for example, a VPN (Virtual Private Network) with security ensured or Internet communication with ID address authentication. The information processing device 200 communicates with the user terminal 300 of a user who uses the service provided by the information processing device 200 via a network N1 (see FIG. 9) such as the Internet. Note that the number of user terminals 300 is not limited to that shown in FIG. 1.

[0014] The copyrighted work management device 100 is a server device that registers and manages copyrighted music information. The copyrighted work management device 100 periodically performs registration of copyrighted music information. The copyrighted work management device 100 extracts a plurality of types of feature amounts from the registered copyrighted music information and transmits the extracted feature amounts to the information processing device 200 via the private network N2.

[0015] The user terminal 300 is an information processing terminal such as a PC (personal computer) or a tablet terminal. Various program applications are installed on the user terminal 300. An application related to music creation is installed on the user terminal 300. For example, the user terminal 300 has an automatic composition function by AI added by a plug-in (extended application) to an application such as a DAW that realizes a comprehensive music production environment. For example, the plug-in can take forms such as VST (Steinberg's Virtual Studio Technology) (registered trademark), AudioUnits, AAX (Avid Audio eXtension), etc. Also, the user terminal 300 is not limited to DAW, and for example, mobile applications such as iOS may be used.

[0016] The user terminal 300 activates and executes the DAW and the automatic composition function by AI, and communicates with the information processing device 200 to receive the music information composed in the information processing device 200.

[0017] The user of the user terminal 300 is either an administrator who operates and manages the entire system, a composer who creates music, a producer such as an arranger or a studio engineer, or a general user who receives music information through the automatic composition function. In this embodiment, it is described that the user terminal 300 is used by the producer Uc.

[0018] The information processing device 200 is a server device that executes information processing related to the automatic composition function by AI of the user terminal 300. For example, the information processing device 200 is a so-called cloud server, and executes automatic composition by AI according to the instruction information from the user terminal 300, and provides the generated music information to the user terminal 300.

[0019] The information processing device 200 generates a composition model for music generation by performing machine learning. For example, the information processing device 200 uses a Markov model or the like to provide music information automatically composed to the user terminal 300.

[0020] The information processing apparatus 200 uses style information (music piece feature information) as learning data for the composition model. The style information is information in which a plurality of types of feature quantities such as chord progressions, melodies, and bass progressions and predetermined identification information are associated with each other, which are extracted from music piece information as a plurality of types of feature quantities, and is used in composition processing using machine learning. The information processing apparatus 200 obtains a plurality of types of feature quantities from the copyrighted music piece information or the music piece information created by the producer, and for each music piece information, by collectively assigning each feature quantity with a style information ID (predetermined identification information), a plurality of style information is generated and stored in a database.

[0021] FIG. 2 is a diagram showing an example of the data configuration of the style information according to the embodiment. The style information has a style information ID 710 which is identification information of the style information, style palette sequence information 720 (music piece order information), style palette information 730 (music piece form information), score information 740, and lyric information 750.

[0022] The score information 740 includes a plurality of types of feature quantities extracted from the music piece. The score information 740 has a score ID, melody information, chord progression information, bass information, and drum information. The score ID is identification information of the score information. The melody information is a melody in a measure of a specified length. The chord progression information is information indicating the chord progression in a measure of a specified length. The bass information is information indicating the progression of the bass notes in a measure of a specified length. The drum information is information indicating the progression of the drum sounds (drum patterns and tempo) in a measure of a specified length.

[0023] The lyric information 750 includes a lyric ID and lyric information. The lyric ID is identification information of the lyric information. The lyric information is information indicating the lyrics in a measure of a specified length. The lyric information is, for example, a phrase or a character keyword that is the source of the lyrics. The information processing apparatus 200 can also automatically generate lyrics by using a plurality of lyric information 750 of the style information 700.

[0024] The style palette information 730 is information in which a style palette ID, which is identification information of the style palette information, is registered in association with a score ID of the score information 740 and a lyric ID of the lyric information 750 for the same measure.

[0025] The style palette sequence information 720 is information indicating the order of the style palette information 730. The style palette sequence information 720 has, as a set, a style palette ID that uniquely indicates the style palette information 730 and a bar index, and by having a plurality of such sets, it becomes information for managing the order in the music piece of the style palette information 730. For example, in the case of the example shown in FIG. 2, it is defined that the style palette ID 731a corresponds to measures 1 to 4 of the music piece, the style palette ID 731b corresponds to measures 5 to 8, and the style palette ID 731z corresponds to measures x to y.

[0026] The information processing apparatus 200 performs machine learning using this style information 700 as learning data and performs composition processing. Therefore, the information processing apparatus 200 does not learn the music piece information itself, but learns style information composed of a plurality of types of feature amounts such as chord progressions, melodies, and bass progressions extracted from the music piece information. That is, since the information processing apparatus 200 learns a plurality of feature amounts extracted in advance from the music piece information, the information processing load is smaller than when learning the music piece information itself, and the music piece information can be efficiently provided to the user.

[0027] Specifically, the process of music piece creation by the producer will be described. FIGS. 3 to 7 are diagrams showing an example of the display screen of the user terminal 300 according to the embodiment. When the producer activates the automatic composition function on the user terminal 300, the window 370 shown in FIG. 3 is displayed on the user terminal 300.

[0028] Note that the window 370 includes a composition parameter setting unit 371, a style information display unit 372, a composition control unit 373, and a produced music display / editing unit 374. The composition parameter setting unit 371 is an area where parameters such as note length and complexity can be set. The style information display unit 372 is an area where style information to be used for composition can be selected by keyword input or pull-down selection. The composition control unit 373 is an area where a composition instruction can be given by selecting a composition execution instruction button. The produced music display / editing unit 374 is an area where a plurality of piano rolls on which melodies and lyrics are displayed are shown.

[0029] Then, as shown in FIG. 4, when the style palette selection pull-down 372a is selected, the chord progressions of each style information included in the information processing apparatus 200 are listed as candidates.

[0030] The candidates for the chord progressions may be displayed in any order, such as alphabetical order, the order of the most frequently used by the producer, the order of the most frequently used by all users, the order of generation of the style information, etc. All of the style information included in the information processing apparatus 200 may be displayed for the chord progressions, or only a part thereof may be displayed. When there are many candidates for the chord progressions, the display area can be selected using a pager. Also, when the producer inputs a desired chord progression in the search keyword input field 372b, the information processing apparatus 200 may extract the style information including this chord progression and list the chord progression information of each extracted style information in the style palette selection pull-down 372a.

[0031] The producer selects a desired chord progression from the chord progressions presented in the style palette selection dropdown 372a and selects the composition execution instruction button. As a result, the information processing device 200 extracts style information having the selected chord progression, performs machine learning using the extracted style information 700 as learning data, and performs a composition process. Then, the information processing device 200 provides music information to the user terminal 300. As a result, the producer can receive the provision of music information generated according to this chord progression only by selecting a desired chord progression from the chord progressions presented in the style palette selection dropdown 372a.

[0032] In addition, since the style information 700 includes lyric information as a feature amount, the producer can receive the presentation of style information matching this lyric only by inputting a desired lyric. Specifically, as shown in FIG. 5, when the producer inputs a desired lyric in the search keyword input field 372b, the information processing device 200 extracts style information having this lyric or a lyric similar to this lyric, and may list and display the lyric information of each extracted style information in the style palette selection dropdown 372a.

[0033] The producer selects desired lyric information from the lyric information presented in the style palette selection dropdown 372a and selects the composition execution instruction button. As a result, the information processing device 200 extracts style information having the selected lyric information, performs machine learning using the extracted style information 700 as learning data, and performs a composition process to provide music information to the user terminal 300.

[0034] As a result, the producer can receive the provision of music information generated according to this lyric only by selecting a desired lyric from the lyrics presented in the style palette selection dropdown 372a. At this time, the information processing device 200 may also automatically generate lyrics according to the generated music and provide music information in which the melody and the lyrics are associated to the user terminal 300. In this case, on the screen of the user terminal 300, the melody display piano roll 374a in FIG. 6 displays the melody and the lyrics corresponding to the melody.

[0035] Also, when the producer inputs lyrics, as shown in FIG. 7, the user terminal 300 may display a list of candidates for the chord progression of the style information presented from the information processing apparatus 200 to assist the producer in composing music.

[0036] In this way, the information processing apparatus 200 generates style information having feature amounts of a plurality of types of music information as a data set for learning the composition model, and causes the composition model to learn. Thereby, the information processing apparatus 200 provides the producer with music information composed according to the characteristics of the music. Hereinafter, the flow of the style information generation process among the information processing according to the present embodiment will be described with reference to FIG. 1.

[0037] As shown in FIG. 1, when new copyrighted music information is registered in the copyrighted work management apparatus 100 (step S11), the copyrighted work management apparatus 100 extracts feature amounts of a plurality of types from each newly registered copyrighted music information (step S12). At this time, the copyrighted work management apparatus 100 extracts the feature amounts by using, for example, 12-tone analysis technology. Then, the information processing apparatus 200 acquires feature amounts of a plurality of types from the copyrighted work management apparatus 100 (step S13), and for each music information, summarizes each feature amount and assigns a style information ID to generate style information corresponding to each music information (step S14).

[0038] Also, when a new music is created through input of feature amounts such as chord progression, melody, and bass progression by the producer Uc of the user terminal 300, the information processing apparatus 200 acquires music information including each feature amount (step S21). Then, the information processing apparatus 200 extracts feature amounts such as chord progression, melody, and bass progression from the acquired music information (step S22). Then, the information processing apparatus 200 summarizes each feature amount for each music information and assigns a style information ID to generate style information corresponding to each music information (step S23). The information processing apparatus 200 generates a plurality of style information 700 by performing the processing shown in FIG. 1 and creates a database.

[0039] In the information processing according to this embodiment, it is also possible to update style information. FIG. 8 is a conceptual diagram showing the flow of information processing according to the embodiment. As shown in FIG. 8, during music creation by the producer Uc, when style information is selected by the producer Uc (step S31), the information processing apparatus 200 performs composition processing by learning this style information and outputs music information to the user terminal 300 (step S32). The user Uc further performs performance based on the provided music information. When the information processing apparatus 200 receives this performance information (step S33), it adds the performance information to the style information used for composition and updates this style information (step S34).

[0040] In this way, in the information processing according to this embodiment, the style information being used is updated according to the music creation of the producer Uc. As a result, the information processing apparatus 200 can make this style information approach the style of the producer Uc's music creation and compose and provide music information that suits the style of the producer Uc.

[0041] The overall outline of the flow of the information processing according to this embodiment has been described above. From FIG. 9 onwards, the configuration of the information processing system 1 including the work management apparatus 100, the information processing apparatus 200, and the user terminal 300 will be described, and the details of various processes will be described in order.

[0042] [1-2. Configuration of the Information Processing System According to the Embodiment] FIG. 9 is a diagram showing an example of the information processing system 1 according to the embodiment. As shown in FIG. 6, the information processing system 1 includes user terminals 300-1 to 300-3, an information processing apparatus 200, and a work management apparatus 100. The information processing system 1 functions as a work management system and a learning model information management system. In the example of FIG. 9, three user terminals 300-1 to 300-3 are illustrated, but when there is no particular distinction in the description, they are referred to as the user terminal 300.

[0043] The information processing apparatus 200 and the user terminal 300 are communicably connected to each other via the network N1, either wired or wirelessly. Further, the information processing apparatus 200 and the work management apparatus 100 are communicably connected to each other via the private network N2, either wired or wirelessly.

[0044] The work management apparatus 100 manages music work information. The work management apparatus 100 has music work information registered therein periodically. The work management apparatus 100 extracts a plurality of types of feature amounts from the registered music work information, and transmits the extracted feature amounts to the information processing apparatus 200.

[0045] The user terminal 300 transmits music information created by a producer to the information processing apparatus 200, and receives the provision of music information composed by the information processing apparatus 200 when the automatic composition function is activated.

[0046] The information processing apparatus 200 generates style information, which is learning data, from music work information or music information created by a producer, and generates a composition model by performing machine learning. The information processing apparatus 200 provides music information automatically composed using the generated model to the user terminal 300.

[0047] [1-3. Configuration of the Work Management Apparatus According to the Embodiment] Next, with reference to FIG. 10, the configuration of the work management apparatus 100 shown in FIG. 9 will be described. FIG. 10 is a diagram showing a configuration example of the work management apparatus 100 according to the embodiment. As shown in FIG. 10, the work management apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

[0048] The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 110 is connected to the private network N2, either wired or wirelessly, and transmits and receives information to and from the information processing apparatus 200 via the private network N2.

[0049] The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 stores various data used for information processing. The storage unit 120 includes a copyrighted music information storage unit 121, a music storage unit 122, and a feature information storage unit 123.

[0050] The copyrighted music information storage unit 121 stores information regarding copyrighted music, which is a work created in the past. FIG. 11 is a diagram showing an example of the copyrighted music information storage unit 121 according to the embodiment.

[0051] As shown in FIG. 11, the copyrighted music information storage unit 121 stores a copyrighted music ID, copyrighted music meta information, and an MP3 file name. The copyrighted music ID is, for example, identification information for uniquely identifying a copyrighted music such as The Beatles or Michael Jackson. The copyrighted music meta information is additional information regarding the copyrighted music, such as the title of the copyrighted music, the artist name, the album name, the release date, the genre, the mood, the lyrics, and the like. The MP3 file name is a file name for identifying the MP3 file of the copyrighted music.

[0052] The music storage unit 122 stores music information of the copyrighted music. FIG. 12 is a diagram showing an example of the music storage unit 122 according to the embodiment. As shown in FIG. 12, the music storage unit 122 stores the MP3 files of each copyrighted music. This MP3 file can be identified by the MP3 file name stored in the copyrighted music information storage unit 121.

[0053] The feature information storage unit 123 stores a plurality of types of feature amounts of the copyrighted music. FIG. 13 is a diagram showing an example of the feature information storage unit 123 according to the embodiment. As shown in FIG. 13, in the feature information storage unit 123, the copyrighted music feature amounts are stored in association with the copyrighted music IDs. The copyrighted music feature information includes overall information indicating features of the entire copyrighted music and time-series information indicating features that change over time.

[0054] FIG. 14 is a diagram showing an example of overall information according to the embodiment. As shown in FIG. 14, the overall information has, for example, tempo information, beat information, and music genre information such as genre and mood. The music genre information is, for example, information such as the delegate channel of the 12-tone analysis technique.

[0055] FIG. 15 is a diagram showing an example of time series information constituting the copyrighted music feature information according to the embodiment. As shown in FIG. 15, the time series information has, for example, beat information, chord progression information, music structure information, and melody information. The beat information has, for example, the time position information of each beat and the time position information of each measure, such as a metronome. The chord progression information is information on the time series progression of the accompaniment chords according to the flow of the music, for example, information indicating the progression of chords such as "C-F-Am-G7". The music structure information is the structure of the entire music, for example, information such as "A melody - B melody - refrain - A melody - refrain". The melody information has, for example, information on the melody line inferred from the audio information of the music.

[0056] Returning to FIG. 10, the description will be continued. The control unit 130 is realized, for example, by a program stored inside the user terminal 300 being executed with a RAM (Random Access Memory) or the like as a work area by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. Further, the control unit 130 is a controller and may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0057] The control unit 130 has a management unit 131, an extraction unit 132, and a transmission unit 133, and realizes or executes the information processing functions and operations described below.

[0058] The management unit 131 manages various information related to the copyrighted music work management device 100. For example, the management unit 131 stores various information in the storage unit 120 and appropriately updates the stored information. Specifically, the management unit 131 stores a new copyrighted music piece in the copyrighted music work information storage unit 121 and updates the information related to the new copyrighted music piece.

[0059] The extraction unit 132 extracts a plurality of types of feature quantities from the copyrighted music work information. The extraction unit 132 periodically acquires the copyrighted music work information that is registered, at a preset timing, and extracts code progression information, beat information, melody information, and drum information as feature quantities from the acquired copyrighted music work information. The extraction unit 132 extracts the music feature quantities of the MP3 file of each newly registered copyrighted music piece to obtain feature information. The extraction unit 132 extracts various feature quantities from the music information using, for example, 12-tone analysis technology and registers them in the feature information storage unit 123 as copyrighted music work feature information.

[0060] The extraction unit 132 performs a feature quantity extraction process on the copyrighted music work information by receiving a copyrighted music work extraction instruction from an administrator terminal (not shown) used by the system administrator via the information processing device 200. Alternatively, the extraction unit 132 performs a feature quantity extraction process on the copyrighted music work information by receiving a copyrighted music work extraction instruction from the information processing device 200.

[0061] The transmission unit 133 transmits the feature quantities of the copyrighted music work information of a plurality of types extracted by the extraction unit 132 to the information processing device 200 together with information related to the copyrighted music work information, such as music structure information and copyrighted music work meta information.

[0062] [1-4. Configuration of the Information Processing Apparatus According to the Embodiment] Next, with reference to FIG. 16, the configuration of the information processing apparatus 200 shown in FIG. 9 will be described. FIG. 16 is a diagram showing a configuration example of the information processing apparatus 200 according to the embodiment. As shown in FIG. 16, the information processing apparatus 200 includes a communication unit 210, a storage unit 220, and a control unit 230. Note that the information processing apparatus 200 may include an input unit (e.g., a keyboard, a mouse, etc.) that receives various operations from an administrator or the like who manages the information processing apparatus 200, and a display unit (e.g., a liquid crystal display, etc.) for displaying various information.

[0063] The communication unit 210 is realized by, for example, a NIC or the like. The communication unit 210 is connected to the network N1 and the private network N2 by wire or wirelessly, and transmits and receives information to and from the user terminal 300, the copyright management apparatus 100, etc. via the network N or the private network N2.

[0064] The storage unit 220 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 220 stores various data used for information processing.

[0065] As shown in FIG. 16, the storage unit 220 includes a user information storage unit 221, a style information storage unit 222, an ownership information storage unit 223, a production information storage unit 224, and an operation history information storage unit 225.

[0066] The user information storage unit 221 stores various information (user information) about the user. FIG. 17 is a diagram showing an example of the user information storage unit 221 according to the embodiment.

[0067] The user information storage unit 221 stores user information including a user ID, user meta information, and authority information. The user information storage unit 221 stores the user meta information and authority information corresponding to each user ID in association with each user ID.

[0068] The user ID indicates identification information for uniquely identifying a user. For example, the user ID indicates identification information for uniquely identifying users such as producers, general users, and system administrators. User meta-information is additional information about the user, such as the user's name and address. Authority information stores values for identifying authorities, such as system administrator authority information, producer authority information, and general user authority information. Note that the user information storage unit 221 is not limited to the above, and may store various types of information according to the purpose. The user meta-information may store various types of information related to the user. For example, when the user is a natural person, the user meta-information may store demographic attribute information such as the user's gender and age, and psychographic attribute information.

[0069] The style information storage unit 222 stores information related to the composition model. FIG. 18 is a diagram showing an example of the style information storage unit 222 according to the embodiment.

[0070] The style information storage unit 222 stores learning model information including a model information ID, a creator ID, model information meta-information, style information 700, a work ID, and shareability information. The style information storage unit 222 stores the creator ID, model information meta-information, style information, work ID, and shareability information corresponding to each model information ID in association with each model information ID.

[0071] The model information ID indicates identification information for uniquely identifying the composition model information. The creator ID indicates identification information for uniquely identifying the creator of the corresponding composition model information. For example, the creator ID indicates identification information for uniquely identifying users such as system administrators, producers, and general users.

[0072] The model information meta-information is information representing the characteristics of the work to be learned, for example. The learning model information meta-information is information such as the tempo, genre, atmosphere such as light and dark, the structure of the song such as A melody, B melody, and refrain, chord progression, scale, and church mode of the music.

[0073] The style information 700 is learning data of a composition model generated by a generation unit 233 (described later) included in the information processing apparatus 200. As described with reference to FIG. 2, the style information is information in which a plurality of types of feature amounts such as chord progressions, melodies, and bass progressions extracted from music information are associated with predetermined identification information.

[0074] The shareability information indicates, for example, whether the corresponding learning model can be shared. The shareability information stores, for example, a value for specifying whether or not the corresponding learning model can be shared.

[0075] Note that the style information storage unit 222 is not limited to the above, and may store various types of information according to the purpose. For example, the composition model information meta-information may store various additional information regarding the composition model, such as information regarding the date and time when the composition model was created.

[0076] The ownership information storage unit 223 stores various types of information regarding the style information selected by the producer who creates the music at the time of music creation. FIG. 19 is a diagram showing an example of the ownership information storage unit 223 according to the embodiment. In the ownership information storage unit 223, the user ID of the producer who creates the music and the style information ID selected by this producer are stored in association with each other.

[0077] The production information storage unit 224 stores various types of information regarding the produced music. FIG. 20 is a diagram showing an example of the production information storage unit 224 according to the embodiment. As shown in FIG. 20, the production information storage unit 224 stores the user ID of the producer who created the music and the score ID created by the producer in association with each other.

[0078] The operation history information storage unit 225 stores operation history information of the producer with respect to the user terminal 300. The operation history information storage unit 225 stores the operation history corresponding to each user ID in association with each user ID. The operation history information indicates the operation history of the producer. For example, the operation history information may include various information related to the producer's operation, such as the content of the operation performed by the producer and the date and time when the operation was performed. For example, operations include selection of style information presented from the information processing apparatus 200, selection of a composition execution instruction button, reproduction and editing of music information received from the information processing apparatus 200, and the like.

[0079] Returning to FIG. 16, the description will be continued. The control unit 230 is realized, for example, by a program (for example, an information processing program according to the present embodiment) stored inside the information processing apparatus 200 being executed with a RAM or the like as a work area by a CPU, an MPU, or the like. Further, the control unit 130 is a controller and may be realized by an integrated circuit such as an ASIC or an FPGA.

[0080] The control unit 230 includes an acquisition unit 231, an extraction unit 232, a generation unit 233, a reception unit 234, a selection unit 235, a transmission unit 236, a composition unit 237, and an update unit 238, and realizes or executes the functions and operations of information processing described below.

[0081] The acquisition unit 231 acquires music information. The acquisition unit 231 acquires music information created by a producer by communicating with the user terminal 300 via the network N1. This music information is created by the producer using an application related to the creation of music installed on the user terminal 300, that is, an automatic composition function, and includes feature quantities related to the music, such as chord progressions, melodies, bass progressions, and drum sound progressions. Further, the acquisition unit 231 receives, together with information related to the copyrighted music information, the feature quantities of a plurality of types of copyrighted music information extracted by the extraction unit 132 of the copyrighted work management device 100 by communicating with the copyrighted work management device 100 via the private network N2. That is, the acquisition unit 231 receives, from the copyrighted work management device 100 via the private network N2, the copyrighted music feature information for each newly registered copyrighted music.

[0082] The extraction unit 232 extracts a plurality of types of feature quantities from the music information. The extraction unit 132 extracts chord progression information, beat information, melody information, and drum information as feature quantities from the music information created by the producer. The extraction unit 232 extracts various feature quantities from the music information using, for example, 12-tone analysis technology.

[0083] The generation unit 233 generates style information in which a plurality of types of feature quantities extracted by the extraction unit 232 are associated with style information IDs as learning data in the composition process. Regarding the music information created by the producer, the generation unit 233 associates the style information ID of the style information 700 with the user ID of the producer and registers it in the production information storage unit 224. Regarding the music information registered in the copyrighted work management device 100, the generation unit 233 may associate the copyrighted music ID with the style ID of the style information 700.

[0084] The reception unit 234 receives various information transmitted from the user terminal 300. For example, the reception unit 234 receives information about the producer who uses the automatic composition function on the user terminal 300 and information regarding the style information selected by the producer. In addition, the reception unit 234 can also receive the registration of music pieces associated with the style information 700, the editing of style information, and the like.

[0085] When the automatic composition function is activated on the user terminal 300, the selection unit 235 selects all or some of the style information.

[0086] The transmission unit 236 transmits the presentation information of the style information selected by the selection unit 235 to the user terminal 300. As a result, the code progressions or lyric information of each style information are listed as candidates in the style palette selection pull-down 372a of the user terminal 300. Then, when receiving the instruction information instructing the selection of any of the presented style information from the user terminal 300, the selection unit 235 selects the selected style information from the style information storage unit 222.

[0087] Based on the style information selected by the selection unit 235, the composition unit 237 composes music piece information using machine learning and transmits the composed music piece information to the user terminal 300. The composition unit 237 may perform composition using various existing music piece generation algorithms. For example, the composition unit 237 may use a music piece generation algorithm using a Markov chain, or may use a music piece generation algorithm using deep learning. In addition, the composition unit 237 may generate a plurality of music piece information in response to the instruction information transmitted from the user terminal 300. As a result, the producer can receive a plurality of proposals from the composition unit 237, so that the composition work can be advanced using more diverse information.

[0088] When the update unit 238 receives the performance information based on the music piece information composed by the composition unit 237 from the user terminal 300, it adds this performance information to the selected style information and updates the selected style information.

[0089] [Configuration of User Terminal According to Embodiment 1-5] Next, with reference to FIG. 21, the configuration of the user terminal 300 shown in FIG. 9 will be described. FIG. 21 is a diagram showing a configuration example of the user terminal 300 according to the embodiment. As shown in FIG. 21, the user terminal 300 includes a communication unit 310, an input unit 320, an output unit 330, a storage unit 340, a control unit 350, and a display unit 360.

[0090] The communication unit 310 is realized by, for example, a NIC, a communication circuit, or the like. The communication unit 310 is connected to the network N1 by wire or wirelessly, and transmits and receives information to and from other devices such as the information processing device 200 and other terminal devices via the network N1.

[0091] The input unit 320 receives various operations from the user. The input unit 320 includes a keyboard and a mouse connected to the user terminal 3000. The input unit 320 receives input from the user. The input unit 320 receives input from the user via the keyboard and the mouse. The input unit 320 may have a function of detecting sound. In this case, the input unit 320 may include a microphone for detecting sound.

[0092] Various information may be input to the input unit 320 via the display unit 360. In this case, the input unit 320 may have a touch panel that can realize functions equivalent to those of a keyboard and a mouse. In this case, the input unit 12 receives various operations from the user via the display screen by the function of the touch panel realized by various sensors. Note that, in a tablet terminal, the capacitance method is mainly adopted as the detection method for the user's operation by the input unit 320, but any detection method such as the resistive film method, the surface acoustic wave method, the infrared method, or the electromagnetic induction method may be adopted as long as it can detect the user's operation and realize the function of the touch panel. Further, the user terminal 300 may have an input unit that also receives operations by buttons or the like.

[0093] The output unit 330 outputs various information. The output unit 330 includes a speaker for outputting sound.

[0094] The storage unit 340 is implemented by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 340 stores various types of information used for displaying information.

[0095] The control unit 350 is implemented, for example, by a CPU, an MPU, etc., when a program stored inside the user terminal 300 is executed using a RAM or the like as a work area. Further, the control unit 350 is a controller and may be implemented by an integrated circuit such as an ASIC or an FPGA. The control unit 350 includes a display control unit 351, a registration unit 352, a transmission / reception unit 353, a selection unit 354, a playback unit 355, and a performance reception unit 356.

[0096] The display control unit 351 controls various displays for the display unit 360. The display control unit 351 controls the display of the display unit 360. The display control unit 351 controls the display of the display unit 360 based on the information received from the information processing device 200. The display control unit 351 controls the display of the display unit 360 based on the information generated by the processing of each component of the control unit 350. The display control unit 351 may control the display of the display unit 360 by an application that displays an image.

[0097] The display control unit 351 causes the display unit 360 to display a window 370 (see FIGS. 3 to 7) or the like by an application having an automatic composition function by a DAW and AI. Further, when the display control unit 351 receives the presentation information of the style information from the information processing device 200, it causes the code progression and lyrics of the presented style information to be displayed in the style palette selection pull-down 372a (see FIG. 4) of the window 370. Further, when the display control unit 351 receives the music information and lyric information generated by the information processing device 200, it causes the melody display piano roll 374a (see FIG. 7) of the window 370 to display the melody and the lyrics corresponding to the melody.

[0098] The registration unit 352 accepts the registration of various types of information. For example, the registration unit 352 accepts the registration of drum patterns, chord progressions, and melodies set by the user when the DAW is started or the like. For example, the drum patterns, chord progressions, and melodies are registered via an application that displays images IM11, IM21, IM31, IM41 (FIGS. 26 and 27).

[0099] The transmission / reception unit 353 communicates with the information processing device 200 and transmits and receives various types of information. The transmission / reception unit 353 transmits music information including drum patterns, chord progressions, and melodies received by the registration unit 352 to the information processing device 200. Also, when the automatic composition function is activated, the transmission / reception unit 353 receives presentation information of style information transmitted from the information processing device 200. The transmission / reception unit 353 transmits instruction information for instructing the selection of style information to the information processing device 200. Then, the transmission / reception unit 353 receives music information and lyric information generated by the information processing device 200. Also, the transmission / reception unit 353 transmits performance information regarding the performance received by the user terminal 300 to the information processing device 200.

[0100] The selection unit 354 selects any one of the style information presented from the information processing device 200. For example, among the chord progressions displayed in the style palette selection pull-down 372a (see FIG. 4) of the window 370, any chord progression is selected by the operation of the input unit 320 by the user. Thereby, the selection unit 354 transmits instruction information for selecting the style information corresponding to the selected chord progression from the transmission / reception unit 353 to the information processing device 200.

[0101] The playback unit 355 plays back the music information generated by the information processing device 200. Specifically, the playback unit 255 sets arbitrary instrument information for each of the melody, chords, and bass sounds included in the music data and plays back each data. Note that the playback unit 255 may play back by combining each of the melody, chords, and bass sounds.

[0102] The performance reception unit 356 receives a performance by the producer when the producer performs a composition using the automatic composition function. For example, the performance reception unit 356 receives performance information that is performed in accordance with the reproduction of the music information generated by the information processing apparatus 200 by the automatic composition function.

[0103] The display unit 360 displays various types of information. The display unit 360 is realized by, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. The display unit 360 displays various types of information in accordance with the control by the display control unit 351. The display unit 360 can also display information such as an image provided from the information processing apparatus 200.

[0104] [1-6. Procedures of Information Processing According to Embodiment] [1-6-1. Generation Process of Style Information of Copyright Music Information] Next, with reference to FIG. 22, the procedures of various information processing according to the embodiment will be described. First, as one of the information processing according to the present embodiment, the automatic generation process of style information regarding copyright music information will be described. FIG. 22 is a sequence diagram showing the procedures of the information processing according to the embodiment.

[0105] As shown in FIG. 22, when new copyright music information is registered in the work management apparatus 100 (step S101), the work management apparatus 100 extracts a plurality of types of feature amounts from each newly registered copyright music information (step S102). At this time, the work management apparatus 100 performs the extraction of the feature amounts using, for example, the 12-tone analysis technique. Then, the work management apparatus 100 transmits, via the private network N2, the copyright music feature information including a plurality of types of feature amounts extracted from the copyright music information and information regarding the copyright music information to the information processing apparatus 200 (step S103).

[0106] The information processing apparatus 200 automatically creates style information 700 based on the received copyrighted music feature information (step S104). The information processing apparatus 200 can generate score information 740, for example, from the beat information, chord progression information, and melody information of the copyrighted music feature information. The information processing apparatus 200 can generate lyric information 750, for example, from the lyric information of the copyrighted music meta information.

[0107] The information processing apparatus 200 can generate style palette information 730 by, for example, grouping those with similar chord progressions among these multiple score information 740 and lyric information 750. Similar chord progressions are, for example, those with exactly the same chord progression. Alternatively, similar chord progressions may be those where each chord is classified into Tonic (T), Sub-dominat (S), and Dominat (D), and the sequence of T, S, and D is the same. In the case of C major and A minor, T is C / Em / Am, S is F, Dm, and D is G, Dm7-5. And since the chord progressions C-D-G-C and Em-Dm-Bm7-5-Am are both T-S-D-T, they can be considered the same chord progression. Also, similar chord progressions can be classified based on, for example, similar chord progressions using machine learning or deep learning rather than using music theory.

[0108] Also, the information processing apparatus 200 may register the automatically generated style palette information 730 alone in the style palette sequence information 720. The information processing apparatus 200 may generate and register style palette sequence information 720 by arranging a plurality of style palette information 730. When arranging a plurality of style palette information 730, the information processing apparatus 200 can arrange them with reference to the music structure information.

[0109] Subsequently, the information processing apparatus 200 registers the generated style information in association with the identification information of the copyrighted music information (step S105) and stores it in the style information storage unit 222.

[0110] [1-6-2. Generation Process of Style Information for Music Information Created by the Producer] Style information can also be generated for the music information created by the producer. Therefore, with reference to FIG. 23, the generation process of style information regarding the music information created by the producer will be described. FIG. 23 is a sequence diagram showing the procedure of the information processing according to the embodiment.

[0111] As shown in FIG. 23, the producer operates the user terminal 300 to start a music composition function such as a DAW (step S111). When the creation of style information is selected (step S112), composition start information indicating the start of composition is transmitted from the user terminal 300 (step S113). Then, when the information processing apparatus 200 receives the composition start information, it transmits the information necessary for composition (step S114).

[0112] Subsequently, according to the operation by the producer in accordance with the UI instruction, the user terminal 300 registers a drum pattern (for example, tempo, number of bars, beat positions where Hight Hat, Bass Drumus, and Snare sound) (step S115). When step S115 ends, according to the operation by the producer, the user terminal 300 registers the chord progression (step S116).

[0113] After step S116 ends, since the music composition function app automatically plays the drums and the chord progression, accordingly, the producer inputs a melody into the user terminal 300 multiple times. By this input operation, the user terminal 300 registers the input melody (step S117). The user may additionally input lyric information using the music composition function. The user terminal 300 continues the melody registration until the input of the melody by the user ends. When the registration of the melody by the user ends (step S118), the user terminal 300 transmits the music information created by the producer to the information processing apparatus 200 (step S119). The music information includes feature quantities such as a drum pattern, a chord progression, and a melody.

[0114] The information processing apparatus 200 extracts each feature amount included in the music information to generate score information and lyric information, and generates style palette information from the score information and the lyric information (step S120).

[0115] Then, the information processing apparatus 200 obtains each style palette sequence information in the music information, and associates the style information ID with the score information, the lyric information, the style palette information, and the style palette sequence information to generate style information 700 (step S121).

[0116] The information processing apparatus 200 registers the style information 700 in the style information storage unit 222 (step S122). Along with this, the information processing apparatus 200 registers data in which this style information ID is associated with the user ID of the producer in the ownership information storage unit 223. Then, the information processing apparatus 200 registers data in which the score ID is associated with the user ID of the producer in the production information storage unit 224, and ends the style information generation process. Note that the style palette sequence information may be generated by the producer arranging a plurality of style palette information using the composition function.

[0117] By executing the processes of FIGS. 22 and 23, a plurality of style information 700 is accumulated in the information processing apparatus 200. A user such as a producer can select a preferred one from these style information and use automatic composition.

[0118] [1-6-3. Update Process of Style Information] The style information can also be updated by the producer. Therefore, with reference to FIG. 24, the generation process of the style information regarding the music information created by the producer will be described. FIG. 24 is a sequence diagram showing the procedure of the information processing according to the embodiment.

[0119] When the information processing apparatus 200 receives composition start information (step S132) upon activation of the automatic composition function on the user terminal 300 by the producer (step S131), it selects style information (step S133) and transmits presentation information of the style information to the user terminal 300 (step S134). For example, the information processing apparatus 200 selects all style information, style information whose usage count by the producer exceeds a predetermined number, or style information whose usage count by all users exceeds a predetermined number from the style information storage unit 222, and transmits presentation information of the selected style information.

[0120] Then, based on the presentation information, the user terminal 300 displays a list of style information (step S135). For example, the user terminal 300 displays a list with the chord progressions of the style information as candidates. When style information is selected by the producer on the user terminal 300 (step S136), the user terminal 300 transmits selection information indicating the selected style information to the information processing apparatus 200 (step S137).

[0121] The information processing apparatus 200 extracts the selected style information, performs machine learning using the extracted style information as learning data to perform composition processing (step S138), and provides music information to the user terminal 300 (step S139). Note that the information processing apparatus 200 extracts feature amounts from the composed music information by the extraction unit 232, stores new score information including these feature amounts in the storage unit 220, and registers it in the ownership information storage unit 223.

[0122] When the user terminal 300 plays the provided music (step S140), it accepts performance processing (step S141). When the producer performs, for example, using a MIDI keyboard, the performance information becomes MIDI information. When the information processing apparatus 200 receives the transmission of this performance information (step S142), it extracts feature amounts from the performance information (step S143).

[0123] The information processing apparatus 200 updates the style information by adding, as score information, the feature amounts extracted from the performance information to the style information selected by the producer (step S144). In this way, since the actual performance by the producer is added to the style information selected by the producer, the music information automatically composed using this style information approaches the style of the producer. That is, the style information can be made to approach the style of the producer who performed the music. Until the producer finishes performing, the processes of steps S140 to S142 are repeated. Then, the score information generated in the repetition process of steps S140 to S142 repeated until the music is completed is stored in the storage unit 220.

[0124] In this way, in the information processing system 1, since the actual performance by the producer is added to the style information selected by the producer, the music information automatically composed using this style information approaches the style of the producer. That is, in the information processing system 1, the style information can be made to approach the style of the producer who performed the music.

[0125] Then, when the performance by the producer ends (step S145: Yes), the producer may operate the user terminal 300 to perform, for example, arrangement processing (step S146), mixing or mastering processing (step S147).

[0126] [1-7. Conceptual diagram of the configuration of the information processing system] Here, using FIG. 25, each function, hardware configuration, and data in the information processing system are conceptually shown. FIG. 25 is a diagram showing an example of a conceptual diagram of the configuration of the information processing system. Specifically, FIG. 25 is a schematic diagram showing an outline of the functions of a system which is an example of the application of the information processing system 1.

[0127] [1-7-1. Regarding the overall configuration] The copyrighted music management server device shown in FIG. 25 corresponds to the copyrighted work management device 100 in the information processing system 1. The server device shown in FIG. 25 corresponds to the information processing device 200 in the information processing system 1. Also, the system administrator app section shown in FIG. 25 corresponds to the app installed on the terminal used by the system administrator. Also, the music producer app section shown in FIG. 25 corresponds to the app installed on the user terminal 300 in the information processing system 1. Also, the general user app section shown in FIG. 25 corresponds to the app installed on the terminal used by the general user. In the example of FIG. 25, one system administrator app section, one music producer app section, and one general user app section are each illustrated, but these may include a plurality according to the number of corresponding terminals.

[0128] The control unit of the copyrighted music management server device shown in FIG. 25 corresponds to the control unit 130 of the copyrighted work management device 100. For example, the copyrighted music management section of the copyrighted music management server device corresponds to the management section 131 of the copyrighted work management device 100. The copyrighted music feature information analysis function of the copyrighted music management server device corresponds to the extraction section 132 of the copyrighted work management device 100. The in-server database section of the copyrighted music management server device corresponds to the storage section 120 of the copyrighted work management device 100.

[0129] The learning processing unit and the control unit of the server device shown in FIG. 25 correspond to the control unit 230 of the information processing device 200. For example, the learning processing unit of the server device corresponds to the composition section 237 of the information processing device 200. The in-server database section of the server device corresponds to the storage section 220 of the information processing device 200.

[0130] The display operation unit and the control unit of the music producer app section shown in FIG. 25 correspond to the control unit 350 of the user terminal 300. For example, the display operation unit of the music producer app section corresponds to the display control unit 351 of the user terminal 300. The display operation units and the control units of the system administrator app section and the general user app section correspond to the control units of the terminal devices used by their respective users.

[0131] As shown in FIG. 25, the server device is connected to the application section for system administrators, the application section for music producers, and the application section for general users via a network N1 such as the Internet. Further, the server device is connected to the copyrighted music management server device via a private network N2.

[0132] [1-7-2. Regarding the Copyrighted Music Management Server Device] First, the configuration related to the copyrighted music management server device will be described.

[0133] The copyrighted music management server device has a control section and a database section within the copyrighted music management server. The control section of the music production management server device has a copyrighted music management function and a copyrighted music feature information analysis function.

[0134] [1-7-3. Regarding the Server Device] First, the configuration related to the server device will be described.

[0135] The server device has a control section, a learning processing section, and a database section within the server. The control section of the server device has a music production information management function, a style information management function, a user operation history information management function, and a copyrighted music analysis function. The learning processing section of the server device has a machine learning processing function and a deep learning processing function.

[0136] [1-7-4. Regarding the Application Section for Music Producers] Next, the configuration related to the application section for music producers will be described.

[0137] The application section for music producers has a display operation section and a control section. The display operation section of the application section for music producers has a music production information display function and a style information display editing function. The application section for music producers has a style information sharing function and a user operation history information transmission function.

[0138] The music producer app part is, for example, music editing software (such as DAW), and can display music information, for example, through the produced music information display function. If the DAW has, for example, an AI-assisted music production function, new music information can be produced while using the learning model information display and editing function. The system administrator app part and the general user app part also have a similar configuration, but the user's permissions for the system are different.

[0139] [1-8.UI (User Interface)] Here, with reference to FIGS. 26 and 27, the details of the information display and other automatic composition functions by the app (music composition app) will be described. FIGS. 26 and 27 are diagrams showing an example of the user interface according to the embodiment.

[0140] FIG. 26 shows an example of the user interface when the music composition app is displayed on the screen of the user terminal 300.

[0141] In the example shown in FIG. 26, the user interface IF11 displays the music data received by the music composition app. Although the details will be described later, the music data in the music composition app is composed of three different types of data: melody, chord, and bass sound. The user interface IF11 shown in FIG. 26 displays the data related to the melody among the three different types of data.

[0142] The setting information ST11 displays information related to the style palette, which is an example of the setting information in the automatic composition function. The style palette is the designation information for designating the style information that becomes the learning data of machine learning.

[0143] The setting information ST12 displays information about harmony, which is an example of the setting information in the automatic composition function. The information about harmony is, for example, information for determining the probability that the constituent sounds included in the chord appear in the melody in the music data composed by the information processing apparatus 200. For example, when the user sets the information about harmony to "strict", the probability that the constituent sounds included in the chord appear in the melody in the automatically composed music data increases. On the other hand, when the user sets the information about harmony to "loose", the probability that the constituent sounds included in the chord appear in the melody in the automatically composed music data decreases. In the example of FIG. 26, the user indicates that the information about harmony has been applied more "strictly".

[0144] The setting information ST13 displays note length information, which is an example of the setting information in the automatic composition function. The note length information is, for example, information for determining the note length in the music data composed by the information processing apparatus 200. For example, when the user sets the note length information to "long", the probability that notes with a relatively long sound length (for example, whole notes, half notes, etc.) appear in the automatically composed music data increases. On the other hand, when the user sets the note length information to "short", the probability that notes with a relatively short sound length (for example, eighth notes, sixteenth notes, etc.) appear in the automatically composed music data increases.

[0145] The setting information ST14 displays information for determining the types and amounts of material music other than the material music included in the specified information (the style palette specified by the user), which is an example of the setting information in the automatic composition function. Such information is, for example, information for determining whether to strictly perform learning based on the music included in the style palette specified by the user in the music data composed by the information processing device 200. For example, when the user sets such information to "never", in the learning in automatic composition, the tendency for music other than the music included in the style palette to be used becomes lower. On the other hand, when the user sets such information to "only", in the learning in automatic composition, the tendency for music other than the music included in the style palette to be used becomes higher.

[0146] The music data MDT1 displays the specific music data transmitted from the information processing device 200. In the example of FIG. 26, the music data MDT1 includes information indicating chord progressions such as Cm, information indicating the pitch and note lengths within a measure, the change in the pitch of the notes (in other words, the melody), and the like. Also, as shown in FIG. 26, the music data MDT1 may include, for example, four different types of content. That is, the information processing device 200 may transmit not only one type of automatically composed music data but also a plurality of music data. Thereby, the user can select the music data that the user likes from among the plurality of generated music data candidates, or can compose a favorite music by combining a plurality of music data.

[0147] Note that the user interface IF11 shown in FIG. 26 displays data related to the melody among the three different types of data, namely, the melody, chords, and bass notes included in the music data, while the other data is displayed on other user interfaces. This point will be described with reference to FIG. 27.

[0148] As shown in FIG. 27, in addition to the user interface IF11 that displays data related to the melody, the user terminal 300 may display on the screen a user interface IF12 that displays data related to the chords and a user interface IF13 that displays data related to the bass notes. Although not shown in FIG. 27, different note information from the music data MDT1 in the user interface IF11 is displayed in the user interface IF12 and the user interface IF13. Specifically, the user interface IF12 displays note information related to the chords corresponding to the melody of the music data (for example, the constituent notes of the chord Cm, etc.). Also, the user interface IF13 displays note information related to the bass notes corresponding to the melody and chords of the music data (for example, the note "C" if the chord is Cm, etc.).

[0149] The user can perform operations such as selecting the information to be copied or editing a part of the bass notes from among the displayed user interface IF11, user interface IF12, and user interface IF13.

[0150] [2. Effects of this Embodiment] As described above, the information processing apparatus (information processing apparatus 200 in the embodiment) according to this embodiment includes an acquisition unit (acquisition unit 231 in the embodiment), an extraction unit (extraction unit 232 in the embodiment), and a generation unit (generation unit 233 in the embodiment). The acquisition unit acquires music information. The extraction unit extracts a plurality of types of feature amounts from the music information acquired by the acquisition unit. The generation unit generates information in which a plurality of types of feature amounts extracted by the extraction unit are associated with predetermined identification information as music feature information (style information in the embodiment) to be used as learning data in the composition process using machine learning.

[0151] In this way, in the information processing apparatus according to the present embodiment, style information having feature amounts of a plurality of types of music information can be generated as a dataset for learning a composition model. In the information processing apparatus according to the present embodiment, by causing the composition model to learn this style information, music information composed according to the features of a music piece can be provided to each user including the producer. Therefore, according to the information processing apparatus according to the present embodiment, the convenience of the music composition function by the user can be improved.

[0152] Further, the acquisition unit acquires music information by receiving, from the terminal device (user terminal 300 in the embodiment), music information including feature amounts related to a music piece created by the producer using an application related to the creation of the music piece installed in the terminal device. The extraction unit extracts a plurality of types of feature amounts included in the music information. The generation unit associates the identification information of the producer with the music feature information. Thereby, the information processing apparatus can generate music feature information related to the music information created by the producer.

[0153] Further, the feature amounts related to the music piece created by the producer are chord progression information indicating the chord progression, melody information indicating the melody, and a bass signal indicating the bass progression in a measure of a prescribed length. Therefore, for the music information created by the producer, the information processing apparatus can generate music feature information using the feature amounts related to the music piece created by the producer, and thus can generate the music feature information quickly.

[0154] Further, the feature amounts related to the music piece created by the producer are drum progression information indicating the drum progression in a measure of a prescribed length. Therefore, the information processing apparatus can generate music feature information including the drum progression information.

[0155] In addition, the acquisition unit acquires the copyrighted music piece information registered regularly at a preset timing. The extraction unit extracts a plurality of types of feature amounts from the copyrighted music piece information. The generation unit associates the identification information of the copyrighted music piece information with the style information. As a result, the information processing apparatus 200 can automatically generate music piece feature information regarding the copyrighted music piece information registered regularly.

[0156] In addition, the information processing apparatus has a transmission unit (transmission unit 236 in the embodiment) that transmits presentation information of the music piece feature information in accordance with instruction information received from a terminal device in which an application related to music composition is installed. When the information processing apparatus receives a selection of music piece feature information from the terminal device, it has a composition unit (composition unit 237 in the embodiment) that composes music piece information using machine learning based on the selected feature information and transmits the composed music piece information to the terminal device. In this way, by the information processing apparatus presenting music piece feature information corresponding to the instruction information to the terminal device, the producer can select desired music piece feature information therefrom. Then, the information processing apparatus can provide music piece information composed based on the music piece feature information desired by the producer.

[0157] In addition, when the information processing apparatus receives performance information based on the music piece information transmitted by the composition unit from the terminal device, it further has an update unit (update unit 238 in the embodiment) that adds the performance information to the selected music piece feature information and updates the selected music piece feature information. In this way, the information processing apparatus can make the music piece feature information closer to the style of the producer who performed the music by adding the performance information by the producer to the selected music piece feature information.

[0158] Further, the extraction unit extracts, as feature quantities, chord progression information indicating a chord progression, melody information indicating a melody, and bass information indicating a bass progression in a measure of a specified length from the music information. The generation unit generates score information having chord progression information indicating a chord progression, melody information indicating a melody, and bass information indicating a bass sound progression in a measure of a specified length, and uses it as a component of the music feature information. Thereby, the information processing apparatus can generate music feature information having chord progression information, melody information, and bass information. Then, at the time of composition, the information processing apparatus can efficiently provide music information to the user because it learns feature quantities such as chord progression information, melody information, and bass information instead of the music information itself.

[0159] The extraction unit extracts, as a feature quantity, drum information indicating a drum sound progression in a measure of a specified length from the music information. The generation unit further adds drum progression information to the score information. Thereby, the information processing apparatus can generate music feature information having chord progression information, melody information, bass information, and drum information.

[0160] The generation unit generates lyric information indicating lyrics in a measure of a specified length from the lyric information added to the music information, and uses it as a component of the music feature information. Thereby, when lyrics are searched for in the terminal device, the information processing apparatus can extract music feature information including this lyric or a lyric similar to this lyric and present it to the terminal device, so that the convenience of the music creation function by the user can be improved. In addition, the information processing apparatus can also automatically generate lyrics.

[0161] The generation unit generates music format information for registering by associating the identification information of the score information and the identification information of the lyric information for the same measure, and uses it as a component of the music feature information. By learning this music feature information, the information processing apparatus can further provide music information desired by the user.

[0162] The generation unit adds and registers, to the music form information, the identification information of score information having code progression information similar to the code progression information of the score information registered in the music form information. Thereby, the information processing apparatus can compose music information along the structure of the music.

[0163] The generation unit generates music order information indicating the order of the music form information and uses it as a component of the music feature information. Since the information processing apparatus can also learn the order of the music form information, the learning accuracy can be further improved.

[0164] [3. Other Embodiments] The processes according to the above-described embodiments and modification examples may be implemented in various different forms (modification examples) other than the above-described embodiments and modification examples.

[0165] [3-1. Other Configuration Examples] Each of the above configurations is an example, and the information processing system 1 may have any system configuration as long as the above-described information processing can be realized. For example, the work management apparatus 100 and the information processing apparatus 200 may be integrated.

[0166] [3-2. Others] Also, among the processes described in each of the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

[0167] Also, each component of each illustrated apparatus is conceptually functional and does not necessarily have to be physically configured as shown. That is, the specific form of the distribution and integration of each apparatus is not limited to that shown, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads and usage situations.

[0168] Also, each of the above-described embodiments and modifications can be appropriately combined as long as the processing contents do not conflict with each other.

[0169] Also, the effects described in this specification are merely examples and are not limited, and there may be other effects.

[0170] [4. Hardware Configuration] Information devices such as the information processing apparatus 200, the work management apparatus 100, and the user terminal 300 according to each of the above-described embodiments and modifications are realized by a computer 1000 having a configuration as shown in FIG. 28, for example. FIG. 28 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the information processing apparatus 200, the work management apparatus 100, and the user terminal 300. Hereinafter, the information processing apparatus 200 according to the embodiment will be described as an example. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each part of the computer 1000 is connected by a bus 1050.

[0171] The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each part. For example, the CPU 1100 expands a program stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0172] The ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started up, and programs dependent on the hardware of the computer 1000.

[0173] The HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by the CPU 1100, data used by such programs, and the like. Specifically, the HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0174] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (for example, the Internet). For example, the CPU 1100 receives data from other devices or transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0175] The input / output interface 1600 is an interface for connecting an input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard and a mouse via the input / output interface 1600. Also, the CPU 1100 transmits data to output devices such as a display, a speaker, and a printer via the input / output interface 1600. Further, the input / output interface 1600 may function as a media interface that reads programs and the like recorded on a predetermined recording medium (media). The media is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0176] For example, when the computer 1000 functions as the information processing apparatus 200 according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 and the like by executing the information processing program loaded on the RAM 1200. Further, the HDD 1400 stores the information processing program according to the present disclosure and the data in the storage unit 120. Note that the CPU 1100 reads and executes the program data 1450 from the HDD 1400. As another example, these programs may be acquired from other devices via the external network 1550.

[0177] Note that the present technology can also adopt the following configuration. (1) An acquisition unit that acquires music information; An extraction unit that extracts a plurality of types of feature amounts from the music information acquired by the acquisition unit; A generation unit that generates, as music feature information, information in which the plurality of types of feature amounts extracted by the extraction unit are associated with predetermined identification information and that is used as learning data in music composition processing using machine learning; An information processing apparatus, characterized by comprising the above. (2) The acquisition unit acquires music information by receiving, from the terminal device, music information including feature amounts related to music created by a producer using an application related to the creation of music installed in the terminal device. The extraction unit extracts a plurality of types of feature amounts included in the music information. The generation unit associates identification information of the producer with the music feature information. The information processing apparatus according to (1), characterized by the above. (3) The feature amounts related to the music created by the producer are chord progression information indicating chord progressions, melody information indicating melodies, and bass signals indicating bass progressions in measures of a specified length. The information processing apparatus according to (2), characterized by the above. (4) The feature amount related to the music piece created by the producer is drum progress information indicating the progress of drums in a measure of a specified length, and is characterized by the information processing apparatus according to (3). (5) The acquisition unit acquires the copyrighted music piece information registered regularly at a preset timing, The extraction unit extracts the feature amounts of the plurality of types from the copyrighted music piece information, The generation unit associates the identification information of the copyrighted music piece information with the music piece feature information and is characterized by the information processing apparatus according to (1). (6) A transmission unit that transmits presentation information of the music piece feature information according to instruction information received from a terminal device in which an application related to music piece creation is installed, When receiving a selection of the music piece feature information from the terminal device, a composition unit that composes music piece information using machine learning based on the selected music piece feature information and transmits the composed music piece information to the terminal device, and is further characterized by the information processing apparatus according to (1). (7) When receiving performance information based on the music piece information transmitted by the composition unit from the terminal device, an update unit that adds the performance information to the selected music piece feature information and updates the selected music piece feature information, and is further characterized by the information processing apparatus according to (6). (8) The extraction unit extracts, as feature amounts, chord progress information indicating chord progress, melody information indicating a melody, and a bass signal indicating the progress of a bass in a measure of a specified length from the music piece information, The generation unit generates score information having chord progress information indicating the progress of chords, melody information indicating a melody, and bass information indicating the progress of bass notes in a measure of the specified length, and uses the score information as a component of the music piece feature information and is characterized by the information processing apparatus according to (1). (9) The extraction unit extracts, as a feature amount, drum information indicating the progression of drum sounds in a measure of the specified length from the music information. The generation unit further adds the drum information to the score information. The information processing apparatus according to (8), characterized in that. (10) The generation unit generates lyric information indicating lyrics in a measure of the specified length from the lyric information added to the music information, and makes it a component of the music feature information. The information processing apparatus according to (8), characterized in that. (11) The generation unit generates music format information for registering by associating the identification information of the score information and the identification information of the lyric information for the same measure, and makes it a component of the music feature information. The information processing apparatus according to (10), characterized in that. (12) The generation unit adds and registers the identification information of score information having chord progression information similar to the chord progression information of the score information registered in the music format information to the music format information. The information processing apparatus according to (11), characterized in that. (13) The generation unit generates music order information indicating the order of the music format information, and makes it a component of the music feature information. The information processing apparatus according to (11), characterized in that. (14) An information processing method executed by a computer, comprising: acquiring music information; extracting a plurality of types of feature amounts from the acquired music information; generating, as music feature information used in music composition processing using machine learning, information associating the extracted plurality of types of feature amounts with predetermined identification information. The information processing method, characterized in that. (15) acquiring music information; extracting a plurality of types of feature amounts from the acquired music information; Generate information associating the extracted feature quantities of the plurality of types with predetermined identification information as music feature information used in music composition processing using machine learning. An information processing program characterized by causing a computer to execute the program.

Explanation of Signs

[0178] 1 Information processing system 100 Work management device 110, 210, 310 Communication unit 120, 220, 340 Storage unit 130, 230, 350 Control unit 121 Copyright music information storage unit 122 Music storage unit 123 Feature information storage unit 131 Management unit 132 Extraction unit 133 Transmission unit 200 Information processing device 221 User information storage unit 222 Style information storage unit 223 Ownership information storage unit 224 Production information storage unit 225 Operation history information storage unit 231 Acquisition unit 232 Extraction unit 233 Generation unit 234 Reception unit 235 Selection unit 236 Transmission unit 237 Composition unit 238 Update unit 300 User terminal 320 Input unit 330 Output unit 351 Display control unit 352 Registration unit 353 Transmission / reception unit 354 Selection unit 355 Reproduction unit 356 Performance reception unit 360 Display unit

Claims

1. An acquisition unit that acquires music information; An extraction unit that extracts a plurality of types of feature amounts from the music information acquired by the acquisition unit; A generation unit that generates music feature information in which the plurality of types of feature amounts extracted by the extraction unit are associated with predetermined identification information; A transmission unit that transmits presentation information of the music feature information according to instruction information received from a terminal device in which an application related to music creation is installed; When receiving a selection of the music feature information from the terminal device, a composition unit that composes music information using machine learning based on the selected music feature information and transmits the composed music information to the terminal device; An information processing apparatus characterized by including the above.

2. The acquisition unit acquires music information by receiving, from the terminal device, music information created by a producer using an application related to music creation installed in the terminal device, The extraction unit extracts a plurality of types of feature amounts included in the music information, The generation unit associates identification information of the producer with the music feature information The information processing apparatus according to claim 1, characterized by the above.

3. The music information created by the producer includes chord progression information indicating a chord progression, melody information indicating a melody, and a bass signal indicating a bass progression in a measure of a specified length. The information processing apparatus according to claim 2, characterized by the above.

4. The music information created by the producer includes drum progression information indicating a drum progression in a measure of a specified length. The information processing apparatus according to claim 3, characterized by the above.

5. The acquisition unit acquires regularly registered copyrighted music information at a preset timing, The extraction unit extracts the plurality of types of feature amounts from the copyrighted music information, The generation unit associates the identification information of the copyrighted music information with the music feature information. The information processing apparatus according to claim 1, characterized in that.

6. When receiving performance information based on the music information transmitted by the composition unit from the terminal device, an update unit that adds the performance information to the selected music feature information and updates the selected music feature information. The information processing apparatus according to claim 1, further comprising.

7. The extraction unit extracts, as feature amounts, chord progression information indicating a chord progression, melody information indicating a melody, and a bass signal indicating a bass progression in a measure of a specified length from the music information. The generation unit generates score information having chord progression information indicating a chord progression, melody information indicating a melody, and bass information indicating a bass sound progression in a measure of the specified length, and uses the score information as a component of the music feature information. The information processing apparatus according to claim 1, characterized in that.

8. The extraction unit extracts, as a feature amount, drum information indicating a drum sound progression in a measure of the specified length from the music information. The generation unit further adds the drum information to the score information. The information processing apparatus according to claim 7, characterized in that.

9. The generation unit generates lyric information indicating lyrics in a measure of the specified length from the lyric information added to the music information, and uses the lyric information as a component of the music feature information. The information processing apparatus according to claim 7, characterized in that.

10. The generation unit generates music format information for registering by associating the identification information of the score information and the identification information of the lyric information for the same measure, and uses the music format information as a component of the music feature information. The information processing apparatus according to claim 9, characterized in that.

11. The generation unit adds and registers, to the music format information, identification information of score information having code progression information similar to the code progression information of the score information registered in the music format information. The information processing apparatus according to claim 10, characterized in that.

12. The generation unit generates music order information indicating the order of the music format information and uses it as a component of the music feature information. The information processing apparatus according to claim 10, characterized in that.

13. The acquisition unit acquires music information by receiving, from the terminal device, music information created by a producer using an application related to the creation of the music. The information processing apparatus according to claim 1, characterized in that.

14. The music information includes at least one of code progression information, beat information, melody information, and drum progression information. The information processing apparatus according to claim 13, characterized in that.

15. An information processing method executed by a computer, comprising: acquiring music information; extracting a plurality of types of feature amounts from the acquired music information; generating music feature information in which the extracted plurality of types of feature amounts are associated with predetermined identification information; transmitting presentation information of the music feature information according to instruction information received from a terminal device in which an application related to the creation of music is installed; when receiving a selection of the music feature information from the terminal device, composing music information using machine learning based on the selected music feature information and transmitting the composed music information to the terminal device. The information processing method, characterized in that.

16. acquiring music information; extracting a plurality of types of feature amounts from the acquired music information; Generate music feature information by associating the extracted feature quantities of the plurality of types with predetermined identification information. Transmit the presentation information of the music feature information according to the instruction information received from the terminal device on which the application related to music creation is installed. When receiving the selection of the music feature information from the terminal device, compose music information using machine learning based on the selected music feature information, and transmit the composed music information to the terminal device. An information processing program, characterized in that it causes a computer to execute the above steps.

17. An acquisition unit for acquiring music information. An extraction unit for extracting feature quantities of a plurality of types from the music information acquired by the acquisition unit. A generation unit for generating music feature information by associating the feature quantities of the plurality of types extracted by the extraction unit with predetermined identification information. A transmission unit for transmitting the presentation information of the music feature information according to the instruction information received from the terminal device on which the application related to music creation is installed. A composition unit that, when receiving the selection of the music feature information from the terminal device, composes music information using machine learning based on the selected music feature information, and transmits the composed music information to the terminal device. An information processing system, characterized by comprising the above components.

Images