Information processing device, information processing method, information processing program, and information processing system

Abstract

To propose an information processing device, an information processing method, an information processing program, and an information processing system that can improve convenience of a function of user's music creation.SOLUTION: An information processing device according to the present disclosure includes: an acquisition unit that acquires music information; an extraction unit that extracts a plurality of types of feature quantities from the music information acquired by the acquisition unit; and a generation unit that generates information in which the plurality of types of feature quantities extracted by the extraction unit and predetermined identification information are associated, as music feature information to be used as learning data in music composition processing through use of machine learning.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, an information processing program, and an information processing system. [Background technology]

[0002] With the advancement of AI (Artificial Intelligence), the use of computers in the arts field is advancing. For example, a technology is known in which existing music pieces are used as training data for machine learning to generate a model for music generation, and the computer is then allowed to compose new music (see, for example, Patent Document 1). This technology uses a Markov model to imitate the characteristics of existing music pieces and generate more natural melodies. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 9,110,817 Summary of the Invention [Problem to be solved by the invention]

[0004] According to conventional technology, users can use music information proposed (generated) by AI in composing, allowing them to compose music based on a wider variety of perspectives.

[0005] The AI-based automatic composition function is set up for general users, who can receive automatically created music information simply by setting an image such as bright or dark. However, music creators often set specific song characteristics such as chord progressions and bass progressions during the song creation process, and so they have requested that music information be provided that matches the song's characteristics rather than an image.

[0006] Therefore, the present disclosure proposes an information processing device, an information processing method, an information processing program, and an information processing system that can improve the convenience of the music creation function for users. [Means for solving the problem]

[0007] In order to solve the above problem, an information processing device of one embodiment according to the present disclosure includes an acquisition unit that acquires music information, an extraction unit that extracts multiple types of feature quantities from the music information acquired by the acquisition unit, a generation unit that generates music feature information that associates the multiple types of feature quantities extracted by the extraction unit with predetermined identification information, a transmission unit that transmits presentation information of the music feature information in accordance with instruction information received from a terminal device on which an application related to music creation is installed, and a composition unit that, upon receiving a selection of the music feature information from the terminal device, composes music information using machine learning based on the selected music feature information and transmits the composed music information to the terminal device. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a conceptual diagram illustrating a flow of information processing according to the embodiment. [Figure 2] FIG. 3 is a diagram illustrating an example of a data configuration of style information according to the embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal according to the embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal according to the embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal according to the embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal according to the embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal according to the embodiment. [Figure 8] FIG. 2 is a conceptual diagram illustrating a flow of information processing according to the embodiment. [Figure 9]FIG. 1 is a diagram illustrating an example of an information processing system according to an embodiment. [Figure 10] The configuration of a copyrighted material management device according to an embodiment will be described. [Figure 11] FIG. 2 is a diagram illustrating an example of a copyrighted music information storage unit according to the embodiment. [Figure 12] FIG. 2 is a diagram illustrating an example of a music storage unit according to the embodiment. [Figure 13] FIG. 4 is a diagram illustrating an example of a feature information storage unit according to the embodiment. [Figure 14] FIG. 2 is a diagram illustrating a configuration of overall information according to the embodiment. [Figure 15] FIG. 2 is a diagram showing an example of time-series information constituting the song feature information according to the embodiment. [Figure 16] FIG. 1 is a diagram illustrating an example of the configuration of an information processing device according to an embodiment. [Figure 17] FIG. 4 is a diagram illustrating an example of a user information storage unit according to the embodiment. [Figure 18] FIG. 4 is a diagram illustrating an example of a style information storage unit according to the embodiment. [Figure 19] FIG. 4 is a diagram illustrating an example of a possession information storage unit according to the embodiment. [Figure 20] FIG. 2 is a diagram illustrating an example of a production information storage unit according to the embodiment. [Figure 21] FIG. 2 is a diagram illustrating an example of the configuration of a user terminal according to the embodiment. [Figure 22] FIG. 2 is a sequence diagram showing a procedure of information processing according to the embodiment. [Figure 23] FIG. 2 is a sequence diagram showing a procedure of information processing according to the embodiment. [Figure 24] FIG. 2 is a sequence diagram showing a procedure of information processing according to the embodiment. [Figure 25] FIG. 1 is a diagram illustrating an example of a conceptual diagram of a configuration of an information processing system. [Figure 26] FIG. 2 is a diagram illustrating an example of a user interface according to the embodiment. [Figure 27] FIG. 2 is a diagram illustrating an example of a user interface according to the embodiment. [Figure 28]FIG. 2 is a hardware configuration diagram showing an example of a computer that realizes the functions of an information processing device, a copyrighted work management device, and a user terminal. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the information processing device, information processing method, and information processing program according to the present application are not limited to these embodiments. In addition, in the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0010] The present disclosure will be described in the following order: 1. Embodiment 1-1. Example of information processing according to the embodiment 1-2. Configuration of information processing system according to embodiment 1-3. Configuration of copyrighted material management device according to the embodiment 1-4. Configuration of the information processing device according to the embodiment 1-5. Configuration of user terminal according to the embodiment 1-6. Information processing procedure according to the embodiment 1-6-1.Generation of style information for copyrighted music information 1-6-2. Generation process of style information for music information created by a producer 1-6-3. Style information update process 1-7. Conceptual diagram of information processing system configuration 1-7-1. Overall structure 1-7-2. Copyrighted music management server device 1-7-3. Server equipment 1-7-4. About the app for music creators 2. Effects of this embodiment 3. Other embodiments 3-1.Other configuration examples 3-2.Other 4. Hardware Configuration

[0011] [1. Embodiment] [1-1. Example of information processing according to the embodiment] First, an example of information processing according to the present disclosure will be described with reference to Fig. 1. Fig. 1 is a conceptual diagram showing the flow of information processing according to the embodiment. The information processing according to the embodiment is executed by a copyrighted work management device 100, an information processing device 200, and a user terminal 300.

[0012] In this embodiment, the case where the information processing device 200 is an information processing device that provides a service (also simply referred to as a "service") related to the creation of content (information) as a work of authorship is shown as an example. Note that, although a musical piece (music content) is shown below as an example of content, the content is not limited to a musical piece, and may be various other content such as video content such as a movie or text content such as a book (novels, etc.). Furthermore, the musical piece referred to here is not limited to a completed piece of music (whole piece), but is a concept that includes various musical information such as a partial sound source that constitutes a piece of music (piece of music) and short sounds used for sampling.

[0013] The information processing device 200 communicates with the copyrighted work management device 100, which manages copyrighted music information, using a private network N2 (see FIG. 9). The private network N2 is, for example, a secure VPN (Virtual Private Network) or Internet communication with ID address authentication. The information processing device 200 communicates with user terminals 300 of users who use services provided by the information processing device 200 using a network N1 (see FIG. 9) such as the Internet. The number of user terminals 300 is not limited to that shown in FIG. 1.

[0014] The copyrighted work management device 100 is a server device that registers and manages copyrighted music information. The copyrighted work management device 100 periodically registers copyrighted music information. The copyrighted work management device 100 extracts multiple types of features from the registered copyrighted music information and transmits the extracted features to the information processing device 200 via the private network N2.

[0015] The user terminal 300 is an information processing terminal such as a PC (personal computer) or a tablet terminal. Various program applications are installed on the user terminal 300. An application related to music creation is installed on the user terminal 300. For example, the user terminal 300 has an AI-based automatic composition function added by a plug-in (extension application) to an application such as a DAW that provides a comprehensive music production environment. For example, the plug-in can take the form of VST (Steinberg's Virtual Studio Technology) (registered trademark), AudioUnits, AAX (Avid Audio eXtension), etc. Furthermore, the user terminal 300 is not limited to a DAW, and may also use a mobile application such as iOS.

[0016] The user terminal 300 starts and executes the DAW and AI-based automatic composition function, and communicates with the information processing device 200 to receive information about music composed by the information processing device 200.

[0017] The user of the user terminal 300 may be an administrator who operates and manages the entire system, a producer such as a composer, arranger, or studio engineer who creates music, or a general user who receives music information via the automatic composition function. In this embodiment, the user terminal 300 is assumed to be used by a producer Uc.

[0018] The information processing device 200 is a server device that executes information processing related to the AI-based automatic composition function of the user terminal 300. For example, the information processing device 200 is a so-called cloud server that executes AI-based automatic composition in accordance with instruction information from the user terminal 300 and provides the generated music information to the user terminal 300.

[0019] The information processing device 200 generates a composition model for generating music by performing machine learning. For example, the information processing device 200 provides the user terminal 300 with music information that has been automatically composed using a Markov model or the like.

[0020] The information processing device 200 uses style information (music feature information) as learning data for the composition model. Style information is information that associates multiple types of feature quantities, such as chord progressions, melody, and bass progressions, extracted from music information as multiple types of feature quantities with predetermined identification information, and is used in composition processing using machine learning. The information processing device 200 obtains multiple types of feature quantities from copyrighted music information or music information created by a composer, and generates multiple pieces of style information by grouping each feature quantity for each piece of music information and assigning a style information ID (predetermined identification information), thereby creating a database of multiple pieces of style information.

[0021] 2 is a diagram showing an example of the data configuration of style information according to an embodiment. The style information includes a style information ID 710, which is identification information of the style information, style palette sequence information 720 (music sequence information), style palette information 730 (music format information), score information 740, and lyric information 750.

[0022] The score information 740 includes multiple types of feature quantities extracted from the music. The score information 740 has a score ID, melody information, chord progression information, bass information, and drum information. The score ID is identification information for the score information. The melody information is the melody for a measure of a specified length. The chord progression information is information indicating the chord progression for a measure of a specified length. The bass information is information indicating the progression of bass notes for a measure of a specified length. The drum information is information indicating the progression of drum notes (drum pattern and tempo) for a measure of a specified length.

[0023] The lyric information 750 includes a lyric ID and lyric information. The lyric ID is identification information for the lyric information. The lyric information is information indicating lyrics for a bar of a specified length. The lyric information is, for example, a phrase or character keyword that is the basis of the lyrics. The information processing device 200 can also perform automatic verbiage by using multiple pieces of lyric information 750 in the style information 700.

[0024] The style palette information 730 is information in which a style palette ID, which is identification information of the style palette information, is registered in association with a score ID of the score information 740 and a lyric ID of the lyric information 750 for the same measure.

[0025] The style palette sequence information 720 is information that indicates the order of the style palette information 730. The style palette sequence information 720 is a pair of a style palette ID that uniquely identifies the style palette information 730 and a bar index, and by having multiple pairs of these, it becomes information that manages the order of the style palette information 730 within a song. For example, in the example shown in Figure 2, it is defined that measures 1 to 4 of the song correspond to style palette ID 731a, measures 5 to 8 correspond to style palette ID 731b, and measures x to y correspond to style palette ID 731z.

[0026] The information processing device 200 performs machine learning using this style information 700 as learning data and performs composition processing. Therefore, the information processing device 200 does not learn the music information itself, but learns style information composed of multiple types of feature amounts extracted from the music information, such as chord progressions, melodies, and bass progressions. In other words, because the information processing device 200 learns multiple feature amounts extracted in advance from the music information, the information processing load is smaller than when learning the music information itself, and music information can be provided to the user efficiently.

[0027] Specifically, the process of a composer creating a piece of music will be described. Figures 3 to 7 are diagrams showing examples of the display screen of the user terminal 300 according to this embodiment. When the composer activates the automatic composition function on the user terminal 300, a window 370 shown in Figure 3 is displayed on the user terminal 300.

[0028] The window 370 has a composition parameter setting section 371, a style information display section 372, a composition control section 373, and a composed music display / edit section 374. The composition parameter setting section 371 is an area where parameters such as note duration and complexity can be set. The style information display section 372 is an area where style information to be used for composition can be selected by keyword input or pull-down selection. The composition control section 373 is an area where composition instructions can be given by selecting a composition execution instruction button. The composed music display / edit section 374 is an area where multiple piano rolls showing melodies and lyrics are displayed.

[0029] As shown in FIG. 4, when the style palette selection pull-down 372a is selected, the chord progressions of each style information stored in the information processing device 200 are displayed as candidates in a list.

[0030] Chord progression candidates may be displayed in any order, such as alphabetical order, most frequently used by the creator, most frequently used by all users, or order in which style information was created. All or only a portion of the style information stored in the information processing device 200 may be displayed as chord progression candidates. When there are many chord progression candidates, a display area may be selectable using a pager. Furthermore, when a creator enters a desired chord progression in the search keyword input field 372b, the information processing device 200 may extract style information that includes this chord progression and display a list of the chord progression information for each extracted style information in the style palette selection pull-down 372a.

[0031] The composer selects a desired chord progression from the chord progressions presented in the style palette selection pull-down 372a and selects the composition execution command button. The information processing device 200 then extracts style information containing the selected chord progression, performs machine learning using the extracted style information 700 as training data, and performs composition processing. The information processing device 200 then provides music information to the user terminal 300. As a result, the composer can receive music information created to match a desired chord progression simply by selecting the desired chord progression from the chord progressions presented in the style palette selection pull-down 372a.

[0032] Furthermore, because the style information 700 includes lyric information as a feature, a producer can simply input desired lyrics and receive a presentation of style information that matches the lyrics. Specifically, as shown in Fig. 5, when a producer inputs desired lyrics into the search keyword input field 372b, the information processing device 200 may extract style information that has the lyrics or lyrics similar to the lyrics, and display a list of the lyric information of each extracted style information in the style palette selection pull-down 372a.

[0033] The creator selects the desired lyric information from the lyric information presented in the style palette selection pull-down 372a and selects the composition execution command button. As a result, the information processing device 200 extracts style information having the selected lyric information, performs machine learning using the extracted style information 700 as learning data, and performs composition processing, thereby providing music information to the user terminal 300.

[0034] As a result, the producer can receive music information generated to match the lyrics simply by selecting the desired lyrics from the lyrics presented in the style palette selection pull-down 372a. At this time, the information processing device 200 may also automatically generate lyrics to match the generated music, and provide music information in which the melody and lyrics are associated with each other to the user terminal 300. In this case, the melody and the lyrics corresponding to the melody are displayed on the screen of the user terminal 300 in the melody display piano roll 374a of FIG. 6.

[0035] Furthermore, when the creator inputs lyrics, as shown in FIG. 7, the user terminal 300 may display a list of possible chord progressions for the style information presented by the information processing device 200 to assist the creator in creating a piece of music.

[0036] In this way, the information processing device 200 generates style information having multiple types of feature quantities of music information as a data set for training a composition model, and causes the composition model to learn the style information. In this way, the information processing device 200 provides music information composed in accordance with the features of the music to the creator. The flow of style information generation processing, which is part of the information processing according to this embodiment, will be described below with reference to FIG. 1.

[0037] As shown in Fig. 1, when new copyrighted music information is registered in the copyrighted work management device 100 (step S11), the copyrighted work management device 100 extracts multiple types of feature quantities from each newly registered copyrighted music information (step S12). In this case, the copyrighted work management device 100 extracts the feature quantities using, for example, twelve-tone analysis technology. The information processing device 200 then acquires the multiple types of feature quantities from the copyrighted work management device 100 (step S13), compiles the feature quantities for each piece of music information, and assigns a style information ID to generate style information corresponding to each piece of music information (step S14).

[0038] Furthermore, when a new piece of music is created by inputting feature quantities such as chord progression, melody, and bass progression through the operation of the creator Uc on the user terminal 300, the information processing device 200 acquires music information including each feature quantity (step S21). The information processing device 200 then extracts feature quantities such as chord progression, melody, and bass progression from the acquired music information (step S22). The information processing device 200 then compiles each feature quantity for each piece of music information and assigns a style information ID to generate style information corresponding to each piece of music information (step S23). The information processing device 200 generates multiple pieces of style information 700 and stores them in a database by performing the process shown in FIG. 1.

[0039] The information processing according to this embodiment also makes it possible to update style information. FIG. 8 is a conceptual diagram showing the flow of information processing according to this embodiment. As shown in FIG. 8, when the producer Uc selects style information while creating a piece of music (step S31), the information processing device 200 performs composition processing by learning this style information and outputs music information to the user terminal 300 (step S32). The user Uc then performs a performance based on the provided music information. When the information processing device 200 receives this performance information (step S33), it adds the performance information to the style information used in the composition and updates this style information (step S34).

[0040] In this way, in the information processing according to this embodiment, the style information used is updated in accordance with the music creation of the producer Uc. This allows the information processing device 200 to bring this style information closer to the music creation style of the producer Uc, and to compose and provide music information that matches the style of the producer Uc.

[0041] The above has outlined the overall flow of information processing according to this embodiment. From Fig. 9 onwards, the configuration of the information processing system 1 including the copyrighted work management device 100, the information processing device 200, and the user terminal 300 will be explained, and various processes will be explained in detail.

[0042] [1-2. Configuration of information processing system according to embodiment] Fig. 9 is a diagram showing an example of an information processing system 1 according to an embodiment. As shown in Fig. 6, the information processing system 1 includes user terminals 300-1 to 300-3, an information processing device 200, and a work management device 100. The information processing system 1 functions as a work management system and a learning model information management system. In the example of Fig. 9, three user terminals 300-1 to 300-3 are shown, but when there is no need to distinguish between them, they will be referred to as user terminal 300.

[0043] The information processing device 200 and the user terminal 300 are connected via a network N1 so as to be able to communicate with each other via wired or wireless communication. The information processing device 200 and the copyrighted work management device 100 are connected via a private network N2 so as to be able to communicate with each other via wired or wireless communication.

[0044] The copyrighted work management device 100 manages copyrighted music information. The copyrighted music information is periodically registered in the copyrighted work management device 100. The copyrighted work management device 100 extracts multiple types of feature quantities from the registered copyrighted music information and transmits the extracted feature quantities to the information processing device 200.

[0045] The user terminal 300 transmits information about music pieces created by a composer to the information processing device 200, and also receives information about music pieces composed by the information processing device 200 when the automatic composition function is activated.

[0046] The information processing device 200 generates style information, which is learning data, from copyrighted music information or music information created by a composer, and generates a composition model by performing machine learning. The information processing device 200 provides the user terminal 300 with music information automatically composed using the generation model.

[0047] [1-3. Configuration of copyrighted material management device according to the embodiment] Next, the configuration of the work management device 100 shown in Fig. 9 will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the configuration of the work management device 100 according to the embodiment. As shown in Fig. 10, the work management device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

[0048] The communication unit 110 is realized by, for example, a network interface card (NIC), etc. The communication unit 110 is connected to the private network N2 by wire or wirelessly, and transmits and receives information to and from the information processing device 200 via the private network N2.

[0049] The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 stores various data used for information processing. The storage unit 120 has a copyrighted music information storage unit 121, a music storage unit 122, and a feature information storage unit 123.

[0050] The copyrighted music information storage unit 121 stores information about copyrighted music, which is copyrighted work created in the past. Fig. 11 is a diagram showing an example of the copyrighted music information storage unit 121 according to the embodiment.

[0051] As shown in Fig. 11, the copyrighted music information storage unit 121 stores copyrighted music IDs, copyrighted music meta information, and MP3 file names. The copyrighted music ID is identification information for uniquely identifying copyrighted music by, for example, the Beatles, Michael Jackson, etc. The copyrighted music meta information is additional information related to the copyrighted music, such as the copyrighted music title, artist name, album name, release date, genre, mood, and lyrics. The MP3 file name is a file name that identifies the MP3 file of the copyrighted music.

[0052] The music storage unit 122 stores music information for copyrighted music pieces. Fig. 12 is a diagram showing an example of the music storage unit 122 according to an embodiment. As shown in Fig. 12, the music storage unit 122 stores an MP3 file for each copyrighted music piece. This MP3 file can be identified by the MP3 file name stored in the copyrighted music information storage unit 121.

[0053] The characteristic information storage unit 123 stores characteristic quantities of multiple types of copyrighted music pieces. Fig. 13 is a diagram showing an example of the characteristic information storage unit 123 according to an embodiment. As shown in Fig. 13, the characteristic information storage unit 123 stores copyrighted music piece characteristic quantities in association with copyrighted music piece IDs. The copyrighted music piece characteristic information includes overall information indicating the characteristics of the copyrighted music piece as a whole, and time-series information indicating the characteristics that change over time.

[0054] 14 is a diagram showing an example of overall information according to an embodiment. As shown in FIG. 14, the overall information includes, for example, tempo information, beat information, and song type information such as genre and mood. The song type information is, for example, information such as a random channel based on 12-tone analysis technology.

[0055] FIG. 15 is a diagram showing an example of time series information constituting the authored musical composition characteristic information according to the embodiment. As shown in FIG. 15, the time series information includes, for example, beat information, chord progression information, musical composition structure information, and melody information. The beat information includes, for example, time position information for each beat and time position information for each measure, like a metronome. The chord progression information is information on the time series progression of accompaniment chords according to the flow of the musical composition, for example, information indicating a chord progression such as "CF-Am-G7." The musical composition structure information is information on the overall structure of the musical composition, for example, "A melody-B melody-chorus-A melody-chorus." The melody information includes, for example, information on a melody line inferred from the audio information of the musical composition.

[0056] 10, the explanation will be continued. The control unit 130 is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing a program stored inside the user terminal 300 using a RAM (Random Access Memory) or the like as a working area. The control unit 130 is also a controller, and may be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0057] The control unit 130 has a management unit 131, an extraction unit 132, and a transmission unit 133, and realizes or executes the functions and actions of information processing described below.

[0058] The management unit 131 manages various types of information related to the copyrighted work management device 100. For example, the management unit 131 stores various types of information in the storage unit 120 and updates the stored information as appropriate. Specifically, the management unit 131 stores new copyrighted music pieces in the copyrighted music information storage unit 121 and updates information related to the new copyrighted music pieces.

[0059] The extraction unit 132 extracts multiple types of feature quantities from the copyrighted music information. The extraction unit 132 acquires periodically registered copyrighted music information at a preset timing and extracts chord progression information, beat information, melody information, and drum information as feature quantities from the acquired copyrighted music information. For each newly registered copyrighted music, the extraction unit 132 extracts music feature quantities from the MP3 file of each copyrighted music to obtain feature information. The extraction unit 132 extracts various feature quantities from the music information using, for example, twelve-tone analysis technology, and registers the extracted feature quantities in the feature information storage unit 123 as copyrighted music feature information.

[0060] The extraction unit 132 performs feature extraction processing on the copyrighted music information by receiving a copyrighted music extraction instruction from an administrator terminal (not shown) used by a system administrator via the information processing device 200. Alternatively, the extraction unit 132 performs feature extraction processing on the copyrighted music information by receiving a copyrighted music extraction instruction from the information processing device 200.

[0061] The transmitting unit 133 transmits the feature quantities of the multiple types of copyrighted music information extracted by the extracting unit 132 to the information processing device 200 together with information related to the copyrighted music information, such as music structure information and copyrighted music meta information.

[0062] [1-4. Configuration of information processing device according to embodiment] Next, the configuration of the information processing device 200 shown in Fig. 9 will be described with reference to Fig. 16. Fig. 16 is a diagram showing an example of the configuration of the information processing device 200 according to the embodiment. As shown in Fig. 16, the information processing device 200 has a communication unit 210, a storage unit 220, and a control unit 230. Note that the information processing device 200 may also have an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator who manages the information processing device 200, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

[0063] The communication unit 210 is realized by, for example, an NIC etc. The communication unit 210 is connected to the network N1 and the private network N2 by wire or wirelessly, and transmits and receives information to and from the user terminal 300, the copyright management device 100 etc. via the network N1 or the private network N2.

[0064] The storage unit 220 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 220 stores various data used in information processing.

[0065] As shown in FIG. 16, the storage unit 220 includes a user information storage unit 221, a style information storage unit 222, a property information storage unit 223, a production information storage unit 224, and an operation history information storage unit 225.

[0066] The user information storage unit 221 stores various types of information related to users (user information). Fig. 17 is a diagram illustrating an example of the user information storage unit 221 according to the embodiment.

[0067] User information including a user ID, user meta information, and authority information is stored in the user information storage unit 221. The user information storage unit 221 stores the user meta information and authority information corresponding to each user ID in association with each user ID.

[0068] The user ID indicates identification information for uniquely identifying a user. For example, the user ID indicates identification information for uniquely identifying a user such as a creator, a general user, or a system administrator. The user meta information is additional information about the user, such as the user's name and address. The authority information stores values for identifying authority, such as system administrator authority information, creator authority information, and general user authority information. Note that the user information storage unit 221 is not limited to the above, and may store various types of information depending on the purpose. The user meta information may store various types of information about the user. For example, if the user is a natural person, the user meta information may store demographic attribute information such as the user's gender and age, psychographic attribute information, etc.

[0069] The style information storage unit 222 stores information related to the composition model. Fig. 18 is a diagram showing an example of the style information storage unit 222 according to the embodiment.

[0070] The style information storage unit 222 stores learning model information including a model information ID, a creator ID, model information meta information, style information 700, a work ID, and sharability information. The style information storage unit 222 stores the creator ID, model information meta information, style information, work ID, and sharability information corresponding to each model information ID in association with each model information ID.

[0071] The model information ID indicates identification information for uniquely identifying composition model information. The creator ID indicates identification information for uniquely identifying the creator of the corresponding composition model information. For example, the creator ID indicates identification information for uniquely identifying a user such as a system administrator, a producer, or a general user.

[0072] The model information meta information is information that describes the characteristics of the work being studied, such as the tempo, genre, mood (e.g., light and dark), song structure (e.g., verse A, verse B, chorus), chord progression, scale, church mode, etc.

[0073] The style information 700 is learning data for a composition model generated by a generating unit 233 (described later) of the information processing device 200. As explained in Fig. 2, the style information is information that associates a plurality of types of feature quantities, such as chord progression, melody, and bass progression, extracted from music information with predetermined identification information.

[0074] The sharing possibility information indicates, for example, whether the corresponding learning model can be shared. For example, the sharing possibility information stores a value for identifying whether the corresponding learning model can be shared.

[0075] The style information storage unit 222 may store various types of information according to the purpose, not limited to the above. For example, the composition model information meta information may store various additional information related to the composition model, such as information on the date and time when the composition model was created.

[0076] The ownership information storage unit 223 stores various information related to style information selected by a creator who creates a piece of music when creating the piece of music. Fig. 19 is a diagram showing an example of the ownership information storage unit 223 according to the embodiment. The ownership information storage unit 223 stores the user ID of the creator who creates a piece of music and the style information ID selected by the creator in association with each other.

[0077] The production information storage unit 224 stores various information related to the produced music. Fig. 20 is a diagram showing an example of the production information storage unit 224 according to the embodiment. As shown in Fig. 20, the production information storage unit 224 stores the user ID of the producer who created the music and the score ID created by the producer in association with each other.

[0078] The operation history information storage unit 225 stores operation history information of the producer on the user terminal 300. The operation history information storage unit 225 stores operation history corresponding to each user ID in association with the user ID. The operation history information indicates the operation history of the producer. For example, the operation history information may include various information related to the producer's operations, such as the content of the operations performed by the producer and the date and time of the operations. For example, the operations include selecting style information presented by the information processing device 200, selecting the composition execution instruction button, and playing or editing music information received from the information processing device 200.

[0079] 16, the explanation will be continued. The control unit 230 is realized, for example, by a CPU, an MPU, or the like executing a program (for example, the information processing program according to this embodiment) stored inside the information processing device 200 using a RAM or the like as a work area. The control unit 130 is a controller, and may be realized, for example, by an integrated circuit such as an ASIC or an FPGA.

[0080] The control unit 230 has an acquisition unit 231, an extraction unit 232, a generation unit 233, a reception unit 234, a selection unit 235, a transmission unit 236, a composition unit 237, and an update unit 238, and realizes or executes the functions and actions of information processing described below.

[0081] The acquisition unit 231 acquires music information. The acquisition unit 231 acquires music information created by a composer by communicating with the user terminal 300 via the network N1. This music information was created by the composer using a music creation application, i.e., an automatic composition function, installed on the user terminal 300, and includes music feature quantities such as chord progressions, melody, bass progressions, and drum progressions. The acquisition unit 231 also communicates with the work management device 100 via the private network N2 to receive feature quantities of multiple types of authored music information extracted by the extraction unit 132 of the work management device 100, along with information related to the authored music information. That is, the acquisition unit 231 receives authored music feature information for each newly registered authored music from the work management device 100 via the private network N2.

[0082] The extraction unit 232 extracts multiple types of feature quantities from the music information. The extraction unit 132 extracts chord progression information, beat information, melody information, and drum information as feature quantities from the music information created by the producer. The extraction unit 232 extracts various feature quantities from the music information using, for example, 12-tone analysis technology.

[0083] The generating unit 233 generates style information in which the plurality of types of feature quantities extracted by the extracting unit 232 are associated with style information IDs, as learning data for the composition process. For music information created by a producer, the generating unit 233 associates the style information ID of the style information 700 with the user ID of the producer and registers the associated information in the production information storage unit 224. For music information registered in the copyrighted work management device 100, the generating unit 233 may associate the style ID of the style information 700 with the authored music ID.

[0084] The receiving unit 234 receives various types of information transmitted from the user terminal 300. For example, the receiving unit 234 receives information about the creator who uses the automatic composition function on the user terminal 300, and information about the style information selected by the creator. The receiving unit 234 can also register music to be linked to the style information 700, edit the style information, and so on.

[0085] When the automatic composition function is activated in the user terminal 300, the selection unit 235 selects all or part of the style information.

[0086] The transmitting unit 236 transmits the presentation information of the style information selected by the selecting unit 235 to the user terminal 300. As a result, the chord progression or lyric information of each style information is displayed as a list of candidates in the style palette selection pull-down 372a of the user terminal 300. Then, upon receiving instruction information from the user terminal 300 instructing the selection of one of the presented style information, the selecting unit 235 selects the selected style information from the style information storage unit 222.

[0087] The composing unit 237 composes music information using machine learning based on the style information selected by the selecting unit 235, and transmits the composed music information to the user terminal 300. The composing unit 237 may compose music using various existing music generation algorithms. For example, the composing unit 237 may use a music generation algorithm using a Markov chain, or a music generation algorithm using deep learning. The composing unit 237 may also generate multiple pieces of music information in response to instruction information transmitted from the user terminal 300. This allows the creator to receive multiple suggestions from the composing unit 237, allowing the creator to use a wider variety of information to advance the composition work.

[0088] When the updating section 238 receives performance information based on music information composed by the composition section 237 from the user terminal 300, it adds the performance information to the selected style information and updates the selected style information.

[0089] [1-5. Configuration of user terminal according to embodiment] Next, the configuration of the user terminal 300 shown in Fig. 9 will be described using Fig. 21. Fig. 21 is a diagram showing an example of the configuration of the user terminal 300 according to the embodiment. As shown in Fig. 21, the user terminal 300 has a communication unit 310, an input unit 320, an output unit 330, a storage unit 340, a control unit 350, and a display unit 360.

[0090] The communication unit 310 is realized by, for example, a NIC, a communication circuit, etc. The communication unit 310 is connected to the network N1 by wire or wirelessly, and transmits and receives information to and from the information processing device 200 and other devices such as other terminal devices via the network N1.

[0091] The input unit 320 receives various operations input from the user. The input unit 320 has a keyboard and a mouse connected to the user terminal 3000. The input unit 320 accepts input from the user. The input unit 320 accepts input from the user via the keyboard and mouse. The input unit 320 may have a function for detecting voice. In this case, the input unit 320 may include a microphone for detecting voice.

[0092] Various information may be input to the input unit 320 via the display unit 360. In this case, the input unit 320 may have a touch panel that can achieve the same functions as a keyboard or a mouse. In this case, the input unit 320 accepts various operations from the user via the display screen using touch panel functions realized by various sensors. Note that, although a capacitive method is mainly adopted for detecting user operations by the input unit 320 in tablet terminals, other detection methods such as a resistive film method, a surface acoustic wave method, an infrared method, and an electromagnetic induction method may be adopted as long as they can detect user operations and achieve the function of a touch panel. The user terminal 300 may also have an input unit that accepts operations using buttons, etc.

[0093] The output unit 330 outputs various types of information and includes a speaker that outputs sound.

[0094] The storage unit 340 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 340 stores various information used for displaying information.

[0095] The control unit 350 is realized by, for example, a CPU, an MPU, or the like executing a program stored in the user terminal 300 using a RAM or the like as a work area. The control unit 350 is also a controller, and may be realized by, for example, an integrated circuit such as an ASIC or an FPGA. The control unit 350 has a display control unit 351, a registration unit 352, a transmission / reception unit 353, a selection unit 354, a playback unit 355, and a performance reception unit 356.

[0096] The display control unit 351 controls various displays on the display unit 360. The display control unit 351 controls the display on the display unit 360. The display control unit 351 controls the display on the display unit 360 based on information received from the information processing device 200. The display control unit 351 controls the display on the display unit 360 based on information generated by processing by each component of the control unit 350. The display control unit 351 may control the display on the display unit 360 by an application that displays images.

[0097] The display control unit 351 displays a window 370 (see FIGS. 3 to 7) and the like on the display unit 360 using an application for the DAW and AI-based automatic composition function. Furthermore, when the display control unit 351 receives presentation information for style information from the information processing device 200, it displays the chord progression and lyrics of the presented style information in a style palette selection pull-down 372a (see FIG. 4) in the window 370. Furthermore, when the display control unit 351 receives music information and lyric information generated by the information processing device 200, it displays the melody and the lyrics corresponding to the melody in a melody display piano roll 374a (see FIG. 7) in the window 370.

[0098] The registration unit 352 accepts registration of various information. For example, the registration unit 352 accepts registration of drum patterns, chord progressions, and melodies set by the user when the DAW is started, etc. For example, the drum patterns, chord progressions, and melodies are registered via an app that displays images IM11, IM21, IM31, and IM41 (FIGS. 26 and 27).

[0099] The transmitting / receiving unit 353 communicates with the information processing device 200, transmitting and receiving various information. The transmitting / receiving unit 353 transmits to the information processing device 200 music information including drum patterns, chord progressions, and melodies accepted by the registration unit 352. When the automatic composition function is activated, the transmitting / receiving unit 353 receives presentation information of style information sent from the information processing device 200. The transmitting / receiving unit 353 transmits to the information processing device 200 instruction information instructing the selection of style information. The transmitting / receiving unit 353 then receives music information and lyric information generated by the information processing device 200. The transmitting / receiving unit 353 also transmits to the information processing device 200 performance information relating to a performance accepted by the user terminal 300.

[0100] The selection unit 354 selects one of the style information presented by the information processing device 200. For example, one of the chord progressions displayed in the style palette selection pull-down 372a (see FIG. 4) of the window 370 is selected by the user operating the input unit 320. In response to this, the selection unit 354 transmits instruction information for selecting the style information corresponding to the selected chord progression from the transmission / reception unit 353 to the information processing device 200.

[0101] The playback unit 355 plays back the music information generated by the information processing device 200. Specifically, the playback unit 255 sets arbitrary instrument information for each of the melody, chords, and bass notes included in the music data, and plays back each piece of data. Note that the playback unit 255 may play back a combination of the melody, chords, and bass notes.

[0102] When the creator performs a performance together with a composition using the automatic composition function, the performance accepting unit 356 accepts the performance by the creator. For example, the performance accepting unit 356 accepts performance information to be played in accordance with the playback of music information generated by the information processing device 200 using the automatic composition function.

[0103] The display unit 360 displays various types of information. The display unit 360 is realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 360 displays various types of information in accordance with the control of the display control unit 351. The display unit 360 can also display information such as images provided from the information processing device 200.

[0104] [1-6. Information Processing Procedure According to the Embodiment] [1-6-1. Generation process of style information for copyrighted music information] Next, various information processing procedures according to the embodiment will be described with reference to Fig. 22. First, as one of the information processing procedures according to the embodiment, automatic generation processing of style information related to copyrighted music information will be described. Fig. 22 is a sequence diagram showing the information processing procedures according to the embodiment.

[0105] As shown in Fig. 22, when new copyrighted music information is registered (step S101), the copyrighted work management device 100 extracts multiple types of feature quantities from each newly registered copyrighted music information (step S102). In this case, the copyrighted work management device 100 extracts the feature quantities using, for example, twelve-tone analysis technology. Then, the copyrighted work management device 100 transmits copyrighted music feature information including the multiple types of feature quantities extracted from the copyrighted music information and information related to the copyrighted music information to the information processing device 200 via the private network N2 (step S103).

[0106] The information processing device 200 automatically creates style information 700 based on the received authored music characteristic information (step S104). The information processing device 200 can generate score information 740 from, for example, beat information, chord progression information, and melody information of the authored music characteristic information. The information processing device 200 can generate lyric information 750 from, for example, lyric information of the authored music meta information.

[0107] The information processing device 200 can generate style palette information 730 by bundling together chord information with similar chord progressions from the multiple score information 740 and lyric information 750. Similar chord progressions are, for example, those with identical chord progressions. Alternatively, similar chord progressions can be those in which the order of T, S, and D is the same, with each chord classified as Tonic (T), Sub-Dominat (S), and Dominat (D). In the case of C major and A minor, T is C / Em / Am, S is F and Dm, and D is G and Dm7-5. The chord progressions CDGC and Em-Dm-Bm7-5-Am are both TSDT, and can therefore be considered the same chord progression. Furthermore, similar chord progressions can be classified based on similar chord progressions using, for example, machine learning or deep learning, rather than using music theory.

[0108] Furthermore, the information processing device 200 may register automatically generated style palette information 730 independently in the style palette sequence information 720. The information processing device 200 may generate and register style palette sequence information 720 in which multiple pieces of style palette information 730 are arranged. When arranging multiple pieces of style palette information 730, the information processing device 200 can arrange them with reference to music structure information.

[0109] Subsequently, the information processing device 200 registers the generated style information in association with the identification information of the copyrighted music information (step S105), and stores the same in the style information storage unit 222.

[0110] [1-6-2. Generation process of style information for music information created by a producer] Style information can also be generated for music information created by a producer. Therefore, a process for generating style information for music information created by a producer will be described with reference to Fig. 23. Fig. 23 is a sequence diagram showing the procedure of information processing according to the embodiment.

[0111] 23, the creator operates the user terminal 300 to activate a composition function such as a DAW (step S111) and selects creation of style information (step S112), which causes composition start information indicating the start of composition to be transmitted from the user terminal 300 (step S113). Then, upon receiving the composition start information, the information processing device 200 transmits information required for composition (step S114).

[0112] Next, the user terminal 300 registers a drum pattern (for example, tempo, number of bars, and beat positions for playing high hat, bass drums, and snare) through operations by the producer in accordance with UI instructions (step S115). After step S115 is completed, the user terminal 300 registers a chord progression through operations by the producer (step S116).

[0113] After step S116 is completed, the composition function application automatically plays the drums and chord progression, and the creator inputs the melody into the user terminal 300 multiple times in time with the performance. Through this input operation, the user terminal 300 registers the input melody (step S117). The user may also input additional lyric information using the composition function. The user terminal 300 continues to register the melody until the user has finished inputting the melody. When the user has finished registering the melody (step S118), the user terminal 300 transmits the composition information of the creator to the information processing device 200 (step S119). The composition information includes features such as the drum pattern, chord progression, and melody.

[0114] The information processing device 200 extracts each feature amount included in the music information to generate score information and lyric information, and generates style palette information from the score information and lyric information (step S120).

[0115] Then, the information processing device 200 obtains each piece of style palette sequence information in the music information, and generates the style information 700 by associating the style information ID with the score information, lyrics information, style palette information, and style palette sequence information (step S121).

[0116] The information processing device 200 registers the style information 700 in the style information storage unit 222 (step S122). At the same time, the information processing device 200 registers data associating this style information ID with the user ID of the creator in the ownership information storage unit 223. Then, the information processing device 200 registers data associating the score ID with the user ID of the creator in the production information storage unit 224, and ends the style information generation process. Note that the style palette sequence information may be generated by the creator arranging multiple pieces of style palette information using a composition function.

[0117] 22 and 23 are executed, a plurality of pieces of style information 700 are accumulated in the information processing device 200. A user such as a composer can select a preferred piece of style information from among these pieces of style information and use the automatic composition.

[0118] [1-6-3. Style information update process] The style information can also be updated by the creator. Therefore, a process for generating style information related to music information created by a creator will be described with reference to Fig. 24. Fig. 24 is a sequence diagram showing the procedure of information processing according to the embodiment.

[0119] When the creator activates the automatic composition function on the user terminal 300 (step S131), the information processing device 200 receives composition start information (step S132), selects style information (step S133), and transmits presentation information of the style information to the user terminal 300 (step S134). For example, the information processing device 200 selects from the style information storage unit 222 all style information, style information that has been used by the creator more than a predetermined number of times, or style information that has been used by all users more than a predetermined number of times, and transmits the presentation information of the selected style information.

[0120] The user terminal 300 then displays a list of style information based on the presented information (step S135). The user terminal 300 displays, for example, a list of chord progressions of the style information as candidates. When the producer selects style information (step S136), the user terminal 300 transmits selection information indicating the selected style information to the information processing device 200 (step S137).

[0121] The information processing device 200 extracts the selected style information, performs machine learning using the extracted style information as learning data to perform composition processing (step S138), and provides the music information to the user terminal 300 (step S139). Note that the information processing device 200 extracts features from the composed music information using the extraction unit 232, stores new score information including these features in the storage unit 220, and registers the score information in the owned information storage unit 223.

[0122] The user terminal 300 plays the provided music (step S140) and accepts performance processing (step S141). If the creator plays the music using a MIDI keyboard, for example, the performance information will be MIDI information. When the information processing device 200 receives this performance information (step S142), it extracts features from the performance information (step S143).

[0123] The information processing device 200 updates the style information selected by the producer by adding the feature quantities extracted from the performance information as score information to the style information selected by the producer (step S144). In this way, the producer's actual performance is added to the style information selected by the producer, so that the music information automatically composed using this style information approaches the producer's style. In other words, the style information can be made to approach the style of the producer who performed the music. The processing of steps S140 to S142 is repeated until the producer finishes playing. Then, the score information generated during the process of steps S140 to S142 repeated until the music is completed is saved in the storage unit 220.

[0124] In this way, in the information processing system 1, the actual performance of the producer is added to the style information selected by the producer, so that the music information automatically composed using this style information becomes closer to the style of the producer. In other words, the information processing system 1 can make the style information closer to the style of the producer who performed the music.

[0125] Then, when the producer has finished performing the music (step S145: Yes), the producer may operate the user terminal 300 to perform, for example, arranging processing (step S146), mixing, or mastering processing (step S147).

[0126] [1-7. Conceptual diagram of information processing system configuration] Here, each function, hardware configuration, and data in the information processing system are conceptually shown using Fig. 25. Fig. 25 is a diagram showing an example of a conceptual diagram of the configuration of the information processing system. Specifically, Fig. 25 is a schematic diagram showing an outline of the functions of a system that is an example of application of the information processing system 1.

[0127] [1-7-1. Overall structure] The copyrighted music management server device shown in Figure 25 corresponds to the copyrighted work management device 100 in the information processing system 1. The server device shown in Figure 25 corresponds to the information processing device 200 in the information processing system 1. The system administrator application section shown in Figure 25 corresponds to the application installed on the terminal used by the system administrator. The producer application section shown in Figure 25 corresponds to the application installed on the user terminal 300 in the information processing system 1. The general user application section shown in Figure 25 corresponds to the application installed on the terminal used by the general user. In the example of Figure 25, one system administrator application section, one music producer application section, and one general user application section are shown, but multiple of these may be included depending on the number of corresponding terminals.

[0128] 25 corresponds to the control unit 130 of the copyrighted music management server device. For example, the copyrighted music management unit of the copyrighted music management server device corresponds to the management unit 131 of the copyrighted music management device 100. The copyrighted music feature information analysis function of the copyrighted music management server device corresponds to the extraction unit 132 of the copyrighted music management device 100. The database unit in the copyrighted music management server of the copyrighted music management server device corresponds to the memory unit 120 of the copyrighted music management device 100.

[0129] 25 corresponds to the control unit 230 of the information processing device 200. For example, the learning processing unit of the server device corresponds to the composition unit 237 of the information processing device 200. An internal database unit of the server device corresponds to the storage unit 220 of the information processing device 200.

[0130] The display operation unit and control unit of the music producer application unit shown in Figure 25 correspond to the control unit 350 of the user terminal 300. For example, the display operation unit of the music producer application unit corresponds to the display control unit 351 of the user terminal 300. The display operation unit and control unit of the system administrator application unit and the general user application unit correspond to the control units of the terminal devices used by the respective users.

[0131] 25, the server device is connected to a system administrator application section, a music producer application section, and a general user application section via a network N1 such as the Internet, and is also connected to a copyrighted music management server device via a private network N2.

[0132] [1-7-2. Copyrighted music management server device] First, the configuration related to the copyrighted music management server device will be described.

[0133] The authored music management server device has a control unit and an internal database unit for authored music management. The control unit of the authored music management server device has an authored music management function and an authored music feature information analysis function.

[0134] [1-7-3. Server equipment] First, the configuration related to the server device will be described.

[0135] The server device has a control unit, a learning processing unit, and a server database unit. The control unit of the server device has a produced music information management function, a style information management function, a user operation history information management function, and a copyrighted music analysis function. The learning processing unit of the server device has a machine learning processing function and a deep learning processing function.

[0136] [1-7-4. About the app for music creators] Next, the configuration related to the music creator application section will be described.

[0137] The music producer application unit has a display operation unit and a control unit. The display operation unit of the music producer application unit has a function to display information about created music and a function to display and edit style information. The music producer application unit has a function to share style information and a function to transmit user operation history information.

[0138] The music producer app is, for example, music editing software (DAW, etc.), and can display music information using a music production information display function. If the DAW has, for example, an AI-assisted music production function, new music information can be created using the learning model information display and editing function. The system administrator app and general user app also have similar configurations, but users have different system access rights.

[0139] [1-8.UI (User Interface)] Here, details of the automatic composition function, such as information display by the application (music creation application), will be described with reference to Fig. 26 and Fig. 27. Fig. 26 and Fig. 27 are diagrams showing an example of a user interface according to the embodiment.

[0140] FIG. 26 shows an example of a user interface when the music creation application is displayed on the screen of the user terminal 300.

[0141] In the example shown in Fig. 26, the user interface IF11 displays music data received by a music creation application. Note that, as will be described in detail later, music data in a music creation application is made up of three different types of data: melody, chords, and bass notes. The user interface IF11 shown in Fig. 26 displays data related to the melody among the three different types of data.

[0142] The setting information ST11 displays information about a style palette, which is an example of setting information for the automatic composition function. The style palette is specification information for specifying style information that serves as learning data for machine learning.

[0143] The setting information ST12 displays information about harmony, which is an example of setting information for the automatic composition function. The information about harmony is, for example, information for determining the probability that constituent notes included in a chord will appear in a melody in music data composed by the information processing device 200. For example, if the user sets the information about harmony to "strict," the probability that constituent notes included in a chord will appear in a melody in the automatically composed music data increases. On the other hand, if the user sets the information about harmony to "loose," the probability that constituent notes included in a chord will appear in a melody in the automatically composed music data decreases. The example in FIG. 26 indicates that the user has applied the information about harmony to a "strict" setting.

[0144] The setting information ST13 displays note length information, which is an example of setting information for the automatic composition function. Note length information is, for example, information for determining the length of notes in music data composed by the information processing device 200. For example, if the user sets the note length information to "long," the automatically composed music data is more likely to contain notes with relatively long sound lengths (e.g., whole notes, half notes, etc.). On the other hand, if the user sets the note length information to "short," the automatically composed music data is more likely to contain notes with relatively short sound lengths (e.g., eighth notes, sixteenth notes, etc.).

[0145] The setting information ST14 is an example of setting information for the automatic composition function, and displays information for determining the type and amount of material music pieces other than those included in the specified information (the style palette specified by the user). This information is, for example, information for determining whether or not learning will be strictly based on music pieces included in the style palette specified by the user, in music data composed by the information processing device 200. For example, if the user sets this information to "never," there is a low tendency for music pieces other than those included in the style palette to be used in learning with the automatic composition. On the other hand, if the user sets this information to "only," there is a high tendency for music pieces other than those included in the style palette to be used in learning with the automatic composition.

[0146] The music data MDT1 indicates specific music data transmitted from the information processing device 200. In the example of FIG. 26, the music data MDT1 includes information indicating chord progressions such as Cm, information indicating the pitches and note durations within a measure, and information on note pitch transitions (in other words, the melody). Also, as shown in FIG. 26, the music data MDT1 may include, for example, four different types of content. That is, the information processing device 200 may transmit multiple pieces of music data instead of transmitting only one type of automatically composed music data. This allows the user to select their preferred music data from the multiple generated music data candidates, or to combine multiple pieces of music data to compose their favorite music.

[0147] The user interface IF11 shown in Fig. 26 displays data related to the melody among the three different types of data contained in the music data: melody, chords, and bass notes, but other data is displayed in other user interfaces. This point will be explained using Fig. 27.

[0148] As shown in Fig. 27, the user terminal 300 may display on its screen a user interface IF12 that displays data related to chords and a user interface IF13 that displays data related to bass notes, in addition to a user interface IF11 that displays data related to melody. Although not shown in Fig. 27, the user interface IF12 and the user interface IF13 display note information that is different from the music data MDT1 in the user interface IF11. Specifically, the user interface IF12 displays note information related to chords corresponding to the melody of the music data (e.g., constituent notes of a Cm chord). The user interface IF13 displays note information related to bass notes corresponding to the melody and chords of the music data (e.g., a "C" note for a Cm chord).

[0149] The user can select information to copy from the displayed user interface IF11, user interface IF12, and user interface IF13, and can perform operations such as editing a part of the bass sound, for example.

[0150] [2. Effects of this embodiment] As described above, the information processing device according to this embodiment (the information processing device 200 in the embodiment) has an acquisition unit (the acquisition unit 231 in the embodiment), an extraction unit (the extraction unit 232 in the embodiment), and a generation unit (the generation unit 233 in the embodiment). The acquisition unit acquires music information. The extraction unit extracts multiple types of feature quantities from the music information acquired by the acquisition unit. The generation unit generates information that associates the multiple types of feature quantities extracted by the extraction unit with predetermined identification information, as music feature information (style information in the embodiment) to be used as learning data in a composition process using machine learning.

[0151] In this way, the information processing device according to this embodiment can generate style information having multiple types of feature quantities of music information as a data set for training a composition model. By training the composition model with this style information, the information processing device according to this embodiment can provide each user, including the composer, with music information composed to match the characteristics of the music. Therefore, the information processing device according to this embodiment can improve the convenience of the music composition function for users.

[0152] The acquisition unit acquires music information by receiving, from a terminal device (user terminal 300 in this embodiment), music information including features related to music created by a creator using a music creation application installed on the terminal device. The extraction unit extracts multiple types of features included in the music information. The generation unit associates the music feature information with the creator's identification information. This allows the information processing device to generate music feature information related to music information created by a creator.

[0153] Furthermore, the feature quantities related to the music created by the composer are chord progression information indicating the chord progression, melody information indicating the melody, and bass signal indicating the bass progression in a bar of a specified length. Therefore, for music information created by a composer, the information processing device can generate music feature information using the feature quantities related to the music created by the composer, thereby enabling the music feature information to be generated quickly.

[0154] Furthermore, the feature quantity related to the music piece created by the composer is drum progression information indicating the progression of the drums in a measure of a specified length. Therefore, the information processing device can generate music piece feature information including the drum progression information.

[0155] The acquisition unit acquires periodically registered copyrighted music information at a preset timing. The extraction unit extracts multiple types of feature quantities from the copyrighted music information. The generation unit associates identification information of the copyrighted music information with style information. This allows the information processing device 200 to automatically generate music feature information related to periodically registered copyrighted music information.

[0156] The information processing device also has a transmission unit (transmission unit 236 in the embodiment) that transmits presentation information of song feature information in accordance with instruction information received from a terminal device on which a song creation application is installed. The information processing device also has a composing unit (composing unit 237 in the embodiment) that, upon receiving a selection of song feature information from the terminal device, composes song information using machine learning based on the selected feature information and transmits the composed song information to the terminal device. In this way, the information processing device presents song feature information to the terminal device in accordance with the instruction information, allowing the creator to select desired song feature information from among the information. The information processing device can then provide song information composed based on the song feature information desired by the creator.

[0157] Furthermore, the information processing device further includes an update unit (the update unit 238 in this embodiment) that, when receiving performance information based on the music information transmitted by the composition unit from the terminal device, adds the performance information to the selected music feature information and updates the selected music feature information. In this way, by adding the performance information by the producer to the selected music feature information, the information processing device can bring the music feature information closer to the style of the producer who performed the music.

[0158] The extraction unit also extracts, as features from the music information, chord progression information indicating a chord progression, melody information indicating a melody, and bass information indicating a bass progression for a measure of a specified length. The generation unit generates score information having chord progression information indicating a chord progression, melody information indicating a melody, and bass information indicating a bass progression for a measure of a specified length, and uses these score information as components of the music feature information. This allows the information processing device to generate music feature information having chord progression information, melody information, and bass information. Furthermore, since the information processing device learns feature amounts such as chord progression information, melody information, and bass information, rather than the music information itself, during composition, it can efficiently provide music information to users.

[0159] The extraction unit extracts, as a feature, drum information indicating the progression of drum sounds in a measure of a specified length from the music information. The generation unit further adds the drum progression information to the score information. This allows the information processing device to generate music feature information including chord progression information, melody information, bass information, and drum information.

[0160] The generation unit generates lyrics information indicating lyrics for bars of a specified length from the lyrics information to be added to the music information, and uses the information as a component of the music feature information. This allows the information processing device, when lyrics are searched on the terminal device, to extract music feature information including the lyrics or lyrics similar to the lyrics and present it on the terminal device, thereby improving the convenience of the music creation function for users. The information processing device also becomes capable of automatically generating lyrics.

[0161] The generation unit generates music format information that associates and registers score information identification information and lyric information identification information for the same measure, and uses the generated music format information as a component of the music feature information. By learning this music feature information, the information processing device can further provide music information that the user desires.

[0162] The generation unit adds identification information of score information having chord progression information similar to the chord progression information of the score information registered in the music format information to the music format information, thereby enabling the information processing device to compose music information that conforms to the structure of the music.

[0163] The generation unit generates song order information indicating the order of the song format information and sets the information as a component of the song feature information. Since the information processing device can also learn the order of the song format information, it can further improve learning accuracy.

[0164] [3. Other embodiments] The processing according to the above-described embodiment and modified examples may be implemented in various different forms (modified examples) other than the above-described embodiment and modified examples.

[0165] [3-1. Other configuration examples] The above configurations are merely examples, and the information processing system 1 may have any system configuration as long as it can realize the above-described information processing. For example, the copyrighted work management device 100 and the information processing device 200 may be integrated.

[0166] [3-2.Other] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. Furthermore, the information, including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings, can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0167] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0168] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0169] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0170] [4. Hardware Configuration] Information devices such as the information processing device 200, the work management device 100, and the user terminal 300 according to the above-described embodiments and modifications are realized by a computer 1000 having a configuration such as that shown in FIG. 28, for example. FIG. 28 is a hardware configuration diagram showing an example of the computer 1000 that realizes the functions of the information processing device 200, the work management device 100, and the user terminal 300. The information processing device 200 according to the embodiment will be described below as an example. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0171] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0172] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000 .

[0173] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0174] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0175] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined recording medium. Examples of media include optical recording media such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disk), magneto-optical recording media such as an MO (Magneto-Optical disk), tape media, magnetic recording media, and semiconductor memories.

[0176] For example, when the computer 1000 functions as the information processing device 200 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200, thereby realizing functions of the control unit 130 and the like. The information processing program according to the present disclosure and data in the storage unit 120 are stored in the HDD 1400. The CPU 1100 reads and executes program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from another device via an external network 1550.

[0177] The present technology can also be configured as follows. (1) an acquisition unit that acquires music information; an extracting unit that extracts a plurality of types of feature quantities from the music information acquired by the acquiring unit; a generating unit that generates information in which the plurality of types of feature quantities extracted by the extracting unit are associated with predetermined identification information as music feature information to be used as learning data in a composition process using machine learning; An information processing device comprising: (2) the acquisition unit acquires music information by receiving, from the terminal device, music information including features related to music created by a creator using a music creation application installed in the terminal device; the extraction unit extracts a plurality of types of feature amounts included in the music information; The generating unit associates the identification information of the creator with the music feature information. The information processing device according to (1) above. (3) The information processing device described in (2) is characterized in that the features related to the music created by the producer are chord progression information indicating the chord progression, melody information indicating the melody, and bass signal indicating the bass progression in a bar of a specified length. (4) The information processing device according to (3), wherein the feature quantity relating to the music created by the composer is drum progression information indicating the progression of drums in a bar of a specified length. (5) The acquisition unit acquires copyrighted music information that is periodically registered at a predetermined timing, the extraction unit extracts the plurality of types of feature quantities from the copyrighted music information; The generating unit associates the identification information of the copyrighted music information with the music feature information. The information processing device according to (1) above. (6) a transmitting unit that transmits presentation information of the music feature information in accordance with instruction information received from a terminal device on which an application related to music creation is installed; a composition unit that, upon receiving a selection of the music feature information from the terminal device, composes music information using machine learning based on the selected music feature information and transmits the composed music information to the terminal device; The information processing device according to (1) further comprises: (7) an updating unit that, when receiving performance information based on the music information transmitted by the composing unit from the terminal device, adds the performance information to the selected music feature information and updates the selected music feature information; The information processing device according to (6) further comprises: (8) the extraction unit extracts, from the music information, chord progression information indicating a chord progression, melody information indicating a melody, and a bass signal indicating a bass progression in a measure of a specified length, as feature quantities; The generating unit generates score information having chord progression information indicating a chord progression, melody information indicating a melody, and bass information indicating a bass note progression for the measure of the specified length, and sets the score information as components of the music feature information. The information processing device according to (1) above. (9) the extraction unit extracts, from the music information, drum information indicating a progression of drum sounds in a measure of the specified length as a feature; The generation unit further adds the drum information to the score information. The information processing device according to (8) above. (10) The generating unit generates lyrics information indicating lyrics for a measure of the specified length from lyrics information to be added to the music information, and sets the lyrics information as a component of the music feature information. The information processing device according to (8) above. (11) The generating unit generates music format information that associates and registers the identification information of the score information and the identification information of the lyric information for the same measure, and uses the music format information as a component of the music feature information. The information processing device according to (10) above. (12) The generation unit adds identification information of score information having chord progression information similar to chord progression information of score information registered in the music format information to the music format information and registers the same. The information processing device according to (11) above. (13) The generating unit generates song order information indicating the order of the song format information, and sets the song order information as a component of the song feature information. The information processing device according to (11) above. (14) 1. A computer-implemented information processing method, comprising: Get song information, extracting a plurality of types of feature amounts from the acquired music information; The extracted information associating the plurality of types of feature quantities with predetermined identification information is generated as music feature information to be used in a composition process using machine learning. 1. An information processing method comprising: (15) Get song information, extracting a plurality of types of feature amounts from the acquired music information; The extracted information associating the plurality of types of feature quantities with predetermined identification information is generated as music feature information to be used in a composition process using machine learning. An information processing program characterized by causing a computer to execute the above. [Explanation of symbols]

[0178] 1. Information Processing Systems 100 Copyright management device 110,210,310 Communications Department 120,220,340 Storage section 130,230,350 Control unit 121 Copyrighted music information storage unit 122 Music storage section 123 Feature information storage unit 131 Management Department 132 Extraction part 133 Transmitter 200 Information processing device 221 User information storage unit 222 Style information storage unit 223 Ownership information storage unit 224 Production information storage section 225 Operation history information storage unit 231 Acquisition Department 232 Extraction part 233 Generation part 234 Reception Department 235 Selection Section 236 Transmitter 237 Composition Department 238 Update Department 300 User terminal 320 Input section 330 Output section 351 Display control unit 352 Registration Department 353 Transmitter / Receiver 354 Selection Section 355 Playback Department 356 Performance Reception 360 display

Claims

1. an acquisition unit that acquires music information; an extracting unit that extracts a plurality of types of feature quantities from the music information acquired by the acquiring unit; a generation unit that generates music feature information in which the plurality of types of feature amounts extracted by the extraction unit are associated with predetermined identification information; a transmitting unit that transmits presentation information of the music feature information in accordance with instruction information received from a terminal device on which an application related to music creation is installed; a composition unit that, upon receiving a selection of the music feature information from the terminal device, composes music information using machine learning based on the selected music feature information and transmits the composed music information to the terminal device; An information processing device comprising:

2. the acquisition unit acquires music information by receiving, from the terminal device, music information created by a creator using a music creation application installed on the terminal device; the extraction unit extracts a plurality of types of feature amounts included in the music information; The generating unit associates the identification information of the creator with the music feature information.

2. The information processing apparatus according to claim 1, wherein:

3. 3. The information processing device according to claim 2, wherein the music information created by the composer includes chord progression information indicating a chord progression, melody information indicating a melody, and a bass signal indicating a bass progression in a bar of a specified length.

4. 4. The information processing apparatus according to claim 3, wherein the music information created by the composer includes drum progression information indicating the progression of drums in a measure of a specified length.

5. The acquisition unit acquires copyrighted music information that is periodically registered at a predetermined timing, the extraction unit extracts the plurality of types of feature quantities from the copyrighted music information; The generating unit associates the identification information of the copyrighted music information with the music feature information.

2. The information processing apparatus according to claim 1, wherein:

6. an updating unit that, when receiving performance information based on the music information transmitted by the composing unit from the terminal device, adds the performance information to the selected music feature information and updates the selected music feature information; 2. The information processing apparatus according to claim 1, further comprising:

7. the extraction unit extracts, from the music information, chord progression information indicating a chord progression, melody information indicating a melody, and a bass signal indicating a bass progression in a measure of a specified length, as feature quantities; The generating unit generates score information having chord progression information indicating a chord progression, melody information indicating a melody, and bass information indicating a bass note progression for the measure of the specified length, and sets the score information as components of the music feature information.

2. The information processing apparatus according to claim 1, wherein:

8. the extraction unit extracts, from the music information, drum information indicating a progression of drum sounds in a measure of the specified length as a feature; The generation unit further adds the drum information to the score information.

8. The information processing apparatus according to claim 7,

9. The generating unit generates lyrics information indicating lyrics for a measure of the specified length from lyrics information to be added to the music information, and sets the lyrics information as a component of the music feature information.

8. The information processing apparatus according to claim 7,

10. The generating unit generates music format information that associates and registers the identification information of the score information and the identification information of the lyric information for the same measure, and uses the music format information as a component of the music feature information.

10. The information processing apparatus according to claim 9,

11. The generation unit adds identification information of score information having chord progression information similar to chord progression information of score information registered in the music format information to the music format information and registers the same.

11. The information processing apparatus according to claim 10,

12. The generating unit generates song order information indicating the order of the song format information, and sets the song order information as a component of the song feature information.

11. The information processing apparatus according to claim 10,

13. the acquisition unit acquires music information by receiving, from the terminal device, music information created by a creator using an application related to the creation of the music; 2. The information processing apparatus according to claim 1, wherein:

14. The music information includes at least one of chord progression information, beat information, melody information, and drum progression information.

14. The information processing apparatus according to claim 13,

15. 1. A computer-implemented information processing method, comprising: Get song information, extracting a plurality of types of feature amounts from the acquired music information; generating music feature information in which the extracted feature quantities of the plurality of types are associated with predetermined identification information; transmitting presentation information of the music feature information in accordance with instruction information received from a terminal device on which an application related to music creation is installed; When the selection of the song feature information is accepted from the terminal device, song information is composed using machine learning based on the selected song feature information, and the composed song information is transmitted to the terminal device.

1. An information processing method comprising:

16. Get song information, extracting a plurality of types of feature amounts from the acquired music information; generating music feature information in which the extracted feature quantities of the plurality of types are associated with predetermined identification information; transmitting presentation information of the music feature information in accordance with instruction information received from a terminal device on which an application related to music creation is installed; When the selection of the song feature information is accepted from the terminal device, song information is composed using machine learning based on the selected song feature information, and the composed song information is transmitted to the terminal device. An information processing program characterized by causing a computer to execute the above.

17. an acquisition unit that acquires music information; an extracting unit that extracts a plurality of types of feature quantities from the music information acquired by the acquiring unit; a generation unit that generates music feature information in which the plurality of types of feature amounts extracted by the extraction unit are associated with predetermined identification information; a transmitting unit that transmits presentation information of the music feature information in accordance with instruction information received from a terminal device on which an application related to music creation is installed; a composition unit that, upon receiving a selection of the music feature information from the terminal device, composes music information using machine learning based on the selected music feature information and transmits the composed music information to the terminal device; An information processing system comprising:

Images