Information processing method, information processing program, and information processing device

Abstract

An information processing device 100 provides an interface with which it is possible to generate musical piece information by editing some elements forming the musical piece information, and on the basis of multiple pieces of musical piece information created using the interface, generates new musical piece information by a machine learning model.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method, an information processing program, and an information processing device. [Background technology]

[0002] With the advancement of AI (Artificial Intelligence), the use of computers in the field of art is increasing.

[0003] For example, there is a known technology that uses existing music pieces as training data to perform machine learning to generate a training model for music generation, and then has a computer compose new music pieces (see, for example, Patent Document 1). This technology uses a Markov model to imitate the characteristics of existing music pieces and generate more natural melodies. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 9,110,817 Summary of the Invention [Problem to be solved by the invention]

[0005] According to conventional technology, users can use music data proposed (generated) by AI in composing, allowing them to compose music based on a wider variety of perspectives.

[0006] With the AI ​​automatic composition function, users can receive automatically created music information by setting an image such as bright or dark, but there is a demand for an easy way to edit music information and receive music information that suits the user's preferences.

[0007] Therefore, the present disclosure proposes an information processing method, an information processing program, and an information processing device that can improve the convenience of AI-based automatic composition functions. [Means for solving the problem]

[0008] In order to solve the above problem, in an information processing method according to one embodiment of the present disclosure, a computer each Forming song information The key Basic Among the chord progression information, bass information, rhythm information and melody information, the chord progression information, bass information and rhythm information are fixed, and the melody information is By editing Multiple providing an interface capable of generating the music information, and as learning data to perform machine learning of a machine learning model to learn the relationship between the chord progression information, bass information, and rhythm information and the melody information, and Machine learning models By inputting the chord progression information, bass information, and rhythm information, New music information is generated. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a conceptual diagram illustrating a flow of information processing according to the embodiment. [Figure 2] FIG. 3 is a diagram illustrating an example of a data configuration of style information according to the embodiment. [Figure 3] FIG. 1 is a diagram (1) showing an example of a display screen when the creator terminal according to the embodiment generates style information. [Figure 4] FIG. 10 is a diagram (2) showing an example of a display screen when the creator terminal according to the embodiment generates style information. [Figure 5] FIG. 10 is a diagram (3) showing an example of a display screen when the creator terminal according to the embodiment generates style information. [Figure 6] FIG. 4 is a diagram (4) showing an example of a display screen when the creator terminal according to the embodiment generates style information. [Figure 7] FIG. 5 is a diagram (5) showing an example of a display screen when the creator terminal according to the embodiment generates style information. [Figure 8] FIG. 1 is a diagram (1) showing an example of a display screen when the creator terminal according to the embodiment executes composition processing. [Figure 9]FIG. 10 is a diagram (2) showing an example of a display screen when the creator terminal according to the embodiment executes composition processing. [Figure 10] FIG. 10 is a diagram (3) showing an example of a display screen when the producer terminal according to the embodiment executes composition processing. [Figure 11] FIG. 1 is a diagram illustrating an example of an information processing system according to an embodiment. [Figure 12] FIG. 1 is a diagram illustrating an example of the configuration of an information processing device according to an embodiment. [Figure 13] FIG. 4 is a diagram illustrating an example of user information according to the embodiment. [Figure 14] FIG. 4 is a diagram illustrating an example of a style information table according to the embodiment. [Figure 15] FIG. 4 is a diagram illustrating an example of a style information management table according to the embodiment. [Figure 16] FIG. 10 is a diagram illustrating an example of a produced music information table according to the embodiment. [Figure 17] FIG. 2 is a diagram illustrating an example of the configuration of a creator terminal according to the embodiment. [Figure 18] 1 is a flowchart (1) showing a procedure of the information processing device 100 according to the embodiment. [Figure 19] 10 is a flowchart (2) showing the procedure of the information processing device 100 according to the embodiment. [Figure 20] FIG. 1 is a diagram illustrating an example of a conceptual diagram of a configuration of an information processing system. [Figure 21] FIG. 10 is a diagram showing an example of a catalog display. [Figure 22] FIG. 10 is a diagram showing an example of mapping using three-dimensional graphics. [Figure 23] FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of an information processing device and a creator terminal. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an information processing method, an information processing program, and an information processing device disclosed in the present application will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to these embodiments.

[0011] The present disclosure will be described in the following order: 1. Embodiment 1-1. Example of information processing according to the embodiment 1-2. Configuration of information processing system according to embodiment 1-3. Configuration of the information processing device according to the embodiment 1-4. Configuration of the creator terminal according to the embodiment 1-5. Information processing procedure according to the embodiment 1-6. Effects of the embodiment 2. Conceptual diagram of information processing system configuration 2-1. Overall structure 2-2. Server equipment 2-3.About each app 3. Other embodiments 4. Hardware Configuration 5. Conclusion

[0012] [1. Embodiment] [1-1. Example of information processing according to the embodiment] First, an example of information processing according to the embodiment will be described with reference to Fig. 1. Fig. 1 is a conceptual diagram showing the flow of information processing according to the embodiment. The information processing according to the embodiment is executed by an information processing device 100 and a creator terminal 200.

[0013] In this embodiment, the case where the information processing device 100 is an information processing device that provides a service (also simply referred to as "service") related to the creation of content as a work is shown as an example. Note that, although a musical piece (music content) is shown below as an example of content, the content is not limited to a musical piece, and may be various types of content such as video content such as a movie or text content such as a book (novels, etc.). Furthermore, the musical piece referred to here is not limited to a completed piece of music (whole piece), but is a concept that includes various types of musical information such as sound sources that make up a piece of music (piece of music) and short sounds used for sampling.

[0014] The information processing device 100 communicates with creator terminals 200 of users who use services provided by the information processing device 100, using a network N such as the Internet (see FIG. 6). Note that the number of creator terminals 200 is not limited to that shown in FIG.

[0015] The information processing device 100 is a server device that executes information processing related to the AI-based automatic composition function of the producer terminal 200. For example, the information processing device 100 is a so-called cloud server that executes AI-based automatic composition in accordance with information instructed by the producer terminal 200 via the network N, and provides the generated music information to the producer terminal 200.

[0016] The producer terminal 200 is an information processing terminal such as a smartphone, a PC (personal computer), or a tablet terminal. Various program applications are installed on the producer terminal 200. Applications related to music creation are installed on the producer terminal 200. For example, the producer terminal 200 has an AI-based automatic composition function added by a plug-in (extension application) to an application such as a DAW (Digital Audio Workstation) that provides a comprehensive music production environment. For example, the plug-in can take the form of VST (Steinberg's Virtual Studio Technology) (registered trademark), AudioUnits, AAX (Avid Audio eXtension), or the like. Furthermore, the producer terminal 200 is not limited to a DAW, and may also use a mobile application such as iOS.

[0017] The flow of information processing will be described with reference to Fig. 1. For example, the information processing device 100 provides an interface for generating style information to the producer terminal 200 (step S11). The style information is learning data used when performing machine learning of a composition model.

[0018] The producer terminal 200 starts up the DAW and displays the interface provided by the information processing device 100 using a plug-in or the like. The producer C1 operates the producer terminal 200 to generate style information and provides it to the information processing device 100 (step SS12).

[0019] Here, the user of the producer terminal 200 will be described as producer C1, but this is not limiting. For example, the user of the producer terminal 200 may be any of the following: an administrator who operates and manages the entire system, producers such as composers, arrangers, and studio engineers who create music, or general users who receive music information via the automatic composition function.

[0020] The information processing device 100 collects style information from the creator terminal 200 (another terminal device not shown) and executes machine learning of a composition model using the collected style information (step S13). For example, the composition model corresponds to a Markov model or the like.

[0021] The information processing device 100 uses the composition model to generate new music information and provides the music information to the creator terminal 200 (step S14).

[0022] Next, an example of style information will be described. The style information includes melody information, chord information, bass information, rhythm information, etc. Fig. 2 is a diagram showing an example of the data structure of style information according to an embodiment. Style information 700 includes a style information ID 710, a style type ID 711, disclosure availability information 712, style palette sequence information 720, style palette information 730, score information 740, style palette meta information 750, and accompaniment audio information 760.

[0023] The style information ID 710 is information for identifying the style information 700. The style type ID 711 is information for identifying the type to which the style information 700 belongs. The disclosure permission information 712 is information for indicating whether the corresponding style information 700 is permitted to be made public to general users. The disclosure permission information 712 may be binary information.

[0024] The style palette sequence information 720 is information that indicates the order of the style palette information 730. The style palette sequence information 720 is a pair of a style palette ID that uniquely identifies the style palette information 730 and a bar index, and by having multiple pairs of these, it becomes information that manages the order of the style palette information 730 within a song. For example, in the example shown in Figure 2, it is defined that measures 1 to 4 of the song correspond to style palette ID 731a, measures 5 to 8 correspond to style palette ID 731b, and measures x to y correspond to style palette ID 731z.

[0025] The style palette information 730 is information in which the score ID of the score information 740 for the same measure is registered in the style palette ID, which is identification information of the style palette information.

[0026] The style palette information 730 may bundle together score IDs of multiple score information 740 whose chord progressions are similar. Similar chord progressions are, for example, those with the exact same chord progression. Alternatively, similar chord progressions may be those in which the order of T, S, and D is the same, with each chord classified as Tonic (T), Sub-Dominat (S), and Dominat (D). In the case of C major and A minor, T is C / Em / Am, S is F and Dm, and D is G and Dm7-5. The chord progressions CDGC and Em-Dm-Bm7-5-Am are both TSDT, and therefore can be considered the same chord progression. Similar chord progressions can also be classified based on similar chord progressions using, for example, machine learning or deep learning, rather than using music theory.

[0027] The score information 740 includes feature quantities of multiple types (elements) extracted from the music. The score information 740 has a score ID, melody information, chord progression information, bass information, and rhythm information. The score ID is identification information for the score information. The melody information is the melody in a measure of a specified length. The chord progression information is information indicating the chord progression in a measure of a specified length. The bass information is information indicating the progression of bass notes in a measure of a specified length. The rhythm information is information indicating the progression of drum sounds (drum pattern and tempo) in a measure of a specified length.

[0028] The style palette meta information 750 includes key signature / scale information, tempo information, and bar number information. The key signature / scale information is information that identifies the key signature of the music corresponding to the style information 700, and is set to either a major key (C major, D major, A major, etc.) or a minor key (A minor, E minor, B minor, etc.). The tempo information is information that indicates the tempo of the music corresponding to the style information 700, and is set in BPM (Beats Per Minute), etc. The bar number information indicates the number of bars corresponding to the style information 700. For example, the bar number information is set to 4 bars, 8 bars, etc.

[0029] The accompaniment audio information 760 includes multiple pieces of MP3 (MPEG-1 Audio Layer-3) information. The MP3 information is music information that producer C1 has recorded independently, corresponding to the melody information, chord progression information, bass information, and rhythm information set in the score information 740.

[0030] The information processing device 100 performs machine learning using the style information 700 described in Fig. 2 as training data to generate a composition model. Therefore, the information processing device 100 does not learn the music information itself, but learns style information composed of time-series feature quantities of multiple types (elements) extracted from the music information, such as melody information, chord progression information, bass information, and rhythm information. In other words, since the information processing device 100 learns multiple feature quantities extracted in advance from the music information, the information processing load is smaller than when learning the music information itself, and music information can be provided to the user efficiently.

[0031] For example, it is assumed that the information processing device 100 is provided with style information to be used as learning data from the creator terminal 200. The information processing device 100 presents an interface for creating style information to the creator terminal 200. The creator C1 generates new style information using the interface displayed on the creator terminal 200 and provides the new style information to the information processing device 100.

[0032] Specifically, the process of music creation by a producer will be described. Figs. 3 to 7 are diagrams showing examples of display screens when the producer terminal 200 according to the embodiment generates style information. The display screens (windows) shown in Figs. 3 to 7 correspond to interfaces provided by the information processing device 100. When the producer activates the automatic composition function on the producer terminal 200, a window 270 shown in Fig. 3 is displayed on the producer terminal 200.

[0033] The window 270 in FIG. 3 has a key signature / scale setting section 270a, a tempo setting section 270b, and a number of bars setting section 270c. The key signature / scale setting section 270a is an area where key signature / scale information can be set. Key signature information is, in English notation, C, D, E, F, G, A, B, etc., and scale information is, for example, minor or major. Key signature / scale information is a combination of key signature information and scale information, and is represented as, for example, C-major or D-minor. The tempo setting section 270b is an area where tempo information can be set. For example, 120 BPM is specified as tempo information. The number of bars setting section 270c is an area where number of bars information can be set. For example, 4 bars, 8 bars, 16 bars, etc. are set as number of bars information. The creator C1 operates the creator terminal 200 to input key signature / scale information, tempo information, and number of bars information. Following the window 270 shown in FIG. 3, a window 271 shown in FIG.

[0034] The window 271 in FIG. 4 includes a rhythm information setting section 271a. The rhythm information setting section 271a is an area where rhythm information can be set. For example, rhythm elements include Hi-Hat (hereinafter, HH), Snare-Drum (hereinafter, SD), and Bass-Drum (hereinafter, BD). In the rhythm information setting section 271a, input buttons for HH, SD, and BD are arranged in a grid for each sixteenth note. The creator C1 can input a rhythm pattern by arbitrarily setting each input button to ON (active) or OFF (inactive). When the input button is ON, the creator C1 can further set the sound strength (hereinafter, velocity). For example, velocity can range from 0 to 127, but tapping the input button inputs a default value of 100. The creator C1 can easily change the velocity value by swiping up or down to increase or decrease the velocity value.

[0035] In the rhythm information setting section 271a, four consecutive input buttons lined up horizontally indicate the rhythm information for one measure. One input button corresponds to one beat. A highlighted input button is active, and an unhighlighted input button is inactive. The active / inactive information and velocity value for each beat corresponding to HH, SD, and BD are the features of the rhythm information. The features of the rhythm information are associated with a bar index for each measure. The producer C1 operates the producer terminal 200 to input rhythm information. The producer terminal 200 saves the input rhythm information as rhythm information in the score information 740. Following the window 271 shown in FIG. 4, a window 272 shown in FIG. 5 is displayed on the producer terminal 200.

[0036] Window 272 in Fig. 5 has a chord progression information setting section 272a. Chord progression information setting section 272a is an area where chord progression information can be set. For example, four consecutive grid-shaped input areas in area 272a-1 indicate the chord progression information for one measure. When the input area in area 272a-1 is selected and a selection button in area 272a-2 is pressed, the chord corresponding to the pressed selection button is set in the selected input area.

[0037] Chords include C, Dm, Em, F, G, Am, Bm, etc., and can be arranged for each measure. For example, in the case of style information for four measures, the chord progression information will be |Em|Am|F|G|, |C|Dm|Em|F|, etc. Chords can also be changed for each beat within a measure.

[0038] For example, when producer C1 operates producer terminal 200 and selects input field 272a-11, and presses selection button 272a-21 for "Em," the chord "Em" is set in input field 272a-11. In this way, producer C1 can easily select basic chords such as C, Dm, Em, F, G, Am, and Bm using the selection buttons, but can also input complex chords such as CM7, C7, and Csus4.

[0039] The order of the chords included in area 272a-1 becomes the feature of the chord progression information. The feature of the chord progression information is associated with a bar index for each bar. The producer terminal 200 saves the input chord progression information as chord progression information in the score information 740. Following window 272 shown in FIG. 5, window 273 shown in FIG. 6 is displayed on the producer terminal 200.

[0040] Window 273 in FIG. 6 has a bass information setting section 273a. Bass information setting section 273a is an area where bass information can be set. For example, four consecutive input areas in area 273a-1 indicate bass information for one measure. When an input area in area 273a-1 is selected and a selection button in area 273a-2 is pressed, the chord corresponding to the selection button is set in the selected input area. Basically, the bass note corresponds to the chord name; for example, for a C chord, the bass note is C; for a Dm chord, the bass note is D; and for an Em chord, the bass note is E.

[0041] It should be noted that the composer C1 can set a base note that is different from the chord name. For example, he / she can specify a base note of F for a C chord, a base note of B for a Dm chord, and a base note of F for an Em chord. For example, when the chord and the base note are different, they are written as a fraction, with the base note as the denominator and the chord name as the numerator. For example, if a base note of F is specified for a C chord, the notation will be "C / F."

[0042] When producer C1 operates the producer terminal 200 to select the input area 273a-11 and presses the "F" selection button 273a-21, the bass note "F" is set in the input area 273a-11. If a chord name C has already been set in the same input area, "C / F" is set. The order of the bass notes included in the area 273a-1 becomes the feature of the bass information. The feature of the bass information is associated with the bar index for each measure. The producer terminal 200 saves the input bass information as bass information in the score information 740. Following the window 273 shown in FIG. 6, a window 274 shown in FIG. 7 is displayed on the producer terminal 200.

[0043] 7 has a melody information setting section 274a. The melody information setting section 274a is an area where melody information can be set. For example, when producer C1 presses the play button 240c, the producer terminal 200 plays an accompaniment corresponding to the chord progression information, bass information, and rhythm information set in FIGS. 3 to 5.

[0044] While listening to the accompaniment, producer C1 operates producer terminal 200 to press the image of the piano keyboard in melody information setting section 274a, thereby inputting (performing) melody information according to the press time and the position of the pressed key. The input melody information is displayed in partial area 274b. In partial area 274b, the range of one bar is range 274b-1. The melody information is visually displayed by bars, the position of the bar corresponding to the scale, and the length of the bar indicating the length of the note. The scale and note length in each bar are the feature quantities of the melody information. The feature quantities of the melody information are associated with a bar index for each bar.

[0045] The producer terminal 200 saves the input melody information as melody information in the score information 740. Note that the producer C1 can input the melody information again without registering it in the score information 740 by making a retry request.

[0046] Here, in the window 274 of Fig. 7, the producer C1 can repeatedly generate multiple pieces of melody information while keeping the chord progression information, bass information, and rhythm information fixed. The producer terminal 200 generates the style information 700 described in Fig. 2 based on the information input in the windows of Figs. 3 to 7, and provides the generated style information 700 to the information processing device 100.

[0047] An example of the process in which the producer terminal 200 generates style information 700 will be described below. The producer terminal 200 combines the chord progression information, bass information, and rhythm information generated in Figures 4 to 6 with one piece of melody information generated in Figure 7 into one piece of score information 740 and assigns a unique score ID to it. When the producer terminal 200 has generated multiple pieces of melody information, it generates score information 740 for each piece of melody information and assigns a score ID to each piece of score information 740.

[0048] When a plurality of pieces of score information 740 are generated, the producer terminal 200 bundles similar pieces of chord progression information into one piece of style palette information 730 based on the chord progression information of the plurality of pieces of score information 740, and assigns a unique style palette ID to it.

[0049] The producer terminal 200 refers to the score information 470 corresponding to the score ID of the style palette information, and identifies the bar index corresponding to the melody information, chord progression information, bass information, and rhythm information, thereby identifying the relationship between the bar index and the style palette ID, and generates the style palette sequence information 720. The relationship between the bar index and the style palette ID may be specified by the producer C1 as appropriate.

[0050] The producer terminal 200 sets the key signature / scale information, tempo information, and number of bars information input in Fig. 3 as style palette meta information 750. When the producer terminal 200 specifies style information and accepts input of MP3 information, it sets the MP3 information in accompaniment audio information 760.

[0051] When generating style information 700, the creator terminal 200 assigns a unique style information ID 710. The creator terminal 200 sets a style type ID 711 and disclosure permission information 712 in response to input from the creator C1.

[0052] As described above, the information processing device 100 provides the producer terminal 200 with an interface that enables the creation of music information by fixing some elements among the multiple elements that make up music information and editing the remaining elements. The information processing device 100 generates new music information based on the multiple pieces of music information created using the interface. For example, the music information corresponds to the score information 740 of the style information 700. The fixed elements are chord progression information, bass information, and rhythm information. The other elements are melody information, and when multiple pieces of melody information are generated, multiple pieces of score information 740 corresponding to the melody information are generated.

[0053] This allows the producer C1 to easily create style information, and the style information can be easily provided to the information processing device 100. The information processing device 100 uses the provided plurality of style information to perform machine learning of a composition model.

[0054] Subsequently, when the above-mentioned processing is completed, the producer C1 can perform AI-based composition processing by executing data communication with the information processing device 100. FIGS. 8 to 10 are diagrams showing examples of display screens when the producer terminal 200 according to the embodiment executes composition processing. The display screens (windows) shown in FIGS. 8 to 10 correspond to interfaces provided by the information processing device 100. When the producer C1 activates the automatic composition function on the producer terminal 200, a window 280 shown in FIG. 8 is displayed on the producer terminal 200.

[0055] Window 280 in FIG. 8 includes a style information list 280a. Creator C1 operates creator terminal 200 to select one of the style information pieces included in style information list 280a. In the example shown in FIG. 8, style information names 280a1 to 280a7 are displayed in style information list 280a. For example, it is assumed that style information names 280a1 to 280a7 are each associated with a style information ID. The style information list 280a may also display the creation date and time of the style information, ratings, comments, and the like in association with the style information name. Creator C1 operates creator terminal 200 to select the name of the style information. When one of style information names 280a1 to 280a7 displayed in style information list 280a is selected, window 281 shown in FIG. 9 is displayed on the creator terminal 200.

[0056] Window 281 in Fig. 9 has a chord progression information setting section 281a and a melody information display section 281b. Similar to the chord progression information setting section 272a described in Fig. 5, chord progression information setting section 281a has an area 281a-1 for setting chord progression information and an area 281a-2 for displaying chord selection buttons. The producer terminal 200 sets the chord progression information of the style information selected in window 280 in Fig. 8 in area 281a-1. The producer terminal 200 displays the melody information of the style information selected in window 280 in Fig. 8 in melody information display section 281b.

[0057] The producer C1 can change the chord progression set in area 281a-1 to suit his / her taste. For example, by operating the selection buttons in area 281a-2 and area 281a-1, the producer C1 can change the chord progression |C|Dm|Em|F| to a chord progression |C|Dm7|Em7|FM7|. Information about the changed chord progression is notified from the producer terminal 200 to the information processing device 100.

[0058] When the chord progression is changed as described above, the information processing device 100 can instantly create new style information. For example, if the style information 700A selected in the window 280 of FIG. 8 includes chord progression information (|C|Dm|Em|F|), new style information 710A' including chord progression information (|C|Dm7|Em7|FM7|) is generated by executing the process described in FIG. 9. The style information 700A and the style information 710A' share information in common except for the style information ID and chord progression information. The style information 710A' is added to the style information list 280a. When the change of the chord progression information in the window 281 is completed, the window 282 of FIG. 10 is displayed on the producer terminal 200.

[0059] 10 has a bass information setting section 273a and a melody information display section 282b. Similar to the bass information setting section 273a described in FIG. 6, the bass information setting section 273a has an area 282a-1 for setting bass information and an area 282a-2 for displaying bass note selection buttons. The producer terminal 200 sets the bass information of the style information selected in the window 280 of FIG. 8 in the area 282a-1. The producer terminal 200 displays the melody information of the style information selected in the window 280 of FIG. 8 in the melody information display section 282b.

[0060] The producer C1 can change the bass note set in the area 282a-1 to suit his / her taste. For example, the producer C1 changes the bass information by operating the selection button in the area 282a-2 and the area 282a-1. When the bass information is changed, new style information is generated and added to the style information list 280a in the same manner as when the chord progression information is changed. When the change of the bass information is completed in the window 282, the changed style information is notified to the information processing device 100.

[0061] The information processing device 100 performs machine learning of a composition model based on the updated style information. When the information processing device 100 receives a composition request for the updated style information from the producer terminal 200, the information processing device 100 automatically creates new melody information for the updated style information using the composition model. The information processing device 100 notifies the producer terminal 200 of the new melody information and displays the new melody information in a window. For example, the information processing device 100 displays the new melody information in the melody information display area 281b included in the window 281 in FIG. 9.

[0062] Here, new melody information generated by the composition model of the information processing device 100 can also be arranged. The producer C1 operates the producer terminal 200 to specify the measures to be arranged and makes an arrangement request to the information processing device 100. The measures to be arranged are referred to as the measures to be arranged. Explaining this with reference to FIG. 9, for example, the producer C1 operates the producer terminal 200 to specify the measure 281b-1 to be arranged.

[0063] The information processing device 100 arranges the melody information of the bars to be arranged out of all the bars of the melody information. For example, the information processing device 100 may have an arrangement table that associates parameters with pre-learned arrangement rules, and when parameters are specified by the producer C1, the information processing device 100 may execute an arrangement according to the specified parameters. The information processing device 100 saves score information 740 of the arranged melody information as a history.

[0064] The above process can be repeated, and the style information of the creator C1 is accumulated in the creator C1 preference score information 740, and the style information is optimized to the creator C1 preference.

[0065] [1-2. Configuration of information processing system according to embodiment] Fig. 11 is a diagram showing an example of an information processing system according to an embodiment. As shown in Fig. 11, the information processing system includes an information processing device 100, creator terminals 200-1 to 200-3, and a service server 300. In the example shown in Fig. 11, creator terminals 200-1 to 200-3 are illustrated, but when there is no need to distinguish between them, they will be referred to as creator terminal 200. The information processing device 100 is connected to the creator terminal 200 via a network N. The information processing device 100 is connected to the service server 300 via the network N.

[0066] The information processing device 100 provides the producer terminal 200 with an interface for generating style information, and acquires the style information from the producer terminal 200. The information processing device 100 generates new music information based on the acquired style information and provides it to the producer terminal 200.

[0067] The creator terminal 200 receives an interface from the information processing device 100 and displays the windows described with reference to Figures 1 to 7. The creator terminal 200 receives input of information related to style information from the creator and provides the style information to the information processing device 100.

[0068] The creator terminal 200 receives an interface from the information processing device 100 and displays the style information list shown in Fig. 8 etc. The creator terminal 200 receives selection of style information and update of the selected style information from the creator, and notifies the information processing device 100 of the updated style information.

[0069] The service server 300 provides services related to various SNS (Social Networking Services). For example, the service server 300 receives score information 740 included in style information from the information processing device 100, and generates an MP3 file based on the score information 740 and MIDI (Musical Instrument Digital Interface) information. The service server 300 posts the generated MP3 file on various SNS sites and collects evaluation information from users of the SNS sites.

[0070] [1-3. Configuration of information processing device 100 according to embodiment] Next, the configuration of the information processing device 100 shown in Fig. 11 will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. As shown in Fig. 12, the information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

[0071] The communication unit 110 is realized by, for example, a network interface card (NIC) etc. The communication unit 110 is connected to a network N by wire or wirelessly, and transmits and receives information to and from the creator terminal 200 and the service server 300 via the network N.

[0072] The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 stores various data used in information processing. The storage unit 120 has user information 121, a style information table 122, a style information management table 123, a produced music information table 124, and composition model data 125.

[0073] The user information 121 stores various types of information related to users (user information). Fig. 13 is a diagram showing an example of the user information 121 according to the embodiment.

[0074] The user information 121 includes a user ID, user meta information, and authority information. The user information 121 stores the user meta information and authority information corresponding to each user ID in association with each user ID.

[0075] The user ID indicates identification information for uniquely identifying a user. For example, the user ID indicates identification information for uniquely identifying a user such as a creator, a general user, or a system administrator. The user meta information is additional information about the user, such as the user's name and address. The authority information stores values ​​for identifying authority, such as system administrator authority information, creator authority information, and general user authority information. Note that the user information 121 is not limited to the above, and various information may be stored depending on the purpose. The user meta information may store various information about the user. For example, if the user is a natural person, the user meta information may store demographic attribute information such as the user's gender and age, psychographic attribute information, etc.

[0076] The style information table 122 holds a plurality of pieces of style information provided from the creator terminal 200. FIG. 14 is a diagram showing an example of the style information table 122 according to an embodiment. As shown in FIG. 14, the style information table 122 includes a plurality of pieces of style information 700A to 700d. The style information table 122 may include style information other than the style information 700A to 700d. The description of the style information 700a to 700d is the same as the description of the style information 700 shown in FIG. 2.

[0077] The style information management table 123 holds information that associates style information with the creator who created the style information. FIG. 15 is a diagram showing an example of the style information management table 123 according to an embodiment. As shown in FIG. 15, the style information management table 123 associates a user ID with a style information ID. The user ID indicates identification information for uniquely identifying a user. The style information ID is information that identifies style information. For example, the first row of the style information management table 123 indicates that the style information with style information ID 710A and the style information with style information ID 710B were created by a creator with user ID "C1."

[0078] The produced music information table 124 holds information that associates score information with the creator who produced the score information. FIG. 16 is a diagram showing an example of the produced music information table 124 according to the embodiment. As shown in FIG. 16, the produced music information table 124 associates a user ID with a score ID. The user ID indicates identification information for uniquely identifying a user. The score ID is information that identifies the score information.

[0079] The composition model data 125 is data for a composition model generated by executing machine learning using multiple style information included in the style information table 122 as input. For example, such a composition model learns the relationship between the time-series feature values ​​of chord progression information, bass information, and rhythm information and the time-series feature values ​​of melody information. Therefore, when chord progression information, bass information, and rhythm information are input to the composition model, melody information is predicted from the input chord progression information, bass information, and rhythm information, and thus music information is automatically generated. The composition model corresponds to a Markov model, etc.

[0080] In the embodiment, a case where melody information is predicted using a composition model is described, but this is not limited to this, and the composition model data 125 may be generated by performing machine learning to predict chord progression information, bass information, and rhythm information.

[0081] Returning to the explanation of Fig. 12, the control unit 130 is realized by, for example, a central processing unit (CPU) or a micro processing unit (MPU) executing a program stored inside the information processing device 100 using a random access memory (RAM) or the like as a working area. The control unit 130 is also a controller, and may be realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0082] The control unit 130 includes an interface control unit 131 , a machine learning unit 132 , a composition unit 133 , an arrangement unit 134 , and a service server cooperation unit 135 .

[0083] The interface control unit 131 is a processing unit that provides an interface capable of generating music information such as style information to the producer terminal 200. First, the processing of the interface control unit 131 relating to the windows 270 to 274 described with reference to Figs. 3 to 7 will be described.

[0084] The interface control unit 131 transmits information of the windows 270 to 274 described in FIGS. 3 to 7 to the creator terminal 200. The creator terminal 200 displays the windows 270 to 274 described in FIGS. 3 to 7 and accepts input of style information from the creator. The interface control unit 131 receives the style information from the creator terminal 200. For example, the style information is associated with a user ID.

[0085] The interface control unit 131 registers the style information received from the creator terminal 200 in the style information table 122. The interface control unit 131 registers the relationship between the style information ID included in the style information and the user ID in the style information management table 123.

[0086] The style information that the interface control unit 131 receives from the producer terminal 200 includes multiple pieces of melody information in addition to fixed chord progression information, bass information, and rhythm information.

[0087] When any score information is specified among the multiple score information included in the style information, the interface control unit 131 associates the score ID of the specified score information with the user ID and registers it in the produced music information table 124. For example, the producer operates the producer terminal 200 to specify favorite score information.

[0088] Next, a description will be given of the processing of interface control unit 131 relating to windows 280 to 282 described in Figures 8 to 10. Interface control unit 131 causes information on windows 280 to 282 described in Figures 8 to 10 to be displayed on creator terminal 200. For example, interface control unit 131 causes names corresponding to style information registered in style information table 122 to be displayed in style information list 280a of window 280 in Figure 8.

[0089] When the creator selects one of the names of style information displayed in the style information list 280a of the window 280, the creator terminal 200 notifies the interface control unit 131 of the style information ID corresponding to the selected name.

[0090] When the interface control unit 131 receives the style information ID, it acquires the style information corresponding to the style information ID from the style information table 122 and transmits the acquired style information to the producer terminal 200. By performing this process, the interface control unit 131 sets the chord progression information of the style information in the area 281a-1 of the window 281 in FIG.

[0091] When the chord progression information is changed in the window 281 of FIG. 9, the interface control unit 131 generates new style information and registers the generated new style information in the style information table 122.

[0092] Furthermore, when the interface control unit 131 transmits the style information to the creator terminal 200, the base information of the style information is set in the area 282a-2 of the window 282 in Fig. 10. Even when the base information is changed in the window 282 in Fig. 10, the interface control unit 131 generates new style information and registers the generated new style information in the style information table 122.

[0093] Here, when the interface control unit 131 receives a request from the creator terminal 200 to perform composition processing using changed style information, it outputs the changed style information to the composition unit 153. The interface control unit 131 acquires melody information corresponding to the changed style information from the composition unit 153 and transmits the acquired melody information to the creator terminal 200. As a result, the new melody information is displayed in the melody information display unit 281b etc. included in the window 281 of Fig. 9. When the interface control unit 131 receives a request from the creator terminal 200 to add style information having new melody information, it registers the style information having the new melody information in the style information table 122.

[0094] When the interface control unit 131 receives designation of parameters and measures to be arranged, it outputs melody information, parameters, and information about the measures to be arranged to the arranging unit 154. The interface control unit 131 acquires arranged melody information from the arranging unit 154 and transmits the acquired melody information to the producer terminal 200. When the interface control unit 131 receives a request from the producer terminal 200 to add style information having the arranged melody information, it registers the style information having the arranged melody information in the style information table 122.

[0095] As the interface control unit 131 executes the above processing, new style information is accumulated in the style information table 122.

[0096] The machine learning unit 152 generates composition model data 125 based on the style information registered in the style information table 122. For example, the machine learning unit 152 generates the composition model data 125 by machine learning the relationship between the time-series feature amounts of chord progression information, bass information, and rhythm information and the time-series feature amounts of melody information. The machine learning unit 152 may generate the composition model data 125 using a music generation algorithm that uses a Markov chain, or may generate the composition model data 125 using a music generation algorithm that uses deep learning. The machine learning unit 152 may acquire new style information from the interface control unit 131 and re-learn the composition model.

[0097] The composition unit 133 generates melody information based on the style information and the composition model data 125. For example, when the composition unit 133 receives changed style information from the interface control unit 131, it inputs the time-series feature quantities of the chord progression information, bass information, and rhythm information included in the style information into the composition model, and predicts (composes) the time-series feature quantities of the melody information. The composition unit 133 outputs the predicted melody information (new music information) to the interface control unit 131.

[0098] Upon receiving melody information, parameters, and information about bars to be arranged, the arranging unit 134 arranges the time-series features of the bars to be arranged in the melody information. For example, the arranging unit 134 has an arrangement table that associates parameters with pre-learned arrangement rules, and performs an arrangement according to the parameters specified by the producer. The arranging unit 134 outputs the arranged melody information to the interface control unit 131.

[0099] The service server cooperation unit 135 notifies the service server 300 of the style information and provides various SNS-related services. The service server cooperation unit 135 provides the service server 300 with score information 740 included in the style information in the style information table 122. The service server cooperation unit 135 receives evaluation information related to the score information 470 from the service server 300, and stores the received evaluation information in association with the style information. The interface control unit 131 may display the evaluation information in association with the name of the style information displayed in the style information list 280a of the window 280.

[0100] [1-4. Configuration of creator terminal 200 according to the embodiment] Next, the configuration of the creator terminal 200 shown in Fig. 11 will be described with reference to Fig. 17. Fig. 17 is a diagram showing an example of the configuration of the creator terminal 200 according to an embodiment. As shown in Fig. 17, the creator terminal 200 includes a communication unit 210, an input unit 220, an output unit 230, a storage unit 240, a control unit 250, and a display unit 260.

[0101] The communication unit 210 is realized by, for example, a NIC, a communication circuit, etc. The communication unit 210 is connected to a network N by wire or wirelessly, and transmits and receives information to and from the information processing device 100 and other devices such as other terminal devices via the network N.

[0102] The input unit 220 receives various operations input from the user. The input unit 220 has a keyboard and a mouse connected to the creator terminal 2000. The input unit 220 accepts input from the user. The input unit 220 accepts input from the user via the keyboard and mouse. The input unit 220 may have a function for detecting audio. In this case, the input unit 220 may include a microphone for detecting audio.

[0103] Various information may be input to the input unit 220 via the display unit 260. In this case, the input unit 220 may have a touch panel that can achieve the same functions as a keyboard or a mouse. In this case, the input unit 220 accepts various operations from the user via the display screen using touch panel functions realized by various sensors. Note that, while a capacitance method is mainly adopted for detecting user operations by the input unit 220 in tablet terminals, other detection methods such as a resistive film method, a surface acoustic wave method, an infrared method, and an electromagnetic induction method may be adopted as long as they can detect user operations and achieve the function of a touch panel. In addition, the creator terminal 200 may have an input unit that accepts operations using buttons, etc.

[0104] The producer may also use the input unit 220 to input MP3 information to be set in the accompaniment audio information 760 .

[0105] The output unit 230 outputs various types of information and includes a speaker that outputs sound.

[0106] The storage unit 240 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 240 stores various information used for displaying information. The storage unit 240 stores operation history information 241.

[0107] The operation history information 241 is information indicating the history of operations performed on the producer terminal 200 by a producer who creates a piece of music when an application is launched. For example, the operation history information 241 may include various information related to the producer's operations, such as the content of the operations performed by the producer and the date and time of the operations. The operations include selecting style information presented by the information processing device 100, selecting the composition execution instruction button, and playing, editing, and producing music information received from the information processing device 100.

[0108] The control unit 250 is realized, for example, by a CPU, an MPU, or the like executing a program stored in the producer terminal 200 using RAM or the like as a work area. The control unit 250 is also a controller, and may be realized, for example, by an integrated circuit such as an ASIC or an FPGA. The control unit 250 has a display control unit 251, a transmission / reception unit 252, a request unit 253, and a playback unit 254.

[0109] The display control unit 251 controls various displays on the display unit 260. The display control unit 251 controls the display on the display unit 260. The display control unit 251 controls the display on the display unit 260 based on information received from the information processing device 100. The display control unit 251 controls the display on the display unit 260 based on information generated by processing by each component of the control unit 250. The display control unit 251 may control the display on the display unit 260 by an application that displays images.

[0110] The display control unit 251 displays windows 270 to 274, 280 to 283 (see FIGS. 3 to 10) and the like on the display unit 260 using the DAW and the AI ​​automatic composition function application. For example, information on the windows 270 to 274, 280 to 283 is transmitted from the interface control unit 131. The display control unit 251 displays newly composed melody information and arranged melody information received from the information processing device 100 by a request unit 253 (described later) in an area 282b of the window 282 and the like.

[0111] The transmitting / receiving unit 252 communicates with the information processing device 100, transmitting and receiving various information. When the automatic composition function is activated, the transmitting / receiving unit 252 receives style information transmitted from the information processing device 100. The transmitting / receiving unit 252 transmits a style information ID selected by the creator to the information processing device 100. The transmitting / receiving unit 252 then receives the style information from the information processing device 100. The transmitting / receiving unit 252 also transmits music information (style information) such as a melody arranged and composed by the creator to the information processing device 100.

[0112] The request unit 253 issues a composition request and an arrangement request to the information processing device 100. For example, when new style information is generated in the window 282 and an instruction for a composition request is received from the producer, the request unit 253 transmits composition request information including chord progression information, bass information, and rhythm information to the information processing device 100. In response to this, the information processing device 100 generates new melody information and transmits the new melody information to the producer terminal 200.

[0113] Furthermore, when the request unit 253 receives instructions from the producer regarding the measures to be arranged and the parameters, it transmits an arrangement request including the melody information, the measures to be arranged, and the parameters to the information processing device 100. In response to this, the information processing device 100 arranges the measures to be arranged in the melody information, and transmits the new arranged melody information to the producer terminal 200.

[0114] The playback unit 254 plays back the music information generated by the information processing device 100. Specifically, the playback unit 254 sets arbitrary instrument information for each of the melody, chords, and bass notes included in the music data, and plays back each piece of data. Note that the playback unit 254 may play back a combination of the melody, chords, and bass notes.

[0115] The control unit 250 also accepts the performance by the producer when the producer performs a performance along with the composition provided by the automatic composition function.The control unit 250 also accepts the producer's arrangement of the composition provided by the automatic composition function and processing related to the production of the music.

[0116] The display unit 260 displays various types of information. The display unit 260 is realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 260 displays various types of information in accordance with the control of the display control unit 251. The display unit 260 can also display information such as an image (interface) provided from the information processing device 100.

[0117] [1-5. Information Processing Procedure According to the Embodiment] Next, various information processing procedures according to the embodiment will be described with reference to Fig. 18. Fig. 18 is a flowchart (1) showing the procedures of the information processing device 100 according to the embodiment. The processing in Fig. 18 corresponds to the processing described with reference to Figs. 3 to 7. The interface control unit 131 of the information processing device 100 starts data communication with the creator terminal 200 and transmits interface information (step S101).

[0118] The interface control unit 131 receives information on the key / scale, tempo, and number of bars selected by the producer from the producer terminal 200 in the window 270 of Fig. 3 (step S102). The interface control unit 131 receives rhythm information input by the producer from the producer terminal 200 in the window 271 of Fig. 4 (step S103). The interface control unit 131 receives chord progression information input by the producer from the producer terminal 200 in the window 272 of Fig. 5 (step S104).

[0119] The interface control unit 131 receives bass information input by the producer from the producer terminal 200 in window 273 of Fig. 6 (step S105). The interface control unit 131 receives melody information input by the producer from the producer terminal 200 in window 274 of Fig. 7 (step S105). If a retry of the melody information is to be performed (step S107, Yes), the process proceeds to step S106. If a retry of the melody information is not to be performed (step S107, No), the process proceeds to step S108.

[0120] The interface control unit 131 executes a data saving process (step S108). In step S108, the interface control unit 131 generates style information based on the key / scale, tempo, number of bars, rhythm information, chord progression information, bass information, melody information, etc. received from the producer terminal 200, and registers the generated style information in the style information table 122. Note that the style information may be generated on the producer terminal 200 side, and the interface control unit 131 may receive the style information.

[0121] If additional recording of melody information is to be performed (step S109, Yes), the process proceeds to step S106. On the other hand, if additional recording of melody information is not to be performed (step S109, No), the interface control unit 131 ends the process.

[0122] Fig. 19 is a flowchart (2) showing the procedure of the information processing device 100 according to the embodiment. The processing in Fig. 19 corresponds to the processing described with reference to Figs. 8 to 10 etc. The interface control unit 131 of the information processing device 100 starts data communication with the producer terminal 200 and transmits interface information (step S201).

[0123] 8, the interface control unit 131 receives the style information ID of the selected style name from the creator terminal 200. The interface control unit 131 transmits the style information corresponding to the style information ID to the creator terminal 200 (step S202).

[0124] When a change to the chord progression information is received in the window 281 of Fig. 9, the interface control unit 131 updates the chord progression information (step S203). When a change to the bass note is received in the window 282 of Fig. 10, the interface control unit 131 updates the bass information (step S204).

[0125] The composition unit 133 of the information processing device 100 executes the AI ​​composition process based on the updated style information (step S205). The interface control unit 131 transmits the AI-composed music information (melody information) to the creator terminal 200 (step S206).

[0126] If the interface control unit 131 has not received an arrangement request from the producer terminal 200 (step S207, No), it proceeds to step S210. On the other hand, if the interface control unit 131 has received an arrangement request from the producer terminal 200 (step S207, Yes), it proceeds to step S208.

[0127] The interface control unit 131 receives the parameters and the designation of the bars to be arranged (step S208). The arrangement unit 134 of the information processing device 100 executes the arrangement process on the bars to be arranged of the melody information (step S209). The interface control unit 131 transmits the arranged music information (melody information) to the producer terminal 200 (step S210).

[0128] The interface control unit 131 registers the new style information in the style information table 122 (step S220). If the interface control unit 131 has received a request to compose another melody information (step S212, Yes), the interface control unit 131 proceeds to step S205. On the other hand, if the interface control unit 131 has not received a request to compose another melody information (step S212, No), the interface control unit 131 ends the process.

[0129] [1-6. Effects of the embodiment] As described above, the information processing device 100 provides the producer terminal 200 with an interface that enables the producer to generate music information by fixing some of the multiple elements that make up music information and editing the remaining elements. The information processing device 100 generates new music information based on multiple pieces of music information created using the interface. For example, the music information corresponds to the score information 740 of the style information 700. The fixed elements are chord progression information, bass information, and rhythm information. The other elements are melody information, and when multiple pieces of melody information are generated, multiple pieces of score information 740 corresponding to the melody information are generated. This allows the producer to easily create style information (multiple pieces of melody information, chord progression information, bass information, and rhythm information), and the information processing device 100 can easily receive the style information.

[0130] The information processing device 100 generates new music information by inputting multiple pieces of music information into the learning model. This makes it easy to create new music information based on multiple pieces of music information. For example, new melody information can be generated based on chord progression information, bass information, and rhythm information included in style information.

[0131] The information processing device 100 retrains the learning model based on multiple pieces of music information created by the creator, thereby generating new music information that meets the creator's preferences.

[0132] The information processing device 100 allows a plurality of style information to be selected, displays them on the producer terminal 200, and visually displays the feature quantities of the melody information, chord progression information, bass information, and rhythm information contained in the selected style information. This allows editing work to be performed based on style information that has already been created.

[0133] When the feature quantities of the melody information, chord progression information, bass information, and rhythm information of the style information are updated, the information processing device 100 registers the updated information as new style information. This makes it possible to easily create new style information from a clone of the style information.

[0134] When a range is specified for melody information in style information and an arrangement request is received, the information processing device 100 performs arrangement processing for the specified range. This makes it easy to create melody information that is an arrangement of existing melody information. The information processing device 100 may also perform arrangement processing when a range is specified not only for melody information but also for chord progression information, bass information, or rhythm information.

[0135] Every time new style information is generated, the information processing device 100 stores the style information in association with the user ID. This allows the information processing device 100 to accumulate multiple pieces of style information generated by the composer through AI composition work.

[0136] The plurality of style information stored in the information processing device 100 can be used in other services such as SNS by linking with the service server 300. For example, the service server 300 can publish the style information, thereby collecting evaluation information on the style information and displaying the evaluation information in association with the style information. Note that accompaniment audio information 760 is set in the style information, allowing for variations in performance.

[0137] [2. Conceptual diagram of information processing system configuration] Here, the functions, hardware configuration, and data in the information processing system are conceptually shown using diagrams. Fig. 20 is a diagram showing an example of a conceptual diagram of the configuration of the information processing system. Specifically, Fig. 20 is a schematic diagram showing an outline of the functions of the information processing system in Fig. 11.

[0138] [2-1. Overall structure] The server device shown in Fig. 20 corresponds to information processing device 100. The system administrator application unit shown in Fig. 20 corresponds to an application installed on a terminal used by a system administrator. The creator mobile application unit shown in Fig. 20 corresponds to creator terminal 200 in the information processing system and corresponds to an application installed on creator terminal 200. The creator plug-in application unit shown in Fig. 20 corresponds to creator terminal 200 in the information processing system and corresponds to an application installed on creator terminal 200. In the example of Fig. 20, one system administrator application unit, one creator mobile application unit, and one creator plug-in application unit are illustrated, but a plurality of these may be included depending on the number of corresponding terminals.

[0139] 20 corresponds to the control unit 130 of the information processing device 100. For example, the learning processing unit of the server device corresponds to the composition unit 133 of the information processing device 100. An internal database unit of the server device corresponds to the storage unit 120 of the information processing device 100.

[0140] The display operation section, control section, DAW application section, mobile display operation section, mobile control section, plug-in display operation section, and plug-in control section of the creator application section shown in FIG.

[0141] The SNS server shown in FIG.

[0142] 20, the server device is connected to a system administrator application unit, a creator mobile application unit, and a creator plug-in application unit via a network N such as the Internet. The server device is also connected to an SNS server.

[0143] [2-2. Server equipment] First, the configuration related to the server device will be described.

[0144] The server device has a control unit, a learning processing unit, and a server database unit. The control unit of the server device has a produced music information management function, a style information management function, and an arrangement information management function. The learning processing unit of the server device has a machine learning processing function and a deep learning processing function.

[0145] [2-3. About each app] Next, the configuration related to the system administrator application section will be described.

[0146] The system administrator application section has a display operation section, a control section, and a DAW application section. The display operation section has the functions of displaying produced music, displaying and editing style information, and displaying arrangement information. The control section has the functions of managing style information and managing arrangement information.

[0147] The DAW application unit has a DAW display and editing function. The DAW application unit is, for example, music editing software (DAW, etc.), and can display, for example, music information using a copyright information display function. If the DAW has, for example, an AI-assisted music production function, new music information can be created using the style information display and editing function. The system administrator application unit also has a similar configuration, but the user's system access rights are different.

[0148] This section explains the mobile app section for creators.

[0149] The creator application unit has a mobile display operation unit and a mobile control unit. The mobile display operation unit has a created music display function, a style information display and editing function, and an arrangement information display function. The mobile control unit has a style information management function and an arrangement information management function.

[0150] The plug-in application section for creators will now be explained.

[0151] The producer's plug-in application section has a plug-in display and operation section, a plug-in control section, and a DAW application section. The plug-in display and operation section has a produced music display function, a style information display and editing function, and an arrangement information display function. The plug-in control section has a style information management function and an arrangement information management function. The DAW application section is the same as the description above regarding the DAW display and editing function.

[0152] [3. Other embodiments] The processing according to the above-described embodiment and modified examples may be implemented in various different forms (modified examples) other than the above-described embodiment and modified examples.

[0153] In the above embodiment, chord progression information, bass information, and rhythm information are set, and when input of melody information is accepted, it is recorded as score information. However, it is possible to save style information all at once by having the DAW display editing unit of the DAW application unit edit the chord progression information, bass information, rhythm information, and multiple pieces of melody information, and then reading out the information from the DAW application unit using the plug-in control unit and uploading it to the control unit 130 of the information processing device 100.

[0154] In the example described in Figure 9, the creator himself edits the chord progression information, but he may also communicate with the information processing device 100 to request an arrangement of the chord progression information and recommend the arranged chord progression information.

[0155] In the above-described embodiment, melody information is arranged, but it is also possible to fix the melody information and vary the chord progression information through arranging processing. For example, a chord progression of |C|Dm|Em|F| can be arranged into a chord progression of |C|FM7|G7|F| through arranging processing.

[0156] In the above-described embodiment, the information processing device 100 provides the score information 740 to the service server 300. However, a composition contest may be held among multiple creators, and general users may submit score information to the contest. The service server 300 may compile evaluation information, rank the contestants, and determine a winner. The service server 300 may also notify the information processing device 100 of the score information submitted to the contest, and the information processing device 100 may use the notified score information to generate and save new style information.

[0157] In the above-described embodiment, information processing device 100 displays window 280 in Fig. 8 to allow the creator to select style information, but there are various other display methods for displaying style information candidates. For example, information processing device 100 may display a list of style information categorized by category, may display a card-format catalog with an image of the style information as a background, or may map style information onto a three-dimensional graphic to allow the user to search for it.

[0158] Fig. 21 is a diagram showing an example of a catalog display. As shown in Fig. 21, in a window 280-1, a card-format catalog with an image of style information as a background is displayed in an area 280-11.

[0159] In the process of mapping onto a three-dimensional graphic and allowing the creator to search, the information processing device 100 plots points corresponding to the style information in a space consisting of X, Y, and Z axes based on the score information of the style information (feature quantities of melody information, chord progression information, bass information, and rhythm information). The X axis indicates dark (minor) or bright (major), with the positive value indicating brighter. The Y axis indicates simple (simple) or complex (tension), with the positive value indicating more complex. The Z axis indicates slow or fast BPM tempo, with the positive value indicating faster tempo.

[0160] When a point plotted on the three-dimensional graphic is selected, the information processing device 100 acquires style information corresponding to the selected point from the style information table 122, and transmits the acquired style information to the creator terminal 200 for display. This allows the creator to visually grasp the characteristics of the style information, select style information that suits their taste, and perform various edits.

[0161] Fig. 22 is a diagram showing an example of mapping using three-dimensional graphics. As shown in Fig. 22, in a window 280-2, points corresponding to the features of style information are set in a three-dimensional space 280-21.

[0162] [4. Hardware Configuration] Information devices such as the information processing device 100 and the creator terminal 200 according to the above-described embodiments and modifications are realized by a computer 1000 configured as shown in FIG. 23, for example. FIG. 23 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100 and the creator terminal 200. The following description will be given taking the information processing device 100 according to the embodiment as an example. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0163] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0164] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000 .

[0165] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0166] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0167] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined recording medium. Examples of media include optical recording media such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disk), magneto-optical recording media such as an MO (Magneto-Optical disk), tape media, magnetic recording media, and semiconductor memories.

[0168] For example, when the computer 1000 functions as the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200, thereby realizing functions of the control unit 130 and the like. The information processing program according to the present disclosure and data in the storage unit 120 are stored in the HDD 1400. The CPU 1100 reads and executes program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from another device via an external network 1550.

[0169] [5. Conclusion] The information processing device is an information processing method executed by a computer, which provides an interface that enables generation of music information by fixing some elements among the multiple elements that form the music information and editing the other elements, and generates new music information based on the multiple pieces of music information created using the interface. This allows a producer to easily create style information (multiple pieces of melody information, chord progression information, bass information, rhythm information), and the information processing device can easily receive the style information.

[0170] The information processing device generates the new music information by inputting the plurality of pieces of music information into a learning model, thereby making it possible to easily create new music information based on the plurality of pieces of music information.

[0171] The information processing device retrains the learning model based on the plurality of pieces of music information, thereby generating new music information that meets the tastes of the creator.

[0172] The information processing device provides the interface and acquires fixed features for some elements and multiple features for other elements included in the multiple pieces of music information, thereby enabling efficient collection and use of multiple pieces of music information.

[0173] The information processing device displays different pieces of music information in a selectable manner through the interface, and when a selection of music information is received from the different pieces of music information, visually displays the feature quantities of the multiple elements that form the selected music information, thereby enabling editing work to be performed based on the style information that has already been created.

[0174] When the feature quantities of the elements forming the selected music information are updated, the information processing device generates new music information having the updated feature quantities, thereby making it possible to easily create new style information from a clone of style information.

[0175] When a range is specified for the feature quantities of elements forming the selected music information, the information processing device arranges the feature quantities included in the range and generates new music information having the arranged feature quantities. This makes it easy to create melody information that is an arrangement of existing music information (melody information, etc.).

[0176] Each time new music information is generated, the information processing device stores the new music information in association with the user's identification information. This allows the AI ​​composition process to accumulate multiple styles of music created by the composer.

[0177] The information processing device maps points corresponding to the music information into a multidimensional space based on the feature quantities contained in the music information, and displays the mapped points, allowing the creator to visually grasp the features of the style information, select style information that suits their taste, and perform various editing operations. [Explanation of symbols]

[0178] 100 Information processing device 110,210 Communications Department 120,240 storage section 121 User information 122 Style Information Table 123 Style Information Management Table 124 Production Music Information Table 125 composition model data 130,250 Control unit 131 Interface control section 132 Machine Learning Department 133 Composition Department 134 Arrangement Section 135 Service Server Collaboration Unit 200 Creator Terminal 220 Input section 230 Output section 241 Operation History Information 251 Display control unit 252 Transmitter / Receiver 253 Request part 254 Playback Department 260 Display

Claims

1. An information processing method for generating music information, comprising: an interface that allows a user to generate a plurality of pieces of music information by editing melody information while keeping the chord progression information, bass information, and rhythm information fixed, among the chord progression information, bass information, rhythm information, and melody information that are elements that form the music information, respectively; Using a plurality of pieces of music information created using the interface as learning data, machine learning of a machine learning model is performed to learn the relationship between the chord progression information, bass information, and rhythm information and the melody information, and new music information is generated by inputting the chord progression information, bass information, and rhythm information into the machine learning model generated by the machine learning. An information processing method that performs processing.

2. The information processing method according to claim 1 , wherein the machine learning model is retrained based on the plurality of pieces of music information.

3. The information processing method according to claim 1 , wherein the interface is provided, and fixed features for the chord progression information, the bass information, and the rhythm information, and a plurality of features for the melody information are acquired.

4. The information processing method according to claim 1, wherein the interface displays different pieces of music information in a selectable manner, and when a selection of music information is received from the different pieces of music information, the characteristics of the multiple elements forming the selected music information are visually displayed.

5. 5. The information processing method according to claim 4, wherein when the feature quantities of the elements forming the selected music information are updated, new music information having the updated feature quantities is generated.

6. An information processing method as described in claim 3, wherein when a measure to be arranged is specified for the features of elements forming selected music information, the features contained in the measure to be arranged are arranged and new music information having the arranged features is generated.

7. 2. The information processing method according to claim 1, wherein each time new music information is generated, the new music information is stored in association with the user's identification information.

8. 4. The information processing method according to claim 3, wherein points corresponding to the music information are mapped onto a multidimensional space based on the feature amounts included in the music information, and the mapped points are displayed.

9. Computer, an interface control unit that provides an interface that can generate a plurality of pieces of music information by editing melody information while keeping the chord progression information, bass information, and rhythm information fixed, among chord progression information, bass information, rhythm information, and melody information, which are elements that form the music information; a composing unit that uses a plurality of pieces of music information created using the interface as learning data to perform machine learning on a machine learning model to learn the relationship between the chord progression information, bass information, and rhythm information and the melody information, and that generates new music information by inputting the chord progression information, bass information, and rhythm information into the machine learning model generated by the machine learning. An information processing program that functions as a

10. An interface control unit that provides an interface capable of generating a plurality of pieces of music information by editing melody information while keeping the chord progression information, bass information, and rhythm information fixed, among chord progression information, bass information, rhythm information, and melody information, which are elements that form the music information; a composition unit that uses a plurality of pieces of music information created using the interface as learning data to perform machine learning on a machine learning model to learn the relationship between the chord progression information, bass information, and rhythm information and the melody information, and that generates new music information by inputting the chord progression information, bass information, and rhythm information into the machine learning model generated by the machine learning; and An information processing device having the above.

Images