Text provision method, program, and text provision device

Abstract

A text providing method according to an embodiment comprises: providing chord input data in which chords are arranged in chronological order to a learned model having learned a relation between chord sequence data in which chords are arranged in chronological order and explanatory text concerning the chords included in the chord sequence data; and acquiring text corresponding to the chord input data from the learned model.

Description

[Technical field]

[0001] The present disclosure relates to a text providing method. [Background technology]

[0002] The impression given to a listener by the multiple chords that make up a piece of music changes depending on their combination (for example, a chord progression arranged in chronological order). A typical listener receives an impression from a piece of music intuitively. The listener can verify this impression by analyzing the piece of music based on music theory, such as chord progressions. For example, Patent Document 1 discloses a technology that detects cadences in a musical score that shows the chord progression of a piece of music, displays arrow symbols in the cadence parts, and uses different colors depending on the type of cadence. The arrow symbols and colors allow the user to recognize the parts of the chords included in the piece of music that correspond to the cadence and the type of cadence. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-56938 A Summary of the Invention [Problem to be solved by the invention]

[0004] There are various types of chord progressions included in music. Understanding the types of chord progressions is an important factor in supporting the impression of a music piece. According to the technology described in Patent Document 1, a user can recognize the part and type of cadence included in the score of a music piece from image information such as arrow symbols and colors. However, unless the user has a certain level of knowledge of music theory, the user cannot understand the meaning from the image information and cannot utilize the obtained information.

[0005] One of the objectives of this disclosure is to provide explanatory text for a number of codes arranged in chronological order. [Means for solving the problem]

[0006] According to one embodiment of the present disclosure, there is provided a text providing method that includes obtaining text corresponding to code input data in which codes are arranged in chronological order, based on a relationship between code string data in which codes are arranged in chronological order and explanatory text regarding the codes included in the code string data. Effect of the Invention

[0007] According to the present disclosure, explanatory text regarding a number of codes arranged in chronological order can be provided. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 illustrates a text providing system according to an embodiment. [Diagram 2] 10 is a flowchart illustrating a text providing process according to an embodiment. [Diagram 3] FIG. 13 is a diagram for explaining a chroma vector representing a code in one embodiment. [Figure 4] FIG. 13 is a diagram for explaining a chroma vector representing a code in one embodiment. [Diagram 5] 13 is a diagram for explaining an example of an explanatory text obtained from code input data. FIG. [Figure 6] 1 is a flowchart illustrating a model generation process according to an embodiment. [Figure 7] FIG. 10 is a diagram for explaining an example of a teacher data set; [Figure 8] FIG. 10 is a diagram for explaining an example of a teacher data set; [Figure 9] FIG. 10 is a diagram for explaining an example of a teacher data set; [Figure 10] FIG. 13 is a diagram for explaining a chord progression detected as two-five-one. [Figure 11] 13 is a diagram for explaining an example of an explanatory text obtained from code input data. FIG. [Figure 12] 13 is a diagram for explaining an example of an explanatory text obtained from code input data. FIG. [Figure 13] FIG. 13 is a diagram for explaining modified examples of a chord progression detected as two-five-one. [Figure 14] FIG. 13 is a diagram for explaining modified examples of a chord progression detected as two-five-one. [Figure 15] FIG. 2 is a diagram illustrating a music database according to an embodiment. [Figure 16] FIG. 13 is a diagram for explaining a method for calculating chord progression importance. [Figure 17] 10 is a flowchart illustrating a process for generating code input data in one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. The embodiment shown below is an example, and the present disclosure is not to be interpreted as being limited to these embodiments. In the drawings referred to in this embodiment, the same parts or parts having similar functions are given the same or similar symbols (symbols with A, B, etc. added after the numbers), and repeated explanations may be omitted.

[0010] [1-1. Text provision system] 1 is a diagram showing a text providing system according to an embodiment. The text providing system 1000 includes a text providing server 1 (text providing device) and a model generation server 3 connected to a network NW such as the Internet. A communication terminal 9 is a smartphone, tablet PC, laptop PC, desktop PC, or the like, and is connected to the network NW to perform data communication with other devices.

[0011] The text providing server 1 receives data on a song from the communication terminal 9 via the network NW, and transmits explanatory text corresponding to the chord progression included in the song to the communication terminal 9. The communication terminal 9 can display the explanatory text on a display. The text providing server 1 generates explanatory text using a trained model obtained by machine learning. When the trained model 155 receives chord input data in which the chords constituting the song are arranged in chronological order, it outputs explanatory text on the chord progression by computation using a neural network. The model generation server 3 executes machine learning processing using a teacher dataset, and generates a trained model to be used in the text providing server 1. The text providing server 1 and the model generation server 3 will be described below.

[0012] [1-2. Text Providing Server] The text providing server 1 includes a control unit 11, a communication unit 13, and a storage unit 15. The control unit 11 includes a CPU (processor), RAM, and ROM. The control unit 11 executes a program stored in the storage unit 15 using the CPU, thereby performing processing according to instructions written in the program. The program includes a program 151 for performing a text providing process, which will be described later.

[0013] The communication unit 13 includes a communication module, and is connected to the network NW to transmit and receive various data to and from other devices.

[0014] The storage unit 15 includes a storage device such as a non-volatile memory, and stores the program 151 and the trained model 155. In addition, various data used in the text providing server 1 is stored. The storage unit 15 may store a music database 159. The music database 159 will be described in another embodiment. The program 151 may be provided to the text providing server 1 in a state stored in a computer-readable recording medium such as a magnetic recording medium, an optical recording medium, a magneto-optical recording medium, or a semiconductor memory, as long as it is executable by a computer. In this case, the text providing server 1 may be provided with a device for reading the recording medium. The program 151 may be provided by downloading it via the communication unit 13.

[0015] The trained model 155 is generated by machine learning in the model generation server 3 and provided to the text providing server 1. When code input data is provided, the trained model 155 outputs explanatory text about the code by computation using a neural network. In this example, the trained model 155 is a model that uses a recurrent neural network (RNN). The trained model 155 uses Seq2Seq (Sequence To Sequence), that is, includes an encoder and a decoder, which will be described later. The code input data and explanatory text are examples of data described in a time series, and details will be described later. For this reason, it is preferable that the trained model 155 adopts a model that is advantageous for handling time series data.

[0016] The trained model 155 may be a model using a Long Short Term Memory (LSTM) or a Gated Recurrent Unit (GRU). The trained model 155 may be a model using a Convolutional Neural Network (CNN), Attention (Self-Attention, Source Target Attention), or the like. The trained model 155 may be a model combining a plurality of models. The trained model 155 may be stored in another device connected via a network NW. In this case, the text providing server 1 may be connected to the trained model 155 via the network NW.

[0017] [1-3. Model generation server] The model generation server 3 includes a control unit 31, a communication unit 33, and a storage unit 35. The control unit 31 includes a CPU (processor), a RAM, and a ROM. The control unit 31 executes a program stored in the storage unit 35 using the CPU, thereby performing processing according to instructions described in the program. This program includes a program 351 for performing a model generation process, which will be described later. The model generation process is a process for generating a trained model 155 using a teacher dataset.

[0018] The communication unit 33 includes a communication module, and is connected to the network NW to transmit and receive various data to and from other devices.

[0019] The storage unit 35 includes a storage device such as a non-volatile memory, and stores the program 351 and the teacher dataset 355. In addition, various data used in the model generation server 3 is stored. The program 351 may be provided to the model generation server 3 in a state stored in a computer-readable recording medium such as a magnetic recording medium, an optical recording medium, a magneto-optical recording medium, or a semiconductor memory, as long as it is executable by a computer. In this case, the text providing server 1 may be provided with a device for reading the recording medium. The program 351 may be provided by downloading it via the communication unit 33.

[0020] The storage unit 35 may store a plurality of teacher datasets 355. The teacher dataset 355 is data in which the code string data 357 and the explanatory sentence data 359 are associated with each other, and is used when generating the trained model 155. Details of the teacher dataset 355 will be described later.

[0021] [2. Text provision processing] Next, a description will be given of a text providing process (text providing method) executed by the control unit 11 in the text providing server 1. The text providing process is started in response to a request from the communication terminal 9, for example.

[0022] FIG. 2 is a flowchart showing a text providing process in an embodiment. The control unit 11 waits until it receives music code data from the communication terminal 9 (step S101; No). The music code data is data in which a plurality of chords constituting a music piece are arranged in chronological order. For example, the music code data is described as, for example, "CM7-Dm7-Em7-...". When arranged in chronological order, each chord may be arranged in a predetermined unit period (for example, one measure, one beat, etc.) or may be arranged in order without considering the unit period. For example, assuming that each chord is arranged in a unit of one measure in the above example, when the first chord in the above example continues for two measures, the music code data is described as "CM7-CM7-Dm7...". On the other hand, assuming that the number of measures is not considered, the music code data is described as "CM7-Dm7-..." as in the above example.

[0023] When a user operates the communication terminal 9 to instruct transmission of music code data, the communication terminal 9 transmits the music code data to the text providing server 1. When the text providing server 1 receives the music code data, the control unit 11 generates code input data from the music code data (step S103). The code input data is written by converting each code included in the music code data into a predetermined format. Specifically, the code input data is data in which each code is described by a chroma vector.

[0024] 3 and 4 are diagrams for explaining chroma vectors representing chords in one embodiment. As shown in FIG. 3 and FIG. 4, the chroma vectors are described by "1" indicating the presence or absence of a note corresponding to each note name (C, C#, D, ...) or "0" indicating the absence of a note. In this example, each chord is converted into data (hereinafter referred to as conversion data) that combines a chroma vector corresponding to a constituent note, a chroma vector corresponding to a bass note, and a chroma vector corresponding to a tension note. In this example, the conversion data is data in which three chroma vectors are described as matrix data (3 x 12). The conversion data may be described as vector data in which three chroma vectors are connected in series.

[0025] FIG. 3 is an example of the chord "CM7" represented as conversion data. FIG. 4 is an example of the chord "C / B" represented as conversion data. "CM7" and "C / B" have the same constituent tones, but different bass tones and tension tones. Therefore, the conversion data can distinguish between "CM7" and "C / B". In other words, the conversion data can clearly represent the function of the chord. The conversion data only needs to include at least the chroma vectors of the constituent tones, and does not need to include at least one or both of the bass tones and tension tones. The structure of the conversion data can be set appropriately depending on the desired result.

[0026] The chord input data is data in which conversion data is arranged in chronological order. As in the above example, if the music chord data is "CM7-Dm7-...", the chord input data is written as data in which the conversion data corresponding to "CM7", the conversion data corresponding to "Dm7", and so on are arranged in that order.

[0027] Returning to FIG. 2, the explanation will be continued. The control unit 11 provides the chord input data to the trained model 155 (step S105). The control unit 11 executes calculation processing by the trained model 155 (step S107), and acquires text output data from the trained model 155 (step S109). The control unit 11 transmits the acquired text output data to the communication terminal 9 (step S111). The text output data corresponds to the above-mentioned explanatory text, and includes a character group indicating an explanation regarding the chord defined by the chord input data. The explanatory text includes at least one of a first character group explaining the chord progression, a second character group explaining the function of the chord, and a third character group explaining the connection technique between the chords. In this example, the explanatory text includes the first character group, the second character group, and the third character group.

[0028] FIG. 5 is a diagram for explaining an example of an explanatory sentence obtained from code input data. The trained model 155 includes an encoder (also called an input layer) that generates intermediate state data by computing the provided code input data with an RNN, and a decoder (also called an output layer) that outputs sentence output data by computing the intermediate state data with an RNN. More specifically, the encoder is provided with a plurality of conversion data included in the code input data in chronological order. The decoder outputs a plurality of characters (character group) arranged in chronological order as explanatory sentence. The character here may mean a word (morpheme) classified by morphological analysis. The intermediate state may also be called a hidden state or a hidden layer.

[0029] The chord input data shown in Fig. 5 is shown as music chord data "CM7-Dm7-Em7-...", but as described above, each chord is described as conversion data. In the chord input data, an end marker (EOS: End Of Sequence) is added to the part where the chord ends. When this chord input data is provided to the trained model 155, the trained model 155 outputs text output data including the explanatory text exemplified in Fig. 5.

[0030] According to the code input data shown in FIG. 5, the text output data, i.e., the explanatory text, is composed of a combination of the following character groups. "In the first half, the diatonic chords ascend in sequence, forming a two-five between Fm7 and Bb7. Bb7 functions as a substitute chord for the subdominant minor chord Fm7, while it also functions as a back chord for the dominant 7th chord E7 that follows Am7. In the second half, the substitute chord Am7 for the tonic chord CM7 and the substitute chord AbM7 for the subdominant minor chord Fm7 are repeated, resulting in a progression in which the root note goes up and down by a semitone while the 3rd and 7th notes remain the same."

[0031] In the explanatory text shown in Figure 5, the first group of letters (explaining the chord progression) corresponds to "a two-five is formed between Fm7 and Bb7."

[0032] The second group of letters (explaining the function of the chords) in the explanatory text shown in Figure 5 corresponds to "Bb7 functions as a substitute chord for the subdominant minor chord Fm7", "Bb7 functions as a hidden chord for the dominant 7th chord E7 for the following Am7", "Am7 as a substitute chord for the tonic chord CM7", and "AbM7 as a substitute chord for the subdominant minor chord Fm7". In fact, the explanatory text combines the explanations of the two functions of Bb7 into one, expressing it as "Bb7 functions as..., while it functions as... for the following Am7."

[0033] The third group of characters (explanation of the connecting technique between chords) in the explanatory text shown in Figure 5 corresponds to "The diatonic chords ascend sequentially," and "By repeating Am7 and AbM7, the root note goes up and down by a semitone while the 3rd and 7th notes are kept the same." In fact, in the explanatory text, the ascending of the diatonic chords sequentially is expressed as "The diatonic chords ascend sequentially, ..." to connect with the next sentence.

[0034] The text output data obtained in this manner is transmitted to the communication terminal 9 that transmitted the music code data. As a result, an explanatory text corresponding to the music code data is provided to the user of the communication terminal 9. This concludes the description of the text providing process.

[0035] [3. Model generation process] Next, a description will be given of the model generation process (model generation method) executed by the control unit 31 in the model generation server 3. The model generation process is started in response to a request from a terminal or the like used by an administrator of the model generation server 3. The model generation process may also be started in response to a user request, i.e., a request from the communication terminal 9.

[0036] 6 is a flowchart showing a model generation process in one embodiment. The control unit 31 acquires the teacher data set 355 from the storage unit 35 (step S301). As described above, the teacher data set 355 includes code string data 357 and commentary sentence data 359 that are associated with each other. The code string data 357 is written in the same format as the code input data. That is, the code string data 357 is written as data in which the codes represented by the conversion data are arranged in chronological order.

[0037] The explanatory text data 359 is data including explanatory text as shown in FIG. 5. This explanatory text is text that explains the chord defined by the chord string data 357. As described above, the explanatory text includes at least one of a first character group that explains the chord progression, a second character group that explains the function of the chord, and a third character group that explains the connecting technique between the chords. In this example, the explanatory text data 359 is provided with an identifier for identifying words that are obtained by dividing the explanatory text by morphological analysis. Each word is described by a "One Hot Vector". The analyzed text may be described by a word representation such as "word2vec" or "GloVe".

[0038] In this example, the chord string data 357 included in the teacher data set 355 includes a sequence of chords corresponding to one piece of music, and is marked with at least one end marker EOS. The teacher data set 355 may take a variety of forms. Several examples of the teacher data set 355 will be described with reference to Figures 7 to 9.

[0039] 7 to 9 are diagrams illustrating an example of a teacher data set. In teacher data set 355 described here, chord string data 357 corresponding to chords in a song is shown in multiple sections (music sections CL(A) to CL(E)). Here, music sections CL(A) to CL(E) each correspond to a divided range of a phrase or the like that makes up a song, for example, a range of 8 bars, and each includes multiple chords arranged in chronological order. Each music section does not have to be the same length as the other music sections.

[0040] The chord string data 357 shown in FIG. 7 has a format in which chords corresponding to the music sections CL(A) to CL(E) are written in a series, and includes an end marker EOS only at the end of the data.

[0041] The chord string data 357 in Fig. 8 has a format in which the chords corresponding to the music sections CL(A)-CL(E) are written by dividing them into each music section. An end marker EOS is written at the division position. A section divided by an end marker EOS is called a division area. One division area may contain multiple music sections. However, in this example, one music section is not included in multiple division areas.

[0042] The chord string data 357 in Fig. 9 has a format in which the chords corresponding to the music sections CL(A)-CL(E) as shown in Fig. 8 are divided into each music section, and then the chords of the music sections before and after the music section are added to each divided area. That is, in the chord string data 357 in Fig. 9, multiple consecutive music sections are arranged in one divided area, and at least one music section is included in multiple divided areas. In this example, three consecutive music sections are arranged in each divided area, and only the first and last divided areas have two consecutive music sections. The number of consecutive music sections is not limited to this example.

[0043] The explanatory text data 359 includes explanatory text ED(A)-ED(E) corresponding to the music sections CL(A)-CL(E), respectively. For example, the explanatory text ED(A) includes a group of characters explaining the chord corresponding to the music section CL(A). The explanatory text data 359 shown in Figures 8 and 9 is divided by an end marker EOS, similar to the chord string data 357.

[0044] Returning to Fig. 6, the explanation will be continued. The control unit 31 inputs the code string data 357 to a model for machine learning (herein referred to as a training model) (step S303). The training model is a model that performs calculation processing using the same neural network (RNN in this example) as the trained model 155. The training model may be the trained model 155 stored in the text providing server 1.

[0045] The control unit 31 executes machine learning by backpropagation using the values ​​output from the training model in response to the input of the code string data and the commentary sentence data 359 (step S305). Specifically, the weight coefficients in the neural network of the training model are updated by machine learning. If there are other teacher data sets 355 to learn from (step S307; Yes), machine learning is executed using the remaining teacher data sets 355 (steps S301, S303, S305). If there are no other teacher data sets 355 to learn from (step S307; No), the control unit 31 ends the machine learning.

[0046] Control unit 31 generates a training model that has undergone machine learning as a learned model (step S309), and ends the model generation process. The generated learned model is provided to text providing server 1 and used as learned model 155. In this way, learned model 155 is a model that has learned the correlation between the codes defined in code string data 357 and the explanatory text related to those codes.

[0047] When the chord string data 357 input in machine learning includes an end marker EOS in the middle of the data as in Figures 8 and 9, the control unit 31 resets the intermediate state at the point of the end marker EOS. That is, in machine learning, chords in a specific divided area and chords in other divided areas divided from that area are not treated as continuous time-series data. In the example shown in Figure 8, chords in a specific music section and chords in music sections other than that music section are treated as time-series data independent of each other. On the other hand, chords included in one music section are treated as time-series data.

[0048] 9, music sections that are separate from each other, for example, music sections CL(B) and CL(E), are not included in the same division area, but are treated as independent time series data. Meanwhile, music sections CL(B) and CL(C) may be included in the same division area, or in different division areas. Therefore, depending on the division area, the chords of music section CL(B) and music section CL(C) may be treated as a series of time series data, or as independent time series data.

[0049] The three exemplary teacher datasets are summarized as follows. The first example is a teacher dataset in which no divided regions are set, as shown in FIG. 7. The second example is a teacher dataset in which multiple divided regions are set, as shown in FIG. 8, and none of the music sections are included in the multiple divided regions. The third example is a dataset in which multiple divided regions are set, as shown in FIG. 9, and at least one music section is included in the multiple divided regions.

[0050] In particular, as in the first and third examples, by increasing the number of codes treated as time-series data, it is possible to realize highly accurate machine learning that broadly considers the context of the code sequence. As in the third example, by narrowing the scope of the context, it is possible to exclude parts that are too far apart from each other and have a weak relationship from the machine learning target, thereby realizing more accurate machine learning. In machine learning, only one of these examples may be used, or multiple examples may be used in combination.

[0051] [4. Code interpretation example] Next, we will explain in more detail the correlation between the interpretation of chords and the explanatory text. Here, we will explain an example where two-five-one (II-VI) is detected as a typical chord progression.

[0052] FIG. 10 is a diagram for explaining a chord progression detected as two-five-one. In FIG. 10, examples of two-five-one chord progressions within the scale of Cmaj or Amin (basic form, derived form) and other examples outside the scale (alternate chord, postponed solution) are shown. "Alternate chord" means that an alternate chord is used as part of the chord progression. Derived forms and alternate chords corresponding to the basic form are surrounded by a single range of dashed lines. In the derived forms and alternate chords, the parts that differ from the basic form are underlined. "Postponed solution" is an example that takes the form of two-five-one, but is changed by inserting the chord shown in parentheses.

[0053] The trained model 155 generated by the above-mentioned machine learning can output explanatory text that two-five-one exists even in a chord progression expressed other than the basic form. In the sequence of chords that make up a piece of music, a chord progression that corresponds to two-five-one may be included by chance without intending to include two-five-one. Even in such a case, the trained model 155 generated by machine learning including the context can output explanatory text that takes into account whether the chord progression corresponds to two-five-one or not.

[0054] Figures 11 and 12 are diagrams for explaining examples of explanatory text obtained from chord input data. Both Figures 11 and 12 have the chord sequence "Em7-A7-GbM7-Ab7", but in Figure 12, DbM7 is added to the end. In other words, the chord located at the end of the time series by the end marker EOS is Ab7 in Figure 11, whereas it is DbM7 in Figure 12, which is different.

[0055] The trained model 155 infers that the element “Em7-A7-Ab7” in the chord input data shown in FIG. 11 is related to two-five-one, and outputs the following explanatory text as text output data. "Em7-A7-Ab7 is a diatonic chord of the Cmaj scale, which is a derivative of II-VI (Em7-A7-Dm7) when it is likened to the Dmaj scale, with Dm7 changed to a back chord. GbM7 is a two-five for Ab7 in Dbmaj, and is inserted to temporarily delay the resolution to Ab7."

[0056] On the other hand, the trained model 155 infers that the elements “GbM7-Ab7-DbM7” in the chord input data shown in FIG. 12 are related to two-five-one, and further outputs the following explanatory text, which also mentions the elements “Em7-A7-Ab7,” as text output data. "The key temporarily changes from Cmaj to Dmaj. GbM7-Ab7-DbM7 is the Dbmaj scale, II-VI (Ebm7-Ab7-DbM7) with the II changed to the same subdominant IV (Ebm7→GbM7). To make the key change smoother, a kind of two-five-one is incorporated using the hidden chords Em7-A7-Ab7."

[0057] In this way, by performing machine learning as described above using many training data sets 355, even if the code sequences contained in the code input data are similar, the trained model 155 can output text output data containing appropriate explanatory text by taking into account the context of the similar parts.

[0058] 13 and 14 are diagrams for explaining modified chord progressions detected as two-five-one. In the example shown in Fig. 13, when a linking technique called a descending baseline is applied to the basic form of the two-five-one chord progression "Bm7(-5)-E7-Am7", the chord sequence becomes, for example, "Bm7(-5)-Bm7(-5) / F-E7-E7 / G#-Am7". Even in this case, it is possible to recognize that the chord progression is two-five-one without being affected by the change in the baseline.

[0059] In the example shown in Figure 14, when the passing diminished technique is applied to the basic chord progression of two-five-one, "Dm7-Db7-CM7," the chord sequence becomes, for example, "Dm7-Ddim7-Db7-CM7." Even in this case, it is possible to recognize that it is a two-five-one chord progression, regardless of the addition of Ddim7.

[0060] [5. Extraction of specific sections] In the above-described embodiment, the chord input data may specify the sequence of all chords included in the music chord data, or may specify the sequence of a part of chords extracted from the music chord data. In the following description, the section of the music corresponding to the chords included in the chord input data is referred to as a specific section. The specific section may be set by the user, or may be set by a predetermined method, the example of which is given below.

[0061] An example of the predetermined method will be described. The chord input data provided to the trained model 155 does not have to be all of the music chord data, and if a characteristic part of the music can be used, an explanatory text characteristic of the music can be obtained. Therefore, it is preferable to set such a characteristic part of the music as a specific section. The characteristic part of the music can be set by various methods, and an example will be described.

[0062] In the example described here, the control unit 11 divides a song into a plurality of predetermined judgment sections (for example, the above-mentioned song sections) and sets the judgment sections that satisfy a predetermined condition as specific sections. In this example, the chord progression importance of each judgment section is calculated, and the judgment sections that have a chord progression importance exceeding a predetermined threshold are set as specific sections.

[0063] The chord progression importance is calculated based on various data registered in the music database 159 and the chord progression in the determination section. An example of this calculation method will be described.

[0064] 15 is a diagram for explaining a music database in one embodiment. The music database 159 is stored, for example, in the storage unit 15 of the text providing server 1. The music database 159 registers information about a plurality of music pieces, such as genre information, scale information, chord appearance rate data, and chord progression appearance rate data that are associated with each other.

[0065] Genre information is information that indicates the genre of music, for example, "rock", "pops", "jazz", etc. Scale information is information that indicates scales (including keys in this example), such as "C major scale", "C minor scale", "C# major scale", etc. Each scale has set notes that make up the scale (hereinafter referred to as scale constituent notes).

[0066] The chord appearance rate data indicates the ratio of each type of chord to the total number of chords in all songs registered in the song database. For example, if the total number of chords is "10,000" and the number of the chord "Cm" is "100," the appearance rate of that chord is "0.01."

[0067] In calculating the chord appearance rate, the identity of similar chords may be determined using any of the following criteria: If the chord names are different, they may be treated as different chords ("CM7" is different from "C / B"); If the constituent notes are the same, they may be treated as the same chord ("CM7" is the same as "C / B"); If the constituent notes and bass notes are the same, they may be treated as the same chord ("CM7" is the same as "G / C"); Even if the constituent notes are different, they may be treated as the same chord if everything except the tension notes is the same ("CM7" is the same as "C").

[0068] The chord progression appearance rate data indicates the ratio of each type of chord progression to the total number of chord progressions in all songs registered in the song database. The chord progressions here are set in advance by the user, etc. For example, if the total number of chord progressions is "20,000" and the number of chord progressions "Dm-G7-CM7" is "400," the appearance rate of the chord progression is "0.02."

[0069] The criteria for determining whether chords are the same may be the same as the method for determining the chord appearance rate described above. The criteria for determining whether chord progressions are the same may be any of the criteria listed below. Chord progressions that are similar to each other may be treated as the same chord progression. For example, derived forms and forms that use hidden chords for the basic form shown in Figure 10 may be treated as the same chord progression.

[0070] Chord progressions that have at least two identical chord progressions may be treated as the same chord progression. For example, if the chord progression is "Dm-G7-CM7", "*-G7-CM7", "Dm-*-CM7" and "Dm-G7-*" may be treated as the same chord progression. Here, "*" indicates an unspecified chord (any of all chords).

[0071] The chord appearance rate data and the chord progression appearance rate data include data for all songs. In this example, the chord appearance rate data and the chord progression appearance rate data further include data defined corresponding to each genre defined in the genre information. For example, the chord appearance rate data and the chord progression appearance rate data corresponding to the genre "rock" may include the appearance rates of chords and the appearance rates of chord progressions obtained only from songs corresponding to the genre "rock". The parameters of the appearance rate (the total number of chords and the total number of chord progressions) may be data for all songs.

[0072] The frequency of occurrence of chords and chord progressions in the genre "rock" is different from that in the genre "jazz". Therefore, by having the frequency of occurrence of chords and the frequency of occurrence of chord progressions for each genre, the characteristic parts of a song can be determined more accurately. Genre information does not necessarily need to be used, and in this case, the frequency of occurrence of chords and the frequency of occurrence of chord progressions for each genre may not be present.

[0073] FIG. 16 is a diagram for explaining a method for calculating chord progression importance. The example shown in FIG. 16 shows each index value and importance when the chord progression in the judgment section is "C-Cm-CM7-Cm7". The index value includes the chord progression rarity (CP) determined for the chord progression, the scale element (S) determined for each chord that constitutes the chord progression, and the chord rarity (C). Based on these indexes, the chord importance (CS) for each chord and the chord progression importance (CPS) for the chord progression are calculated. Both the index value and the importance have values ​​ranging from "0" to "1". The higher the value, the more characteristic the element is.

[0074] In this example, the key of the song is C, the scale is major scale, and the genre is pops. These pieces of information may be set in advance by the user, or may be set by analyzing the song chord data. When analyzing the song chord data, for example, the song chord may be set in relation to similar chords by comparison with songs registered in the song database 159, or may be estimated from the sequence of chords using a trained model obtained by machine learning or the like.

[0075] The scale element (S) is set to "0" if all of the constituent tones of the chord are included in the scale constituent tones, and is set to "1" if any of the constituent tones of the chord are not included in the scale constituent tones. This is because chords that include tones that are not included in the scale constituent tones can be said to be characteristic parts of a piece of music.

[0076] Chord rarity (C) is calculated by a predetermined formula. The formula is set so that the higher the chord occurrence rate, the smaller the chord rarity (C). In the case of the C major scale, C and CM7 have a relatively high chord occurrence rate, so the chord rarity (C) is set to a relatively small value.

[0077] Chord progression rarity (CP) is calculated using a predefined formula. The formula is set so that the higher the frequency of occurrence of a chord progression, the lower the chord progression rarity (CP). In this example, the frequency of occurrence of the chord progression "C-Cm-CM7-Cm7" is extremely low, so the chord progression rarity (CP) is set to a high value of "1."

[0078] Chord importance (CS) is calculated using scale factor (S), chord rarity (C) and chord progression rarity (CP). In this example, the formula is CS=a×S+b×C+c×CP, where a=1 / 4, b=1 / 4, c=1 / 2. Chord progression importance (CPS) is the average of chord importance (CS).

[0079] The higher the chord progression importance score (CPS) obtained in this way (the closer to "1"), the more unusual the chord progression is compared to other songs. In other words, a section with a high chord progression importance score (CPS) can be said to be a distinctive part of a song.

[0080] The above method of calculating the index value and importance is an example, and various calculation methods can be used as long as the importance of the chord progression as a whole (characteristic parts of the music) can be obtained. Next, a method of generating chord input data using a specific section will be described. This method of generating chord input data can be replaced with the process in step S103 shown in FIG. 2, for example.

[0081] 17 is a flowchart showing a process for generating chord input data in one embodiment. The control unit 11 sets the key, scale, and genre (step S1031). As described above, the key, scale, and genre may be acquired by receiving them from the communication terminal 9 according to user settings, or may be acquired by analyzing music chord data. The control unit 11 divides the music into a plurality of judgment sections (step S1033), and calculates the chord progression importance (CPS) of each judgment section (step S1035).

[0082] The control unit 11 sets at least one judgment section as a specific section based on the chord progression importance (CPS) calculated for each judgment section (step S1037). Here, a judgment section with a chord progression importance (CPS) greater than a predetermined threshold is set as the specific section. A predetermined number of judgment sections may be set as the specific sections in descending order of the chord progression importance (CPS) of the judgment section with the largest chord progression importance (CPS).

[0083] The control unit 11 generates code input data corresponding to the specific interval (step S1039). In the code input data, an end marker EOS may be placed for each specific interval so that one specific interval is placed in one divided area, or when multiple consecutive determination intervals are set as multiple specific intervals, the multiple specific intervals may be placed in one divided area.

[0084] In this way, by providing the generated chord input data to the trained model 155, the trained model 155 can generate explanatory text for the chord progression that represents the characteristic parts of the song and output text output data.

[0085] [6. Modifications] The present disclosure is not limited to the above-described embodiment, and includes various other modified examples. For example, the above-described embodiment has been described in detail to clearly explain the present disclosure, and is not necessarily limited to an embodiment having all of the configurations described. Other configurations may be added to, deleted from, or substituted for part of the configuration of the embodiment. Some modified examples will be described below.

[0086] (1) In the above-described embodiment, the text providing server 1 uses the trained model 155 to generate explanatory text from the code input data, but a model that does not use a neural network (e.g., a rule-based model) may be used. According to the trained model 155, the accuracy of explanatory text can be improved by using many teacher data sets 355 for machine learning.

[0087] According to the rule-based model, it is necessary to set a rule for generating explanatory text from chord input data, that is, a correspondence between information corresponding to the chord string data 357 and information corresponding to the explanatory text data 359. This rule requires a large amount of information. For example, as described above, various types of chord sequences that are determined to be the chord progression of two-five-one are assumed. Therefore, in order to improve the accuracy of the explanatory text, it is necessary to set explanatory texts corresponding to the many types that can be assumed. In order to reduce the amount of information, it may be necessary to simplify the explanatory text compared to when the trained model 155 is used. Although it is expected that the efficiency may be lower than when the trained model 155 is used, it is possible to generate explanatory text from chord input data using the rule-based model.

[0088] (2) The chord appearance rate data and the chord progression appearance rate data may be defined to be equivalent regardless of the key of the song. For example, the chord appearance rate data may be interpreted such that the chord "CM7" when the song is in the key of "C" and the chord "EM7" when the key is "E" are the same chord. The chord progression appearance rate data may be interpreted such that the chord progression "Dm-G7-CM7" when the song is in the key of "C" and the chord progression "Fm-B7-EM7" when the key is "E" are the same chord progression.

[0089] In other words, both the chord appearance rate data and the chord progression appearance rate data may be defined by a chord that is expressed relative to the key of the song. The relative expression may be, for example, a conversion when the key is "C", or may be converted to descriptions such as "I" and "II". For example, the chord "Em7" in the key of "C" is expressed as "IIIm7".

[0090] In this case, the control unit 11 converts the chord appearance rate data and chord progression appearance rate data defined by the chords of relative expression into chords of absolute expression based on the set key of the music piece. The control unit 11 calculates the chord importance (CS) and chord progression importance (CPS) based on the appearance rate of the converted chords.

[0091] (3) The text providing server 1 may use a computation model such as SVM (Support Vector Machine) or HMM (Hidden Markov Model) instead of using the trained model 155. In this case, the control unit 11 uses this computation model to acquire a specific chord progression, for example, "two-five-one", from the chord input data. The control unit 11 generates an explanatory text by combining the acquired chord progression with a predetermined template. The predetermined template is, for example, "XXXX is used in this chord progression." By using the acquired chord progression ("two-five-one" in the above example) in the "XXXX" part, an explanatory text saying "Two-five-one is used in this chord progression" is generated. In the case of HMM, the chords included in the chord input data may be input sequentially. In the case of SVM, a predetermined number of chords included in the chord input data may be input all at once.

[0092] (4) A server storing a plurality of trained models 155 may be connected to the network NW. This server may be the model generation server 3. The text providing server 1 may select one of the plurality of trained models 155 stored in this server and execute the above-mentioned text providing process. The text providing server 1 may download the trained model 155 used in the text providing process from the server and store it in the storage unit 15, or may transmit code input data and receive text output data by communicating with the server that stores the trained model 155 without downloading it.

[0093] The multiple trained models 155 differ from each other in at least some of the teacher data sets 355 used during machine learning. For example, when machine learning is performed using multiple teacher data sets 355 classified by genre (jazz, classical, etc.), multiple trained models 155 corresponding to multiple genres are generated. The teacher data sets 355 may be classified by genre type or by instrument type. According to this classification, the chord string data and the explanatory text data are specialized for that classification. The teacher data set 355 may be classified by the creator of the explanatory text included in the explanatory text data 359.

[0094] For example, highly accurate explanatory text can be obtained by providing chord input data corresponding to a piece of music classified as jazz to the trained model 155 corresponding to jazz. The target into which the piece of music corresponding to the chord input data is classified may be set by the user or may be set by analyzing the piece of music.

[0095] A single code input data may be provided to a plurality of trained models 155 to obtain a plurality of types of explanatory text. For example, by using a plurality of trained models 155 corresponding to a plurality of creators, the obtained plurality of types of explanatory text can be compared to select the one suitable for the user. A new explanatory text may be generated based on commonalities among the explanatory texts obtained from the plurality of trained models 155.

[0096] (5) The chord input data and chord string data 357 are not limited to being described as chroma vectors. For example, as long as the constituent notes of a chord are represented by data including vectors, they may be represented in other ways. Chords may also be described using expressions that apply "word2vec," "GloVe," or the like.

[0097] The above is a description of the modified example.

[0098] As described above, according to one embodiment of the present disclosure, a text providing method is provided that includes obtaining text corresponding to code input data in which codes are arranged in chronological order, based on the relationship between code string data in which codes are arranged in chronological order and explanatory text regarding the codes included in the code string data.

[0099] Obtaining the sentence may include obtaining the sentence from a trained model by providing the code data to the trained model that has learned the relationship.

[0100] The chords included in the chord string data may include at least constituent notes of the chord and a bass note.

[0101] The chords included in the chord string data may include at least constituent notes of the chord and tension notes of the chord.

[0102] The code may be represented by data including a vector.

[0103] The chord may be represented by data including a first chroma vector corresponding to the constituent notes of the chord.

[0104] The chord may be represented by data including a second chroma vector corresponding to a base note of the chord.

[0105] The chord may be represented by data including a third chroma vector corresponding to a tension note of the chord.

[0106] The explanatory text may include a first character group that explains the chord progression.

[0107] The explanatory text may include a second group of characters that explains the function of the code.

[0108] The explanatory text may include a third character group that explains the linking technique between the codes.

[0109] The method may include obtaining music code data in which the chords of a song are arranged in chronological order, and extracting, from the music code data, a sequence of the chords in a specific section that satisfies a predetermined condition as the chord input data.

[0110] The predetermined condition may include a condition using the chord included in the music code data and an importance of the chord determined according to a key of the music.

[0111] The predetermined condition may include a condition using the chord included in the music code data and an importance of the chord determined according to the genre of the music.

[0112] A program for causing a computer to execute the text providing method may be provided. A text providing device may be provided, the device including a memory unit for storing instructions of the program and a processor for executing the instructions. [Explanation of symbols]

[0113] 1...text providing server, 11...control unit, 13...communication unit, 15...storage unit, 151...program, 155...trained model, 159...music database, 3...model generation server, 31...control unit, 33...communication unit, 35...storage unit, 351...program, 355...teacher data set, 357...code string data, 359...explanatory text data, 9...communication terminal, 1000...text providing system

Claims

1. providing code input data in which codes are arranged in chronological order to a trained model that has been trained to learn the relationship between code string data in which codes are arranged in chronological order and explanatory text regarding the codes included in the code string data; Obtaining a sentence corresponding to the code input data from the trained model. A method of providing text, including:

2. The text providing method according to claim 1 , wherein the code is represented by data including a vector.

3. The text providing method according to claim 1 or 2, wherein the chord is represented by data including a first chroma vector corresponding to a constituent note of the chord.

4. The text providing method according to claim 1 , wherein the chord is represented by data including a second chroma vector corresponding to a base note of the chord.

5. 5. The text providing method according to claim 1, wherein the chord is represented by data including a third chroma vector corresponding to a tension note of the chord.

6. The text providing method according to claim 1 , wherein the explanatory text includes a first character group explaining a chord progression.

7. The text providing method according to claim 1 , wherein the explanatory text includes a second character group explaining a function of the code.

8. The text providing method according to claim 1 , wherein the explanatory text includes a third character group explaining a connection technique between codes.

9. Acquire song code data in which the codes of the song are arranged in chronological order; extracting, from the music chord data, a sequence of the chords in a specific section that satisfies a predetermined condition as the chord input data; Further comprising: The text providing method according to any one of claims 1 to 8.

10. the predetermined condition includes a condition using the chord included in the music code data and an importance of the chord determined according to a key of the music. The text providing method according to claim 9.

11. the predetermined condition includes a condition using the chord included in the music code data and an importance of the chord determined according to the genre of the music; The text providing method according to claim 9 or 10.

12. Acquiring text corresponding to code input data in which codes are arranged in chronological order based on a relationship between code string data in which codes are arranged in chronological order and explanatory text regarding the codes included in the code string data A program for causing a computer to execute the above, The program, wherein obtaining the sentence includes obtaining the sentence from a trained model that has learned the relationship by providing the code input data to the trained model.

13. 13. The program according to claim 12, wherein the chords included in the chord string data include at least constituent notes of the chord and a bass note.

14. 14. The program according to claim 12, wherein the chords included in the chord string data include at least constituent notes of the chords and tension notes of the chords.

15. The program according to claim 12 , wherein the code is represented by data including a vector.

16. 16. The program according to claim 12, wherein the chord is represented by data including a first chroma vector corresponding to a constituent note of the chord.

17. 17. The computer-readable medium according to claim 12, wherein the chord is represented by data including a second chroma vector corresponding to a bass note of the chord.

18. 18. The program according to claim 12, wherein the chord is represented by data including a third chroma vector corresponding to a tension note of the chord.

19. 19. The program according to claim 12, wherein the explanatory text includes a first character group explaining a chord progression.

20. a control unit that acquires text corresponding to code input data in which codes are arranged in chronological order based on a relationship between code string data in which codes are arranged in chronological order and explanatory text relating to the codes included in the code string data; The control unit includes obtaining the sentence from a trained model that has learned the relationship by providing the code input data to the trained model.

Images