Musical piece information processing device, system, musical piece information processing method, and program

Abstract

Provided is a musical piece information processing device comprising: a conversion unit that converts the feature amount of a second musical piece into information indicating the probability of presence of a cue point at each position within the second musical piece by using a converter trained to convert the feature amount of a first musical piece into information indicating the probability of presence of a cue point at each position within the first musical piece; and a cue point-related processing unit that, on the basis of the information indicating the probability of presence of the cue point, presents cue point candidate position information within the second musical piece to a user, or automatically sets the cue point within the second musical piece.

Description

Music information processing device, system, music information processing method and program

[0001] The present invention relates to a music information processing device, a system, a music information processing method, and a program.

[0002] For example, in DJ equipment, a playback start position within a song is designated, and when a predetermined operation is performed, playback of the song begins from the designated playback start position. The designated playback start position is also called a cue point. Technology for playing songs using such cue points is described, for example, in Patent Document 1.

[0003] Patent No. 6263417

[0004] However, for example, beginner DJs with little experience often don't know where in a song they should set cue points, and even if they don't, the task of setting cue points is time-consuming for DJs, so automating some of the process can shorten the time required.

[0005] Therefore, an object of the present invention is to provide a music information processing device, system, music information processing method, and program that enable easy recognition of an appropriate position in a song for setting a cue point.

[0006] [1] A music information processing device comprising: a converter that converts feature quantities of a first music piece into information indicating the likelihood of a cue point being present at each position in the second music piece using a converter trained to convert feature quantities of a first music piece into information indicating the likelihood of a cue point being present at each position in the first music piece; and a cue point-related processing unit that presents candidate cue point position information within the second music piece to a user based on the information indicating the likelihood of a cue point being present, or automatically sets a cue point within the second music piece. [2] The music information processing device of [1], wherein the information indicating the likelihood of a cue point being present for each of the first music piece and the second music piece is a cue point vector including at least elements corresponding to positions where a cue point is likely to be present within the music piece, and values ​​of the elements of the cue point vector represent the likelihood of the cue point being present at each position in the music piece. [3] The music information processing device of [2], wherein the cue point vector includes a plurality of elements for each position where a cue point is likely to be present within the music piece, corresponding to a plurality of types of cue points that can be set. [4] The music information processing device according to any one of [1] to [3], wherein the feature is an audio feature in the vicinity of a development change point of the music. [5] The music information processing device according to any one of [1] to [4], wherein the feature includes a feature indicating a development pattern of the music. [6] The music information processing device according to any one of [1] to [5], wherein the feature includes a feature indicating a genre of the music. [7] The music information processing device according to any one of [1] to [6], wherein the feature includes a feature indicating a user attribute. [8] The music information processing device according to any one of [1] to [7], wherein the conversion unit converts the feature into information indicating the probability that the cue point exists based on attribute information of the user.[9] A system including: a learning device that learns, in response to input of feature quantities of a first song, a converter that converts feature quantities of a song into information indicating the likelihood of a cue point being present at each position within the song, so as to output information indicating the same tendency as a cue point setting result in the first song by a first user; and a song information processing device that uses the converter to convert feature quantities of a second song into information indicating the likelihood of a cue point being present at each position within the second song, and presents candidate cue point position information within the second song to the second user based on the information indicating the likelihood of a cue point being present, or automatically sets cue points within the second song.

[10] The system described in [9], wherein the learning device trains the converter for each attribute information of the first user that is set based on a cue point position pattern set by the first user in the first song, and the song information processing device converts the feature quantities of the second song into information indicating the likelihood of a cue point being present at each position within the second song based on the attribute information of the second user.

[11] The system described in [9] or

[10] , wherein the learning device uses nDCG (normalized discount cumulative gain) to evaluate whether the information output by the converter has the same tendency as the cue point setting results in the first song by the first user.

[12] The music information processing device described in any one of [9] to

[11] , wherein the information indicating the likelihood of the presence of the cue point for each of the first song and the second song is a cue point vector including at least elements corresponding to positions in the songs where a cue point is likely to exist, and the likelihood of the cue point being present at each position in the songs is expressed by the values ​​of the elements of the cue point vector.

[13] A song information processing method comprising the steps of: training a converter that converts song features into information indicating the likelihood of a cue point being present at each position within a song, so that when a first user inputs feature features of a first song, it outputs information that shows the same tendency as a cue point setting result within the first song by the first user; converting feature features of a second song using the converter into information indicating the likelihood of a cue point being present at each position within the second song, and presenting candidate cue point position information within the second song to the second user based on the information indicating the likelihood of a cue point being present, or automatically setting a cue point within the second song;

[14] A program that causes a computer to realize a function of converting feature features of a second song into information indicating the likelihood of a cue point being present at each position within the second song, using a converter that has been trained to convert feature features of a first song into information indicating the likelihood of a cue point being present at each position within the first song, and a function of presenting candidate cue point position information within the second song to the user based on the information indicating the likelihood of a cue point being present, or automatically setting a cue point within the second song.

[0007] 5 is a diagram conceptually illustrating one embodiment of the present invention. FIG. 6 is a diagram conceptually explaining cue points set in a song. FIG. 7 is a diagram conceptually explaining extraction of a feature vector of a song. FIG. 8 is a diagram illustrating an example configuration of a system in which the elements shown in FIG. 1 are implemented. FIG. 9 is a schematic block diagram showing a functional configuration for executing a cue point prediction step in one embodiment of the present invention. FIG. 10 is a flowchart showing an example of processing when a cue point candidate position presentation function is implemented in the functional configuration shown in FIG. 11. FIG. 11 is a flowchart showing an example of processing when a cue point automatic setting function is implemented in the functional configuration shown in FIG. 12. FIG. 12 is a diagram illustrating user attribute classification using cue point vectors in one embodiment of the present invention. FIG. 13 is a diagram illustrating an example of a feature vector of a song including elements representing user attribute classification. FIG. 14 is a diagram illustrating processing when using a feature vector of a song including elements representing user attribute classification. FIG. 15 is a diagram illustrating processing when a converter is trained for each user attribute classification.

[0008] FIG. 1 is a diagram conceptually illustrating one embodiment of the present invention. FIG. 1A illustrates a learning step of a converter 100 using a group of cue point vectors CPs representing the cue point settings made by a user U in songs included in a group of songs MCs. The group of cue point vectors CPs are labels indicating the positions of cue points. As described below, the converter 100 converts feature vectors of songs into cue point vectors, which are examples of information indicating the likelihood that a cue point exists at each position in a song. In the learning step, the converter 100 learns to output a group of predicted cue point vectors CPe in response to input feature vectors of songs in the group of songs MCs, so that the group of predicted cue point vectors CPe tends to be the same as the group of cue point vectors CPs representing the cue point settings made by the user U.

[0009] More specifically, in the learning step, the converter 100 is trained so that, for the same song included in the group of songs MCs, the cue point vectors included in the group of predicted cue point vectors CPe and the cue point vectors included in the group of cue point vectors CPs show the same tendency. This allows the relationship between the feature quantities of the song and the positions of the cue points to be learned. As will be described in the example below, in this embodiment, the cue point vectors include elements corresponding to positions where cue points may exist in the song, specifically development change points. Therefore, it is possible to determine whether the cue point vectors share the same tendency based on the commonality of the magnitude relationships between the elements. An example of an algorithm that can be used for this determination is nDCG (normalized discount cumulative gain).

[0010] 1(b) shows a cue point prediction step using the converter 100 trained in the learning step. In the cue point prediction step, the trained converter 100 is used to predict appropriate positions for placing cue points within a piece of music MCx. The predicted cue point positions are represented by predicted cue point vectors CPx. As will be described later, the predicted cue point positions are used, for example, to present to a user (who may be different from the user U at the time of training) as cue point candidate position information, or to automatically set cue points.

[0011] 2 is a diagram for conceptually explaining cue points set within a song, and shows a portion of a song information display that is displayed on, for example, DJ equipment. In the illustrated example, the song information display includes an icon image 501, a title 502, and a song waveform 503, as well as a cue point display 504 and a song development display 505. The cue point display 504 displays cue points P1 to P5 that have been set in the song. In the illustrated example, there are two types of cue points. Specifically, cue points P1, P2, and P3 are memory cues, and cue points P4 and P5 are hot cues.

[0012] In this specification, a cue point is a point recorded to play a song from a specific position within the song in response to a user operation, and is recorded using position information within the song, specifically, a timestamp. In the example of Figure 2, memory cues are automatically sorted and recorded in chronological order, and when a user operation is performed, the playback standby state is entered, and playback from the cue point is started with the next operation. Hot cues are assigned to cue buttons (not shown) (labeled, for example, "A" to "H") and recorded, and when a user operation is performed, playback from the cue point is started immediately. In this way, multiple types of cue points may be set, each with different recording methods and different behaviors in response to user operations.

[0013] The cue points are not limited to the two types shown in the above example, but may be one type, or three or more types.Furthermore, the actions of the various cue points are not limited to the above examples, and cue points with different actions than those described above may be set, even if they are called hot cues or memory cues.

[0014] Meanwhile, the music development display 505 displays music development sections, specifically, intros, verses, bridges, choruses, etc., identified by, for example, a separately performed music analysis. FIG. 2 shows music development change points S1 to S9. The development change points S1 to S9 are the start points or boundary points of the music development sections, and are recorded using position information within the music, specifically, timestamps. For example, development change point S1 is the start point of the music development section "intro," and development change point S2 is the boundary point between the music development section "intro" and "verse 1." In the illustrated example, cue points are set near the development change points. Specifically, cue points P1, P2, P3, P4, and P5 are set near development change points S1, S2, S3, S5, and S8, respectively. Cue points are not necessarily set at all development change points; in the illustrated example, no cue points are set at development change points S4, S6, S7, and S9. Note that the positions where cue points may exist within a piece of music are not limited to development change points, but may also be beats or bars.

[0015] For example, in the learning step shown in FIG. 1 , if user U sets cue points for a certain song as in the example shown in FIG. 2 , a cue point vector [1, 1, 1, 0, 1, 0, 0, 1, 0] is added to the cue point vector group CPs as a label indicating the cue point setting results for the song by user U. In this example, information indicating the likelihood of a cue point being present at each position in the song is a cue point vector including elements corresponding to development change points in the song. When the cue point vector is used as a label indicating the cue point setting results for the song by user U, the nth element of the cue point vector is set to "0" or "1" depending on whether a cue point has been set near the nth development change point in the song. More specifically, for example, if a cue point has been set within a range of 500 milliseconds before and after the nth development change point in the song, the nth element of the cue point vector is set to "1," and if no cue point has been set within the range, the nth element is set to "0."

[0016] FIG. 3 is a diagram conceptually illustrating the extraction of a feature vector for music. In this embodiment, as described above, the cue point vector indicates whether a cue point is set for each development change point in music. Therefore, the extracted feature vector also includes audio features near the development change point in music. Specifically, for example, audio features of a predetermined length before and after the development change point are extracted as the feature vector. In the example shown in FIG. 3 , audio features of a total of six segments Seg1 to Seg6 are extracted for a section Sec1 of three segments each around the development change point Sn, and these are concatenated to form elements of the feature vector. The number of segments and the length of the sections are not particularly limited. For example, if the bit rate of a music piece is 22.050 kHz, one segment may be 59,049 samples, or 2.68 seconds, and audio features of six segments, or a section of approximately 16 seconds centered around the development change point, may be extracted as the feature vector.

[0017] The feature vector of a song may include additional elements in addition to audio features near each development change point. For example, the feature vector of a song may include features indicating the development pattern of the song. The development pattern of a song is a pattern that classifies the arrangement of song development sections, such as "intro → verse → bridge → chorus → verse → bridge → verse → chorus" in the songs shown in FIGS. 2 and 3 . For example, if p development patterns are set for a song (p≧2), the feature vector of a song may include, as its elements, a one-hot vector in which only one of the p elements is "1" and the rest are "0". Similarly, the feature vector of a song may include features indicating the genre of the song. For example, if q genres are set for a song (q≧2), the feature vector of a song may include, as its elements, a one-hot vector in which only one of the q elements is "1" and the rest are "0".

[0018] The feature vector of a song may also include a feature that indicates the normalized position of a cue point within the song. For example, if a song has a total length of 100 seconds and the cue point is located 25 seconds from the start, the feature vector of the song may include "0.25," which indicates the position of the cue point normalized with the total length set to 1.

[0019] In the learning step of this embodiment, the converter 100 is trained so that the group of predicted cue point vectors CPe, which are the results of the converter 100 predicting the positions of cue points in a song, have the same tendency as the group of cue point vectors CPs, which are the results of settings by the user U. For example, if the results of cue point settings (correct labels) by the user U for a certain song are represented by a cue point vector [1, 1, 1, 0, 1, 0, 0, 1, 0], and when the feature vectors of this song are input to the converter 100, a predicted cue point vector [1.8, 1.2, 1.5, -0.4, 1.5, -1.2, -1.6, 1.1, 0.1] is output, it can be said that the predicted cue point vectors have the same tendency as the correct labels. This is because, when the element values ​​of the predicted cue point vectors are ranked from largest to smallest, the result is [1, 4, 2, 7, 2, 8, 9, 5, 6], which matches the correct labels. In the cue point vector of the setting result, the 1st, 2nd, 3rd, 5th, and 8th elements "1" are tied for 1st place, and the 4th, 6th, 7th, and 9th elements "0" are tied for last place, whereas in the predicted cue point vector, the 1st, 2nd, 3rd, 5th, and 8th elements are ranked from 1st to 5th, and the 4th, 6th, 7th, and 9th elements are ranked from 6th to 9th, so the rankings match.

[0020] nDCG (normalized discount cumulative gain) is a known method for calculating the degree of agreement between two rankings. The nDCG calculated for two rankings takes a value between 0 and 1, approaching 1 if the two rankings show the same trend and approaching 0 if they show different trends. The nDCG value calculated for the two cue point vectors is 1 because the two rankings perfectly match. On the other hand, if the two rankings show completely opposite rankings, the nDCG value is 0. When nDCG is used in the learning step, the converter 100 is trained so that the nDCG calculated for the cue point vectors included in the predicted cue point vector group CPe and the cue point vectors included in the cue point vector group CPs for the same song included in the song group MCs approaches 1. Note that the algorithm used in the learning step is not limited to nDCG; for example, any other algorithm capable of evaluating the commonality of rankings or any algorithm used in supervised learning may be used.

[0021] FIG. 4 is a diagram showing an example of the configuration of a system in which the elements shown in FIG. 1 are implemented. The system 10 shown in FIG. 4 includes a PC (Personal Computer) 11, a DJ controller 12, a speaker 13, and a server 14. The PC 11 is a device that stores, processes, and plays music audio data. It is not limited to a PC, but may also be a terminal device such as a tablet or smartphone. The DJ controller 12 is connected to the PC 11 via a communication means such as a USB (Universal Serial Bus) and acquires user input related to music playback using a channel fader, crossfader, performance pad, jog dial, and various knobs and buttons. The audio data is played using the speaker 13 under the control of the DJ controller 12. The server 14 is connected to the PC 10 via a network as needed and provides various functions. The functions described below are implemented as software in the PC 11, the DJ controller 12, or the server 14 by the computer operating according to programs stored in memory.

[0022] 1 , i.e., the converter 100 and the determination function for learning, are implemented, for example, in the server 14. In this case, the server 14 collects the results of cue point setting operations for songs performed by the user U on the PC 11 or DJ controller 12 connected via a network as a group of cue point vectors CPs, and uses these results, along with feature vectors of songs in a group of songs MCs that have been separately extracted and collected, for learning the converter 100. In another example, the converter 100 and the determination function for learning may be implemented in the PC 11, and the learning steps may be performed by downloading some or all of the group of cue point vectors CP stored in the server 14.

[0023] Meanwhile, the function for executing the cue point prediction step shown in FIG. 1 is implemented, for example, in the PC 11. In this case, for example, when a user operates the PC 11 or the DJ controller 12 to select a song MCx, the converter 100 inputs the feature vectors of the song MCx and uses the predicted cue point vectors CPx obtained to present cue point candidate position information and automatically set cue points. Some or all of these processes may be executed by a server 14 connected to the PC 11 via a network. At least part of the cue point prediction step may be executed in advance for songs that the user can play. For example, the predicted cue point vectors CPx obtained by inputting the feature vectors of songs that the user can play into the converter 100, or information for presenting cue point candidate position information and automatically setting cue points based on the predicted cue point vectors CPx, may be stored in association with each song in the PC 11, the DJ controller 12, or the server 14.

[0024] 1 are not limited to the above example. In another example, the system may include DJ equipment such as a mixer or an all-in-one DJ system (a digital audio player with communication and mixing functions) instead of or in addition to the PC 11, and at least some of the learning step and cue point prediction step may be performed on these DJ equipment. Furthermore, the learning step and cue point prediction step do not have to be performed on a single device, but may be distributed and performed across multiple devices.

[0025] 5 is a schematic block diagram showing the functional configuration for executing the cue point prediction step in one embodiment of the present invention. In the illustrated example, the following functions are implemented: a feature vector acquisition unit 110, a converter 100 (conversion unit), a cue point candidate position presentation processing unit 120, and a cue point automatic setting processing unit 130. The feature vector acquisition unit 110 acquires feature vectors of the music to be processed in the cue point prediction step. The feature vectors of the music are extracted, for example, by a music analysis performed in advance, and stored in the PC 11 or the server 14 in association with the music. Alternatively, the feature vector acquisition unit 110 may include a music analysis function and extract feature vectors from the audio data of the music or data of the music development section.

[0026] The converter 100 (conversion unit) converts the feature vector of a song acquired by the feature vector acquisition unit 110 into a cue point vector that represents the positions of cue points within the song. As already described, the converter 100 used in the cue point prediction step has been trained through a learning step and can output a cue point vector that represents the positions of cue points predicted to be appropriate for the input feature vector of the song. For example, if the converter 100 outputs a cue point vector [1.8, 1.2, 1.5, -0.4, 1.5, -1.2, -1.6, 1.1, 0.1] for the input feature vector of the song, converting the top five elements to "1" and the remaining elements to "0" would result in a cue point vector [1, 1, 1, 0, 1, 0, 0, 1, 0], resulting in a prediction that it would be appropriate to set cue points at the first, second, third, fifth, and eighth development change points of the song. In the above example, the number of top elements to be set to "1" may be selectable by the user, or may be automatically determined based on the upper limit of the number of cue points that can be set. As another example, the average value of all elements of the cue point vector or another predetermined value may be used as a reference value, and elements equal to or greater than the reference value may be converted to "1" and elements less than the reference value may be converted to "0." Note that the prediction result does not necessarily have to be expressed using "1" and "0."

[0027] The cue point candidate position presentation processor 120 and the cue point automatic setting processor 130 are examples of cue point-related processors that perform processing using cue point vectors acquired by the converter 100. Either one of them may be implemented, or both may be implemented. The cue point candidate position presentation processor 120 presents the positions of development change points in a song indicated by the cue point vectors to the user as cue point candidate position information. The cue point candidate position information is displayed on the display of the PC 11 or the DJ controller 12, for example, in a form similar to the cue point display 504 of the song information display shown in FIG. 2. Meanwhile, the cue point automatic setting processor 130 automatically sets cue points at positions in a song that are predicted to be appropriate for setting cue points based on the cue point vectors. The automatically set cue points are displayed on the display of the PC 11 or the DJ controller 12, for example, as the cue point display 504 of the song information display shown in FIG. 2.

[0028] According to the processing of the cue point candidate position presentation processor 120 as described above, the user can select a position in a song where a cue point should be set by being presented with candidates in advance. Similarly, with the automatic cue point setting processor 130, the user can select a cue point to use by trial playback from each of the cue points that have been automatically set in the song. In either case, by predicting an appropriate cue point based on the cue point vector, even a beginner can easily recognize an appropriate position in a song where a cue point should be set.

[0029] FIG. 6 is a flowchart illustrating an example of processing when the cue point candidate position presentation function is implemented in the functional configuration shown in FIG. 5 . In the illustrated example, the user first selects a song by operating the PC 11 or DJ controller 12 (step S101), and cue point vectors for the selected song are acquired (step S102). In this step, for example, the feature vector acquisition unit 110 estimates cue point positions from the feature vectors of the selected song. Alternatively, cue point vectors previously generated using the feature vector acquisition unit 110 and the converter 100 may be read. Once the cue point vectors for the selected song are acquired, the cue point candidate position presentation processing unit 120 presents cue point candidate position information to the user using the display of the PC 11 or DJ controller 12 (step S103). The user sets cue points by referring to the presented information and referring to the PC 11 or DJ controller 12 (step S104). Thereafter, when the user instructs playback from the cue point (step S105), the PC 11 or DJ controller 12 plays the song from the instructed cue point (step S106).

[0030] FIG. 7 is a flowchart showing an example of processing when the automatic cue point setting function is implemented in the functional configuration shown in FIG. 5 . In the illustrated example, as in the example of FIG. 7 , the user selects a song (step S201), and the cue point vectors of the selected song are obtained (step S202). Once the cue point vectors of the selected song are obtained, the automatic cue point setting processor 130 automatically sets cue points within the song (step S203). The positions of the automatically set cue points are presented to the user by being displayed on the display of the PC 11 or DJ controller 12. The user operates the PC 11 or DJ controller 12 while referring to the presented information to instruct playback from the cue point (step S204). Once playback from the cue point is instructed, the PC 11 or DJ controller 12 plays the song from the specified cue point (step S205).

[0031] The algorithm explained so far is an algorithm that predicts general cue point setting trends from the cue point setting trends of a large number of sampled users. Below, we will explain the algorithm for cue point prediction using user attributes. The algorithm for cue point prediction using user attributes is an algorithm that predicts cue point setting trends that reflect the preferences of a specific user, i.e., an individual, based on statistics of that individual's cue point setting trends.

[0032] In one embodiment of the present invention, the cue point prediction algorithm using user attributes obtains a score representing the degree of cue prediction based on the frequency distribution of each type of development change point of the set cue point for each development change point and the beat positions 4, 8, 16, 32, and 64 beats before that cue point. FIG. 8 is a diagram illustrating user attribute classification using cue point vectors in one embodiment of the present invention. The graph shown in FIG. 8 illustrates the frequency distribution of each type of development change point of cue points set in a song by a specific user. In the graph, the vertical axis represents the music development section after the development change point, and the horizontal axis represents the music development section before the development change point, as well as the number of beats between the cue point set near the development change point and the development change point. For example, the area where "CHORUS" on the vertical axis intersects with "INTRO-16" on the horizontal axis indicates the frequency of cue points set 16 beats before the development change point with "INTRO" before it and "CHORUS" after it. The highest frequency is the area where "CHORUS" on the vertical axis intersects with "UP" on the horizontal axis, and the next highest frequency is the area where "CHORUS" on the vertical axis intersects with "CHORUS" on the horizontal axis.

[0033] The scores obtained using the cue point prediction algorithm that does not use user attributes are normalized to [0, 1], and the scores obtained using the cue point prediction algorithm that uses user attributes are also normalized to [0, 1]. The scores obtained by adding these up for each beat are used as the score for that beat. The predicted positions of the cue points are determined in descending order of the beats with the highest scores. Instead of just adding the scores up, weighted scores can also be added, or the scores can be multiplied.

[0034] User attributes can be considered by considering the distribution of cue points frequently set at transitions between different developments, i.e., statistical analysis of cue points. The genre of music frequently played can also be considered. Furthermore, the position within a song where cue points are frequently set can also be considered. For example, a vector can be used to represent trends such as more frequent cue points in the first half of a song or more frequent cue points in the second half. For a single cue point, the position within a song can be represented by a numerical value (0, 1). For multiple cue points, the entire song can be divided into n parts and represented as an n-dimensional vector. For example, a song can be divided into eight parts and represented as an 8-dimensional vector. Specifically, an 8-dimensional vector of [1, 0.3, 0.6, 0.5, 0.1, 0.1, 0.4, 0.3] indicates that the user must set cue points at the beginning of a song and also set cue points in the second half. Clustering such data can be used to classify users. The era of frequently played songs and the type of cue points set (e.g., memory cues, hot cues, loops, colors, etc.) can also be used to classify users.

[0035] FIG. 9 illustrates an example of a song feature vector including elements representing user attribute classification. In the above example, the user attribute classification may be included, for example, as an element of the song feature vector. In this case, the song feature vector includes elements representing user attributes in addition to elements of a vector representing song features. As already mentioned, the song feature vector may include features representing the song's development pattern and features representing the song's genre. Specifically, the vector representing the user attribute classification is a user vector generated from the type of development change point at which a cue point is set. In the example shown in FIG. 8, a user vector with up to 216 dimensions is generated for each combination of the music development section after the development change point (6 combinations) and the music development section before the development change point and the number of beats between the cue point and the development change point set near the development change point (36 combinations). Alternatively, a user vector may be generated by adding up data for each type of cue point. For example, adding up data for two types of cue points, hot cues and memory cues in the example of FIG. 8 results in a user vector with up to 432 dimensions. Furthermore, the user vector may be generated using, for example, the proportion of genres of music owned by the user. The dimension of the user vector may be reduced using SVM, convolution, etc. When the volume of the user vector is large, an increase in the amount of calculation can be suppressed by reducing the dimension.

[0036] FIG. 10 is a diagram illustrating processing when using a feature vector of a song including elements representing a user's attribute classification. In the illustrated example, the user classifier converts user attribute classification information, such as that shown in FIG. 8, into attribute classifications with reduced dimensionality using SVM, convolution, or the like. In the learning step, the converter 100 is trained using attribute classifications generated by the user classifier for user U (see FIG. 1) who provided the cue point setting results and feature quantities of the song for which cue points are to be set. Meanwhile, in the cue point prediction step, the converter 100 predicts the positions of cue points by inputting attribute classifications generated by the user classifier for the user who will use the cue point prediction results and feature quantities of the song for which cue points are to be predicted. For example, by presetting a finite number of attribute classifications, it is possible to generate an appropriate amount of information regarding the user's classification.

[0037] 11 is a diagram illustrating the process of training a converter for each user attribute classification. In the illustrated example, multiple converters 100 are prepared for each user attribute classification. A converter 100 is trained according to the attribute classification generated by the user classifier for user U (see FIG. 1) who provided the cue point setting results. In the cue point prediction step, the positions of cue points are predicted from the features of the music using the trained converter 100 according to the attribute classification generated by the user classifier for the user who will use the cue point prediction results.

[0038] As yet another example, in the learning step shown in FIG. 1 , the converter 100 may be trained for each type of cue point. As described with reference to FIG. 2 , cue points may include multiple types, such as hot cues and memory cues. In the above example, a cue point vector is defined without distinguishing between these types of cue points. However, a cue point vector may be defined to indicate the type of cue point in addition to the position of the cue point within the music. For example, for a music piece in which hot cues and memory cues are set as shown in FIG. 2 , a cue point vector of [1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0] may be defined. In this example, odd-numbered elements of the cue point vector indicate whether a memory cue point is set near a development change point, and even-numbered elements indicate whether a hot cue point is set near a development change point.

[0039] 10...System, 12...DJ controller, 13...Speaker, 14...Server, 100...Converter (conversion unit), 110...Feature vector acquisition unit, 120...Cue point candidate position presentation processing unit, 130...Cue point automatic setting processing unit, 501...Icon image, 502...Title, 503...Music waveform, 504...Cue point display, 505...Music development display, P1 to P5...Cue points, S1 to S9...Development change points, Sec1...Section, Seg1 to Seg6...Segment, Sn...Development change points.

Claims

1. a conversion unit that converts the feature quantities of a second song into information indicating the likelihood of a cue point being present at each position in the second song, using a converter trained to convert the feature quantities of a first song into information indicating the likelihood of a cue point being present at each position in the first song; a cue point related processing unit that presents cue point candidate position information within the second music piece to a user based on information indicating the probability that the cue point exists, or that automatically sets a cue point within the second music piece; A music information processing device comprising:

2. 2. The music information processing device of claim 1, wherein the information indicating the probability that the cue point exists for each of the first song and the second song is a cue point vector that includes at least elements corresponding to positions in the song where a cue point may exist, and the values ​​of the elements of the cue point vector represent the probability that the cue point exists at each position in the song.

3. The music information processing device according to claim 2 , wherein the cue point vector includes a plurality of elements for each position at which a cue point may exist in the music, corresponding to a plurality of types of cue points that can be set.

4. The music information processing device according to claim 1 , wherein the feature amount is an audio feature amount in the vicinity of a development change point of the music.

5. The music information processing device according to claim 1 , wherein the feature amount includes a feature amount indicating a development pattern of the music.

6. The music information processing device according to claim 1 , wherein the feature amount includes a feature amount indicating a genre of the music.

7. The music information processing device according to claim 1 , wherein the feature amount includes a feature amount indicating a user attribute.

8. The music information processing device according to claim 1 , wherein the conversion unit converts the feature amount into information indicating a probability that the cue point exists based on attribute information of the user.

9. a learning device that learns a converter that converts the feature quantities of a song into information indicating the likelihood of a cue point being present at each position within the song, so that when the feature quantities of a first song are input, the converter outputs information that indicates the same tendency as the cue point setting results within the first song by a first user; a music information processing device that converts the feature quantity of a second music piece into information indicating the likelihood of a cue point being present at each position in the second music piece using the converter, and presents candidate cue point position information within the second music piece to a second user based on the information indicating the likelihood of a cue point being present, or automatically sets cue points within the second music piece; A system including:

10. the learning device learns the converter for each attribute information of the first user that is set based on a position pattern of cue points set by the first user in the first song; The system according to claim 9 , wherein the music information processing device converts the feature quantity of the second music piece into information indicating a probability that a cue point exists at each position within the second music piece based on attribute information of the second user.

11. The system described in claim 9 or 10, wherein the learning device uses nDCG (normalized discount cumulative gain) to evaluate whether the information output by the converter has the same tendency as the cue point setting results in the first song by the first user.

12. The system described in claim 9 or 10, wherein the information indicating the probability that the cue point exists for each of the first song and the second song is a cue point vector that includes at least elements corresponding to positions where a cue point may exist within the song, and the values ​​of the elements of the cue point vector represent the probability that the cue point exists at each position within the song.

13. a step of training a converter that converts the feature quantities of a song into information indicating the likelihood of a cue point being present at each position within the song, so that when the feature quantities of a first song are input, the converter outputs information that indicates the same tendency as the cue point setting results within the first song by a first user; converting the feature quantities of the second song into information indicating the likelihood that a cue point exists for each position in the second song using the converter, and presenting candidate cue point position information within the second song to a second user based on the information indicating the likelihood that a cue point exists, or automatically setting a cue point within the second song; A music information processing method including:

14. a function of converting the feature quantities of a second song into information indicating the likelihood that a cue point exists at each position in the second song, using a converter trained to convert the feature quantities of a first song into information indicating the likelihood that a cue point exists at each position in the first song; a function of presenting to a user cue point candidate position information within the second song based on information indicating the probability that the cue point exists, or automatically setting a cue point within the second song; A program to make the above happen on a computer.