Music information processing device, system, music information processing method, and program

Abstract

Provided is a musical piece information processing device comprising: a conversion unit that converts the feature amount of a second musical piece into information indicating the probability of presence of a cue point at each position within the second musical piece by using a converter trained to convert the feature amount of a first musical piece into information indicating the probability of presence of a cue point at each position within the first musical piece; and a cue point-related processing unit that, on the basis of the information indicating the probability of presence of the cue point, presents cue point candidate position information within the second musical piece to a user, or automatically sets the cue point within the second musical piece.

Description

Technical Field

[0001] The present invention relates to a music information processing apparatus, a system, a music information processing method, and a program.

Background Art

[0002] For example, in DJ devices, it is performed that a playback start position is specified in a music piece, and playback of the music piece is started from the specified playback start position when a predetermined operation is performed. The specified playback start position is also referred to as a cue point. A technique for playing back a music piece using such a cue point is described in, for example, Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0004] However, for example, a beginner with little DJ experience often does not know at which position in a music piece a cue point should be set. Even for those who are not beginners, the work of setting cue points by a DJ is time-consuming, so that if part of the work can be automated, for example, the working time can be shortened.

[0005] Accordingly, an object of the present invention is to provide a music information processing apparatus, a system, a music information processing method, and a program that make it possible to easily recognize an appropriate position in a music piece for setting a cue point.

Means for Solving the Problems

[0006] [1] A music information processing device comprising: a conversion unit that converts the feature quantities of a first music into information indicating the probability of cue points existing at each location within the second music using a converter that has been trained to convert the feature quantities of a first music into information indicating the probability of cue points existing at each location within the first music; and a cue point-related processing unit that presents candidate cue point location information within the second music to the user based on the information indicating the probability of cue points existing, or automatically sets cue points within the second music. [2] For each of the first and second pieces of music described above, the information indicating the probability of the existence of the cue point is a cue point vector that includes at least an element corresponding to a position in the piece where the cue point may exist, and the probability of the existence of the cue point at each position in the piece is expressed by the values ​​of the elements of the cue point vector, as described in [1]. [3] The cue point vector described above includes multiple elements for each possible location within the song where a cue point may exist, corresponding to multiple types of configurable cue points, as described in [2]. [4] The above feature quantities are audio feature quantities in the vicinity of the development change points of the music, as described in any one of [1] to [3], for the music information processing device. [5] The above feature quantities include feature quantities that indicate the development pattern of the music, as described in any one of items [1] to [4]. [6] The above feature quantities include feature quantities indicating the genre of the music, as described in any one of items [1] to [5]. [7] The above feature quantities include feature quantities indicating user attributes, as described in any one of items [1] to [6]. [8] The music information processing device according to any one of [1] to [7], wherein the conversion unit converts the above feature quantities into information indicating the probability that the above cue point exists, based on the above user attribute information. [9] A system including a converter that converts the feature quantities of a song into information indicating the probability that a cue point exists at each position within the song, a learning device that learns to output information showing the same trend as the result of a first user setting cue points within the first song in response to the input of the feature quantities of a first song, and a song information processing device that uses the converter to convert the feature quantities of a second song into information indicating the probability that a cue point exists at each position within the second song, and presents candidate cue point location information within the second song to the second user based on the information indicating the probability that a cue point exists, or automatically sets cue points within the second song.

[10] The learning device learns the converter for each attribute information of the first user which is set based on the position pattern of cue points set by the first user in the first song, and the song information processing device converts the feature quantities of the second song into information indicating the probability that a cue point exists at each position in the second song based on the attribute information of the second user, as described in [9].

[11] The learning device, using nDCG (normalized discount cumulative gain), evaluates whether the information output by the converter has the same trend as the cue point setting results in the first song by the first user, as described in [9] or

[10] .

[12] For each of the first and second pieces of music described above, the information indicating the probability of the existence of the cue point is a cue point vector that includes at least an element corresponding to a position in the piece where the cue point may exist, and the probability of the existence of the cue point at each position in the piece is expressed by the values ​​of the elements of the cue point vector, as described in any one of the items from [9] to

[11] . system .

[13] A method for processing song information, comprising the steps of: training a converter that converts the features of a song into information indicating the probability of cue points existing at each location within the song, to output information showing the same trend as the results of cue point setting in the first song by a first user, given the input of the features of a first song; and using the converter, converting the features of a second song into information indicating the probability of cue points existing at each location within the second song, and presenting candidate cue point location information in the second song to a second user based on the information indicating the probability of cue points existing, or automatically setting cue points in the second song.

[14] A program for a computer to implement the following functions: a function to convert the feature quantities of a first song into information indicating the probability of cue points existing at each location within the second song, using a converter that has been trained to convert the feature quantities of a first song into information indicating the probability of cue points existing at each location within the first song; and a function to present candidate cue point location information within the second song to the user based on the information indicating the probability of cue points existing, or to automatically set cue points within the second song. [Brief explanation of the drawing]

[0007] [Figure 1] This is a diagram conceptually illustrating one embodiment of the present invention. [Figure 2] This diagram conceptually explains the cue points set within a song. [Figure 3] This diagram conceptually explains the extraction of feature vectors from musical pieces. [Figure 4] This figure shows an example of a system configuration in which the elements shown in Figure 1 are implemented. [Figure 5] This is a schematic block diagram showing the functional configuration for performing the cue point prediction step in one embodiment of the present invention. [Figure 6] Figure 5 shows a flowchart illustrating an example of processing when the cue point candidate position suggestion function is implemented with the functional configuration shown. [Figure 7] This flowchart shows an example of the process when the automatic cue point setting function is implemented with the functional configuration shown in Figure 5. [Figure 8] This figure illustrates the classification of user attributes using cue point vectors in one embodiment of the present invention. [Figure 9] This figure shows an example of a song feature vector that includes elements representing user attribute classification. [Figure 10] This diagram illustrates the process when using a song feature vector that includes elements representing user attribute classification. [Figure 11] This diagram illustrates the process of learning a converter based on user attribute classifications. [Modes for carrying out the invention]

[0008] Figure 1 is a conceptual diagram illustrating one embodiment of the present invention. Figure 1(a) shows the learning steps of the converter 100 using a group of cue point vectors CPs that represent the results of cue point setting within songs included in a group of songs MCs by user U. The group of cue point vectors CPs are labels indicating the positions of cue points. As will be described later, the converter 100 converts the feature vectors of songs into cue point vectors, which are examples of information indicating the probability that a cue point exists at each position within a song. In the learning steps, the converter 100 is trained so that, in response to the input of the feature vectors of songs in the group of songs MCs, the predicted cue point vector group CPe that it outputs shows the same trend as the group of cue point vectors CPs that represent the results of cue point setting by user U.

[0009] More specifically, in the learning step, the converter 100 is trained so that for the same song included in the song group MCs, the cue point vectors included in the predicted cue point vector group CPe and the cue point vectors included in the cue point vector group CPs show the same trend. This allows the relationship between the song's features and the cue point locations to be learned. As shown in the example described later, in this embodiment, the cue point vectors include elements corresponding to locations where cue points may exist within the song, specifically development change points. Therefore, it is possible to determine whether the cue point vectors have the same trend based on the commonality of the magnitude relationships of each element. For example, an algorithm such as nDCG (normalized discount cumulative gain) can be used for such determination.

[0010] On the other hand, Figure 1(b) shows the cue point prediction step using the converter 100 learned in the learning step. In the cue point prediction step, the learned converter 100 is used to predict the appropriate positions for placing cue points within the song MCx. The predicted cue point positions are represented by the predicted cue point vector CPx. As will be described later, the predicted cue point positions are used, for example, to present cue point candidate position information to the user (which may differ from the user U during learning) or to automatically set cue points.

[0011] Figure 2 is a diagram for conceptually explaining cue points set within a song, and shows a part of the song information display shown on, for example, DJ equipment. In the illustrated example, the song information display includes an icon image 501, a title 502, and a song waveform 503, as well as a cue point display 504 and a song development display 505. The cue point display 504 shows the cue points P1 to P5 set in the song. In the illustrated example, there are two types of cue points. Specifically, cue points P1, P2, and P3 are memory cues, and cue points P4 and P5 are hot cues.

[0012] In the present specification, a cue point is a point recorded for playing back a music piece from a specific position within the music piece in response to a user operation, and is recorded by position information within the music piece, specifically a timestamp. In the example of Figure 2, memory cues are automatically sorted and recorded in chronological order, and when a user operation is performed, a playback standby state is temporarily entered, and then playback from the cue point is started by the next operation. A hot cue is assigned and recorded to a cue button (not shown, for example, labeled with "A" to "H" or the like), and when a user operation is performed, playback from the cue point is started immediately. In this way, a plurality of types of cue points having different recording methods and behaviors in response to user operations may be set.

[0013] Note that the cue points are not limited to the two types in the above examples, and may be one type or three or more types. Further, the operations of the various cue points are not limited to the above examples, and for example, cue points with operations different from those described above may be set even under the names of hot cue and memory cue.

[0014] On the other hand, the song development display 505 displays song development sections identified by, for example, a separately performed song analysis, specifically the intro, verse, bridge, chorus, etc. Figure 2 shows the song development change points S1 to S9. Development change points S1 to S9 are the start or boundary points of song development sections and are recorded by positional information within the song, specifically by timestamps. For example, development change point S1 is the start of the song development section "Intro," and development change point S2 is the boundary point between the song development section "Intro" and "Verse 1." In the illustrated example, cue points are set near the development change points. Specifically, cue points P1, P2, P3, P4, and P5 are set near development change points S1, S2, S3, S5, and S8, respectively. Cue points are not necessarily set for all development change points; in the illustrated example, cue points are not set for development change points S4, S6, S7, and S9. Furthermore, the locations where cue points may exist within a piece of music are not limited to points of change in development; they can also be beats or measures.

[0015] For example, in the learning step shown in Figure 1, if user U sets cue points for a song as shown in the example in Figure 2, the cue point vector [1,1,1,0,1,0,0,1,0] is added to the cue point vector group CPs as a label indicating the result of cue point setting within the song by user U. In this example, the information indicating the probability that a cue point exists at each position within the song is a cue point vector that includes elements corresponding to developmental change points within the song. When using a cue point vector as a label indicating the result of cue point setting within a song by user U, the nth element of the cue point vector is set to "0" or "1" depending on whether a cue point is set near the nth developmental change point of the song. More specifically, for example, if a cue point is set within 500 milliseconds before or after the nth developmental change point of the song, the nth element of the cue point vector is set to "1", and if no cue point is set within the above range, the nth element is set to "0".

[0016] Figure 3 is a diagram for conceptually explaining the extraction of feature vectors for a musical piece. In this embodiment, as described above, the cue point vector indicates whether or not a cue point is set for each development change point in the musical piece, so the feature vector is also extracted to include audio features near the development change points of the musical piece. Specifically, for example, audio features of a predetermined length of section before and after the development change point are extracted as feature vectors. In the example shown in Figure 3, the audio features of a total of six segments, Seg1 to Seg6, are extracted for the section Sec1 consisting of three segments before and after the development change point Sn, and these are concatenated to form elements of the feature vector. The number of segments and the length of the section are not particularly limited, but for example, if the bitrate of the musical piece is 22.050kHz, one segment may be 59049 samples, or 2.68 seconds, and audio features of six segments, or approximately 16 seconds of section centered on the development change point, may be extracted as feature vectors.

[0017] The feature vector of a song may include additional elements in addition to the audio features in the vicinity of each developmental change point. For example, the feature vector of a song may include features that indicate the song's developmental pattern. The song's developmental pattern is a pattern that classifies the sequence of song developmental intervals, such as "intro → verse → bridge → chorus → verse → bridge → verse → chorus" in the song shown in Figures 2 and 3. For example, if there are p song developmental patterns (p≧2), the feature vector of the song may include a one-hot vector as an element, where only one of the p elements is "1" and the rest are "0". Similarly, the feature vector of a song may include features that indicate the song's genre. For example, if there are q song genres (q≧2), the feature vector of the song may include a one-hot vector as an element, where only one of the q elements is "1" and the rest are "0".

[0018] Furthermore, the feature vector of a song may include a feature that indicates the normalized position of the cue point within the song. For example, if a song is 100 seconds long and the cue point is located at 25 seconds from the start, the feature vector of the song may include "0.25", which indicates the position of the cue point normalized with the total length set to 1.

[0019] In the learning step of this embodiment, the converter 100 is trained so that the predicted cue point vector group CPe, which is the prediction result of the converter 100 for the positions of cue points in a song, has the same trend as the cue point vector group CPs, which is the setting result by user U. For example, if the cue point setting result (ground truth label) by user U for a certain song is represented by the cue point vector [1,1,1,0,1,0,0,1,0], and when the feature vector of this song is input to the converter 100, the predicted cue point vector [1.8,1.2,1.5,-0.4,1.5,-1.2,-1.6,1.1,0.1] is output, then it can be said that the predicted cue point vector has the same trend as the ground truth label. This is because when the values ​​of the elements of the predicted cue point vector are ranked from largest to smallest, they become [1,4,2,7,2,8,9,5,6], which matches the ground truth label. In the default cue point vector, elements 1, 2, 3, 5, and 8 ("1") are tied for first place, while elements 4, 6, 7, and 9 ("0") are tied for last place. In contrast, the predicted cue point vector assigns ranks from 1st to 5th to elements 1, 2, 3, 5, and 8, and ranks from 6th to 9th to elements 4, 6, 7, and 9, thus matching the rankings.

[0020] nDCG (normalized discount cumulative gain) is a known method for calculating the degree to which two rankings match. The nDCG calculated for two rankings takes a value between 0 and 1, approaching 1 if the two rankings show the same trend and approaching 0 if they show different trends. The nDCG value calculated for the above two cue point vectors is 1 because the two rankings match perfectly. On the other hand, if the two rankings show completely opposite ranks, the nDCG value is 0. When using nDCG in the learning step, the converter 100 is trained so that the nDCG calculated for the same song in the song group MCs, for the cue point vectors in the predicted cue point vector group CPe and the cue point vectors in the cue point vector group CPs, approaches 1. Note that the algorithm used in the learning step is not limited to nDCG; for example, any other algorithm that can evaluate the commonality of rankings, or any algorithm used in supervised learning, may be used.

[0021] Figure 4 shows an example of a system configuration in which the elements shown in Figure 1 are implemented. The system 10 illustrated in Figure 4 includes a PC (Personal Computer) 11, a DJ controller 12, speakers 13, and a server 14. The PC 11 is a device that stores, processes, and plays back audio data of music, and is not limited to a PC; it may also be a terminal device such as a tablet or smartphone. The DJ controller 12 is connected to the PC 11 via a communication means such as USB (Universal Serial Bus), and acquires user input related to music playback through channel faders, crossfaders, performance pads, jog dials, and various knobs and buttons. The audio data is played back using the speakers 13 according to the control of the DJ controller 12. The server 14 is connected to the PC 10 via a network as needed and provides various functions. The functions described below are implemented in software by the computer operating according to a program stored in memory in the PC 11, DJ controller 12, or server 14.

[0022] For example, the functions for executing the learning steps shown in Figure 1, namely the converter 100 and the judgment function for learning, are implemented, for example, on the server 14. In this case, the server 14 collects the results of cue point setting operations for songs performed by user U on a PC 11 or DJ controller 12 connected via the network as a group of cue point vectors CPs, and uses them together with the feature vectors of songs from a separately extracted and collected group of songs MCs to train the converter 100. In another example, the converter 100 and the judgment function for learning may be implemented on the PC 11, and the learning steps may be executed by downloading some or all of the cue point vectors CPs stored on the server 14.

[0023] On the other hand, the function for executing the cue point prediction step shown in Figure 1 is implemented, for example, on PC 11. In this case, for example, when a user selects a song MCx by operating PC 11 or DJ controller 12, the system inputs the feature vector of the song MCx into the converter 100 and uses the obtained predicted cue point vector CPx to present cue point candidate position information and automatically set cue points. Some or all of these processes may be executed on a server 14 connected to PC 11 via a network. At least a part of the cue point prediction step may be pre-executed for songs that the user can play. For example, the predicted cue point vector CPx obtained by inputting the feature vector of a song that the user can play into the converter 100, or information for presenting cue point candidate position information and automatically setting cue points based on the predicted cue point vector CPx, may be stored in PC 11, DJ controller 12, or server 14 in association with each song.

[0024] Note that the system configuration in which the elements shown in Figure 1 are implemented is not limited to the example above. In other examples, DJ equipment such as a mixer or an all-in-one DJ system (digital audio player with communication and mixing functions) may be included in place of or in conjunction with the PC11, and at least part of the learning step and cue point prediction step may be performed in these DJ devices. Furthermore, the learning step and cue point prediction step do not each have to be performed in a single device, but may be distributed and performed in multiple devices.

[0025] Figure 5 is a schematic block diagram showing the functional configuration for executing the cue point prediction step in one embodiment of the present invention. In the illustrated example, a feature vector acquisition unit 110, a converter 100 (conversion unit), a cue point candidate position suggestion processing unit 120, and a cue point automatic setting processing unit 130 are implemented as functions. The feature vector acquisition unit 110 acquires the feature vectors of the song to be processed in the cue point prediction step. The feature vectors of the song are extracted, for example, by a song analysis performed in advance and stored in the PC 11 or server 14 in association with the song. Alternatively, the feature vector acquisition unit 110 may include a song analysis function and extract feature vectors from the song's audio data or song development section data.

[0026] The converter 100 (conversion unit) converts the feature vector of the song acquired by the feature vector acquisition unit 110 into a cue point vector representing the positions of cue points within the song. As already explained, the converter 100 used in the cue point prediction step is trained in the learning step and can output a cue point vector representing the positions of cue points that are predicted to be appropriate for the input of the song's feature vector. For example, if the converter 100 outputs the cue point vector [1.8,1.2,1.5,-0.4,1.5,-1.2,-1.6,1.1,0.1] for the input of the song's feature vector, then by converting the top 5 elements to "1" and the remaining elements to "0", the cue point vector [1,1,1,0,1,0,0,1,0] is obtained, and the prediction result is that it is appropriate to set cue points at the 1st, 2nd, 3rd, 5th, and 8th development change points of the song. In the above example, the number of elements to set to "1" may be selected by the user, or it may be automatically determined according to the upper limit of the number of settable cue points. As another example, the average value of all elements in the cue point vector, or any other predetermined value, may be used as the baseline, and elements greater than or equal to the baseline may be converted to "1" and elements less than the baseline to "0". Note that the prediction results do not necessarily have to be expressed using "1" and "0".

[0027] The cue point candidate position presentation processing unit 120 and the cue point automatic setting processing unit 130 are examples of cue point-related processing units that perform processing using the cue point vector acquired by the converter 100, and either one or both may be implemented. The cue point candidate position presentation processing unit 120 presents the positions of development change points within the song, indicated by the cue point vector, to the user as cue point candidate position information. The cue point candidate position information is displayed on the display of the PC 11 or DJ controller 12 in a form similar to the cue point display 504 of the song information display shown in Figure 2. On the other hand, the cue point automatic setting processing unit 130 automatically sets cue points at positions within the song where it is predicted that setting cue points is appropriate based on the cue point vector. The automatically set cue points are displayed on the display of the PC 11 or DJ controller 12, for example by the cue point display 504 of the song information display shown in Figure 2.

[0028] According to the processing of the cue point candidate position suggestion processing unit 120 as described above, the user can select a cue point within the song after being presented with a list of candidate positions in advance. Similarly, with the automatic cue point setting processing unit 130, the user can select a cue point to use by trying out playback from each of the cue points that have been automatically set within the song. In either case, because an appropriate cue point is predicted based on the cue point vector, even a beginner can easily recognize an appropriate position within the song to set a cue point.

[0029] Figure 6 is a flowchart illustrating an example of processing when the cue point candidate position presentation function is implemented with the functional configuration shown in Figure 5. In the illustrated example, first the user selects a song by operating the PC 11 or DJ controller 12 (step S101), and a cue point vector is acquired for the selected song (step S102). In this step, for example, the feature vector acquisition unit 110 estimates the cue point position from the feature vector of the selected song. Alternatively, a cue point vector that has been pre-generated using the feature vector acquisition unit 110 and the converter 100 may be read. Once the cue point vector of the selected song is acquired, the cue point candidate position presentation processing unit 120 presents cue point candidate position information to the user using the display of the PC 11 or DJ controller 12 (step S103). The user sets the cue point by referring to the PC 11 or DJ controller 12 while referring to the presented information (step S104). Subsequently, when playback from a cue point is instructed by the user (step S105), the PC 11 or DJ controller 12 plays the song from the instructed cue point (step S106).

[0030] Figure 7 is a flowchart illustrating an example of the process when the automatic cue point setting function is implemented with the functional configuration shown in Figure 5. 6Similar to the previous example, the user selects a song (step S201), and the cue point vector of the selected song is obtained (step S202). Once the cue point vector of the selected song is obtained, the automatic cue point setting processing unit 130 automatically sets the cue points within the song (step S203). The positions of the automatically set cue points are presented to the user by being displayed on the display of the PC 11 or DJ controller 12. The user operates the PC 11 or DJ controller 12 while referring to the presented information to instruct playback from the cue point (step S204). When playback from the cue point is instructed, the PC 11 or DJ controller 12 plays the song from the instructed cue point (step S205).

[0031] The algorithm described so far is an algorithm that predicts general cue point setting trends from the cue point setting trends of a large number of sampled users. Next, we will explain the algorithm for cue point prediction using user attributes. The algorithm for cue point prediction using user attributes is an algorithm that predicts cue setting trends that reflect the preferences of a particular user, i.e., an individual, by statistically analyzing the cue point setting trends of that individual.

[0032] In one embodiment of the present invention, the algorithm for cue point prediction using user attributes obtains a score representing the degree of cue prediction based on the frequency distribution of each type of cue point development change point set for each development change point and the beat positions 4, 8, 16, 32, and 64 beats before it. Figure 8 is a diagram illustrating the classification of user attributes using cue point vectors in one embodiment of the present invention. The graph shown in Figure 8 shows the frequency distribution of each type of cue point development change point set by a specific user in a song. In the graph, the vertical axis shows the song development section after the development change point, and the horizontal axis shows the song development section before the development change point, and the number of beats between the development change point and the cue point set near the development change point. For example, the region where "CHORUS" on the vertical axis and "INTRO-16" on the horizontal axis intersect shows the frequency at which a cue point was set 16 beats before the development change point, where "INTRO" is the preceding point and "CHORUS" is the following point. The region with the highest frequency is where the vertical axis "CHORUS" intersects with the horizontal axis "UP," followed by the region where the vertical axis "CHORUS" intersects with the horizontal axis "CHORUS."

[0033] The scores obtained from the Algol rhythm for cue point prediction without using user attributes are normalized to [0,1], and the scores obtained from the Algol rhythm for cue point prediction using user attributes are also normalized to [0,1]. The scores are then added together for each beat to determine the score for that beat. The beats with the highest scores are then used as the predicted cue point positions. In addition to summation, the scores can be weighted and added together, or multiplied.

[0034] When considering user attributes, we consider the distribution of when cue points are frequently set, i.e., a statistical analysis of cue points. We may also consider the genres of music that users frequently use. Furthermore, we may consider where within a song cue points are frequently placed. For example, tendencies such as being more common in the first half of a song, or also in the second half, can be represented by a vector. For a single cue point, its position within the song can be represented by the numerical value (0,1). For multiple cue points, the entire song can be divided into n parts and represented by an n-dimensional vector. For example, a song can be divided into 8 parts and represented by an 8-dimensional vector. Specifically, if we have an 8-dimensional vector [1,0.3,0.6,0.5,0.1,0.1,0.4,0.3], we can see that the user always places cue points at the beginning of the song, and also places them in the second half. Users can be classified by clustering such data. In addition, the era of music that users frequently use, or the type of cue points they set (e.g., memory cue, hot cue, loop, color, etc.) can also be used to classify users.

[0035] Figure 9 shows an example of a song feature vector that includes elements representing user attribute classification. In the example above, user attribute classification may be included, for example, as an element of the song feature vector. In this case, the song feature vector includes elements representing user attributes in addition to the elements of the vector representing the song's features. As already mentioned, the song feature vector may include features that indicate the song's development pattern or features that indicate the song's genre. Specifically, the vector representing the user attribute classification is a user vector generated from the types of development change points for which cue points are set. In the example shown in Figure 8, a user vector of up to 216 dimensions is generated for each combination of the song development interval after the development change point (6 types) and the number of beats between the cue point set near the development change point and the development change point (36 types). Alternatively, a user vector may be generated by summing the data for each type of cue point. For example, if data for two types, hot cues and memory cues, is summed in the example of Figure 8, a user vector of up to 432 dimensions is generated. Furthermore, the user vector may be generated using the proportion of different genres of songs owned by the user. The dimensionality of the user vector can be reduced using methods such as SVM or convolution. If the user vector has a large capacity, reducing the dimensionality can suppress the increase in computational complexity.

[0036] Figure 10 illustrates the processing when using a feature vector of a song that includes elements representing the user's attribute classification. In the illustrated example, the user classifier converts user attribute classification information, such as that shown in Figure 8, into attribute classifications with reduced dimensionality using methods such as SVM or convolution. In the learning step, the converter 100 is trained using the attribute classification generated by the user classifier for user U (see Figure 1) who provided the cue point setting result, and the feature quantities of the song for which the cue point is to be set. On the other hand, in the cue point prediction step, the attribute classification generated by the user classifier for users who will use the cue point prediction result, and the feature quantities of the song for which the cue point is to be predicted are input to the converter 100 to predict the position of the cue point. For example, by pre-setting a finite number of attribute classifications, the amount of information related to the user's classification can be generated within an appropriate range.

[0037] Figure 11 illustrates the process of learning a transformer for each user attribute classification. In the illustrated example, multiple transformers 100 are prepared for each user attribute classification. For user U (see Figure 1) who provided the cue point setting result, a transformer 100 corresponding to the attribute classification generated by the user classifier is learned. In the cue point prediction step, for users who will use the cue point prediction result, the learned transformer 100 corresponding to the attribute classification generated by the user classifier is used to predict the cue point position from the song's features.

[0038] As yet another example, in the learning step shown in Figure 1, the converter 100 may be learned for each type of cue point. As explained with reference to Figure 2, cue points can include multiple types, such as hot cues and memory cues. In the example above, the cue point vector was defined without distinguishing between these types of cue points, but the cue point vector may be defined to represent the type of cue point in addition to its position in the song. For example, for a song with hot cues and memory cues set as shown in Figure 2, the cue point vector [1,0,1,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0] may be defined. In this example, the odd-numbered elements of the cue point vector indicate whether a memory cue point is set near an expansion change point, and the even-numbered elements indicate whether a hot cue point is set near an expansion change point. [Explanation of symbols]

[0039] 10...System, 12...DJ controller, 13...Speaker, 14...Server, 100...Converter (Conversion Unit), 110...Feature Vector Acquisition Unit, 120...Cue Point Candidate Position Presentation Unit, 130...Automatic Cue Point Setting Unit, 501...Icon Image, 502...Title, 503...Song Waveform, 504...Cue Point Display, 505...Song Development Display, P1~P5...Cue Points, S1~S9...Development Change Points, Sec1...Section, Seg1~Seg6...Segments, Sn...Development Change Point.

Claims

1. A conversion unit that converts the feature quantities of a first song into information indicating the probability of cue points existing at each location within the second song, using a converter trained to convert the feature quantities of a first song into information indicating the probability of cue points existing at each location within the first song, A cue point-related processing unit that either presents the user with cue point candidate position information within the second song based on information indicating the probability of the existence of the cue point, or automatically sets the cue point within the second song. A music information processing device equipped with the following features.

2. The music information processing device according to claim 1, wherein, for each of the first and second music pieces, the information indicating the probability of the existence of the cue point is a cue point vector that includes at least an element corresponding to a position in the music where the cue point may exist, and the probability of the existence of the cue point at each position in the music is expressed by the value of the elements of the cue point vector.

3. The music information processing device according to claim 2, wherein the cue point vector includes a plurality of elements for each position in the music where a cue point may exist, corresponding to a plurality of types of configurable cue points.

4. The music information processing device according to any one of claims 1 to 3, wherein the aforementioned feature quantities are audio feature quantities in the vicinity of the development change points of the music.

5. The music information processing device according to any one of claims 1 to 3, wherein the aforementioned feature quantity includes a feature quantity indicating the development pattern of the music.

6. The music information processing device according to any one of claims 1 to 3, wherein the aforementioned feature quantity includes a feature quantity indicating the genre of the music.

7. The music information processing device according to any one of claims 1 to 3, wherein the aforementioned feature quantity includes a feature quantity indicating user attributes.

8. The music information processing device according to any one of claims 1 to 3, wherein the conversion unit converts the feature quantity into information indicating the probability that the cue point exists based on the user's attribute information.

9. A converter that transforms the feature quantities of a song into information indicating the probability that a cue point exists at each position within the song, and a learning device that learns to output information showing the same trend as the cue point setting results within the first song by a first user in response to the feature quantities of a first song, A music information processing device that uses the converter to convert the feature quantities of a second music into information indicating the probability of a cue point existing at each location within the second music, and presents candidate cue point location information within the second music to a second user based on the information indicating the probability of a cue point existing, or automatically sets cue points within the second music. A system that includes this.

10. The learning device learns the converter for each attribute information of the first user, which is set based on the position pattern of cue points set by the first user within the first song. The system according to claim 9, wherein the music information processing device converts the feature quantities of the second music into information indicating the probability that a cue point exists at each position within the second music, based on the attribute information of the second user.

11. The system according to claim 9 or 10, wherein the learning device uses nDCG (normalized discount cumulative gain) to evaluate whether the information output by the converter has the same trend as the cue point setting results in the first song by the first user.

12. The system according to claim 9 or 10, wherein, for each of the first and second pieces of music, the information indicating the probability of the existence of the cue point is a cue point vector that includes at least an element corresponding to a position in the piece of music where the cue point may exist, and the probability of the existence of the cue point at each position in the piece of music is expressed by the value of the element of the cue point vector.

13. A converter that transforms the feature quantities of a song into information indicating the probability that a cue point exists at each position within the song is trained to output information showing the same trend as the cue point setting results within the first song by a first user, given the feature quantities of a first song as input. Using the converter, the feature quantities of the second song are converted into information indicating the probability of a cue point existing at each location within the second song, and based on the information indicating the probability of a cue point existing, candidate cue point location information within the second song is presented to the second user, or cue points within the second song are automatically set. A method for processing music information, including the information itself.

14. A function that converts the feature quantities of a first song into information indicating the probability of cue points existing at each location within the second song, using a converter trained to convert the feature quantities of a first song into information indicating the probability of cue points existing at each location within the first song, Based on information indicating the probability of the existence of the aforementioned cue point, the system presents the user with information on potential cue point locations within the second song, or automatically sets the cue point within the second song. A program to make a computer realize this.

Images