Method for comparing musical interpretations, computer program product, and system

Abstract

The invention relates to a method (100) for comparing musical interpretations of a segment (3) of a piece of music (1) by users (70), wherein the users (70) can record the interpretations as actual data sets (14) and these are compared with a reference data set (2) for matches or deviations and a ranking between the interpretations of the segments (3) is thus determined.

Description

Method for comparing musical interpretations, computer program product, and system

[0001] The present invention relates to a method for comparing musical interpretations.

[0002] Furthermore, the invention relates to a computer program product for executing the method on a user's terminal device, as well as to a system comprising the terminal device and a server. State of the art

[0003] State-of-the-art methods for comparing musical recordings or interpretations involve displaying notes on a device or screen, and asking the user to play or sing along to the notes on an instrument. The process simultaneously determines any similarities between the notes played and the expected notes, and in the event of discrepancies, incorrectly played notes are colored red. Correctly played notes are colored green. However, such methods do not allow for a comparison of performance with that of other users; they merely serve to provide feedback to the user while singing or playing the instrument. Due to the lack of comparability, users cannot classify their performance and thus become demotivated. Disclosure of the invention

[0004] The present invention therefore has the object of providing a method for comparing musical interpretations, which allows a user to compare his interpretation of a piece of music with those of other users and is encouraged to specifically improve his musical skills.

[0005] The invention solves the stated problem through the subject matter of the independent patent claims. Preferred embodiments emerge from the dependent patent claims and the embodiments presented below.

[0006] According to a first step of the method according to the invention, a user selects a piece of music. At least one reference data set is assigned to the piece of music as a digital representation. The reference data set is preferably a digital representation of a temporal sequence of tones and / or musical notes, which can be stored on a digital storage medium.

[0007] A piece of music or musical work within the meaning of the present invention is understood to be a structured arrangement of sounds or tones that are arranged in such a way as to convey musical ideas, themes and / or feelings. A piece of music is considered to be an independent and self-contained composition that comprises one or more voices. A (musical) voice is defined as a coherent sequence of tones, regardless of whether they are produced or performed by a singing voice or an instrument. The voices can be provided parallel or simultaneous in the composition and thus sound together when the piece of music is performed, which can result in a (possibly harmonious) sound. By arranging the voices and the tones within the voices, various elements such as melody, harmony, rhythm, dynamics and timbre can be created, all of which are interwoven to form the piece of music.

[0008] The reference data set can serve as a digitally storable, structured collection of musical information, preferably encoded in a digital format. This reference data set contains, in particular, a series of commands / instructions or data points that represent the characteristics and elements of the composition. Such elements can include musical notation, timing, pitch, duration, dynamics, and various other parameters that enable precise reproduction or playback of the piece of music. The data points can also be digital samples of a recorded audio signal.

[0009] In particular, one or more reference data sets can be assigned to the piece of music, with each reference data set preferably representing an independent interpretation or performance of the piece of music. For example, one reference data set can correspond to a first interpretation by an expert or professional, and a second reference data set can correspond to a simplified version of the piece of music. It is also conceivable that different reference data sets correspond to interpretations using different instruments.

[0010] Furthermore, at least one segment is assigned to the piece of music, whereby the segment is defined by a start and end point within the reference data set. The segment represents a specific part of the piece of music, which includes the sequence of tones or notes between the start and end points. The segment can be assigned to a sentence, a section, a theme, or a phrase of the piece of music. correspond. Extensive pieces of music or compositions (such as symphonies or sonatas) can consist of individual movements, with each movement forming a closed section with its own character, tempo and / or key. Pieces of music or the individual movements of pieces of music can, in turn, have individual, definable sections (e.g. introduction, verse, chorus, bridge or coda) which differ in their structural or thematic features, or correspond to a single phrase of the piece of music or parts thereof. Themes are melodic or rhythmic ideas which appear repeatedly throughout the course of a piece of music and exhibit various variations. Phases are smaller musical units which are often made up of several motifs.

[0011] Especially when practicing or learning a piece of music, the units to be practiced or repeated are divided into such segments and learned independently of each other. For example, it can be helpful to study particularly difficult segments (movements, sections, themes, phrases, etc.) individually and then combine them with the remaining parts to create the piece.

[0012] According to the invention, in a further method step, an actual data record is recorded during a musical interpretation of at least a portion of the piece of music by the user. The portion of the piece of music recorded by the user comprises the entire segment. The actual data record is—analogous to the reference data record—preferably a digital representation of the portion of the piece of music interpreted by the user, in particular again in the form of a temporal sequence of tones and / or musical notes, which can be stored on a digital storage medium.

[0013] The actual data set can serve as a digitally storable, structured collection of musical information, preferably encoded in a digital format, and in particular containing a series of commands / instructions or data points representing the characteristics and elements of the composition (notation, timing, pitch, duration, dynamics, etc.) to enable reproduction of the recorded interpretation of the part of the musical piece. The data points of the actual data set can also be digital samples of a recorded audio signal.

[0014] In this context, the musical interpretation of a part of the piece of music is understood to mean the playing of notes by the user on an instrument or the singing of notes by the user, whereby the notes correspond to those notes of the piece of music that are assigned to the interpreted part of the piece of music.

[0015] According to the invention, in a subsequent method step, the actual data set is analyzed or compared with the reference data set in the area of ​​the segment for similarities or discrepancies in at least one musical parameter. By analyzing the similarities or discrepancies in the at least one musical parameter between the actual and reference data sets, the similarities or differences in the user's interpretations or playing styles of the piece of music can be determined. The musical parameter can be a parameter that can be used to determine whether and to what extent the piece of music interpreted by the user corresponds to the piece of music according to the reference data set in one or more characteristics of the composition (such as timing, note length, pitch, duration, dynamics, etc.).

[0016] According to the invention, in a further method step, a score is calculated based on the similarities or discrepancies in the at least one musical parameter. The score can be implemented as a mathematical function and can cumulatively consider the similarities or discrepancies in one or more musical parameters.

[0017] A method for comparing musical interpretations can thus be created which enables the user to compare his performance when interpreting a part of a piece of music (in the area of ​​a segment), for example when practicing or rehearsing the piece of music, and thus to create incentives for improvement or to enable an evaluation of his performance development over time.

[0018] According to the invention, a method can thus be created which, by comparing the interpretations, gives the user an incentive for further practice and improvement.

[0019] Further embodiments of the invention are presented below. Unless otherwise stated, the illustrated embodiments, or individual aspects of the embodiments, can be combined with one another in any way.

[0020] According to one embodiment of the invention, in a further method step, a ranking of the actual data set can be determined in comparison to other actual data sets based on the calculated rating. The ranking can reflect the order of interpretations or playbacks of the segment of the piece of music by users, with higher-ranked interpretations exhibiting a higher correspondence with the piece of music according to the reference data set. than those ranked lower. This allows the user to compare their performance with previous interpretations or attempts, or with the performances of other users. By ranking the interpretations according to their agreement with the reference data set, an incentive or incentive can be created for the user to improve their interpretation of the piece of music and perfect their rendition.

[0021] According to one embodiment of the invention, the at least one musical parameter can be selected from a list containing: played note, pitch, frequency, duration, volume, dynamics, timing, tempo, rhythm, or the like. The analysis of the correspondences between the actual data set and the reference data set can thus be performed based on these parameters and, for example, consider whether notes were played at all or if notes are incorrectly played in terms of their pitch, timing, duration, dynamics, or rhythm.

[0022] According to one embodiment, the rating can, for example, have a single value (overall rating) which reflects the overall degree of agreement between the piece of music interpreted by the user and the piece of music according to the reference data set.

[0023] According to a further embodiment, the score can be calculated as an overall score from several sub-scores. Each sub-score can, in turn, reflect a degree of agreement in a specific musical parameter. The sub-scores can, in particular, be selected from a list containing: a note-played score, a pitch score, a volume score, a dynamic score, a timing score, a tempo score, a rhythm score, or the like. For example, if all notes were played correctly at their pitch, a high or good pitch score can be achieved. However, if no note was played, played, or sung at the correct time, a low or poor timing score can be achieved. Furthermore, a note-played score can reflect whether all specified notes were played at all, or whether individual notes were omitted.All partial scores can then be included in the overall score and taken into account accordingly.

[0024] According to one variant, the sub-scores can be multiplied by a weighting factor and added to the overall score. These weighting factors allow the individual sub-scores to be weighted against each other depending on their importance to the overall score, thus exerting different influences on the overall score.

[0025] According to a further variant, the partial scores can be multiplied with each other to obtain the overall score.

[0026] According to yet another variant, the overall score can be calculated based on the sub-scores using a predefined formula. The individual sub-scores can be multiplied or added together, or additionally multiplied by weighting factors.

[0027] According to one implementation variant, the overall score and / or the subscores can assume a value between 0 and 100 percent agreement. A value of 0 percent would correspond to no agreement at all, while a value of 100 percent would correspond to complete agreement between the actual data set and the reference data set.

[0028] According to one embodiment of the invention, the reference data set can comprise at least one temporal progression of a musical parameter. For example, the reference data set can comprise, as one component, the temporal progression of one of the musical parameters described above, such as the presence of tones, the change in pitch, or the variation in volume over the course of the piece of music.

[0029] According to one embodiment of the invention, the reference data set can be a digital notation of the piece of music or a musical interpretation of the piece of music, in particular by an expert.

[0030] The digital notation can contain symbolic representations of notes or note sequences, such as MIDI (Musical Instrument Digital Interface) data or similar digital formats. Such symbolic representations of notes capture musical events and parameters or commands, such as pitch, duration, velocity, etc.; these events and parameters can then be processed accordingly by a synthesizer (e.g., a MIDI synthesizer) or a sound module to generate an audio signal. The synthesizer interprets the (MIDI) commands and, based on the specified parameters, generates one or more sounds, which combine to play the piece of music.

[0031] Such a reference data set can be created, for example, by recording MIDI data during an expert's interpretation of the piece of music. The expert can, for example, play the piece on an instrument with a MIDI interface.

[0032] Alternatively, such a reference dataset can be obtained by analyzing musical events and parameters from a recorded interpretation of the piece of music by an expert. This analysis can be performed digitally on a device or server using appropriate signal processing.

[0033] Alternatively, such a reference data set can also be formed by a (digital) sheet music that defines or specifies the musical events and parameters or commands, such as pitch, duration, velocity, etc., in musical notation. Such a sheet music can also be available, for example, as MIDI data.

[0034] According to one embodiment of the invention, the reference data set can be a digitized audio signal of a musical interpretation of the piece of music, in particular by an expert.

[0035] The digitized audio signal can consist of digital samples of an audio signal. An (analog) audio signal is captured, for example, by a microphone and converted into digital samples of the audio signal by an analog-to-digital converter (ADC). The samples can be recorded by the converter at a specific sampling frequency and represent the amplitude of the audio signal at a specific (discrete) point in time. By arranging the samples at the corresponding sampling frequency and outputting them, for example, via loudspeakers, the audio signal can be played back or reproduced.

[0036] According to one embodiment of the invention, the actual data set can be a digital representation of the musical interpretation of at least a portion of the piece of music, which completely encompasses the segment. In particular, the portion of the piece of music interpreted by the user can also encompass larger portions of the piece of music, as long as the segment itself is completely captured by the user's interpretation. For example, the user can, on the one hand, only play the portion of the piece of music that encompasses the segment and thus compete with other users. On the other hand, the user can also play a complete interpretation of the piece of music and thus simultaneously include one or more segments assigned to the piece of music.

[0037] According to one variant, after interpreting the entire piece of music, the actual data set can be compared with the reference data set in all segments assigned to the piece of music. The user can then obtain a ranking for a large number of segments simultaneously and thus compare their performance with that of other users.

[0038] According to one embodiment, the segments with the lowest ranking or with the lowest ratings can then be displayed to the user so that the user can further practice and improve on these segments.

[0039] According to one embodiment of the invention, the user can select or create the piece of music on a terminal device. For example, the user can select a desired piece of music if it has already been created by another user or by an expert, or create a desired piece of music themselves so that other users can also select this piece of music. To do so, the user can, for example, select a sheet music version of a piece of music via the terminal device, with the reference data set being created and saved from the sheet music version. Likewise, the user can select an audio file containing an interpretation of the piece of music, from which the reference data set is created and saved.

[0040] According to one embodiment of the invention, the user can select or create the segment on the device. If, for example, a piece of music is already available for selection but no segment has yet been assigned to it, the user can define and create a desired segment by specifying a start and end point. If the desired segment is already available, the user can select it on the device.

[0041] According to one embodiment of the invention, segments can be automatically created and / or suggested from the actual data set in one method step. If segments are automatically suggested to the user, or if segments are automatically created, for example, after recording an actual data set, the user can receive a list of their ratings in various segments during their first interpretation of a piece of music, even if no segments have been created previously.

[0042] According to one embodiment, the actual data set can be analyzed for potentially interesting segments that meet certain criteria (e.g., length, number of notes, note density, tempo, dynamics, difficulty, etc.) to automatically create segments. If a potentially interesting segment is found, for example, one characterized by a particular difficulty, a corresponding segment can either be created automatically or suggested to the user for creation.

[0043] According to a further embodiment, the segment selection can also be performed automatically. For this purpose, the actual data set can be compared with the reference data set to determine which segments of the piece of music are completely contained in the actual data set. These completely contained segments can then preferably be automatically selected for comparison with the reference data set.

[0044] According to one embodiment, the terminal device can have a display and / or a microphone. If the terminal device has a display, the user can be shown a selection of possible pieces of music or their associated segments, which the user can select for interpretation. Furthermore, the display can show the instructions or notes for playing the piece of music, so that the user can begin their interpretation of the piece of music directly without any additional aids. If the terminal device has a microphone, the user can easily record their interpretation of the piece of music directly on the terminal device.

[0045] According to one embodiment, the actual data set can be recorded via the device's microphone. This allows the user to begin interpreting the piece of music or segment after selecting it. The user's interpretation can then be recorded directly by the device's microphone as the actual data set, which can then be compared with the reference data set after the recording is complete.

[0046] The terminal device can be designed, for example, as a smartphone, a tablet, a smartwatch, a notebook computer, a computer, a games console or the like.

[0047] According to one implementation variant, the analysis of the reference data set and the actual data set for matches can be performed on the device itself. Furthermore, the scores and rankings can also be calculated on the device.

[0048] According to one variant, the result of the analysis, the rating(s) and / or the ranking can then be displayed on the display of the end device.

[0049] According to one embodiment, the actual data set can be transferred from the terminal device to a server connected to the terminal device, with the analysis for similarities or discrepancies between the actual data set and the reference data set being performed on the server. Optionally, the calculation of the score and / or the determination of the ranking can also be performed on the server.

[0050] The server can be a server connected to the end device via a network or the Internet, with the data being exchanged between the end device and the server via the network / Internet.

[0051] The server can also be another device belonging to the user that is connected to the end device. For example, the other device could be a smartphone, while the end device is a smartwatch. The actual data set can then be recorded via the smartwatch, for example, and transferred to the smartphone, where it is analyzed and a score or ranking is calculated. The analysis / score / ranking can then be displayed on the end device (the smartwatch) or on the server (the smartphone).

[0052] According to one embodiment, the similarities or discrepancies between the actual data set and the reference data set, and / or the score and / or the ranking, can be transmitted back from the server to the device and displayed graphically on the device. The graphical display on the device provides the user with a simple visual representation of their performance, encouraging them to further improve their interpretation or practice.

[0053] According to one embodiment of the method, a spectrogram of the reference data set and / or the actual data set, in particular a discrete, time- and frequency-resolved spectrogram, can be calculated to analyze the actual data set and the reference data set for similarities or discrepancies. A spectrogram is a visual representation of the frequency content of an audio signal over time. Frequency spectra (for discrete time periods) are displayed in sequence along a time axis, resulting in a 2-dimensional frequency spectrogram with time resolution.

[0054] According to one embodiment, the intensity values ​​of the time- and frequency-resolved spectrogram of the reference data set can be scaled to values ​​between a minimum and a maximum value, with the maximum value being greater than the minimum value. The intensity values ​​can then vary between the two extremes, whereby the height of the intensity value can reflect that a certain tone is expected with a certain probability at a certain point in time in the piece of music.

[0055] A high value in the frequency spectrogram (near the maximum value) of the reference data set can signal that at a certain point in time in the piece of music, a certain note is expected. A low value in the frequency spectrogram (close to the minimum value) of the reference data set, on the other hand, can signal that no note is expected at a certain point in time. At a given point in time, the frequency spectrogram can also show several high values ​​at different frequencies. In this case, according to the piece of music, several notes can be expected at the same time. Since notes in a piece of music are usually held for a certain period of time, expected notes in the frequency spectrogram are usually represented as bars elongated along the time axis. The individual bars can partially overlap or - depending on the playing style and dynamics of the piece of music - merge into one another at their ends. Since the loudness of the notes can vary over the entire holding period, the intensity values ​​in the frequency spectrogram can also vary along the bars.

[0056] According to one embodiment, the intensity values ​​of the spectrogram of the reference data set and / or the spectrogram of the actual data set can be given by a function l(t, f) according to the following formula: where p(t, f) corresponds to the probability of the presence of a tone with frequency f at time t and s(t, f) corresponds to a score weight at frequency f at time t.

[0057] For example, the probability p(t, f) can take the value 1 within the time and frequency window of an expected tone and the value 0 outside of this time and frequency window. It is also conceivable that, in addition to the actually expected tone, other secondary tones can have a probability p(t, f) greater than 0, which allows tone deviations to be taken into account during playback.

[0058] The evaluation weighting s(t, f), in turn, can be used to broaden or weaken the function p(t, f). In an extreme case, p(t, f) can be designed as a step function that only returns a value other than 0 in the range of expected notes. Such a "hard" function would not allow any playing deviation and would greatly reduce the achievable score for individual errors. Through the evaluation weighting, the "hardness" of the probability weighting p(t, f) can be reduced, for example, by broadening or weakening the steps or flanks. This broadening can vary depending on the importance of the note or the position in the piece of music, so that a "harder" evaluation of playing deviations can be achieved in some areas and a "milder" evaluation of playing deviations in others.

[0059] If the frequency spectrogram of the reference data set is compared with a frequency spectrogram of the actual data set, matches between the reference data set and the actual data set can be positively weighted. If the actual data set contains a tone at a specific time-frequency pair and the intensity value in the frequency spectrogram of the reference data set has a positive value for this same time-frequency pair (i.e., if this tone is expected), this can be incorporated into a positive (matching) evaluation of the analysis or comparison. If, on the other hand, the frequency spectrogram of the reference data set does not contain a positive value for this time-frequency pair (i.e., if this tone is not expected), this can be incorporated into a negative (deviating) evaluation of the analysis or comparison.Alternatively, the frequency spectrogram of the reference data set can also have a zero value, whereby a tone in the actual data set for this time-frequency pair would not be included in the evaluation of the comparison.

[0060] According to one embodiment of the invention, to calculate the respective time- and frequency-resolved spectrogram, the actual data set and / or the reference data set can be divided into discrete time windows, and a frequency spectrum can be calculated for each time window using a time-frequency transformation. The division into discrete time windows can be carried out statically or dynamically. With a static division, the time windows are selected to be of equal size, and the frequency spectrum is calculated from the respective data set for this time window. With a dynamic division into time windows, the time windows can vary in length depending on the frequencies occurring in the data set. This allows the sharpness and resolution of the spectra to be adapted and optimized to the frequencies of the sounds.

[0061] According to one embodiment, the time windows can have different lengths and partially overlap. This allows the sharpness and resolution of the spectra to be optimized even when, for example, several tones with different frequencies occur simultaneously in the data set.

[0062] According to one embodiment, the time-frequency transformation can be a Fourier transformation, a wavelet transformation, a constant-Q transformation, or a Mel transformation. Such time-frequency transformations can efficiently and reliably decompose an audio signal into its spectral components. A Mel transformation is generally understood to be a transformation for obtaining a time-frequency spectrogram, which has been adapted to the pitch perceived by a person by scaling the frequency axis. Such scaling can therefore also be advantageous in the case of tolerances and broadening of the spectrogram, since these can always be done with a constant width in the frequency direction.

[0063] In the case of a Fourier transform, the resolutions in the time and frequency domains depend only on the shape of the time window. It is not possible to achieve high resolution in both the time and frequency domains simultaneously, which is referred to as time-frequency uncertainty. The resolution in the time domain is inversely proportional to the resolution in the frequency domain. However, for data sets that contain both high and low frequencies (pitches), it is desirable to achieve high frequency resolution for low tones (frequencies), as frequencies and frequency changes are crucial for determining the correct pitches, and high time resolution for high tones (frequencies), as frequencies can change quickly due to the short oscillation length.

[0064] In wavelet transformation, in addition to shifting the time window in the time domain, the length of the time window is also scaled or varied. This allows for better temporal resolution at higher frequencies, while improving frequency resolution at lower frequencies.

[0065] A constant-QT transformation is another type of time-frequency transform, related to Morlet wavelet transforms. This transform exhibits a reduction in frequency resolution at higher frequencies, which is desirable for auditory applications because it reflects the natural hearing ability of the human ear. Frequency resolution is better at low frequencies, while temporal resolution is better at higher frequencies.

[0066] According to one embodiment, the edges in the spectrograms of the reference data set can be broadened and / or smoothed along the time and / or frequency axes. Broadening the edges in the spectrogram of the reference data set can, for example, allow for a certain degree of flexibility in the interpretation of the piece of music. Broadening the edges in the time domain (along the time axis) can reduce the “sharpness” of the point in time at which a note should be played, thus allowing certain deviations in timing or duration of a note to still lead to agreements in the analysis between the actual data set and the reference data set. Likewise, broadening the edges in the frequency domain (along the frequency axis) can reduce the “sharpness” of the pitch that a played note should have, thus allowing certain deviations in pitch to still be lead to similarities in the analysis between the actual data set and the reference data set.

[0067] According to one embodiment, broadening in the time or frequency domain can be achieved by applying a distribution function to the spectrogram or to parts of the spectrogram. For example, applying a Gaussian distribution can achieve a bell-curve-like broadening of the bars along the time and / or frequency axis. The intensity values ​​can vary between the maximum values ​​in the center of the bar and the minimum values ​​at the end of the broadening.

[0068] As described above, the broadening can be achieved by applying a weighting function s(t, f).

[0069] According to a further embodiment, the broadening can also be applied to the flanks in the spectrogram of the actual data set, as previously described for the reference data set.

[0070] According to one embodiment, when analyzing the actual data set and the reference data set, their spectrograms can be examined for similarities in at least one musical parameter. The musical parameters (as described above) can be calculated or extracted from the spectrograms.

[0071] A pitch or tone frequency parameter can be derived, for example, by comparing the position of the bars (which represent the played notes) along the frequency axis in the spectrograms of the actual data set and the reference data set.

[0072] A tone length parameter can be derived, for example, by comparing the length of the bars along the time axis in the spectrograms of the actual data set and the reference data set.

[0073] Likewise, a timing parameter can be derived by comparing the start and / or end positions of the bars along the time axis in the spectrograms of the actual data set and the reference data set.

[0074] In a similar way, other parameters can also be derived from the spectrograms.

[0075] In general, it is noted in this context that the spectrogram of the actual data set and / or the reference data set is not merely the result of a pure time-frequency transformation, as described above. Rather, it can preferably be a complete system for determining the probability of whether a particular musical tone is present at a particular time-frequency position in the spectrogram. This includes the entire processing of the spectrograms, in particular including the determination that a single tone can be recognized as a multitude of overtones in the spectrogram (obtained directly through a time-frequency transformation), or how certain tones are represented in the spectrogram.

[0076] According to one embodiment, overtones can be removed from the spectrogram of the actual data set and / or the reference data set before comparing the actual data set and the reference data set. The overtone removal can be performed taking into account various parameters, such as the timbre of the instrument or voice used to reproduce the piece of music, or the spatial conditions during the reproduction of the piece of music. By removing the overtones and reducing the spectrograms of the actual data set and the reference data set to the fundamental tones, better comparability can be achieved, as the comparison of the actual data set and the reference data set can be carried out independently of instruments, reproduction formats, and spatial conditions.

[0077] According to one variant, the removal or identification of overtones in the spectrograms of the actual data set or the reference data set can be achieved, for example, by applying AI models or machine learning methods. For example, the characteristic overtones can be learned by analyzing the interpreted piece of music and later removed from the spectrograms accordingly.

[0078] According to a further embodiment, a mathematical model for describing the overtones as a function of a fundamental tone can also be used to remove the overtones from the spectrograms of the actual data set and the reference data set.

[0079] According to one embodiment, an AI model (Artificial Intelligence) and a mathematical model can also be combined to remove the overtones from the spectrograms of the actual data set and the reference data set.

[0080] According to one embodiment of the invention, to compare the actual data set and the reference data set, the intensity values ​​of the spectrograms for each discrete time-frequency pair can be multiplied to form intensity products. In this way, the comparison between the data sets can be carried out particularly easily using a mathematical function. The intensity product for a time-frequency pair can then assume a positive, negative, or zero value, with a high value representing a high agreement and a low value representing a low agreement between the reference data set and the actual data set.

[0081] According to a further embodiment, the spectrograms of the reference data set and the actual data set can be displayed and stored as a matrix with discrete time and frequency axes. This allows the intensity products between the matrices of the reference data set and the actual data set to be calculated by multiplying the matrices point by point. This creates a technically particularly simple method.

[0082] According to a further embodiment, the intensity products of all discrete time-frequency pairs can be summed to form an intensity product sum to calculate the score. A higher intensity product sum is associated with a higher ranking in the ranking list. After the individual intensity products have been calculated, a score can be derived particularly easily by simply adding all intensity products together. Negative product sums reduce the score, while positive product sums increase it. Zero values ​​have no effect on the score.

[0083] According to one embodiment, the time resolution of the spectrogram of the reference data set and / or the actual data set can be converted into a relative time component and a tempo component. This can be used, in particular, to decouple the time component of the spectrogram from the actual tempo of the interpretation (playing speed / reproduction speed). The tempo component reflects the actual tempo of the interpretation, while the relative time component is an artificial time component determined by the length of the respective data set.

[0084] According to one embodiment, the relative time components of the reference data set and the actual data set can be synchronized before comparing the actual data set and the reference data set. This synchronization can be performed using fixed points in the reference data set and the actual data set. The relative time components are synchronized to the start and end times of the segment. The start and end points of the segment thus have identical relative time values ​​in the reference data set and the actual data set. This allows the time components in those parts of the reference data set and the actual data set that are attributable to the segment to be directly compared. This scaling can be easily performed for the reference data set, since the segment itself is defined by a start and end time in the reference data set. For the actual data set, the start and end times of the segment may first need to be determined.

[0085] According to one embodiment, the start and end times of the segment in the actual data set can be determined in a separate step, in which the actual data set is analyzed accordingly to locate the segment. This can be done, for example, by recognizing various tone sequence patterns in the actual data set.

[0086] According to a further embodiment of the invention, the start and end times of the segment can also be determined by the user. For example, the user can mark the start and end of the segment in the actual data record.

[0087] According to one embodiment, a start delay can be removed from the actual data record before comparing the actual data record and the reference data record. The start time in the actual data record is shifted relative to the start time in the reference data record. Such a start delay by the user can create a pause at the beginning of the recorded actual data record, which complicates a direct comparison between the reference data record and the actual data record. By removing the start time delay, the actual beginning of the actual data record segment can be determined, which can further improve the synchronization between the relative time components of the reference data record and the actual data record.

[0088] According to one embodiment, an end delay can also be removed from the actual data set before comparing the actual data set and the reference data set. This can occur, for example, if the user ends the recording too late after interpreting the piece of music or segment, resulting in a pause in the playing. Removing an end delay can also further improve the synchronization between the relative time components of the reference data set and the actual data set.

[0089] According to one embodiment, the removal of the start delay and / or the end delay can be determined by applying a cross-correlation function between The actual data set and the reference data set are compared. Using the cross-correlation function, the start and end points of the segment can be reliably determined in the actual data set, allowing the actual data set to be adjusted for the start and / or end delay, so that the relative time components of the reference data set and the actual data set within the segment match as closely as possible without any shifts.

[0090] According to one embodiment of the invention, the reference data set and / or the actual data set can be divided into discrete time windows, wherein a delay between the actual data set and the reference data set can be calculated in each time window. This allows a time-dependent delay component to be calculated, which exhibits a (temporally local) delay compared to the target data set at each point in time of the actual data set, in particular corresponding to the relative time component. The delay component can thus reflect whether the user was too early or too late, or too fast or too slow in interpreting the piece of music, at each point in time.

[0091] According to one implementation variant, the discrete time windows can be selected based on musical events in the reference data set. For example, the beginning of the time windows can coincide with notes to be played or expected in the reference data set.

[0092] According to one embodiment, the discrete time windows can be selected with a constant length, preferably 2 s. According to an alternative embodiment, the length of the time windows can also be adapted to musical events in the reference data set—for example, depending on the length of the notes to be reproduced.

[0093] According to one implementation variant, the discrete time windows can overlap. For example, an overlap of 50% or 75% can further improve the resolution when calculating the delay.

[0094] According to one embodiment, the time-dependent delay component can be determined by applying a (temporally local) cross-correlation function between the actual data set and the reference data set.

[0095] The time-dependent delay component can be smoothed according to one embodiment, for example by applying a smoothing function to remove statistical outliers.

[0096] According to one embodiment, before comparing the actual data set and the reference data set, the local temporal delay can be removed from the actual data set corresponding to the delay component. This completely synchronizes the actual data set and the reference data set along the entire temporal sequence, so that the respective musical events in the actual data set and the reference data set occur at the same times, corresponding to the relative time component. This also reliably eliminates shifts within the piece of music during interpretation by the user, for example, because the user took an unintended pause during interpretation or because the user's timing was incorrect. This enables a comparison of the actual data set and the reference data set without taking into account the timing and any delays in playback.

[0097] The time-dependent delay component can represent the temporal progression of a musical parameter.

[0098] According to one embodiment, a timing score can be calculated from the time-dependent delay component, both overall and for each individual note.

[0099] In addition, according to another embodiment, the actual data set can be restricted to the area containing the segment before comparing the actual data set and the reference data set. This removes data from the actual data set that is not required for the comparison, for example, if the actual data set contains other parts of the piece of music in addition to the segment.

[0100] According to one embodiment, the actual data set can be shifted and / or scaled along the frequency axis before comparing the actual data set and the reference data set in order to eliminate any possible transposition. Transposition here means changing the pitch of notes by a specific interval or shifting an entire piece of music into a different key with the same interval. This can be advantageous, for example, if the user is playing the piece of music on a different instrument, in a different key, or in a different vocal range, so that the target data set is shifted in pitch or frequencies compared to the actual data set. In this case, the frequency axes of the actual data set and the reference data set can be scaled and / or shifted so that they are in the same key / vocal range. This makes it possible to compare the reference data set and the actual data set even if the interpretations differ.

[0101] According to one embodiment, the tempo components of the actual data set and the reference data set can be used to calculate the tempo score. The tempo components can, for example, be simple speed values ​​after the relative time components of the reference data set and the actual data set have been synchronized. These speed values, usually recorded in bpm ("beats per minute"), can then be directly compared, and a tempo score can be calculated based on the degree of agreement.

[0102] The invention further achieves the stated object by a computer program product comprising instructions which, when the program is executed by a user's terminal, cause the user to execute a method according to the embodiment variants described above.

[0103] The invention also achieves the stated object by a system comprising a user's terminal and a server connected to the terminal, wherein the terminal and the server are programmed to execute a method according to the embodiments described above.

[0104] The embodiments described above with reference to the method according to the invention are equally applicable to the computer program product executable on the terminal device, as well as to the system comprising the terminal device and the server. To the extent that the above description describes embodiments with reference to a user's terminal device and a server, these explanations also apply analogously to embodiments of the system. Short description of the characters

[0105] Preferred embodiments of the invention are described in more detail below with reference to the drawings. They show: Fig. 1 is a schematic representation of a first method step of the method according to a first embodiment of the invention, Fig. 2 is a schematic representation of a time and frequency resolved spectrogram of the reference data set, Fig. 3 is a schematic representation of a spectrogram of the reference data set broadened along the time and frequency axes, Fig. 4 is a schematic representation of a further method step of the method according to the first embodiment, Fig. 5 is a schematic representation of a time and frequency resolved spectrogram of the actual data set, Fig. 6 is a schematic representation of the superposition of the spectrograms of the reference data set and the actual data set, Fig. 7 is a schematic representation of the superposition of the spectrograms of the Reference data set and actual data set after synchronization of reference data set and actual data set, Fig. 8 is a schematic representation of a further method step of the method according to the first embodiment, and Fig. 9 is a schematic flow diagram of the method according to the first embodiment. Ways to implement the invention

[0106] The invention is described below by way of example with reference to the embodiments shown in the figures.

[0107] Fig. 1 shows a schematic representation of a first method step of a method 100 according to a first embodiment of the invention. A user selects a piece of music 1 on a terminal 50, which in this case is a smartphone. The piece of music 1 is digitally represented by a reference data set 2, which is stored on a server 60 in a suitable file format (e.g., XML, MIDI, etc.).

[0108] Reference data set 2 can contain symbolic representations of notes (instructions) that are expected at a certain point in time to correctly reproduce the piece of music. Alternatively, reference data set 2 can also be a recorded interpretation of piece of music 1, created by an expert or an artist as a target value to be imitated. However, for the sake of simplicity, it is assumed below that piece of music 1 is present in reference data set 2 as symbolic notation.

[0109] According to a further embodiment variant, which is not shown in detail in the figures, the reference data set 2 can also be stored on the terminal 50 itself, and the method steps described below with reference to the server 60 can be carried out on the terminal 50.

[0110] The piece of music 1 has one or more segments 3, each of which is defined by corresponding start and end points 4, 5 in the reference data set 2. As shown in Fig. 1, the piece of music 1 selected by the user comprises a first segment 3.1 and a second segment 3.2, which are present at different times in the piece of music 1. The segments 3 each comprise a partial section of the piece of music 1 and may overlap, although this is not shown in detail in the figures.

[0111] User 70 can also create a new segment 3 if a corresponding segment 3 is not available. The newly created segment 3 can then be shared with other users 70 so that they can select it for an interpretation of the piece of music 1.

[0112] The user 70 can also specifically share a segment 3 with another user so that the user is asked to interpret the piece of music 1 in this segment 3 and the users can compare their performance.

[0113] User 70 selected the second segment 3.2 for interpretation, which is shown in more detail in the detailed view in Fig. 1. Here, the instructions 6 of the musical piece 1 are represented in musical notation according to the reference data set 2. Segment 3.2 comprises three bars of the musical piece 1, with the beginning of segment 3.2 being marked by the starting point 4 and the end of segment 3.2 by the end point 5.

[0114] The instructions 6 in reference data set 2 can also be specified in a table and converted or separated into a relative time component 7 and a tempo component 8. The relative time component 7 specifies the relative time and the relative length of an instruction 6 within segment 3. The tempo component 8, in turn, specifies the absolute speed (clock speed in BPM) at which the instructions follow one another. This tempo can also change during a segment 3, which is represented by the different entries in the table of tempo component 8.

[0115] As shown in the table of the relative time component 7 in Fig. 1, the reference data set 2 essentially consists of instructions 6, which are identified by their start time T rei and their length L. For example, a first instruction 6 can be, at time T re i=0 a tone e 1with a length of 0.5 relative time units and a relative volume of 0.8. A second instruction 6 would be to play at time T re i=0.5 a tone d 2 with a length of 0.5 relative time units and a relative volume of 0.8. A third instruction 6 would be to Time T re i=1 ,0 a tone g 1 with a length of 1.0 relative time units and a relative volume of 0.8, etc.

[0116] The scaling of the relative time component 7 is given in bars, for example. A relative time unit of 1.0 corresponds to a whole note, a relative time unit of 0.5 to a half note, and a relative time unit of 0.25 to a quarter note, etc.

[0117] According to Fig. 2, the reference data set 2, as shown in Fig. 1 according to the table of the relative time component 7, is represented as a time- and frequency-resolved spectrogram 9 of the reference data set 2. The instructions 6 are translated into bars 10 in the time-frequency domain, with the bars 10 being elongated over their relative duration along the time axis 11 and representing the expected notes during the playback or interpretation of the piece of music 1. The time axis 11 is scaled in the relative time component 7.

[0118] The bars 10 are located along the frequency axis 12 at the level of the frequency of the respective tone and also have a certain broadening along the frequency axis 12, which can compensate for any frequency deviations between the same tones (e.g. due to different tuning of instruments).

[0119] Fig. 3 shows a frequency-resolved spectrogram 9.1 of the reference data set 2, wherein the bars 10.1 have been broadened both along the time axis 11 and along the frequency axis 12 compared to the bars 10 from the frequency-resolved spectrogram 9 from Fig. 2. The broadening was created by applying a Gaussian distribution to the bars 10 and thus ensures a blurring of the edges 23 of the bars 10.1 along the axes 11, 12. Due to the blurring, deviations in frequency and time can be disregarded to a certain extent. The Gaussian distribution can have a different decay along the time axis 11 than along the frequency axis. The timing at the beginning of the bars 10.1 can therefore allow for a certain deviation. In the central region of the bars 10.1, the spectrogram 9.1 has maximum intensity values ​​of +1. In the areas outside the bars 10.1 , i.e. also outside the Gaussian distributed transitions, the spectrogram 9.1 intensity values ​​from -1. The spectrogram 9.1 can be represented as a matrix with a discrete time axis 11 and frequency axis 12, where the values ​​of the matrix are the intensity values ​​between +1 and -1. The intensity values ​​can also take into account the loudness or amplitude of the expected sound.

[0120] According to Fig. 4, a further step of the method 100 is shown, in which the instructions 6 for interpreting the piece of music 1 in the area of ​​the selected segment 3 are displayed to the user 70 on the terminal 50. In particular, the instructions 6 are displayed in such a way that the user 70 can interactively read along during their interpretation, while the interpretation is recorded by the terminal 50 via a microphone 13. The terminal 50 stores the interpretation of the piece of music 1 as an actual data set 14 and sends this actual data set 14 to the server 60 for further processing and analysis. The actual data set 14 consists of a recorded digital audio signal 15, which can be represented as intensity over time.

[0121] In a further embodiment, the user 70 can also play or interpret the entire piece of music 1. During the process, the actual data set 14 recorded by the user can then be analyzed for segments 3 it contains, and all segments 3 contained therein can be automatically selected for analysis. The analysis described below in relation to the figures can then be performed separately for each segment 3.

[0122] This actual data set 14 can be converted into a time- and frequency-resolved spectrogram 16 by applying a time-frequency transformation on the server 60, as shown in Fig. 5. In contrast to the frequency-resolved spectrogram 9, 9.1 of the reference data set 2, the frequency-resolved spectrogram 16 of the actual data set 14 does not have discrete bars, but rather continuous intensity distributions along the frequency axis 12. For the sake of simplicity, Fig. 5 only shows the maximum values ​​of the respective frequency spectrum at a given time, which together form a (discontinuous) curve along the time axis 11. The time- and frequency-resolved spectrogram 16 is also converted into a relative time component 7 and a tempo component 8, so that the time axis 11 is scaled along the relative time component 7.

[0123] The spectrogram 16 of the actual data set 14 is, in particular, normalized so that the intensity values ​​vary between 0 and +1. For each time-frequency pair, the spectrogram 16 thus exhibits an intensity value corresponding to the spectral distribution, i.e., the intensity values ​​of the frequency spectrum at this time at the selected frequency. At the maximum values ​​of the spectra (represented as visible lines), the intensity values ​​are approximately +1; at frequencies without measurable intensity, the intensity values ​​are approximately 0. The spectrogram 16 can also be represented as a matrix of intensity values ​​with discrete time and frequency axes 11, 12.

[0124] After calculating the time-frequency transformation, all overtones of the respective identified fundamental tones were removed from spectrogram 16 of the actual data set 14, although this is not shown in detail in the figures. Spectrogram 9.1 of the reference data set 2 represents the expected notes during the playback or interpretation of the musical piece 1 and thus also contains no overtones. The two spectrograms 9.1 and 16 can thus be compared without the correspondence being distorted by any overtones.

[0125] In a further step of the method 100, the spectrograms 9.1, 16 of the reference data set 2 and the actual data set 14 are superimposed and synchronized by applying a cross-correlation function, as shown in Fig. 6. In this case, a start delay 17 and an end delay 18 are removed from the actual data set 14, if necessary, so that both spectrograms 9.1, 16 only comprise that portion of the piece of music 1 represented by the selected segment 3.2. After removing the start and end delays 17, 18, the two spectrograms 9.1, 16 are synchronously superimposed along the time axis 11 and can be compared accordingly.

[0126] According to a further embodiment variant, which is not shown in detail in the figures, a delay between the actual data set 14 and the reference data set 2 can also be calculated at multiple points in time. For this purpose, the reference data set 2 can, for example, be divided into discrete time windows, with a cross-correlation function being applied to the reference data set 2 and the actual data set 14 in each time window in order to calculate the local delay or shift between the actual data set 14 and the reference data set 2. The actual data set 14 or the reference data set 2 can then be shifted selectively or locally in such a way that the delay is removed or reduced. In this way, the reference data set 2 and the actual data set 14 can be synchronized over the entire time period in order to free the actual data set 14 from local time delays that may be caused, for example, by changes in tempo during interpretation.

[0127] If necessary, in a further method step, a shift along the frequency axis between the spectrograms 9.1, 16 is also removed, which shift has arisen, for example, by a transposition of the interpreted piece of music 1, which, however, has not been shown in more detail in the figures.

[0128] In Fig. 7, the spectrograms 9.1, 16 of the reference data set 2 and the actual data set 14 are superimposed and synchronized, i.e., corrected for shifts along the time and / or frequency axis 11, 12. The spectrogram 16 of the actual Data set 14 has also been restricted to the area of ​​segment 3.2. A direct comparison between actual data set 14 and reference data set 16 within the area of ​​the segment is now possible.

[0129] Fig. 8 illustrates a further step of method 100, in which the reference data set 2 and the actual data set 14 are compared based on their calculated spectrograms 9.1, 16. This is preferably done by displaying the intensity values ​​of the spectrograms 9.1, 16 in the form of a matrix representation, as previously described, which, however, was not shown in detail in the figures. The intensity values ​​in the matrices of the spectrogram 9.1 and the spectrogram 16 can then be multiplied point by point (element by element), and the resulting intensity products can be added together to form an intensity product sum.

[0130] As previously described, the intensity values ​​of spectrogram 9.1 of reference data set 2 vary between +1 and -1. The intensity values ​​of spectrogram 16 of actual data set 14 vary between 0 and +1. If these are now multiplied with one another, the result is a maximum intensity product of +1 if the recorded curve in actual data set 14 and blocks 10.1 in reference data set 2 exactly overlap, i.e. in an area where an expected note is heard. If the recorded curve lies far outside of block 10.1, i.e. in an area where no note is expected, the intensity product can take on negative values ​​down to -1. Positive products therefore represent a match between reference data set 2 and actual data set 14, while negative products represent a deviation. Values ​​in between can represent (minor) deviations, playing errors, or poor matches.For example, deviations in the timing of the played notes can lead to reduced values ​​in the products. Likewise, deviations in pitch can result in a reduction in the products. The intensity product sum can thus provide information about the quantity, intensity, and nature of the deviations or similarities.

[0131] The previously described synchronization between reference data set 2 and actual data set 14 by means of a cross-correlation function can utilize the above-mentioned intensity product sum by calculating the correlation based on the intensity product sum and achieving the correct synchronization at the maximum achieved product sum.

[0132] The matches and deviations can also be graphically displayed to the user 70 on their terminal device 50. Thus, matches in expected tones according to reference data set 2 can be displayed in a specific color (e.g., green), and deviations from expected tones can be displayed in a second color (e.g., blue). Red). The user 70 can then immediately recognize at which points in segment 3, 3.2 he made errors in the interpretation of the piece of music 1 and can thus be specifically encouraged to further improve.

[0133] From this comparison, a score 19 can now be calculated. For example, a timing score 19.1 and / or a pitch score 19.2 can be calculated from the intensity product sum. For this purpose, a function (e.g., a smoothing or broadening function) can be applied to the reference data set 2 or the actual data set 14 before the intensity product sum is calculated in order to disregard deviations in certain parameters (which do not affect the score). In addition, a tempo score 19.3 can be calculated from the comparison of the tempo components 8 of the reference data set 2 and the actual data set 14. The individual scores 19.1, 19.2, 19.3 can be combined to form the overall score 19 in order to determine a ranking 20 of the actual data set 14 or the user's interpretation 70. This ranking 20 can be transmitted from the server 60 back to the terminal 50 and displayed graphically there.In addition, the individual ratings 19, 19.1, 19.2, 19.3 can be displayed to the user 70 on the terminal device 50. For example, the user 70 can identify where they made errors in their interpretation, or in which aspect or rating 19, 19.1, 19.2, 19.3 they performed worse, in order to specifically improve their playing or singing through exercises.

[0134] According to one embodiment variant, recommendations for exercises or additional pieces of music 1 or segments 3 can be displayed to the user on the terminal device 50 according to his or her ratings or based on his or her previous interpretations, although this is not shown in more detail in the figures.

[0135] The method 100 described above according to Figs. 1 to 7 can thus comprise the steps 110, 115, 120, 121, 130, 131, 132, 133, 134, 135, 140, 141, 142, 150, 160, 170 described below. These steps do not necessarily have to be completed in the specified order, but can also partially run in parallel, provided they are not interdependent.

[0136] Selection 110 of a piece of music 1 by the user 70 on a terminal device 50, which is represented by a reference data set 2.

[0137] Selection 115 or creation of a segment 3 assigned to the piece of music 1 by the user 70 on the terminal 50.

[0138] Recording 120 of an interpretation of the piece of music 1 by the user 70 via a microphone 13 of the terminal 50 as an actual data record 14, wherein the actual data record 14 completely comprises the segment 3, and simultaneously displaying the instructions 6 on the terminal 50.

[0139] Transfer 121 of the actual data record 14 to the server 60.

[0140] Analysis 130 of the actual data set 14 and the reference data set 2 in the area of ​​the segment 3 on the server 60 for similarities or deviations in at least one musical parameter, wherein the analysis 130 comprises steps 131, 132, 133 and 134.

[0141] Calculating 131 a time- and frequency-resolved spectrogram 9 from the reference data set 2 and converting the time axis into a relative time component 7 and a tempo component 8 and storing the spectrogram 9 on the server 60.

[0142] Applying 132 a Gaussian distribution to the spectrogram 9 to smooth the flanks of the bars 10 and obtaining a broadened spectrogram 9.1 and storing it on the server 60.

[0143] Calculating 133 a time- and frequency-resolved spectrogram 16 from the actual data set 14 and converting the time axis into a relative time component 7 and a tempo component 8 and storing the spectrogram 16 on the server 60.

[0144] Synchronize 134 the spectrograms 9.1 and 16 by removing the start and end delays 17, 18 from the spectrogram 16 of the actual data set 14.

[0145] Multiplication 135 (point-by-point or element-wise) of the spectrograms 9.1 and 16 of the reference data set 2 and the actual data set 2 and obtaining intensity products 21 which represent the agreements or deviations between the reference data set 2 and the actual data set 14.

[0146] Calculating 140 a rating 19 from the intensity products 21, wherein the calculation 140 of the rating 19 comprises the sub-steps 141, 142 and 143.

[0147] Calculating 141 an intensity product sum 22 from the intensity products 21 and calculating a timing score 19.1 and a pitch score 19.2 from the intensity products 21.

[0148] Calculate 142 a tempo rating 19.3 from the comparison of the tempo components 8 of reference data set 2 and actual data set 14.

[0149] Calculate 143 an overall score 19 by weighting the partial scores 19.1, 19.2, 19.3 and save the overall score 19 on the server 60.

[0150] Determining 150 a ranking 20 from the total score 19 by comparing the total score 19 with total scores of interpretations of the same segment 3 by other users and storing the ranking 20 on the server 60.

[0151] Transfer 160 of ranking 20 from server 60 to terminal 50.

[0152] Display 170 or graphical representation of the ranking 20 on the terminal 50. List of reference symbols 1 piece of music 2 Reference data set 3 segments 3.1 First segment 3.2 Second segment 4 Starting point 5 Endpoint 6 Instruction 7 Relative time component 8 Tempo component 9 Spectrogram of the reference data set 9.1 Broadened spectrogram of the reference data set 10 bars 10.1 Widened beams 11 Timeline 12 Frequency axis 13 Microphone 14 Actual data record 15 Audio signal 16 Spectrogram of the actual data set 17 Start delay 18 Final delay 19 overall ranking 19.1 Timing rating 19.2 Pitch Rating 19.3 Tempo rating Ranking Intensity products Intensity product sum flanks End device server user Proceedings Selecting a piece of music Selecting a segment Recording an interpretation of the piece of music Transferring the actual data set to the server Analysis of the actual data set and the reference data set Calculating a spectrogram from the reference data set Applying a Gaussian distribution to the spectrogram Calculating a spectrogram from the actual data set Synchronizing the spectrograms Multiplication of spectrograms Calculating a score Calculating an intensity product sum Calculating a tempo rating, Calculating an overall score, Determining a ranking Transferring the ranking to the end device Display the ranking on the device

Claims

Patent claims 1 . A method for comparing musical interpretations, comprising the steps: - selection (110) of a piece of music (1) by a user (70), to which piece of music (1) a reference data set (2) is assigned as a digital representation and at least one segment (3, 3.1, 3.2), wherein the segment (3, 3.1, 3.2) is defined by a start point (4) and an end point (5) within the reference data set (2), - recording (120) an actual data record (14) during a musical interpretation of at least part of the piece of music (1) by the user (70), wherein the actual data record (14) completely comprises the segment (3, 3.1, 3.2), - analysis (130) of the actual data set (14) and the reference data set (2) in the area of ​​the segment (3, 3.1, 3.2) for similarities or deviations in at least one musical parameter, Calculating (140) a rating (19) based on the similarities or deviations in the at least one musical parameter.

2. Method according to claim 1, characterized in that the method further comprises the step of: determining (150) a ranking (20) of the actual data set (14) in comparison to further actual data sets on the basis of the calculated rating (19).

3. Method according to claim 1 or 2, characterized in that the at least one musical parameter is selected from a list containing: tone played, pitch, tone frequency, tone length, volume, dynamics, timing, tempo, rhythm.

4. Method according to claim 1, 2 or 3, characterized in that the rating (19) is calculated as an overall rating (19) from a plurality of partial ratings (19.1, 19.2, 19.3), wherein the partial ratings (19.1, 19.2, 19.3) are selected from a list comprising: a tone-played rating, a pitch rating (19.2), a volume rating, a dynamic rating, a timing rating (19.1), a tempo rating (19.3), a rhythm rating.

5. Method according to one of claims 1 to 4, characterized in that the reference data set (2) comprises at least one temporal course of a musical parameter.

6. Method according to one of claims 1 to 5, characterized in that the reference data set (2) is a digital notation of the piece of music (1) or a digitized Audio signal of a musical interpretation of the piece of music (1 ), in particular by an expert.

7. Method according to one of claims 1 to 6, characterized in that the actual data set (14) is a digital representation of the musical interpretation of at least a part of the piece of music (1), which completely comprises the segment (3, 3.1, 3.2).

8. Method according to one of claims 1 to 7, characterized in that the user (70) can select or create the piece of music (1) and / or the segment (3, 3.1, 3.2) on a terminal (50).

9. Method according to one of claims 1 to 8, characterized in that the terminal (50) has a display and / or a microphone.

10. The method according to claim 9, characterized in that the terminal (50) is selected from a list comprising: a smartphone, a tablet, a smartwatch, a notebook computer, a computer, a game console.

11. Method according to claim 9 or 10, characterized in that the actual data set (14) is recorded via a microphone (13) of the terminal (50).

12. The method according to claim 11, characterized in that the actual data record (14) is transmitted from the terminal (50) to a server (60) connected to the terminal (50), wherein the analysis (130) for correspondences or deviations between the actual data record and the reference data record and / or the calculation (140) of the evaluation and / or the determination (150) of the ranking takes place on the server.

13. Method according to one of claims 12, characterized in that the correspondences or deviations between the actual data record (14) and the reference data record (2) and / or the evaluation (19, 19.1, 19.2, 19.3) and / or the ranking (20) are transmitted from the server (60) to the terminal (50) and are graphically displayed on the terminal (50).

14. Method according to one of claims 1 to 13, characterized in that for the analysis (130) of the actual data set and the reference data set for matches or deviations, a, in particular discrete, time and frequency resolved spectrogram (9) of the reference data set (2) and / or a, in particular discrete, time and frequency resolved spectrogram (16) of the actual data set (14) is calculated.

15. Method according to claim 14, characterized in that the intensity values ​​of the time and frequency resolved spectrogram (9) of the reference data set (2) are Values ​​are scaled between a minimum value and a maximum value, where the maximum value is greater than the minimum value.

16. Method according to claim 15, characterized in that the minimum value is negative and the maximum value is positive.

17. Method according to one of claims 14 to 16, characterized in that the intensity values ​​I of the time- and frequency-resolved spectrogram (9) of the reference data set (2) are given by a function l(t, f) = p(t, f) * s(t, f), where p(t, f) corresponds to the probability of the presence of a tone with frequency f at time t, and s(t, f) corresponds to a weighting at frequency f at time t.

18. Method according to one of claims 14 to 17, characterized in that for calculating the respective time- and frequency-resolved spectrogram (9, 16) the actual data set (14) and / or the reference data set (2) are divided into discrete time windows and a frequency spectrum is calculated for each time window by means of a time-frequency transformation.

19. Method according to claim 18, characterized in that the time windows have different lengths and partially overlap each other.

20. Method according to claim 18 or 19, characterized in that the time-frequency transformation is a Fourier transformation, a wavelet transformation, a constant-Q transformation or a Mel transformation.

21. Method according to one of claims 14 to 20, characterized in that the edges in the spectrograms (9, 9.1) of the reference data set (2) are broadened and / or smoothed along the time axis (11) and / or the frequency axis (12), in particular by superimposing a distribution function.

22. Method according to one of claims 14 to 21, characterized in that for the analysis (130) of the actual data set (14) and the reference data set (2) for matches or deviations, their spectrograms (9, 16) are examined for matches in the at least one musical parameter.

23. Method according to one of claims 14 to 22, characterized in that for the analysis (130) of the actual data set (14) and the reference data set (2) for agreements or deviations, the intensity values ​​of the spectrograms for each discrete time-frequency pair are multiplied with one another to form intensity products (21).

24. Method according to claim 23, characterized in that for calculating (140) the rating (19) the intensity products (21) of all discrete time-frequency pairs are added to an intensity product sum (22), wherein in particular a higher intensity product sum (22) is associated with a higher ranking (20) in the ranking list.

25. Method according to one of claims 14 to 24, characterized in that the time resolution of the spectrogram (9, 16) of the reference data set (2) and / or the actual data set (14) is converted into a relative time component (7) and into a tempo component (8).

26. Method according to claim 25, characterized in that the relative time components (7) of the reference data set (2) and the actual data set (14) are synchronized for matches or deviations before the analysis (130) of the actual data set (14) and the reference data set (2).

27. Method according to claim 25 or 26, characterized in that before the analysis (130) of the actual data set (14) and the reference data set (2) for matches or deviations, a start delay (17) and / or an end delay (18) is removed from the actual data set (14).

28. Method according to claim 27, characterized in that the removal of the start delay (17) and / or the end delay (18) is carried out by applying a cross-correlation function between the actual data set (14) and the reference data set (2).

29. Method according to one of claims 1 to 28, characterized in that the actual data set (14) is restricted to that area which comprises the segment (3, 3.1, 3.2) before the analysis (130) of the actual data set (14) and the reference data set (2) for matches or deviations.

30. Method according to one of claims 1 to 29, characterized in that the actual data set (14) is shifted and / or scaled along the frequency axis (12) before the analysis (130) of the actual data set (14) and the reference data set (2) for matches or deviations in order to eliminate any transposition.

31. Method according to claim 25, characterized in that the tempo components (8) of the actual data set (14) and the reference data set (2) are used to calculate the tempo rating (19.3).

32. A computer program product comprising instructions which, when the program is executed by a terminal (50) of a user (70), cause the user (70) to execute a method (100) according to one of claims 1 to 31.

33. A system comprising a terminal (50) of a user (70) and a server (60) connected to the terminal (50), wherein the terminal (50) and the server (60) are programmed to carry out a method (100) according to one of claims 1 to 31.

Images