Music score following method, device, apparatus and storage medium

Abstract

The application provides a score following method, device, equipment and storage medium, and the specific implementation scheme is: obtaining audio data played by a target object; determining corresponding audio features according to the audio data; in the case that the audio features match note features corresponding to a specified note in a to-be-played score, determining a next note of the specified note as a to-be-played note in the to-be-played score. According to the technical scheme of the application, automatic following of the score can be realized.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device and storage medium for following musical scores. Background Technology

[0002] With the increasing popularity of musical instruments, more and more people are learning them. For many music students, this means manually turning pages of sheet music during practice. While some sheet music software offers automatic page turning, it operates at a constant speed. This requires the student to match the automatic page-turning speed during performance, placing high demands on the student. Furthermore, the sheet music cannot intelligently keep pace with the student's playing, resulting in poor practice effectiveness. Summary of the Invention

[0003] To address the aforementioned problems, this application proposes a method, apparatus, device, and storage medium for automatically following musical scores.

[0004] According to a first aspect of the embodiments of this application, a method for following musical scores is provided, comprising:

[0005] Obtain the audio data of the target object's performance;

[0006] Determine the corresponding audio features based on the audio data;

[0007] If the audio feature matches the note feature corresponding to a specified note in the score to be played, the note following the specified note in the score to be played is determined as the note to be played.

[0008] According to a second aspect of the embodiments of this application, a music score following device is provided, comprising:

[0009] The acquisition module is used to acquire the audio data of the target object's performance.

[0010] The determining module is used to determine the corresponding audio features based on the audio data;

[0011] The follow module is used to determine the next note in the score to be played as the note following the specified note when the audio feature matches the note feature corresponding to the specified note in the score to be played.

[0012] A third aspect of this application provides an electronic device, comprising:

[0013] Memory and processor;

[0014] The memory is connected to the processor and is used to store programs;

[0015] The processor implements the above-described method of following musical scores by running the program in the memory.

[0016] A fourth aspect of this application provides a storage medium storing a computer program, which, when executed by a processor, implements the above-described method for following musical scores.

[0017] One embodiment of the above application has the following advantages or beneficial effects:

[0018] The system acquires audio data of the target object's performance, determines the corresponding audio features based on the audio data, and identifies the notes played by the target object. When the audio features match the note features corresponding to a specified note in the score to be played, the next note in the score to be played is identified as the note to be played. In this way, the system can accurately identify notes that have been played and notes that are about to be played on the score, thus achieving automatic score following. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a method for following musical scores provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram illustrating the display of musical scores on a display module, as provided in an embodiment of this application.

[0022] Figure 3 A schematic diagram illustrating the specific process of obtaining the note features corresponding to a specified note in the score to be played in another score following method provided in this application embodiment;

[0023] Figure 4 A schematic diagram illustrating the specific process of determining the note feature sequence in another method for following musical scores provided in this application embodiment;

[0024] Figure 5 A schematic diagram illustrating the specific process of obtaining the note feature library in another music score following method provided in this application embodiment;

[0025] Figure 6 A flowchart illustrating yet another method for following musical scores provided in an embodiment of this application;

[0026] Figure 7A schematic diagram of a music score following device provided in an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of this application are applicable to various music score recognition scenarios, such as music practice and online learning. Using the technical solutions of this application, music score following can be achieved.

[0029] The technical solutions of this application can be applied, by way of example, to hardware devices such as processors, electronic devices, and servers (including cloud servers), or packaged as software programs and run. When the hardware device executes the processing procedure of the technical solutions of this application, or when the aforementioned software program is run, the purpose of music score following can be achieved. This application only provides an exemplary description of the specific processing procedure of the technical solutions of this application, and does not limit the specific implementation form of the technical solutions of this application. Any technical implementation form that can execute the processing procedure of the technical solutions of this application can be adopted by this application.

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] Exemplary methods

[0032] Figure 1 This is a flowchart of a music score following method according to an embodiment of this application. In an exemplary embodiment, a music score following method is provided, including:

[0033] S110. Obtain the audio data of the target object's performance;

[0034] S120. Determine the corresponding audio features based on the audio data;

[0035] S130, if the audio feature matches the note feature corresponding to the specified note in the score to be played, the next note in the score to be played is determined as the note to be played.

[0036] In step S110, exemplarily, the target object represents a person playing a musical instrument. Audio data represents the sound produced by the target object playing the instrument. Optionally, the audio data includes audio corresponding to different note types, including: single notes and chord notes, wherein a chord note is composed of multiple single notes, i.e., multiple single notes played simultaneously constitute a chord note. Optionally, the audio data can be music played by the target object in advance, or music captured in real-time while the target object is playing. Optionally, the audio data can be captured directly, or it can be captured through any acquisition device and then obtained from the acquisition device.

[0037] Specifically, acquiring audio data involves taking the envelope of the sound wave, finding the extrema of the envelope to obtain all its minimum and maximum points, and measuring the interval (min, max) between the maximum and minimum values. Then, the interval between the maximum and minimum values ​​is taken as (0.2*max + 0.8*min, 0.8*max + 0.2*min). When the maximum value is higher than (0.8*max + 0.2*min), the range is expanded outwards from this maximum value to find the first minimum point or the first point t0 and t1 reaching (0.2*max + 0.8*min). The waveform between (t0, t1) is taken as the acquisition result of the note. In step S120, for example, audio features are used to represent the frequency and / or amplitude characteristics of the sound wave in the audio data. Optionally, audio features can be obtained by performing spectral analysis on the audio data. Specifically, a Fourier transform is performed on the sound wave of the audio data, and the frequencies are arranged in descending order of amplitude. An amplitude threshold A is obtained through adaptive threshold segmentation. th Record values ​​higher than A th The frequency and amplitude are used as characteristics of the audio data.

[0038] In step S130, exemplarily, the score to be played represents the musical score that the target object is preparing to play. The score to be played is displayed on a terminal with a display module (e.g., a screen). The note to be played represents the next note to be played. Preferably, the note to be played is displayed in the center of the display module, which ensures the continuity of the score in the target object's field of vision. Specifically, a scrolling page-turning method can be used to avoid obvious interruptions before and after page turning. In this embodiment, as... Figure 2 As shown, the cursor marks the note to be played on the display module. Specifically, if the audio characteristics match the note characteristics corresponding to a specified note in the score, the cursor moves from the specified note to the next note. The cursor can be any pre-defined shape, such as a triangle or arrow. This displays the current playing progress on the score with the cursor, keeping the currently played section always centered on the display module.

[0039] Optionally, after determining the score to be played, the note characteristics of all notes in the score are predetermined. The specified note is used to represent a note pre-selected in the score. Optionally, it can be any note in the score selected by the target object. If no note is selected by the target object, the first note of the score is used as the starting point for performance. Optionally, the note characteristics are used to represent the frequency corresponding to the note.

[0040] Specifically, if the frequency corresponding to the audio data matches the frequency corresponding to the specified note, it means that the target object is playing correctly. Then, the next note after the specified note is determined as the note to be played, so that the target object can determine the next note to be played, thereby realizing the automatic following of the musical score.

[0041] In the technical solution of this application, audio data of the target object's performance is acquired, and corresponding audio features are determined based on the audio data to identify the notes played by the target object. When the audio features match the note features corresponding to a specified note in the score to be played, the next note in the score to be played is determined as the note to be played, until the notes to be played in the score to be played are empty. In this way, the notes that have been played and the notes that are being prepared to be played can be accurately identified on the score, realizing automatic following of the score.

[0042] In one implementation, such as Figure 3 As shown, the method for obtaining the note features corresponding to a specified note in the score to be played includes:

[0043] S310. Determine the note feature sequence based on the instrument being played and the score to be played;

[0044] S320. Using the specified note, determine the corresponding note feature in the note feature sequence.

[0045] For example, the term "musical instrument" refers to the instrument used by the target object to play the score. The musical instrument can include instruments such as a piano, violin, etc. The score can exist in image form or digital score form. Optionally, if the score exists in image form, the image is recognized (e.g., using OCR technology) to extract the specific note sequence. If the score exists in digital score form, the specific note sequence is directly generated. The digital score can generally be found online. The note sequence is then displayed on the display module in either simplified musical notation or standard musical notation.

[0046] Specifically, the process iterates through the identified note sequences in the score to be played, determining the note features corresponding to each note in the sequence based on the playing instrument. This generates a note feature sequence corresponding to the playing instrument in the score. The note features corresponding to a specific note are then identified within this sequence. By associating the specified note in the score with the note features in the note feature sequence, the system can follow the notes in the score based on the matching of the acquired audio data with the note features.

[0047] Preferably, such as Figure 4 As shown, step S310, which involves determining the note feature sequence based on the playing instrument and the score to be played, includes:

[0048] S410. Determine the corresponding note feature library based on the musical instrument being played;

[0049] S420. Using the note feature library, determine the note features corresponding to each note in the score to be played, and obtain the note feature sequence.

[0050] For example, the note feature library includes a database of note features determined by playing single notes and chord notes on a musical instrument. Optionally, the note features corresponding to each note can be determined according to different musical instruments. In this way, different musical instruments can correspond to different note feature libraries. For example, a piano corresponds to one note feature library, a violin corresponds to one note feature library, and a cello corresponds to one note feature library.

[0051] Specifically, a corresponding note feature library is selected based on the instrument being played. The note sequences identified in the score to be played are traversed, and the note features corresponding to each note in the note sequence are determined in the note feature library. Based on the determined note features, a note feature sequence is generated according to the note order of the score to be played. In this way, the note feature library selected according to different instruments can make the recognition of audio data more accurate.

[0052] In one implementation, such as Figure 5 As shown, the method for obtaining the note feature library includes:

[0053] S510. Obtain the audio data of single notes and chord notes played by the instrument.

[0054] S520. Perform spectral analysis on the audio data of the single note and the audio data of the chord note to obtain the predicted note frequency;

[0055] S530. By using a preset frequency range to perform spectral calibration on the predicted note frequency, the note characteristics of the single note and the note characteristics of the chord note are obtained.

[0056] For example, the preset frequency range is determined based on the standard frequency corresponding to the note and the frequencies of the two adjacent notes above and below it. The standard frequency corresponding to the note is determined based on the correspondence between each note and its corresponding standard frequency existing in the prior art. The frequency of the note includes the fundamental frequency and the overtone frequency, but generally only the fundamental frequency is used when performing spectrum calibration. In this embodiment, the standard frequency corresponding to the note and its frequency range are pre-acquired and stored in a preloaded scale library. For example, querying the frequency x corresponding to the current note in the preloaded scale library, and the frequencies y and z (z>x>y) of its two adjacent notes above and below it, then ((x+y) / 2, (x+z) / 2) is set as the frequency range of that note.

[0057] Specifically, the display module sequentially displays all possible single-note and chord notes, enabling the target object to play on an instrument based on the displayed single-note or chord notes, and the notes played by the target object are captured. The captured note sound waves are then segmented using VAD (Voice overtone) technology, which is a technique that extracts the effective waveband from the beginning to the end of a sound wave. Spectral analysis is then performed on the segmented effective waveband to determine the fundamental frequency and overtone frequencies as the note characteristics corresponding to that note.

[0058] The fundamental frequency is spectrally compared with the corresponding frequency range of the note in the preloaded scale library. If the fundamental frequency is within the frequency range, the note, along with its fundamental and overtone frequencies, is recorded in the note feature library. If the fundamental frequency is outside the frequency range, a playing error is displayed on the display module, and options for re-acquiring and forced recording are sent to the target user. For example, "Re-acquiring" and "Forced Recording" are displayed on the display module for the target user to choose from. If the target user selects "Re-acquiring," the note is played again, and its sound is acquired and analyzed to obtain a new fundamental frequency. If the target user selects "Forced Recording," the note, along with its fundamental and overtone frequencies, is recorded in the note feature library.

[0059] In one implementation, determining the corresponding audio features based on the audio data includes:

[0060] Determine the note type of the specified note in the score to be played;

[0061] Based on the note type, determine the audio features corresponding to the audio data.

[0062] For example, note types include single-note notes and chord notes. Since a single-note note consists of a single note, while a chord note consists of multiple notes, the synthesized sound wave characteristics are more complex. Because the target object is to play according to the specified notes in the score, the note type of the specified notes in the score is determined in advance, and the corresponding audio feature determination method is determined based on the note type. This ensures both the accuracy and efficiency of audio feature analysis.

[0063] Preferably, when the note type is a chord note, the step of determining the audio features corresponding to the audio data based on the note type includes:

[0064] Perform spectral analysis on the audio data to determine the corresponding first characteristic frequency;

[0065] When the first characteristic frequency corresponds to multiple predicted chord notes, the frequency domain distance between the multiple predicted chord notes and the audio data is calculated respectively.

[0066] The chord note with the smallest frequency domain distance is identified as the target chord note;

[0067] The frequency of the target chord note is determined as the audio feature corresponding to the audio data.

[0068] For example, the frequency domain distance is used to represent the frequency difference between the current chord note and the chord notes in the note feature library. Specifically, for a certain chord C in the note feature library, the characteristic frequency of chord C is compared with the characteristic frequency of the current chord S. Suppose that the features of chord C contain F frequencies. If there are k frequencies among the F frequencies that are common to the current chord S, then the frequency domain distance is denoted as Fk.

[0069] Specifically, because the sound waves of a chord contain the fundamental tones of multiple notes simultaneously, it's impossible to calibrate using a single scale for fundamental tone matching. Furthermore, the overtones of each scale within a chord note may overlap at the same frequency, potentially causing the frequency amplitude of some overtones to approach or even exceed the fundamental tone. Therefore, when analyzing the characteristics of chord notes, multiple predicted chord notes may exist for the same frequency (i.e., the first characteristic frequency). This necessitates frequency domain distance matching among these predicted chord notes, selecting the chord with the smallest frequency domain distance as the target chord note. This allows for more accurate determination of the target chord notes, facilitating the matching of the played chord notes with the notes in the score.

[0070] Optionally, when the first feature frequency corresponds to multiple predicted chord notes, the frequency domain distance between the multiple predicted chord notes and the audio data is calculated respectively, and then the method further includes:

[0071] In the case where there are multiple chord notes with the smallest frequency domain distance, calculate the amplitude distance between each of the multiple chord notes with the smallest frequency domain distance and the audio data;

[0072] The chord note with the smallest amplitude distance is identified as the target chord note.

[0073] Specifically, when there are multiple chord notes with the smallest frequency domain distance, the one with the smallest amplitude distance is selected as the matching result for that chord note.

[0074] For example, for a chord C in the note feature library, if there are k frequencies shared by chord C and the current chord S, then the amplitude distance formula is as follows:

[0075]

[0076] in, These represent the amplitudes of chord C and the current chord S at their respective i-th shared characteristic frequencies. Based on the amplitude distance formula, calculations are performed on multiple chord notes with the smallest frequency domain distances to obtain multiple amplitude distances. The chord note with the smallest amplitude distance is selected as the target chord note. This solves the problem of some overtones having frequency amplitudes close to or even exceeding the fundamental tone, resulting in multiple chord notes with the smallest frequency domain distances, making it impossible to select the target chord note.

[0077] Furthermore, to make the identified target chord notes more accurate, the amplitude distance is further judged. If the amplitude distance is less than a preset threshold, the match result of the target chord note is considered true. The preset threshold can be set according to actual needs. For example, the preset threshold can be set based on the root mean square of the amplitudes of all fundamental tones of the current chord, as shown in the following formula:

[0078] Th = r * M, where M is the root mean square of the amplitudes of all fundamental tones of the current chord, r = 0.3 * s, and s represents the matching strictness, which is a value between (0,1). It should be noted that s can be set as needed; the smaller s is, the stricter the matching.

[0079] Preferably, when the note type is a single note, the step of determining the audio features corresponding to the audio data based on the note type includes:

[0080] The audio data is subjected to spectral analysis to obtain the frequency of the corresponding single note, and the frequency of the single note is determined as the audio feature corresponding to the audio data.

[0081] Specifically, when acquiring audio data, if the note type of the audio data is determined to be a single note, since a single note consists of only one note and its composition is relatively simple, it is only necessary to perform spectral analysis (i.e., perform Fourier transform) on the single note to determine the frequency corresponding to the single note.

[0082] In one implementation, such as Figure 6 As shown, after step S120, the method further includes:

[0083] S140. If the audio feature does not match the note feature corresponding to the specified note in the score to be played, the specified note is determined as the note to be played in the score to be played.

[0084] For example, if the audio characteristics do not match the note characteristics corresponding to a specified note in the score, it indicates that the target player has made a mistake. The note to be played remains unchanged, still the specified note, and is displayed in the center of the display module. This continues until the audio characteristics corresponding to the target player's audio data match the note characteristics corresponding to the specified note in the score. Only then is the next note after the specified note designated as the note to be played. This allows for automatic following and pausing of the score, helping the target player correct playing errors promptly, and provides good error tolerance, especially for performers and trainees.

[0085] In this embodiment, audio data of the target object's performance is collected in real time, the audio data is analyzed to determine the corresponding note features, and the note features are matched with the note features of the note to be played.

[0086] Furthermore, if the current note to be played in the score is a single note, note feature matching is performed directly. If the current note to be played in the score is a chord note, the parsed note features are stored sequentially in the cache module to generate a note feature sequence. The corresponding target chord note is determined based on the note feature sequence (i.e., feature frequency). The note features of the target chord note are then matched with the note features of the current note to be played.

[0087] If a match is successful, the cursor on the display module points to the next note, and the audio data played by the target object is repeatedly collected and parsed until the last note of the score is played. Once the score has finished matching, the cache module is cleared.

[0088] If a match fails, it indicates an error in the target's performance. The cursor stops moving, the last single note or chord note in the cache module is deleted, and the audio data of the target's performance is repeatedly collected and analyzed until a match is successful. Based on this, the system can identify the note currently being played and subsequently the sequence of notes recently played. When the performer plays notes in the correct rhythm and order according to the score, the system can identify that the current note has been accurately played and the next note is being played accurately, and move the cursor to the next note correctly. When the performer makes a mistake or pauses, the system can stop in time and then continue playing the correct note, allowing the cursor to continue moving. When the performer jumps in their playing progress, such as repeating a previously played section or jumping directly to a later section, the system can compare the analyzed note features with the corresponding note feature sequence in the score to identify the jump and move the cursor to the jumped note. This enables automatic page turning of the score.

[0089] Exemplary device

[0090] Correspondingly, Figure 7 This is a schematic diagram of a music score following device according to an embodiment of this application. In an exemplary embodiment, a music score following device is provided, comprising:

[0091] The acquisition module 710 is used to acquire the audio data of the target object's performance;

[0092] The determining module 720 is used to determine the corresponding audio features based on the audio data;

[0093] The follow module 730 is used to determine the next note in the score to be played as the note to be played when the audio feature matches the note feature corresponding to the specified note in the score to be played.

[0094] In one embodiment, the method for obtaining the note features corresponding to a specified note in the score to be played includes:

[0095] Determine the note characteristic sequence based on the instrument being played and the score to be played;

[0096] Using the specified note, the corresponding note feature is determined in the note feature sequence.

[0097] In one implementation, determining the note feature sequence based on the playing instrument and the score to be played includes:

[0098] Determine the corresponding note feature library based on the musical instrument being played;

[0099] The note feature library is used to determine the note feature corresponding to each note in the score to be played, thereby obtaining a note feature sequence.

[0100] In one embodiment, the method for obtaining the note feature library includes:

[0101] Acquire the audio data of single notes and chord notes played by the instrument.

[0102] Spectral analysis is performed on the audio data of the single note and the audio data of the chord note to obtain the predicted note frequency;

[0103] By performing spectral calibration on the predicted note frequencies using a preset frequency range, the note characteristics of the single note and the note characteristics of the chord notes are obtained.

[0104] In one implementation, the determining module 720 is further configured to:

[0105] Determine the note type of the specified note in the score to be played;

[0106] Based on the note type, determine the audio features corresponding to the audio data.

[0107] In one implementation, when the note type is a chord note, the step of determining the audio features corresponding to the audio data based on the note type includes:

[0108] Perform spectral analysis on the audio data to determine the corresponding first characteristic frequency;

[0109] When the first characteristic frequency corresponds to multiple predicted chord notes, the frequency domain distance between the multiple predicted chord notes and the audio data is calculated respectively.

[0110] The chord note with the smallest frequency domain distance is identified as the target chord note;

[0111] The frequency of the target chord note is determined as the audio feature corresponding to the audio data.

[0112] In one implementation, when the first characteristic frequency corresponds to multiple predicted chord notes, the frequency domain distance between the multiple predicted chord notes and the audio data is calculated respectively, and then the method further includes:

[0113] In the case where there are multiple chord notes with the smallest frequency domain distance, calculate the amplitude distance between each of the multiple chord notes with the smallest frequency domain distance and the audio data;

[0114] The chord note with the smallest amplitude distance is identified as the target chord note.

[0115] In one implementation, when the note type is a single note, the step of determining the audio features corresponding to the audio data based on the note type includes:

[0116] The audio data is subjected to spectral analysis to obtain the frequency of the corresponding single note, and the frequency of the single note is determined as the audio feature corresponding to the audio data.

[0117] In one embodiment, the device further includes:

[0118] The processing module is used to determine the specified note as the note to be played in the score when the audio feature does not match the note feature corresponding to the specified note in the score to be played.

[0119] In one embodiment, the note to be played is displayed in the center of the display module.

[0120] The music score following device provided in this embodiment belongs to the same concept as the music score following method provided in the above embodiments of this application. It can execute the music score following method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects of executing the music score following method. Technical details not described in detail in this embodiment can be found in the specific processing content of the music score following method provided in the above embodiments of this application, and will not be repeated here.

[0121] Exemplary electronic devices

[0122] Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 8 As shown, the device includes:

[0123] Memory 800 and processor 810;

[0124] The memory 800 is connected to the processor 810 and is used to store programs;

[0125] The processor 810 is configured to implement the music score following method disclosed in any of the above embodiments by running the program stored in the memory 800.

[0126] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 820, an input device 830, and an output device 840.

[0127] The processor 810, memory 800, communication interface 820, input device 830, and output device 840 are interconnected via a bus. Among them:

[0128] A bus can include a pathway for transmitting information between various components of a computer system.

[0129] The processor 810 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0130] The processor 810 may include a main processor, as well as a baseband chip, modem, etc.

[0131] The memory 800 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 800 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0132] Input device 830 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0133] Output device 840 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0134] The communication interface 820 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0135] The processor 810 executes the program stored in the memory 800 and calls other devices, which can be used to implement the various steps of any of the music score following methods provided in the above embodiments of this application.

[0136] Exemplary computer program products and storage media

[0137] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods for following musical scores according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0138] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0139] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor in the steps of the music score following method according to various embodiments of this application described in the "Exemplary Methods" section above.

[0140] The specific working content of the aforementioned electronic device, as well as the specific working content of the aforementioned computer program product and the computer program on the storage medium being run by the processor, can all be found in the content of the aforementioned method embodiments, and will not be repeated here.

[0141] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0142] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0143] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0144] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0145] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0146] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0147] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0148] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0149] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0150] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0151] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for following musical scores, characterized in that, include: Obtain the audio data of the target object's performance; Determine the corresponding audio features based on the audio data; If the audio feature matches the note feature corresponding to a specified note in the score to be played, the note following the specified note in the score to be played is determined as the note to be played; the score to be played represents the musical score that the target object is preparing to play, and the score to be played is displayed on a terminal with a display module; The method for obtaining the note features corresponding to a specified note in the score to be played includes: Determine the note characteristic sequence based on the instrument being played and the score to be played; Using the specified note, the corresponding note feature is determined in the note feature sequence.

2. The method according to claim 1, characterized in that, The step of determining the note feature sequence based on the playing instrument and the score to be played includes: Determine the corresponding note feature library based on the musical instrument being played; The note feature library is used to determine the note feature corresponding to each note in the score to be played, thereby obtaining a note feature sequence.

3. The method according to claim 2, characterized in that, The method for obtaining the note feature library includes: Acquire the audio data of single notes and chord notes played by the instrument. Spectral analysis is performed on the audio data of the single note and the audio data of the chord note to obtain the predicted note frequency; By performing spectral calibration on the predicted note frequencies using a preset frequency range, the note characteristics of the single note and the note characteristics of the chord notes are obtained.

4. The method according to claim 1, characterized in that, The step of determining the corresponding audio features based on the audio data includes: Determine the note type of the specified note in the score to be played; Based on the note type, determine the audio features corresponding to the audio data.

5. The method according to claim 4, characterized in that, When the note type is a chord note, the step of determining the audio features corresponding to the audio data based on the note type includes: Perform spectral analysis on the audio data to determine the corresponding first characteristic frequency; When the first characteristic frequency corresponds to multiple predicted chord notes, the frequency domain distance between the multiple predicted chord notes and the audio data is calculated respectively. The chord note with the smallest frequency domain distance is identified as the target chord note; The frequency of the target chord note is determined as the audio feature corresponding to the audio data.

6. The method according to claim 5, characterized in that, In the case where the first characteristic frequency corresponds to multiple predicted chord notes, after calculating the frequency domain distance between the multiple predicted chord notes and the audio data, the method further includes: In the case where there are multiple chord notes with the smallest frequency domain distance, calculate the amplitude distance between each of the multiple chord notes with the smallest frequency domain distance and the audio data; The chord note with the smallest amplitude distance is identified as the target chord note.

7. The method according to claim 4, characterized in that, When the note type is a single note, the step of determining the audio features corresponding to the audio data based on the note type includes: The audio data is subjected to spectral analysis to obtain the frequency of the corresponding single note, and the frequency of the single note is determined as the audio feature corresponding to the audio data.

8. The method according to claim 1, characterized in that, The method further includes: If the audio features do not match the note features corresponding to the specified note in the score to be played, the specified note will be determined as the note to be played in the score.

9. The method according to claim 1, characterized in that, in, The notes to be played are displayed in the center of the display module.

10. A musical score tracking device, characterized in that, include: The acquisition module is used to acquire the audio data of the target object's performance. The determining module is used to determine the corresponding audio features based on the audio data; The follow module is used to determine the next note in the score to be played as the note to be played when the audio feature matches the note feature corresponding to the specified note in the score to be played; the score to be played represents the music score that the target object is preparing to play, and the score to be played is displayed on a terminal with a display module; The method for obtaining the note features corresponding to a specified note in the score to be played includes: Determine the note characteristic sequence based on the instrument being played and the score to be played; Using the specified note, the corresponding note feature is determined in the note feature sequence.

11. An electronic device, characterized in that, include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor, by running the program in the memory, implements the method of following musical scores as described in any one of claims 1 to 9.

12. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method for following musical scores as described in any one of claims 1 to 9.

Images