Music score and audio synchronization method and computer readable storage medium

A music notation and audio technology, which is applied in the field of audio processing, can solve problems such as not involving the synchronization of notation and sound, and the inability to realize step-by-step notation, so as to achieve the effect of accurately comparing and viewing the notation, accurate notation, and satisfying diverse needs

Pending Publication Date: 2022-04-29
FUJIAN STAR NET EVIDEO INFORMATION SYST CO LTD
View PDF1 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, the above-mentioned scheme does not involve how to perform sound synchronization in music scores with repeated marks
Moreover, the above-mentioned scheme cannot realize the off-beating one by one

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Music score and audio synchronization method and computer readable storage medium
  • Music score and audio synchronization method and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0071] Please refer to Figure 1-2 , Embodiment 1 of the present invention is: a method for synchronizing score and audio, which can realize the synchronization of score during music or song playback, such as figure 1 shown, including the following steps:

[0072] S1: Obtain the audio to be synchronized, and extract the main melody track of the audio to be synchronized; wherein, the main melody track refers to the track that is consistent with the notes in the music score.

[0073] In this embodiment, the audio to be synchronized is song audio, and the human voice track can be separated from the original singing audio by spleeter as the main melody track.

[0074] S2: Extract the pitch according to the main melody track, and obtain the note sequence and the start and end time of each main melody note in the note sequence.

[0075] Specifically, extract the fundamental frequency sequence from the main melody track, and then perform similar pitch fitting on the fundamental fre...

Embodiment 2

[0110] This embodiment is another implementation of step S313 in step S3 in the first embodiment, that is, another implementation when more than two best matching note strings are found.

[0111] In this embodiment, the audio to be synchronized is song audio, and the main melody track is a vocal track.

[0112] After the step S2, it further includes: according to the main melody track, through the audio-to-text technology, a lyric sequence is generated, and the start and end times of each lyrics in the lyric sequence are obtained; The start and end times of the lyrics determine the correspondence between the notes of the main melody and the lyrics.

[0113] That is to say, in addition to extracting the sequence of notes, audio-to-text technology, such as TDNN (Time Delay Neural Network), can also be used to extract the sequence of lyrics, and then mark the correspondence between notes and lyrics in chronological order.

[0114] Step S313 of this embodiment includes the follow...

Embodiment 3

[0125] This embodiment is a computer-readable storage medium corresponding to the above-mentioned embodiments, on which a computer program is stored, and when the computer program is executed by a processor, each process in the above-mentioned embodiment of the method for synchronizing score and audio is realized, And can achieve the same technical effect, in order to avoid repetition, no more details here.

[0126] To sum up, the present invention provides a method for synchronizing score and audio and a computer-readable storage medium. By identifying the play order of notes in the main melody track and controlling the play order of score, the synchronization of score and sound can be realized with audio as the main component. , not affected by repeated marks in the music score; by highlighting the music note corresponding to the current playing time or the measure in which it is located, accurate notation at the note level can be achieved, and accurate notation at the bar by...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a music score and audio synchronization method and a computer readable storage medium. The method comprises the following steps: extracting a main melody audio track of an audio to be synchronized; according to the main melody track, pitch is extracted, a note sequence and starting and ending time of each group of main melody notes in the note sequence are obtained, and the starting and ending time of the same group of main melody notes is consistent; the note sequence is compared with a music score corresponding to the audio to be synchronized, the corresponding relation between each group of main melody notes in the note sequence and each group of music score notes in the music score is determined, and the same group of music score notes are music score notes needing to be played at the same moment; and when the to-be-synchronized audio is played, according to the starting and ending time of each group of main melody notes and the corresponding relationship, determining a music score note corresponding to the current playing time, and highlighting the corresponding music score note or a bar where the music score note is located in the music score. The invention can realize spectrum and sound synchronization.

Description

technical field [0001] The invention relates to the technical field of audio processing, in particular to a method for synchronizing music score and audio and a computer-readable storage medium. Background technique [0002] During song playback, in order to allow users to intuitively see which tone the audio played in the song is currently playing, the music score and audio will be played synchronously, that is, as the audio is played, it will be synchronized on the music score Go off the spectrum. However, due to various factors, there may be differences in the overall offset (the start time of the first note), local speed, and fragment repetition of the audio resources that are matched with the score, resulting in that when the score and audio are played at the same time, the Sync, unable to align. Therefore, it is necessary to synchronize the score and the audio. [0003] In addition, in some practical applications, the repetition mark is not marked in the score, or t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G10H1/00G10L15/26G10L15/16
CPCG10H1/0008G10H1/0066G10L15/26G10L15/16G10H2220/011G10H2220/015
Inventor 陈勇王子亮
Owner FUJIAN STAR NET EVIDEO INFORMATION SYST CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap