Lyric processing method and related product

Abstract

The embodiment of the invention discloses a lyric processing method and a related product, which introduces a plurality of timestamps such as a first timestamp and a second timestamp for manually marking timestamps for detecting errors. Wherein the higher the alignment confidence coefficient calculated through the compared timestamp deviation is, on one hand, the higher the accuracy of the second timestamp is, on the other hand, the more reasons are explained from the side, and if a large deviation exists between the manually-labeled timestamp and the second timestamp, the more likely the manually-labeled timestamp is the mistakenly-labeled timestamp. According to the embodiment of the invention, a two-time threshold filtering mode is adopted, namely, the alignment confidence coefficient, the unit time deviation and respective threshold values are automatically compared, so that comparison errors caused by complete dependence on one comparison result (namely, one-time filtering) can be reduced, whether the manually labeled timestamp is wrong or not is checked in multiple aspects, the detected result is more reliable, and meanwhile, the detection efficiency is improved. And the efficiency of detecting a large number of problems of manually marking timestamps can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio technology, and in particular, to a method for processing lyrics and related products. Background Art

[0002] When a music player plays a song, the interface can usually scroll to display the lyrics text corresponding to the current song playing progress, and to achieve this effect, a lyrics source file (qrc, Qt Recources) is often required. As a word-by-word timestamp file of the song, the qrc file contains the lyrics corresponding to the song, as well as the start playing time and the continuous playing time (usually in milliseconds ms) of each word in the lyrics corresponding to the song. As shown in the following partial content of the qrc file, the word "跑" will start playing at 21042ms and end after 513ms:

[0003] 跑(21042,513)啊(21555,544)啊(22099,672)啊(22771,208)的(22979,240)山(23219,943)上(24162,872)一(25298,473)朵(25771,487)啊(26258,528)啊(26786,241)的(27027,288)云(27315,517)哟(27832,1218)啊(29394,520)啊(29914,497)啊(30411,473)啊(30884,294)的(31178,288)照(31466,1001)在(32467,808)

[0004] The above qrc file is usually made by manual annotation or machine automatic alignment annotation. Although the existing machine automatic alignment annotation technology (or lyrics timestamp technology) can be more accurate than manual annotation for most lyrics, there are still some lyrics with large deviations, which rarely occur in manual annotation. This results in the current lyrics timestamp technology being unable to directly replace the manual annotation method. It can be seen that the manual annotation method still has certain advantages, but it is still necessary to detect the problematic manual qrcs made in this way to avoid words with inaccurate timing annotations, resulting in out-of-sync between the content sung by the singer and the words scrolled and displayed in the lyrics, affecting the user experience.

[0005] Currently, some users will feedback lyrics with inaccurate timing to the background, and professional personnel in the background will manually review the manually annotated qrc timestamps of the feedback. However, the number of songs in the song library is increasing day by day. Relying on user feedback to detect each problematic lyric in this way is inefficient and not conducive to effectively handling a large number of manual problematic qrcs. Therefore, an effective solution needs to be provided. Summary of the Invention

[0006] The embodiments of the present application provide a lyrics processing method and related products for efficiently detecting problematic qrcs that are artificially incorrectly timestamped.

[0007] A first aspect of an embodiment of the present application provides a lyrics processing method, comprising:

[0008] Obtain the first timestamp, the second timestamp, and the manually annotated timestamp corresponding to each text unit in the pure lyrics text of the target song; wherein the first timestamp, the second timestamp, and the manually annotated timestamp of the text unit respectively annotate the start playing time and the continuous playing time of the text unit in the target song with different ideas; the first timestamp is obtained by identifying the audio features of the target song; the second timestamp is the timestamp corresponding to the text unit after being aligned with the human voice lyrics of the target song;

[0009] Based on the timestamp deviation between the first timestamp and the corresponding second timestamp of each text unit, the alignment confidence between the first timestamp and the corresponding second timestamp is calculated; the participating timestamp is either the first timestamp or the second timestamp, or a third timestamp calculated based on the first timestamp and the second timestamp, and the higher the alignment confidence, the more consistent the first timestamp and the corresponding second timestamp are;

[0010] Calculating a unit time deviation between the second timestamp of each text unit and the corresponding manually annotated timestamp;

[0011] Based on the alignment confidence and the unit time deviation, a labeling anomaly detection result for the manually labeled timestamp is determined.

[0012] Optionally, after determining the anomaly detection result for the manually annotated timestamp, the method further includes:

[0013] If the labeling anomaly detection result indicates that the manually labeled timestamp is an erroneously labeled timestamp, the erroneously labeled timestamp is replaced with another timestamp; the other timestamp is determined based on the first timestamp and the second timestamp of the text unit corresponding to the unit time deviation.

[0014] Optionally, the calculating the alignment confidence between the first timestamp and the corresponding second timestamp based on the timestamp deviation between the first timestamp and the corresponding second timestamp of each text unit includes:

[0015] If the number of words in the lyrics identified by the audio features of the target song is consistent with the number of words in the lyrics text, based on the ratio between the selected value of each of the text units and each of the first timestamps, the alignment confidence between the first timestamp and the corresponding second timestamp is calculated;

[0016] The selected value includes the timestamp deviation of the text unit or a value corresponding to the first timestamp.

[0017] Optionally, the calculating the alignment confidence between the first timestamp and the corresponding second timestamp based on the timestamp deviation between the first timestamp and the corresponding second timestamp of each text unit includes:

[0018] If the number of words in the lyrics identified by the audio features of the target song is inconsistent with the number of words in the lyrics in the pure lyrics text, at least one target phrase is selected from the identified lyrics text; the target phrase refers to a paragraph of lyrics with a preset number of words and which also appears continuously in the pure lyrics text;

[0019] For each text unit in the target word or sentence, based on a ratio result between a selected value of each text unit and the first timestamp of each text unit, calculating an alignment confidence of the target word or sentence;

[0020] The selected value includes the timestamp deviation of the text unit or a numerical value corresponding to the first timestamp.

[0021] Optionally, the selecting at least one target phrase from the recognized lyrics text includes:

[0022] According to a preset sliding step length, at least one section of the target phrase is selected from the recognized lyrics text by sliding from front to back.

[0023] Optionally, if the identified lyrics text does not contain the target phrase within a preset length but contains short phrases, and the short phrases also appear continuously in the pure lyrics text but have fewer words than the target phrase, the method further includes:

[0024] The alignment confidence of the target phrase adjacent to the short phrase is used as the alignment confidence of the short phrase; the alignment confidence of the short phrase is used to participate in the calculation of the alignment confidence of the entire pure lyrics text.

[0025] Optionally, if the target word or sentence does not exist in an adjacent position of the short word or sentence, the method further includes:

[0026] The alignment confidence of the short words is set to be less than a preset threshold.

[0027] Optionally, the selecting at least one target phrase from the recognized lyrics text includes:

[0028] A distance calculation function is used to calculate the change distance between the recognized lyrics text and the pure lyrics text; the change distance is used to represent the cost of transforming a phrase of the preset number of words in one lyrics text into a phrase of the preset number of words in another lyrics text;

[0029] The change distance that is smaller than the preset distance is taken as the target distance, and the phrase taken from the recognized lyrics text and pointed to by the target distance is taken as the target phrase.

[0030] Optionally, the process of determining the ratio result includes:

[0031] Taking the timestamp deviation between the first timestamp and the corresponding second timestamp of each text unit about the start playing time as the selected value;

[0032] The ratio between the sum of the selected values ​​of the various text units and the first duration sum is determined as the ratio result; the first duration sum is the sum of the continuous playback time in the first timestamps of the various text units.

[0033] Optionally, determining the labeling anomaly detection result for the manually labeled timestamp based on the alignment confidence and the unit time deviation includes:

[0034] Compare the alignment confidence and the unit time deviation with the confidence threshold and the unit time threshold respectively;

[0035] If the alignment confidence is greater than the confidence threshold, and the unit time deviation is greater than the unit time threshold, it is determined that the labeling anomaly detection result is that the manually labeled timestamp is an erroneously labeled timestamp.

[0036] A second aspect of an embodiment of the present application provides an electronic device, including: a processor and a memory;

[0037] The processor is configured to communicate with the memory and execute instructions in the memory to implement the method described in the first aspect of the embodiment of the present application or any specific implementation of the first aspect.

[0038] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, they implement the method described in the first aspect of the embodiment of the present application or any specific implementation of the first aspect.

[0039] A fourth aspect of an embodiment of the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they implement the method described in the first aspect of the embodiment of the present application or any specific implementation of the first aspect.

[0040] It can be seen from the above technical solutions that the embodiments of the present application have at least the following advantages:

[0041] In order to detect erroneous manually annotated timestamps, the embodiment of the present application introduces multiple timestamps such as the first timestamp and the second timestamp. Among them, the higher the alignment confidence calculated by the compared timestamp deviation, on the one hand, it can be explained that the accuracy of the second timestamp is higher, and on the other hand, there is more reason to explain from the side that if there is a large deviation between the manually annotated timestamp and the second timestamp, the manually annotated timestamp is more likely to be an incorrectly annotated timestamp. The embodiment of the present application can reduce the comparison errors caused by relying entirely on one result by using the two results of alignment confidence and unit time deviation, and check whether the manually annotated timestamp is wrong in many aspects, so that the detection result is more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0043] It should be noted that, although the steps in the process diagrams (if any) involved in the embodiments are drawn in sequence as indicated by the arrows, unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0044] Figure 1 A schematic diagram of the system architecture of the lyrics processing method according to an embodiment of the present application;

[0045] Figure 2 , Figure 3 This is a flowchart of a lyrics processing method according to an embodiment of the present application;

[0046] Figure 4 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0048] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims and drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] In the following description, similar expressions such as "a specific implementation" or "a specific example" are involved, which describe a subset of all possible embodiments, but it can be understood that "a specific implementation" or "a specific example" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. In the following description, the term multiple refers to at least two. In some specific examples, the numerical value mentioned in this application reaches the threshold value (if any), which may include the case where the former is greater than the latter of the threshold value; if similar expressions such as "any" or "at least one" are mentioned, it can specifically refer to any one of the listed examples or any combination of these examples.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0051] See also Figure 1 , Figure 1 Schematic diagram of the system architecture provided by the embodiment of the present application. Figure 1As shown, the system architecture may include a business server 100 and a terminal cluster, and the terminal cluster may include: terminal devices 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n and other terminal devices. Among them, the above-mentioned business server 100 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud database, cloud service, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform. The terminal device (including terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n) can be a smart phone, tablet computer, laptop computer, desktop computer, PDA, mobile Internet device (mobile internet device, MID), wearable device (such as smart watch, smart bracelet, etc.), smart computer, smart car and other smart terminals. Among them, the business server 100 can establish a communication connection with each terminal device in the terminal cluster, and the terminal devices in the terminal cluster can also establish a communication connection. In other words, the business server 100 can establish a communication connection with each terminal device in the terminal device 200a, the terminal device 200b, the terminal device 200c, ..., the terminal device 200n. For example, a communication connection can be established between the terminal device 200a and the business server 100. A communication connection can be established between the terminal device 200a and the terminal device 200b, and a communication connection can also be established between the terminal device 200a and the terminal device 200c. The above-mentioned communication connection does not limit the connection method, and can be directly or indirectly connected through a wired communication method, or directly or indirectly connected through a wireless communication method, etc., which can be determined according to the actual application scenario, and this application does not limit it here.

[0052] It should be understood that Figure 1 Each terminal device in the terminal cluster shown in the figure may be installed with an application client. When the application client runs in each terminal device, it can be respectively connected to the above Figure 1Data interaction is performed between the business servers 100 shown, so that the business server 100 can receive business data from each terminal device (such as songs uploaded by users through terminal devices). Among them, the application client can be a browser application, a social application, an instant messaging application, a live broadcast application, a game application, a short video application, a video application, a music application, a karaoke application, etc., which has the function of displaying text, images, audio, video and other data information. The specific application can be determined according to the actual application scenario requirements and is not limited here. Among them, the application client can be an independent client or an embedded sub-client integrated in a client (such as an instant messaging client, a social client, etc.). The specific application can be determined according to the actual application scenario and is not limited here.

[0053] Taking the music application as an example, when a user uses the music application through a terminal device, he or she can not only sing his or her favorite songs, but also upload the song audio and manually annotated qrc to the business server 100 through the terminal device. The singer and the uploader of the manually annotated qrc are not necessarily the same user. Each terminal device (terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n) that establishes a communication connection with the business server 100 can play the song audio and manually annotated qrc sent by the user through the above music application. In addition, after the business server 100 receives the song audio and manually annotated qrc sent by the user through the music application, it can process the manually annotated qrc of the song. The processing process may include: calculating the alignment confidence between the first timestamp and the corresponding second timestamp of the lyrics of the song; calculating the unit time deviation between the second timestamp of each text unit and the corresponding manually annotated timestamp, and automatically comparing the alignment confidence, unit time deviation and respective thresholds to detect the timestamps that are incorrectly annotated in the manually annotated qrc (i.e., the problematic qrc). Afterwards, the incorrectly marked timestamps can be repaired to improve the quality of the song qrc and enhance the user's visual and auditory experience of the song information.

[0054] The method provided in the embodiment of the present application can be performed by Figure 1 The service server 100 shown in the figure may also be executed by a terminal device (such as Figure 1 The method may be executed by any one of the terminal devices 200a, 200b, ..., 200n shown in the figure, or may be executed jointly by the terminal device and the service server. The specific method may be determined according to the actual application scenario and is not limited here.

[0055] The method of the present application will be described in further detail below.

[0056] See also Figure 2In a first aspect, the present application provides a specific embodiment of a lyrics processing method, which includes the following operation steps:

[0057] Step S21, obtaining the first timestamp, the second timestamp, and the manually marked timestamp corresponding to each text unit in the pure lyrics text of the target song.

[0058] Among them, the first timestamp, second timestamp and manually marked timestamp of the text unit respectively mark the starting playback time and continuous playback time of the text unit in the target song with different ideas; the first timestamp is obtained by identifying the audio features of the target song; the second timestamp is the timestamp corresponding to the alignment of the text unit to the vocal lyrics of the target song.

[0059] In some examples, both the first timestamp and the second timestamp can be obtained through model output, as described below, and will not be described in detail here. The difference is that the first timestamp is obtained autonomously by the model recognizing audio features, such as the automatic speech recognition (ASR, Automatic Speech Recognition model, which does not rely on or use known pure lyrics text; while the second timestamp is output using known pure lyrics text, which can be recorded as an automatic alignment algorithm. The two are generated based on different bases, so the results are generally different, so there is a need to compare the two. Specifically, in the context of existing lyrics and audio alignment, the automatic alignment algorithm analyzes the acoustic features of the audio signal and matches it with the word pronunciation or syllable pronunciation in the lyrics text to determine the start and end time of each lyric element in the audio, which can be simply understood as finding the playback time of the current pronunciation in the audio word by word. The ASR model mainly receives audio voice files as input, and does not input pure lyrics text. After recognizing the voice content, the ASR model will further decompose the recognition results into words or phrases, that is, it focuses on automatically recognizing the lyrics text from the audio and tries to assign timestamps to them.

[0060] It should be noted that the above-mentioned pure lyrics text refers to pure text content that does not carry lyrics timestamp information. The text unit contains a single word or a single phoneme. Among them, the phoneme can refer to the smallest speech unit divided according to the natural properties of the language. One pronunciation action forms a phoneme. For example, the pronunciation of "ah", "bo", and "de", each pronunciation can correspond to a phoneme. There are many ways to represent phonemes. For example, the phonemes corresponding to the pronunciation of Mandarin can be represented by pinyin. For example, the phonemes corresponding to the pronunciation of the three words "Mandarin" can be represented by "p, u, t, o, ng, h, u, a" respectively, which can be specifically understood as initials and finals; the phonemes corresponding to the pronunciation of English can be represented by phonetic symbols, such as / i / , / I / , / e / , etc.

[0061] The above-mentioned manually annotated timestamp may refer to the timestamp that relevant personnel divide the text unit according to their own experience, such as marking when a certain word starts to be played and how long it continues to be played. Compared with the first timestamp and the second timestamp obtained by the machine, the manually annotated timestamp may have subjective judgment errors, and therefore needs to be checked and corrected.

[0062] Step S22: Calculate the alignment confidence between the first timestamp and the corresponding second timestamp of each text unit based on the timestamp deviation between the first timestamp and the corresponding second timestamp.

[0063] It can be understood here that the first timestamp and the second timestamp are dynamically aligned to determine the alignment confidence. The higher the alignment confidence (or confidence for short), the closer and more consistent the second timestamp is with the first timestamp. It can be understood here that if the results or ideas given by multiple parties are more consistent (that is, the smaller the deviation is), it generally shows that the result or idea is credible, that is, the participating timestamp is more credible. In this case, if a third-party timestamp is compared with this participating timestamp, it can be preliminarily determined to a certain extent that there is a problem with the third-party timestamp. Therefore, in order to further accurately verify whether it is the manually annotated timestamp that is wrong, or the participating timestamp compared with the manually annotated timestamp that is wrong, the credibility of the participating timestamp can be tested to enhance the reliability of the detection of manually annotated timestamps.

[0064] Step S23: Calculate the unit time deviation between the second timestamp of each text unit and the corresponding manually annotated timestamp.

[0065] The timestamp of the lyrics includes two time concepts: the start time and the duration time. Taking the text unit as a single word as an example, the unit time deviation S can be compared word by word. The calculation formula is:

[0066] S=|time_auto_begin-time_man_begin|-|time_auto_end-time_man_end|

[0067] Among them, time_auto_begin and time_auto_end are the word start time (i.e., the start playback time) and the word end time in the timestamp of the evaluation, and time_man_begin and time_man_end are the word start time and the word end time in the manually marked qrc. The word end time here specifically refers to the time point referred to by "continuous playback time" or "start playback time + continuous playback time". Of course, when calculating the unit time deviation S, it is also possible to only consider the word start time without considering the word end time, because in some examples, as long as the starting gap of the word is aligned, the gap at the end can also be naturally aligned. Of course, when calculating the unit time deviation S, it is also possible to only consider the word end time without considering the word start time.

[0068] Similar to the above process of calculating the unit time deviation, the timestamp deviation of each word can be calculated word by word by combining the two time concepts of the start playback time and the continuous playback time as shown in the above formula S; of course, any of the two time concepts can also be used to calculate the timestamp deviation, without limitation. Since the alignment confidence is calculated based on the timestamp deviation, it can be understood that the alignment confidence can also be calculated based on at least one of the above two time concepts, which can be determined according to the actual situation.

[0069] It should be noted that the artificial qrc in the embodiment of the present application mainly refers to the timestamp of the lyrics rather than the lyrics themselves. Accordingly, detecting problematic qrc mainly refers to detecting the timestamps that are artificially misaligned, rather than the lyrics themselves.

[0070] Step S24: Determine the label anomaly detection result for the manually labeled timestamp based on the alignment confidence and the unit time deviation.

[0071] This anomaly detection result can be used to characterize whether the manually annotated timestamp is an erroneous timestamp. For example, based on the threshold (such as confidence threshold, unit time threshold) comparison logic, it can be determined whether the manually annotated timestamp of the current text unit is an erroneous timestamp, as shown below.

[0072] If step S24 involves comparison between the alignment confidence and the confidence threshold, and comparison between the unit time deviation and the unit time threshold, the order of execution of these two comparison processes may not be limited, and they may also be executed simultaneously, depending on the actual situation.

[0073] In summary, in order to detect erroneous manually annotated timestamps, the embodiment of the present application introduces multiple timestamps such as the first timestamp and the second timestamp. Among them, the higher the alignment confidence calculated by the compared timestamp deviation, on the one hand, it can be explained that the accuracy of the second timestamp is higher, and on the other hand, there is more reason to explain from the side that if the manually annotated timestamp has a large deviation from the second timestamp, the manually annotated timestamp is more likely to be an incorrectly annotated timestamp. It can be seen that the embodiment of the present application adopts a two-threshold filtering method, that is, the alignment confidence, unit time deviation and respective thresholds are automatically compared, which can reduce the comparison errors caused by relying entirely on one comparison result (i.e., one filtering), and check whether the manually annotated timestamp is wrong in many aspects, so that the detection result is more reliable. At the same time, it can speed up the problem detection efficiency of a large number of manually annotated timestamps.

[0074] The execution order of the above step S22 and step S23 can be unlimited, and they can also be executed simultaneously. Based on the above example description, some specific possible implementation examples will be provided below. In actual applications, the implementation contents between these examples can be combined or implemented separately according to the corresponding functional principles and application logics. If they are implemented in combination, the execution order between the combined examples can be determined according to their respective processing logics, which can be determined by the actual scenario.

[0075] based on Figure 2 The present application provides another specific embodiment of a lyrics processing method, which includes the following steps:

[0076] Step S21, obtaining the first timestamp, the second timestamp, and the manually marked timestamp corresponding to each text unit in the pure lyrics text of the target song.

[0077] like Figure 3 As shown, the audio of the target song and its corresponding pure lyrics text can be obtained, and then divided into two inputs to output the first timestamp and the second timestamp corresponding to each text unit. Specifically, the first way can be to input the audio of the target song into the ASR (Automatic Speech Recognition) model. The ASR model can automatically output the lyrics text of the target song (which can be referred to as ASR lyrics) and the first timestamp corresponding to the lyrics text by identifying the audio features, that is, the ASR model can generate a qrc file by itself. The timestamp information in the qrc file is generally different from the second timestamp and the manually marked timestamp. The second way can be to input the audio and pure lyrics text of the target song into the automatic alignment model to output the second timestamp corresponding to the pure lyrics text.

[0078] The difference between the above automatic alignment model and the ASR model is that the input and output are different. Specifically, the text of the automatic alignment model is known, and the model only needs to align the human voice and text in the song audio, such as synchronizing the known pure lyrics text with the time progress information of the human voice. The ASR model only inputs the song audio and has no text information. The model needs to recognize the words sung in the song audio to output a version of the lyrics and its corresponding timestamp. In practical applications, in terms of effect, the timestamp output by the ASR model is often more reliable than the timestamp output by the automatic alignment model. If the timestamp of the automatic alignment is very close to the timestamp output by the ASR, it can be considered that the alignment confidence of the automatic alignment result will be very high, but the disadvantage is that the lyrics recognized by the ASR may be inconsistent with the original pure lyrics text. For timestamp tasks, inconsistent word content has little effect, but inconsistent lyrics length can easily cause misalignment problems.

[0079] Step S22: Calculate the alignment confidence between the first timestamp and the corresponding second timestamp of each text unit based on the timestamp deviation between the first timestamp and the corresponding second timestamp.

[0080] See also Figure 3 , the second timestamp can be used as a participating timestamp compared with the manually annotated timestamp, but in some instances, this participating timestamp can be specifically summarized as being determined based on the first timestamp and the second timestamp, for example, participating timestamp = a*first timestamp+b*second timestamp, where a and b are weight values, and when the weight of a certain item is 0, it means that the participating timestamp is the other item (such as b is 0, which means that the participating timestamp is the first timestamp); of course, a and b can also take non-zero values, then the participating timestamp at this time can be a third timestamp, such as one of the time points between the first timestamp and the corresponding second timestamp. In short, the above-mentioned participating timestamp can be specifically determined by actual conditions, and for ease of understanding, the embodiments of the present application mainly use the second timestamp as the participating timestamp for illustrative purposes.

[0081] The text unit may be a single word at the character level, or a single phoneme at the phoneme level, without limitation. Since the length of the lyrics recognized by ASR may be different from the original pure lyrics text, step S22 will be specifically described below in different cases.

[0082] a. As a possible implementation, the specific operation process of step S22 may include: if the number of lyrics words recognized through the audio features of the target song is consistent with the number of lyrics words in the pure lyrics text, based on the ratio between the selected value of each text unit and each first timestamp, calculate the alignment confidence between the first timestamp and the corresponding second timestamp; the selected value includes the timestamp deviation of the text unit or one of the values ​​in the corresponding first timestamp.

[0083] For example, the alignment confidence of each word can be determined word by word, and the alignment confidence of each word can be summed to obtain the alignment confidence of the entire lyrics text. Among them, the alignment confidence of the i-th word can be the ratio between the timestamp deviation of the i-th word and the sum of the continuous playback time of all words (m); this timestamp deviation can be the difference between different model outputs in the timestamp concept about a single concept such as the start playback time or the continuous playback time, or it can be the difference after combining these two single concepts; this sum of continuous playback time can be the sum of the continuous playback time of all words output by any of the above models.

[0084] For example, if the number of words in the lyrics recognized by ASR is consistent with the number of words in the lyrics automatically aligned (i.e., the number of words in the original pure lyrics text), the alignment confidence at this time can be:

[0085]

[0086] Among them, len_asr_word is the sum of the word durations output by the ASR model; di represents the timestamp deviation between the timestamp of the i-th word output by the ASR model and the timestamp of the i-th word output by the automatic alignment model. The calculation formula of the timestamp deviation di can be as above as the unit time deviation S: di = |time_auto_begin-time_ai_begin|-|time_auto_end-time_ai_end|; ai is the duration of the current word output by the ASR model. min(di,ai) is the above-mentioned one-choice value, which means that if the timestamp deviation di is too large, greater than the word duration ai output by the ASR model, then the word duration ai is taken. This is done to ensure that the value range of the alignment confidence value is between 0 and 1. Here, m can be the total number of words in the entire lyrics. Of course, in some cases, the parameter m can be the number of words in part of the lyrics, which can be determined according to the specific situation.

[0087] In some examples, the above alignment confidence conf avg The formula is transformed to obtain other alignment confidence calculation formulas, for example, the above alignment confidence conf avg In the denominator len_ASR_word, you can try to substitute the sum of the start times between the first and mth words output by the ASR model; or, you can replace conf avg The molecules in the formula are replaced by The denominator can also be replaced. In other words, in some examples, the deviation of the continuous playback time in the first timestamp may not be used to calculate the alignment confidence. Specifically, the process of determining the above ratio result may include: taking the timestamp deviation about the start playback time between the first timestamp of each text unit and the corresponding second timestamp as an alternative value, for example, the timestamp deviation di = |time_auto_begin-time_ai_begin|; determining the ratio between the sum of the alternative values ​​of each text unit and the first duration sum as the ratio result; the first duration sum is the sum of the continuous playback time in the first timestamp of each text unit, such as the above len_asr_word.

[0088] Alternatively, the above alignment confidence conf can be replaced by the dynamic time warping formula (DTW, Dynamic TimeWarping) avg formula, because the DTW formula essentially reflects the similarity between two time series (see below for details).

[0089] As described above, if the number of words recognized by ASR is consistent with the number of words in the pure lyrics text (a successful situation), the alignment confidence of the entire lyrics can be calculated, that is, an alignment confidence can be calculated, which can be used for subsequent rapid detection of artificial problem timestamps.

[0090] b. As another possible implementation, the specific operation process of step S22 may include: if the number of words in the lyrics identified through the audio features of the target song is inconsistent with the number of words in the pure lyrics text, selecting at least one target phrase from the identified lyrics text; the target phrase refers to a paragraph of lyrics with a preset number of words and which also appears continuously in the pure lyrics text; for each text unit in the target phrase, calculating the alignment confidence of the target phrase based on the ratio between the selected value of each text unit and the first timestamp of each text unit; wherein the alignment confidence of each segment of the target phrase can be comprehensively used as the alignment confidence of the pure lyrics text as a whole; the selected value includes the timestamp deviation of the text unit or one of the values ​​in the corresponding first timestamp.

[0091] Generally, the actual situation is not as smooth as in case a. ASR recognition may have more or less words, that is, the number of words recognized by ASR is different from the number of words automatically aligned. In this case, you can choose to calculate the alignment confidence of a part of the lyrics instead of the entire lyrics. The calculation formula can be similar to the above alignment confidence conf avg Formula; the difference is that the parameter m is the number of words in each small part of the lyrics (such as 3 words), rather than the total number of words in the entire lyrics. The resulting alignment confidence conf avg is the alignment confidence of each small part of lyrics.

[0092] The above-mentioned situation b will be described in detail below.

[0093] b1. In some examples, the above-mentioned process of "selecting at least one target phrase from the recognized lyrics text" may specifically include: sliding from front to back to select at least one target phrase from the recognized lyrics text according to a preset sliding step.

[0094] For example, the alignment confidence can be calculated by dynamic sliding window when the ASR output length and the actual lyrics length are inconsistent. Specifically, the sliding window length can be defined first, that is, the preset number of words is 3 and the sliding step is 1, and the target words and phrases that meet the sliding window conditions are searched from the ASR results, for example:

[0095] Actual lyrics (or original lyrics): "If You Love Me"

[0096] ASR Lyrics: "If You Love Me"

[0097] The content of the first sliding window is "If oh", which is not consistent in the original lyrics, so the window moves one word to the right, and the content obtained is "If oh you", which is still not consistent in the original lyrics. The window moves one word to the right again until the three words "you love me" meet the conditions (appear in the original lyrics). Then these three words can be used as the target phrase. After that, the alignment confidence of the current three words "you love me" can be calculated according to the calculation method in case a, that is, the alignment confidence conf at this time avg The parameter m in the formula represents the preset word count of 3. Thus, each target word or phrase will have a corresponding alignment confidence. By combining the alignment confidences of all target words or phrases, the alignment confidence of the entire pure lyrics text can be obtained.

[0098] In some examples, as in the above dynamic sliding window method, the following situation may exist, that is, in the recognized lyrics text, there is no target word or sentence within the preset length but there are short words or sentences, and the short words or sentences also appear continuously in the pure lyrics text but the number of words is less than the number of words in the target words or sentences. Then the embodiment of the present application may also include the following operations: the alignment confidence of the target word or sentence adjacent to the short word or sentence is used as the alignment confidence of the short word or sentence; the alignment confidence of the short word or sentence is used to participate in the calculation of the alignment confidence of the pure lyrics text as a whole.

[0099] The above short phrases may refer to a phrase with more characters than the corresponding phrase in the pure lyrics text, such as an additional modal particle "oh". For example, in the original lyrics and ASR lyrics in the above example, it can be assumed that the preset length is 5 characters. If there is no target phrase that meets the conditions but there is a short phrase "if" in the first 5 characters of the ASR lyrics, the alignment confidence of "if" can reuse the alignment confidence of its adjacent target phrase "you love me"; this operation is to avoid filtering out a large number of useful phrases, resulting in low alignment confidence and increased false positive rate.

[0100] As described above, the word recognition results of the entire ASR lyrics can be traversed to obtain multiple alignment confidence values. For words and sentences that do not meet the conditions, the adjacent target word and sentence results can be taken as the alignment confidence of the current word and sentence.

[0101] In some examples, in the above dynamic sliding window method, there may be the following situation, that is, there is no target word or sentence in the adjacent position of the short word or sentence, then the embodiment of the present application may also include the following operation: setting the alignment confidence of the short word or sentence to be less than a preset threshold. This preset threshold may be a confidence threshold.

[0102] For example, it can be assumed that if there is no adjacent alignment confidence after traversing more than 5 words from the front to the back (i.e., the distance sliding window) in the recognized lyrics text, then it can be determined that the alignment confidence of the short word is less than a preset threshold, such as the alignment confidence is set to 0, to indicate that the result is unreliable. Of course, if the preset length is set long enough, such as close to or even the total number of words in the entire lyrics, then at this time, if the sliding window exceeds the preset length and there is still no nearest alignment confidence, it can be determined that the alignment confidence of the entire lyrics is less than the preset threshold, to indicate that the result is unreliable. Therefore, it can be seen that in order to make the confidence result practical and reasonable, the preset length should be set according to the actual situation (such as the total number of words in the original lyrics and / or the ASR lyrics phonemes) to avoid the preset length being too long or too short to affect the confidence result.

[0103] b2. In some examples, the above-mentioned process of "selecting at least one target phrase from the recognized lyrics text" may specifically include: using a distance calculation function to calculate the change distance between the recognized lyrics text and the pure lyrics text; the change distance is used to characterize the cost of transforming a phrase with a preset number of words in a lyrics text into a phrase with a preset number of words in another lyrics text; taking the change distance less than the preset distance as the target distance, and taking the phrase taken from the recognized lyrics text to which the target distance points as the target phrase.

[0104] The above distance calculation function may refer to the dynamic time warping function (DTW). The DTW algorithm can find the optimal deformation path between two time series (which may be of unequal length) through the method of dynamic programming, so that the cumulative distance of the two sequences along this path (usually based on a certain distance metric such as Euclidean distance) can be deformed at the minimum cost. This cumulative distance is the output of the DTW algorithm, which represents the shortest distance between the two time series and reflects the similarity between the two time series. Therefore, it can be understood that in the field of lyrics, the DTW algorithm can output the cost of a sentence in a lyrics text being transformed into a sentence in another lyrics text. The smaller the cost of this deformation, the more similar the two sentences are, so the ASR recognition sentence with a small cost can be selected as the target sentence to calculate the alignment confidence. For example, the DTW value of each sentence formed by 3 words can be calculated, and the sentence corresponding to the smaller DTW value can be used as the target sentence. In some examples, the entire lyrics text can be divided into groups according to punctuation marks, and the DTW value of each phrase formed by n words in each group can be calculated to determine the target phrase with the smallest deformation distance in the group, so that the target phrase of the entire lyrics text can be selected.

[0105] The above DTW algorithm can be as follows:

[0106] D(i,j)=d(Ai,Bj)+min(D(i-1,j),D(i,j-1),D(i-1,j-1))

[0107] Wherein, D(i,j) represents the DTW distance from the 1st element to the ith element of time series A, and from the 1st element to the jth element of time series B, where the time series may refer to a certain phrase, and the element may refer to a text unit such as a single word in the embodiment of the present application; d(Ai,Bj) represents the distance between the ith element of A and the jth element of B, which is usually a simple distance metric, such as the Euclidean distance, or the distance may be the time difference between the ith word recognized by ASR and the ith word automatically aligned, which may be calculated similarly to S calculated by the above deviation formula; min(D(i-1,j),D(i,j-1),D(i-1,j-1)) represents the minimum DTW distance between three adjacent points. The above DTW algorithm allows nonlinear stretching or compression on the time series; the above DTW algorithm may not consider the end time of the text unit.

[0108] Step S23: Calculate the unit time deviation between the second timestamp of each text unit and the corresponding manually annotated timestamp.

[0109] Step S24: Determine the label anomaly detection result for the manually labeled timestamp based on the alignment confidence and the unit time deviation.

[0110] In some examples, the specific process of step S24 may include: comparing the alignment confidence and the unit time deviation with the confidence threshold and the unit time threshold respectively; if the alignment confidence is greater than the confidence threshold, and the unit time deviation is greater than the unit time threshold, then determining that the labeling anomaly detection result is that the manually labeled timestamp is an incorrectly labeled timestamp.

[0111] As described above, when obtaining the alignment confidence value, the unit time deviation between the automatic alignment timestamp and the manual QRC can be compared. If the alignment confidence is greater than the confidence threshold, and one of the unit time deviations is greater than the unit time threshold, it can be considered with high alignment confidence that the timestamp of the manual QRC is indeed problematic, that is, the currently problematic manually annotated timestamp is detected. Manually annotated timestamps without problems can be ignored and not corrected.

[0112] The above confidence threshold and unit time threshold can be set based on experience and historical data. Specifically, an initial confidence threshold can be determined based on the alignment confidence generated by a large number of audio samples, and an initial unit time threshold can be determined based on the unit time deviation generated by these audio samples; then, the above initial confidence threshold and initial unit time threshold can be used to check whether the manually annotated timestamps (i.e., manual QRCs) of a batch of audios are incorrect; if the test results do not meet expectations, the above initial confidence threshold and initial unit time threshold can be continuously adjusted up or down until the expectations are met, thereby setting an ideal confidence threshold and unit time threshold; this expectation can be that, based on the current confidence threshold and unit time threshold, the ratio between the number of detected manual problem QRCs and the total number of actual problematic manual QRCs is greater than a certain percentage value.

[0113] Generally speaking, the higher the threshold setting, the fewer detections and the more reliable it is, but it may also miss some erroneous QRCs; the lower the setting, the more detections and the higher the false detection rate; therefore, the above confidence threshold and unit time threshold should be set reasonably according to actual conditions, and should not be too high or too low, so as not to affect the detection results.

[0114] like Figure 3 From the description, it can be seen that in addition to citing the second timestamp and the manually annotated timestamp for time comparison, the embodiment of the present application also introduces the first timestamp and the second timestamp for time comparison. This timing problem QRC detection strategy based on alignment confidence is based on the automatic alignment technology, introduces the singing ASR technology, and uses the word timestamp generated by the automatic alignment and the word timestamp generated by the singing ASR technology for dynamic comparison to obtain a timestamp alignment confidence. Compared with the existing manual detection method, the embodiment of the present application not only gives the automatic alignment timestamp result, but also increases the alignment confidence of the result, avoiding the limitations of relying entirely or solely on the performance of the automatic alignment algorithm, and through two threshold filtering methods, the detection result obtained is more reliable.

[0115] based on Figure 2 As an example content, as a possible implementation method, after step S24, the embodiment of the present application may also include the following operations (error correction): if the labeling anomaly detection result indicates that the manually labeled timestamp is an erroneously labeled timestamp, the erroneously labeled timestamp is replaced with other timestamps; the other timestamps are determined based on the first timestamp and the second timestamp of the text unit corresponding to the unit time deviation.

[0116] In practical applications, if the manually annotated timestamp of a text unit such as a word is detected to be incorrect, the first timestamp or the second timestamp of the text unit can be used to replace the manually annotated timestamp to achieve timing correction of the artificial question qrc. Because the timestamp output by the ASR model is often more reliable than the timestamp output by the automatic alignment model, it is preferred to replace the incorrect manually annotated timestamp with the first timestamp output by the ASR model. Of course, as a possible implementation method, a compromise timestamp can be calculated based on the first timestamp and the second timestamp to replace the incorrect manually annotated timestamp, which is not limited here. In general, the above-mentioned other timestamps can be summarized as: other timestamp = a*first timestamp + b*second timestamp, where a and b are weight values, and when the weight of a certain item is 0, it means that the other timestamp is another item (such as b is 0, which means that the other timestamp is the first timestamp); of course, a and b can also take non-zero values, which can be determined by the actual situation and are not limited here.

[0117] In summary, the embodiment of the present application proposes a timing problem QRC detection strategy based on alignment confidence. By introducing ASR technology, the alignment confidence of the timestamp generated by the current algorithm can be calculated, and the word time deviation of the manual QRC and the algorithm QRC can be combined to decide whether the manually marked current word timestamp has a problem, thereby reducing the false detection caused by the deviation caused by a single technology (such as relying only on the performance of the automatic alignment algorithm), increasing the probability of detecting problem QRC, and at the same time, helping to reduce labor input costs, quickly and accurately repair more problem QRCs, and promote the improvement of the QRC quality of the music library.

[0118] Compared to Figure 2 The examples shown illustrate that the above-mentioned additional or detailed example contents or possible implementation methods (such as error correction operations) do not necessarily have to be executed during the specific implementation. If more than two examples or possible implementation methods are added, these examples or possible implementation methods can be implemented in combination or separately. If implemented in combination, the execution order between the combined examples can be determined according to their respective processing logic, which can be determined according to the actual scenario.

[0119] See also Figure 4 According to a second aspect of an embodiment of the present application, there is provided an electronic device 400, which may include one or more processors (such as central processing units (CPU)) 401 and a memory 405, wherein the memory 405 stores one or more applications or data.

[0120] The memory 405 may be a volatile storage or a persistent storage. The program stored in the memory 405 may include one or more modules, each of which may include a series of instruction operations in the electronic device. Furthermore, the processor 401 may be configured to communicate with the memory 405 and execute a series of instruction operations in the memory 405 on the electronic device 400.

[0121] The electronic device 400 may also include one or more power supplies 402, one or more wired or wireless network interfaces 403, one or more input and output interfaces 404, and / or one or more operating systems, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0122] The processor 401 can execute the operations performed by the aforementioned first aspect or any specific method embodiment of the first aspect, and the details will not be repeated here.

[0123] The present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described in the first aspect or any specific implementation of the first aspect.

[0124] The present application provides a computer program product comprising instructions or a computer program. When the computer program product is run on a computer, the computer is enabled to execute the method described in the first aspect or any specific implementation of the first aspect.

[0125] It is understood that in various embodiments of the present application, the sequence number of each step does not mean the order of execution, and the execution order of each step should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The operation content added or refined in each example scheme of the above method, system or device (if any) does not necessarily have to be executed in the specific implementation. If more than two operations are added, these operations can be implemented in combination or separately, depending on the actual scenario.

[0126] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system (if any) and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0127] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system or device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0128] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product (or computer program product) is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a business server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A lyrics processing method, characterized in that: include: Obtain the first timestamp, the second timestamp, and the manually annotated timestamp corresponding to each text unit in the pure lyrics text of the target song; wherein the first timestamp, the second timestamp, and the manually annotated timestamp of the text unit respectively annotate the start playing time and the continuous playing time of the text unit in the target song with different ideas; the first timestamp is obtained by identifying the audio features of the target song; the second timestamp is the timestamp corresponding to the text unit after being aligned with the human voice lyrics of the target song; Based on the timestamp deviation between the first timestamp and the corresponding second timestamp of each text unit, calculating the alignment confidence between the first timestamp and the corresponding second timestamp; the higher the alignment confidence, the more consistent the first timestamp and the corresponding second timestamp; Calculating a unit time deviation between the second timestamp of each text unit and the corresponding manually annotated timestamp; Based on the alignment confidence and the unit time deviation, a labeling anomaly detection result for the manually labeled timestamp is determined.

2. The lyrics processing method according to claim 1, characterized in that: After determining the label anomaly detection result for the manually labeled timestamp based on the alignment confidence and the unit time deviation, the method further includes: If the labeling anomaly detection result indicates that the manually labeled timestamp is an erroneously labeled timestamp, the erroneously labeled timestamp is replaced with another timestamp; the other timestamp is determined based on the first timestamp and the second timestamp of the text unit corresponding to the unit time deviation.

3. The lyrics processing method according to claim 1, characterized in that: The calculating the alignment confidence between the first timestamp and the corresponding second timestamp based on the timestamp deviation between the first timestamp and the corresponding second timestamp of each text unit includes: If the number of words in the lyrics identified by the audio features of the target song is consistent with the number of words in the lyrics text, based on the ratio between the selected value of each of the text units and each of the first timestamps, the alignment confidence between the first timestamp and the corresponding second timestamp is calculated; The selected value includes the timestamp deviation of the text unit or a value corresponding to the first timestamp.

4. The lyrics processing method according to claim 1, characterized in that: The calculating the alignment confidence between the first timestamp and the corresponding second timestamp based on the timestamp deviation between the first timestamp and the corresponding second timestamp of each text unit includes: If the number of words in the lyrics identified by the audio features of the target song is inconsistent with the number of words in the lyrics in the pure lyrics text, at least one target phrase is selected from the identified lyrics text; the target phrase refers to a paragraph of lyrics with a preset number of words and which also appears continuously in the pure lyrics text; For each text unit in the target word or sentence, based on a ratio result between a selected value of each text unit and the first timestamp of each text unit, calculating an alignment confidence of the target word or sentence; The selected value includes the timestamp deviation of the text unit or one of the values ​​corresponding to the first timestamp.

5. The lyrics processing method according to claim 4, characterized in that: The selecting at least one target phrase from the recognized lyrics text comprises: According to a preset sliding step length, at least one section of the target phrase is selected from the recognized lyrics text by sliding from front to back.

6. The lyrics processing method according to claim 5, characterized in that: If the identified lyrics text does not contain the target phrase within the preset length but contains short phrases, and the short phrases also appear continuously in the pure lyrics text but have fewer words than the target phrases, the method further includes: The alignment confidence of the target phrase adjacent to the short phrase is used as the alignment confidence of the short phrase; the alignment confidence of the short phrase is used to participate in the calculation of the alignment confidence of the entire pure lyrics text.

7. The lyrics processing method according to claim 6, characterized in that: If the target word or sentence does not exist in the adjacent position of the short word or sentence, the method further includes: The alignment confidence of the short words is set to be less than a preset threshold.

8. The lyrics processing method according to claim 4, characterized in that: The selecting at least one target phrase from the recognized lyrics text comprises: A distance calculation function is used to calculate the change distance between the recognized lyrics text and the pure lyrics text; the change distance is used to represent the cost of transforming a phrase of the preset number of words in one lyrics text into a phrase of the preset number of words in another lyrics text; The change distance that is smaller than the preset distance is taken as the target distance, and the phrase taken from the recognized lyrics text and pointed to by the target distance is taken as the target phrase.

9. The lyrics processing method according to claim 4, characterized in that: The process of determining the ratio result includes: Taking the timestamp deviation between the first timestamp and the corresponding second timestamp of each text unit about the start playing time as the selected value; The ratio between the sum of the selected values ​​of the various text units and the first duration sum is determined as the ratio result; the first duration sum is the sum of the continuous playback time in the first timestamps of the various text units.

10. The lyrics processing method according to claim 1, characterized in that: The determining, based on the alignment confidence and the unit time deviation, a labeling anomaly detection result for the manually labeled timestamp includes: Compare the alignment confidence and the unit time deviation with a confidence threshold and a unit time threshold, respectively; If the alignment confidence is greater than the confidence threshold, and the unit time deviation is greater than the unit time threshold, it is determined that the labeling anomaly detection result is that the manually labeled timestamp is an erroneously labeled timestamp.

11. An electronic device, characterized in that: include: Processor and memory; The processor is configured to communicate with the memory and execute instructions in the memory to implement the method according to any one of claims 1 to 10.

12. A readable storage medium, characterized in that: The readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. A computer program product, characterized in that The computer program product comprises computer instructions, which implement the method according to any one of claims 1 to 10 when executed by a processor.

Images