Singing scoring method, computer device and storage medium

Abstract

The application discloses a singing scoring method, a computer device and a storage medium, which can be applied to various fields or scenes such as artificial intelligence, cloud technology and intelligent transportation. The method comprises the following steps: in response to a singing instruction for a target song, in the process that a user sings the target song, an extraction time window of a target character corresponding to a current singing progress and a standard pitch are acquired; a plurality of target audio segments are sampled and acquired from an initial audio corresponding to the target character according to the extraction time window of the target character; then, the plurality of target audio segments are scored according to the standard pitch of the target character, so as to obtain a scoring result of the target character; and the scoring result of the target character is displayed. Through the method, when the user sings, the scoring result of each character in the lyrics can be obtained in the granularity of the character through the timestamp information and the pitch information of the song, real-time feedback of the scoring result can be realized, and the accuracy of the scoring result is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a singing scoring method, a computer device, and a computer-readable storage medium. Background Technology

[0002] Besides listening to songs, singing is also a form of entertainment. Whether singing online or offline, people want to get real-time scores for their performances, whether high or low, fast or slow. Based on these scores, people can adjust their singing style accordingly. This is especially true for software that allows singing (including practice singing). Users can correct their pronunciation based on the scores to achieve a more satisfactory performance.

[0003] Currently, most singing software provides feedback on the performance line by line when users sing, scoring each line of lyrics individually. However, a single line of lyrics often takes several seconds, resulting in a low feedback frequency and making it difficult to promptly provide scores to users. Furthermore, scoring line by line can lead to inaccurate scores for longer lines. Therefore, how to provide users with timely and accurate feedback on their singing performance has become a pressing technical problem. Summary of the Invention

[0004] This application provides a singing scoring method, computer device, and storage medium. When a user sings, the scoring result of each character in the lyrics can be obtained at the character level by using the timestamp information and pitch information of the song. This enables real-time feedback of the scoring results and improves the accuracy of the scoring results.

[0005] In a first aspect, embodiments of this application disclose a singing scoring method, the method comprising:

[0006] In response to a singing instruction for a target song, during the user's singing of the target song, the extraction time window and standard pitch of the target character corresponding to the current singing progress are obtained;

[0007] Based on the extraction time window of the target character, multiple target audio segments are sampled from the initial audio corresponding to the target character;

[0008] The multiple target audio segments are scored according to the standard pitch of the target character to obtain the score result of the target character;

[0009] Display the rating results for the target character.

[0010] Secondly, embodiments of this application disclose a singing scoring device, which includes:

[0011] The acquisition unit is used to respond to the singing instruction for the target song and, during the process of the user singing the target song, acquire the extraction time window and standard pitch of the target character corresponding to the current singing progress.

[0012] The acquisition unit is further configured to sample and determine multiple target audio segments from the initial audio corresponding to the target character according to the extraction time window of the target character;

[0013] The processing unit is used to score the multiple target audio segments according to the standard pitch of the target character, and obtain the score result of the target character;

[0014] The output unit is used to display the scoring result of the target character.

[0015] Thirdly, embodiments of this application disclose a computer device including a processor adapted to implement one or more computer programs; and a computer storage medium storing one or more computer programs adapted to be loaded and executed by the processor using the singing scoring method described above.

[0016] Fourthly, this application discloses a computer-readable storage medium storing one or more computer programs adapted to be loaded and executed by a processor using the singing scoring method described above.

[0017] Fifthly, embodiments of this application disclose a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the aforementioned singing scoring method.

[0018] In this embodiment, the computer device, in response to a singing instruction for a target song, acquires the extraction time window and standard pitch of the target character corresponding to the current singing progress during the user's singing of the target song. Based on the extraction time window of the target character, multiple target audio segments are sampled from the initial audio corresponding to the target character. Then, the multiple target audio segments are scored according to the standard pitch of the target character to obtain the score result of the target character. Finally, the score result of the target character is displayed in the user interface. This technical solution combines the timestamp of each accurately located character with the standard pitch of that character, comparing the user's audio data with the standard pitch at the character level to obtain a score for the singing result of a single character, and providing feedback on the evaluation results of one or more characters. Compared to solutions that provide scores for the entire lyric, this technical solution can achieve real-time feedback of the score results and improve the accuracy of the score results. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a network architecture diagram of a singing scoring system disclosed in an embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating a singing scoring method disclosed in an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of time related to a target character as disclosed in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the user main interface disclosed in an embodiment of this application;

[0024] Figure 5 This is a schematic flowchart of another singing scoring method disclosed in the embodiments of this application;

[0025] Figure 6 This is a flowchart illustrating a method for scoring target characters as disclosed in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of the structure of a singing scoring device disclosed in an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of the structure of a computer device disclosed in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] To enable users to obtain timely and accurate evaluation results for each character in a sung song, this application proposes a singing scoring method that processes individual characters to obtain a score for each character (fast or slow, high or low, or normal). The singing scoring method provided in this application can be implemented based on cloud technology and / or blockchain technology. Specifically, it can involve one or more of cloud technologies, such as cloud storage, cloud database, and big data. For example, data required to execute the singing scoring method (e.g., relevant information about the target song) can be obtained from a cloud database. Alternatively, the data required to execute the singing scoring method can be stored on the blockchain in the form of blocks; the data generated by executing the singing scoring method (e.g., scoring results, reference feature information, etc.) can be stored on the blockchain in the form of blocks; furthermore, the data processing device executing the singing scoring method can be a node device in a blockchain network.

[0030] Please see Figure 1 , Figure 1 This is a network architecture diagram of a singing scoring system disclosed in the application embodiment, such as... Figure 1 As shown, the singing scoring system 100 may include at least one terminal device 101 and a computer device 102. The terminal device 101 and the computer device 102 can be connected via a wired or wireless connection, which is not limited here. In specific implementation, the terminal device 101 is mainly used to implement the singing scoring method of this application, including obtaining multiple target audio segments input by the user for a target character based on the extraction time window of the target character, where the target character is the character corresponding to the current singing progress; scoring the multiple target audio segments according to the standard pitch of the target character to obtain the scoring result of the target character; and displaying the scoring result in the user interface of the terminal device 101. The computer device 102 can be responsible for storing the extraction time window and standard pitch information of the target song in the cloud, that is, the terminal device 101 obtains the extraction time window and standard pitch information of the target song from the computer device 102.

[0031] In one possible implementation, the terminal device 101 mentioned above includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc.; the computer device 102 mentioned above can be a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Figure 1 This is merely an illustrative network architecture diagram representing a singing scoring system and is not intended to be limiting. For example, Figure 1 The computer device 102 can be deployed as a node in the blockchain network, or the computer device 102 can be connected to the blockchain network, so that the computer device 102 can upload the scoring results to the blockchain network for storage, so as to prevent the internal data from being tampered with and thus ensure data security.

[0032] Based on the aforementioned singing scoring system, the singing scoring method of this application embodiment can generally include: in response to a singing instruction for a target song, during the user's singing of the target song, computer device 102 acquires the extraction time window and standard pitch of the target character corresponding to the current singing progress, and samples multiple target audio segments from the initial audio corresponding to the target character according to the extraction time window of the target character; then, scores the multiple target audio segments according to the standard pitch of the target character to obtain the scoring result of the target character; finally, the scoring result is sent to terminal device 101 so that the scoring result is displayed in the user interface of terminal device 101. This application, by combining the timestamp of accurately locating each character and the standard pitch of that character, compares the user's audio data and the standard pitch at the character level to obtain the scoring result of a single character, and feeds back the evaluation result of one or more characters. Compared with the scheme of feeding back the scoring result of the entire lyric, the technical solution of this application can realize real-time feedback of the scoring result and improve the accuracy of the scoring result. Since the score is obtained immediately after each character is sung, the feedback delay is greatly reduced, allowing users to receive timely feedback during the singing process to correct pronunciation problems of a particular word, thereby improving the user experience.

[0033] It should be noted that in the specific embodiments of this application, the data involved, such as the target song, reference feature information of the target song, and audio data, are all authorized by the user. When the above embodiments of this application are applied to specific products or technologies, the data used must be authorized or agreed to by the user, and the collection, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0034] Please see Figure 2 This is a flowchart illustrating a singing scoring method disclosed in an embodiment of this application. The singing scoring includes, but is not limited to, steps S201-S204:

[0035] S201: In response to a singing instruction for a target song, during the user's singing of the target song, obtain the extraction time window and standard pitch of the target character corresponding to the current singing progress.

[0036] The target song is the song the user is currently singing or is about to sing. This target song is stored in a database. When the user triggers a command to sing or practice the target song, the computer device can obtain relevant information about the target song. The target song can be in any language, including but not limited to Chinese and English. In this embodiment, after the user triggers the command, the computer device obtains the initial feature information of the target song. This initial feature information includes the timestamp information and standard pitch of each character in the target song. The timestamp information for each character refers to the start and end times of each character, indicating its duration. The timestamp information for each character is obtained by parsing the timestamp information of the target song. A character can be a Chinese character or an English word. The standard pitch of each character is determined based on the sheet music information of the target song. This sheet music information contains the pitch of each character and is standard sheet music information that can be directly obtained from the database.

[0037] One major challenge of real-time scoring is achieving perfect temporal alignment between the user's singing and the lyrics. Even if the user sings exactly in the chronological order of the lyrics, the reaction time delay often results in a difference of several to tens of milliseconds between the user's singing and the lyrics' timestamps. Since real-time scoring extracts the fundamental frequency frame by frame, a difference of tens of milliseconds can correspond to an offset of several or even a dozen frames. This can cause significant errors in matching the fundamental frequency with the standard pitch. Therefore, this application employs a fuzzy matching method to extract the audio data of individual characters. Before extracting the audio data of a single character, the target reserved time for fuzzy matching must first be determined. This target reserved time remains constant within the same song. This time should not be too short, otherwise, insufficient reserved time will lead to a large deviation between the timestamps of the singing and the lyrics, even with fuzzy matching. Conversely, this time should not be too long, otherwise, it will not only increase the computational load per frame, potentially causing computational problems, but also significantly encroach on the temporal calculation interval of the next character. In extreme cases, the calculation intervals of three or more characters may overlap. In this embodiment, the target reservation time can be determined based on the timestamp information of each character. Specifically, half the duration of the character with the shortest timestamp in the target song can be used as the target reservation time. For example, if the shortest duration of the character in the target song is 0.2ms, then the target reservation time is 0.1ms. This effectively avoids the extremely complex situation where timestamps of three or more characters overlap, greatly simplifying the complexity of the system framework.

[0038] For example, taking a single character as an example, let the target reserved time be t0. If the timestamp of the target character on the lyrics is t1 to t2, which is the standard duration determined based on the timestamp of the target song, then based on the target reserved time and the timestamp information of the target character, the extraction time window for the target character can be obtained as t1-t0 to t2+t0. Figure 3 As shown, taking the target character as an example, 301 represents the standard duration of the target character, 302 represents the target reserved duration, and 303 represents the extraction time window of the target character.

[0039] Furthermore, the extraction time window for each character is determined based on the timestamp information of each character and the target reserved time, and the extraction time window and standard pitch of each character are used as reference feature information for the target song.

[0040] S202: Based on the extraction time window of the target character, sample multiple target audio segments from the initial audio corresponding to the target character.

[0041] In one possible implementation, during the user's singing of the target song, for the target character, audio data extraction begins when the time reaches the lower limit of the extraction time window for that character, and stops when the time reaches the upper limit of the extraction time window. This yields an initial audio segment for that target character. Since this application aims to score characters and requires obtaining the pitch of the user singing that character, fundamental frequency extraction is necessary from the audio data.

[0042] In one possible implementation, after obtaining the initial audio corresponding to the target character, a fundamental frequency extraction tool is first used to obtain the number of audio frames corresponding to the initial audio. Specifically, the number of audio frames corresponding to the initial audio of the target character is determined based on the extraction time window of the target character and a preset frame length. The preset frame length can be customized, such as 5.8ms. The ratio of the extraction time window length to the preset frame length is determined as the number of audio frames. Then, the number of audio frames is sampled based on a preset sampling method to obtain multiple target audio segments, each with a duration, such as... Figure 3 As shown in 304, each horizontal line represents the duration of a target audio segment.

[0043] The preset sampling method here can be sampling at equal intervals. For example, if the number of audio frames corresponding to the target character is 100, and the number of frames sampled each time is set to 2, 50 target audio segments can be obtained according to this sampling method. In this application, the preset sampling method can be determined based on the timestamp information of the target character. For example, when the timestamp of the target character is t1 to t2, then the number of frames sampled each time = (t2-t1) / preset frame length.

[0044] It's important to note that vowels and voiced consonants in speech have a fundamental frequency, reflecting the frequency of vocal cord vibration, typically ranging from 100 to 400 Hz. Pitch is determined by this fundamental frequency, and the singing practice system evaluates a user's singing performance based on the degree of match between the fundamental frequency of the user's audio data and the standard pitch of the song. The standard pitch of the target song can be obtained from the song's sheet music file, while the user's pitch needs to be obtained by fundamental frequency extraction from the audio data. Various tools exist for fundamental frequency extraction, employing methods from traditional digital signal processing and deep learning. Traditional digital signal processing methods, such as Yin and Pyin, offer strong applicability and low computational cost. Deep learning-based fundamental frequency extraction methods, such as CREPE, while potentially less applicable outside the training set, exhibit high accuracy within the training set (e.g., in noisy environments). Considering the usage scenario and real-time requirements of the singing practice system, this application adopts Pyin, a toolkit based on traditional digital signal processing, for fundamental frequency extraction. The Pyin toolkit supports frame-by-frame extraction, meeting real-time requirements. The Pyin algorithm is an evolution of the Yin algorithm. The Yin algorithm finds the minimum period of the speech waveform by calculating the autocorrelation function. However, it often uses only one peak and valley for estimation, which may lead to local optima and estimation errors. The Pyin algorithm can find multiple candidate peak and valley values ​​as candidate points in each frame of calculation, effectively avoiding local optima. Furthermore, it uses a hidden Markov model to further improve the smoothness of the fundamental frequency curve.

[0045] S203: Scoring multiple target audio segments based on the standard pitch of the target character to obtain the score result of the target character.

[0046] In one possible implementation, to obtain a more accurate score for the target character, multiple target audio frames are obtained, and then these target audio frames are scored according to the standard pitch of the target character to obtain the score for the target character. The score results include too high, too low, and normal; in some possible embodiments, the score results may also include sung too fast, sung too slow, sung too high, and sung too low.

[0047] In one optional embodiment, based on the lyrics timestamp of the target song, after each line of lyrics is sung, the evaluation results of each previous word are summarized into the overall feedback for that line. Simultaneously, Automatic Speech Recognition (ASR) technology is used to calculate the duration of each word in the audio stream and compare it with the standard duration of each word in the lyrics, thereby determining whether a single word (character) was sung too fast or too slow.

[0048] S204: Display the scoring results for the target character.

[0049] Based on the above method, the scoring results of each character in the target song can be obtained, and then the scoring results can be sent to the terminal device so that they can be displayed in the user interface. Through these scoring results, users can correct their pronunciation in a timely manner, thereby obtaining a more satisfactory singing effect. Reacting the scoring results of each character in real time can greatly improve user satisfaction. At the same time, providing feedback by character makes the feedback more efficient. The user interface can be as Figure 4 shown. This interface is for a singing practice software and is mainly used to display the lyrics of the target song and the evaluation results. Among them, various scoring results are displayed in 401, including singing too fast, singing too slow, singing too high, and singing too low. Among them, "singing too fast" can be represented by the symbol ; "singing too slow" can be represented by the symbol ; "singing too high" can be represented by the symbol "△"; "singing too low" can be represented by the symbol "▽". When a line of lyrics is sung, the scoring results can be displayed at the corresponding position of the lyrics. For example, in a complete line of lyrics "Don't be shy, tell me about you", the marked 404 and 405 respectively indicate that the characters "Don't" and "tell" are sung too high. Similarly, during the singing of the segment "tell me about you" in "Don't be shy, tell me about you", the singing results of each character can also be displayed in real time. For example, 402 represents the standard pitch, and 403 represents the current singing pitch. As Figure 4 shown, 403 is below 402, and it can be seen that this character is sung lower than the standard pitch. Correspondingly, "singing too low" can be displayed in a highlighted form in the 401 area.

[0050] It should be noted that after obtaining the scoring result of a single target character, it can be immediately fed back to the interface, or it may not be immediately fed back to the interface. Instead, to ensure the stability of the scoring results and considering the stability of the user's pronunciation during singing, after obtaining the scoring of each character, post-processing can also be performed. Specifically, obtain the scoring results of a preset number of consecutive target characters in the target song; if the scoring results of a preset number of consecutive target characters are the same, display the scoring results of the preset number of consecutive target characters in the user interface. For example, display the same scoring result at the position of the last target character in the consecutive multiple target characters. Among them, the value of the preset number can be custom-set according to specific circumstances. For example, if the scoring results indicate that N (preset number) consecutive characters are sung too high or too low, the scoring result (singing too high or too low) is fed back at the Nth character. N should not be too large, otherwise the sensitivity of scoring may be lost. Taking N = 2 can obtain good feedback results. In this application, the method selected for post-processing is based on cumulative counting. Optionally, other methods can also be considered, such as smoothing algorithms, etc.

[0051] In this embodiment, the extraction time window and standard pitch of each character in the target song are obtained. During the user's singing of the target song, multiple target audio segments input by the user for the target character are obtained based on the extraction time window of the target character. The target character is the character corresponding to the current singing progress. Then, the multiple target audio segments are scored according to the standard pitch of the target character to obtain the score result of the target character. Finally, the evaluation result is sent to the terminal device so that the score result can be displayed in the user interface of the terminal device. This technical solution combines the timestamp of each accurately located character with the standard pitch of that character, comparing the user's audio data with the standard pitch at the character level to obtain a score for the singing result of a single character, and providing feedback on the evaluation result of one or more characters. Compared to solutions that provide feedback on the score result of the entire lyric, this technical solution can achieve real-time feedback of the score result and improve the accuracy of the score result. Since the score result is obtained after each character is finished, the feedback delay can be greatly reduced, allowing the user to receive timely feedback during the singing process to correct their pronunciation problems of a certain word, thereby improving the user experience.

[0052] Based on the above explanation, please refer to Figure 5 This is another singing scoring method disclosed in the embodiments of this application, including but not limited to the following steps:

[0053] S501: Obtain the initial feature information of the target song, which includes the timestamp information and standard pitch of each character in the target song.

[0054] S502: Determine the target reserved time based on the timestamp information of each character.

[0055] S503: Determine the extraction time window for each character based on the timestamp information of each character and the target reserved time, and use the extraction time window and standard pitch of each character as reference feature information of the target song.

[0056] S504: During the user's singing of the target song, multiple target audio segments are obtained based on the target character extraction time window. The target character is the character corresponding to the current singing progress.

[0057] S505: Determine the first reference pitch and the second reference pitch based on the standard pitch of the target character.

[0058] In one possible implementation, the first reference pitch and the second reference pitch are obtained by adjusting the standard pitch. In this application, the first reference pitch can be a pitch that is an octave higher than the standard pitch, and the second reference pitch can be a pitch that is an octave lower than the standard pitch. An octave can be understood as 12 semitones, which means raising the standard pitch by 12 semitones and lowering it by 12 semitones respectively.

[0059] S506: Determine the minimum pitch difference between each target audio segment in a plurality of target audio segments based on the standard pitch, the first reference pitch, and the second reference pitch.

[0060] In one possible implementation, for each target audio segment, three pitch differences are calculated by comparing the target audio segment with the standard pitch, the first reference pitch, and the second reference pitch. The smallest of these three pitch differences is taken as the minimum pitch difference for the target audio segment. Specifically, the pitch difference between the target audio segment and the standard pitch is calculated by comparing each audio frame included in the target audio segment with the standard pitch to obtain a difference, and then summing the absolute values ​​of the differences corresponding to all audio frames to obtain the pitch difference. The calculation methods for the pitch difference between the target audio segment and the first reference pitch, and the pitch difference between the target audio segment and the second reference pitch are the same and will not be described again. The following example illustrates this: For instance, there are 10 audio segments, each with 10 frames. For audio segment 1, first calculate the difference between the pitch of each audio frame in audio segment 1 and the standard pitch (which also corresponds to 10 frames). Then calculate the absolute value of the difference between the first frame of audio segment 1 and the first frame of the standard pitch to obtain the first difference. Repeat this process to obtain 10 differences. Finally, add the absolute values ​​of these 10 differences to obtain the sum of the first differences between audio segment 1 and the standard pitch. Next, calculate the difference between the pitch of each audio frame in audio segment 1 and the first reference pitch (which also corresponds to 10 frames). Then calculate the absolute value of the difference between the first frame of audio segment 1 and the first frame of the first reference pitch. The process begins by calculating the first difference between the audio segments and the first reference pitch. This is repeated for 10 segments, and the sum of these 10 differences yields the second difference. Next, the difference between the pitch of each audio frame within audio segment 1 and the second reference pitch (also 10 frames) is calculated. The absolute value of the difference between the first frame of audio segment 1 and the first frame of the second reference pitch is then calculated, yielding the first difference. This process is repeated for 10 segments, and the sum of these 10 differences yields the third difference. The smallest sum of these three differences is taken as the minimum pitch difference for audio segment 1. This process is repeated for the other audio segments, resulting in 10 minimum pitch differences for each of the 10 audio segments.

[0061] S507: Determine the target pitch difference from the minimum pitch difference of each target audio segment, and use the target audio segment corresponding to the target pitch difference as the scoring audio segment.

[0062] As explained above, for N audio segments, there are N minimum pitch differences. In one possible implementation, when there are N audio segments, the minimum value among the N minimum pitch differences corresponding to these N audio segments can be taken as the target pitch difference, and the audio segment corresponding to the target pitch difference can be used as the scoring audio segment.

[0063] S508: The pitch corresponding to the target pitch difference of the scoring audio segment is determined as the reference pitch, and the pitch corresponding to the target pitch difference of the scoring audio segment is one of the standard pitch, the first reference pitch, and the second reference pitch.

[0064] In this application, an example can be used to explain this. If there are 10 audio segments, and the minimum pitch difference of audio segment 5 is determined to be the smallest based on the minimum pitch difference among these 10 audio segments, then the minimum pitch difference corresponding to audio segment 5 is taken as the target pitch difference. Audio segment 5 is then used as the scoring audio segment. The pitch corresponding to audio segment 5 is then determined as the reference pitch. The pitch corresponding to audio segment 5 may be one of the standard pitch, the first reference pitch, or the second reference pitch.

[0065] It is important to note that the above method, after obtaining the audio segment, uses the method of subtracting it from the standard pitch and summing the absolute values ​​to perform pitch matching. Alternatively, other vector difference comparison techniques, such as Dynamic Time Warping (DTW) or techniques for calculating vector distances and angles, can also be used for pitch matching.

[0066] S509: Determine the scoring result of the target character based on the reference pitch and the pitch of each audio frame included in the scoring audio segment.

[0067] In one possible implementation, the number of audio frames included in the scoring audio segment is first determined. Then, the pitch of each audio frame in the scoring audio segment is compared sequentially with the reference pitch (whose frame count is the same as the target audio segment's frame count) to obtain the comparison results for each audio frame. Based on the comparison results and the number of audio frames, a first proportion of audio frames in the scoring audio segment with pitches higher than the reference pitch and a second proportion of audio frames in the scoring audio segment with pitches lower than the reference pitch are determined. If the first proportion is greater than a set threshold and the second proportion is less than a set threshold, the score for the target character is determined to be too high, meaning the user's performance of the target character is too high. If both the first and second proportions are less than or equal to the set thresholds, the score for the target character is determined to be normal, meaning the user's performance of the target character is normal. If the second proportion is greater than the set threshold and the first proportion is less than the set threshold, the score for the target character is determined to be too low, meaning the user's performance of the target character is too low. If both the first and second proportions are higher than the set threshold, then it is determined which proportion is larger. If the first proportion is larger, the target character's score is determined to be too high; if the second proportion is larger, the target character's score is determined to be too low. The set threshold is a proportion value, such as 0.5, 0.6, etc.

[0068] In addition to the first and second proportions, there is a third proportion, which is the proportion of audio frames in the scoring audio segment whose pitch is equal to the reference pitch to the total number of audio frames in the scoring audio segment. The sum of the first, second, and third proportions is always equal to 1.

[0069] The above steps S506-S510 can be found in [reference needed]. Figure 6 It can be seen from Figure 6It can be seen that for N audio segments of the target character, for any segment, such as audio segment i, the minimum pitch difference of audio segment i is first determined based on the standard pitch, the first reference pitch, and the second reference pitch. This process is repeated sequentially to obtain the minimum pitch differences corresponding to the N audio segments, resulting in a set including: [minimum pitch difference of audio segment 1, minimum pitch difference of audio segment 2, ..., minimum pitch difference of audio segment N]. Then, the smallest pitch difference among these N minimum pitch differences is determined as the target pitch difference, and the audio segment corresponding to this minimum pitch difference is taken as the scoring audio segment. Further, based on the scoring audio segment, a reference pitch is determined, and the pitch of each audio frame included in the scoring audio segment is compared sequentially with the reference pitch to obtain the comparison results of each audio frame. Then, the first proportion of audio frames in the scoring audio segment with pitches higher than the reference pitch and the second proportion of audio frames in the scoring audio segment with pitches lower than the reference pitch are determined. The scoring result is determined based on the comparison between the proportion and the set threshold. Specifically, if the first proportion is greater than the set threshold and the second proportion is less than the set threshold, the target character is determined to be sung too high; if both the first and second proportions are less than or equal to the set thresholds, the target character is determined to be sung normally; if the second proportion is greater than the set threshold and the first proportion is less than the set threshold, the target character is determined to be sung too low. If both the first and second proportions are greater than the set thresholds, it is determined which proportion is larger. If the first proportion is larger, the target character is determined to be sung too high; if the second proportion is larger, the target character is determined to be sung too low.

[0070] S510: Displays the rating results in the user interface.

[0071] Among them, the above steps S501-S505 and steps S510 and Figure 2 The content described in the relevant embodiments shown is consistent, so it will not be repeated here.

[0072] This application's embodiments primarily focus on how the evaluation results of the target characters are obtained. It mainly involves scoring audio frames sequentially based on pitch information, resulting in high accuracy. Simultaneously, the sung song is scored character-by-character, and the evaluation results are promptly fed back to the user, ensuring high feedback efficiency. For the user, receiving timely evaluation results allows for prompt correction of pronunciation, leading to more accurate singing and further enhancing the user experience.

[0073] Based on the above method embodiments, this application also provides a structural schematic diagram of a singing scoring device. See also... Figure 7 This is a schematic diagram of a singing scoring device provided in an embodiment of this application. Figure 7 The singing scoring device 700 shown can operate the following units:

[0074] The acquisition unit 701, in response to a singing instruction for a target song, acquires the extraction time window and standard pitch of the target character corresponding to the current singing progress during the user's singing of the target song; and samples and acquires multiple target audio segments from the initial audio corresponding to the target character according to the extraction time window of the target character.

[0075] The processing unit 702 is used to score the plurality of target audio segments according to the standard pitch of the target character, and obtain the score result of the target character;

[0076] Output unit 703 is used to display the scoring result of the target character.

[0077] In one possible implementation, when the acquisition unit 701 acquires the extraction time window of the target character corresponding to the current singing progress, it is specifically used for:

[0078] Obtain the timestamp information of the target character corresponding to the current singing progress, and determine the target reserved time based on the timestamp of the target character;

[0079] The extraction time window for the target character is determined based on the timestamp information of the target character and the target reserved time.

[0080] In one possible implementation, when the acquisition unit 701 samples the target audio segment from the initial audio corresponding to the target character according to the extraction time window of the target character, it is specifically used for:

[0081] The number of audio frames corresponding to the initial audio data is determined based on the extraction time window of the target character and the preset frame length.

[0082] The audio frames are sampled according to a preset sampling method to obtain multiple target audio segments. In one possible implementation, when the processing unit 702 scores the multiple target audio segments according to the standard pitch of the target character to obtain the score result of the target character, it is specifically used for:

[0083] Determine the first reference pitch and the second reference pitch based on the standard pitch of the target character;

[0084] The minimum pitch difference for each target audio segment is determined based on the standard pitch, the first reference pitch, and the second reference pitch.

[0085] The target pitch difference is determined from the minimum pitch difference of each target audio segment, and the score result of the target character is obtained based on the target pitch difference.

[0086] In one possible implementation, when the processing unit 702 obtains the scoring result of the target character based on the target pitch difference, it is specifically used for:

[0087] The target audio segment corresponding to the target pitch difference is used as the scoring audio segment;

[0088] The pitch corresponding to the target pitch difference of the scoring audio segment is determined as the reference pitch, and the pitch corresponding to the target pitch difference of the scoring audio segment is one of the standard pitch, the first reference pitch, and the second reference pitch.

[0089] The scoring result of the target character is determined based on the reference pitch and the pitch of each audio frame included in the scoring audio segment.

[0090] In one possible implementation, when the processing unit 702 determines the scoring result of the target character based on the reference pitch and the pitches of each audio frame included in the scoring audio segment, it is specifically used for:

[0091] Determine the number of audio frames included in the scoring audio segment;

[0092] The pitch of each audio frame in the scoring audio segment is compared with the reference pitch in turn to obtain the comparison result of each audio frame.

[0093] The score for the target character is determined based on the comparison results and the number of audio frames.

[0094] In one possible implementation, the scoring result includes normal, high, and low. When the processing unit 702 determines the scoring result of the target character based on the comparison result and the number of audio frames, it is specifically used for:

[0095] Based on the comparison results and the number of audio frames, a first proportion of audio frames in the scoring audio segment whose pitch is higher than the reference pitch is determined, and a second proportion of audio frames in the scoring audio segment whose pitch is lower than the reference pitch is determined.

[0096] If the first ratio is greater than a set threshold and the second ratio is less than the set threshold, then the score of the target character is determined to be too high.

[0097] If both the first ratio and the second ratio are less than or equal to the set threshold, then the scoring result of the target character is determined to be normal.

[0098] If the second ratio is greater than the set threshold and the first ratio is less than the set threshold, then the score of the target character is determined to be too low.

[0099] In one possible implementation, when the output unit 703 displays the scoring result of the target character, it is specifically used for:

[0100] Obtain the scoring results of a preset number of consecutive target characters in the target song;

[0101] If a preset number of consecutive target characters have the same score, then the same score will be displayed at the position of the last character of the consecutive target characters.

[0102] According to one embodiment of this application, Figure 2 and Figure 5 The singing scoring method shown can involve various steps that can be achieved by... Figure 7 The individual units in the singing scoring device shown perform this function. For example, Figure 2 In the singing scoring method shown, step S201 can be performed by... Figure 7 The acquisition unit 701 in the singing scoring device shown is used to execute steps S202 and S203, which can be performed by... Figure 7 The processing unit 702 in the singing scoring device shown executes the steps, and step S204 can be performed by... Figure 7 The output unit 703 in the singing scoring device shown is used to perform this.

[0103] According to another embodiment of this application, Figure 7 The singing scoring device shown can be composed of individual or combined units into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the singing scoring device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0104] According to another embodiment of this application, the following can be achieved by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), a device capable of performing operations such as... Figure 2 and Figure 5 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 7The document describes a singing scoring device and a singing scoring method for implementing embodiments of this application. The computer program may be recorded on, for example, a computer-readable storage medium, loaded onto the aforementioned computer device via the same medium, and executed therein.

[0105] In this embodiment, the acquisition unit 701 acquires the extraction time window and standard pitch of each character in the target song; the processing unit 702, during the user's singing of the target song, acquires multiple target audio segments input by the user for the target character based on the extraction time window of the target character, where the target character is the character corresponding to the current singing progress; then, it scores the multiple target audio segments according to the standard pitch of the target character to obtain the score result of the target character; finally, it sends the evaluation result to the terminal device so that the output unit 703 can display the score result in the user interface of the terminal device. This application, by combining the timestamp of accurately locating each character and the standard pitch of that character, compares the user's multiple target audio segments and standard pitch at the character level to obtain a score for the singing result of a single character, and provides feedback on the evaluation result of one or more characters. Compared with the scheme of providing feedback on the score result of the entire lyric, the technical solution of this application can achieve real-time feedback of the score result and improve the accuracy of the score result. Since the score result can be obtained after a single character is finished, the feedback delay can be greatly reduced, allowing the user to receive timely feedback during the singing process to correct their pronunciation problems of a certain word, thereby improving the user experience.

[0106] Based on the above methods and apparatus embodiments, this application provides a computer device, see [link to relevant documentation]. Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 8 The computer device 800 shown includes at least a processor 801, an input interface 802, an output interface 803, a computer storage medium 804, and a memory 805. The processor 801, input interface 802, output interface 803, computer storage medium 804, and memory 805 can be connected via a bus or other means.

[0107] The computer storage medium 804 can be stored in the memory 805 of the computer device 800. The computer storage medium 804 is used to store computer programs, which include program instructions. The processor 801 is used to execute the program instructions stored in the computer storage medium 804. The processor 801 (or CPU (Central Processing Unit)) is the computing and control core of the computer device 800. It is adapted to implement one or more instructions, specifically to load and execute one or more computer instructions to achieve corresponding method flows or corresponding functions.

[0108] This application embodiment also provides a computer storage medium (memory), which is a memory device in a computer device 800 for storing programs and data. It is understood that the computer storage medium here may include the built-in storage medium in the computer device 800, or it may include extended storage media supported by the computer device 800. The computer storage medium provides storage space, which stores the operating system of the computer device 800. Furthermore, this storage space also stores one or more computer programs (including program code) suitable for loading and execution by the processor 801. It should be noted that the computer storage medium here may be a high-speed RAM memory, or a non-volatile memory, such as at least one disk storage device; optionally, it may also be at least one computer storage medium located remotely from the aforementioned processor.

[0109] In one embodiment, the computer storage medium may be loaded and executed by the processor 801, along with one or more computer programs stored in the computer storage medium, to achieve the aforementioned... Figure 2 and Figure 5 The corresponding steps of the singing scoring method. In specific implementation, one or more instructions in the computer storage medium are loaded by the processor 801 and executed as follows:

[0110] In response to a singing instruction for a target song, during the user's singing of the target song, the extraction time window and standard pitch of the target character corresponding to the current singing progress are obtained;

[0111] Based on the extraction time window of the target character, multiple target audio segments are sampled and obtained from the initial audio corresponding to the target character;

[0112] The multiple target audio segments are scored according to the standard pitch of the target character to obtain the score result of the target character;

[0113] Display the rating results for the target character.

[0114] In one possible implementation, when the processor 801 obtains the extraction time window for each character in the target song, it is specifically used for:

[0115] Obtain the timestamp of the target character corresponding to the current singing progress, and determine the target reserved time based on the timestamp of the target character;

[0116] The extraction time window for the target character is determined based on the timestamp of the target character and the target reserved time.

[0117] In one possible implementation, when the processor 801 samples the target audio segment from the initial audio corresponding to the target character according to the extraction time window of the target character, it is specifically used for:

[0118] The number of audio frames corresponding to the initial audio data is determined based on the extraction time window of the target character and the preset frame length.

[0119] The number of audio frames is sampled according to a preset sampling method to obtain multiple target audio segments. In one possible implementation, when the processor 801 scores the multiple target audio segments according to the standard pitch of the target character to obtain the score result of the target character, it is specifically used for:

[0120] Determine the first reference pitch and the second reference pitch based on the standard pitch of the target character;

[0121] The minimum pitch difference for each target audio segment is determined based on the standard pitch, the first reference pitch, and the second reference pitch.

[0122] The target pitch difference is determined from the minimum pitch difference of each target audio segment, and the score result of the target character is obtained based on the target pitch difference.

[0123] In one possible implementation, when the processor 801 obtains the scoring result of the target character based on the target pitch difference, it is specifically used for:

[0124] The target audio segment corresponding to the target pitch difference is used as the scoring audio segment;

[0125] The pitch corresponding to the target pitch difference of the scoring audio segment is determined as the reference pitch, and the pitch corresponding to the target pitch difference of the scoring audio segment is one of the standard pitch, the first reference pitch, and the second reference pitch.

[0126] The scoring result of the target character is determined based on the reference pitch and the pitch of each audio frame included in the scoring audio segment.

[0127] In one possible implementation, when the processor 801 determines the scoring result of the target character based on the reference pitch and the pitches of each audio frame included in the scoring audio segment, it is specifically used for:

[0128] Determine the number of audio frames included in the scoring audio segment;

[0129] The pitch of each audio frame in the scoring audio segment is compared with the reference pitch in turn to obtain the comparison result of each audio frame.

[0130] The score for the target character is determined based on the comparison results and the number of audio frames.

[0131] In one possible implementation, the scoring result includes normal, high, and low. When the processor 801 determines the scoring result of the target character based on the comparison result and the number of audio frames, it is specifically used for:

[0132] Based on the comparison results and the number of audio frames, a first proportion of audio frames in the scoring audio segment whose pitch is higher than the reference pitch is determined, and a second proportion of audio frames in the scoring audio segment whose pitch is lower than the reference pitch is determined.

[0133] If the first ratio is greater than a set threshold and the second ratio is less than the set threshold, then the score of the target character is determined to be too high.

[0134] If both the first ratio and the second ratio are less than or equal to the set threshold, then the scoring result of the target character is determined to be normal.

[0135] If the second ratio is greater than the set threshold and the first ratio is less than the set threshold, then the score of the target character is determined to be too low.

[0136] In one possible implementation, when the processor 801 displays the scoring result of the target character, it is specifically used for:

[0137] Obtain the scoring results of a preset number of consecutive target characters in the target song;

[0138] If a preset number of consecutive target characters have the same score, then the same score will be displayed at the position of the last character of the consecutive target characters.

[0139] In this application example, the processor 801, in response to a singing instruction for a target song, acquires the extraction time window and standard pitch of the target character corresponding to the current singing progress during the user's singing of the target song. Based on the extraction time window of the target character, it samples multiple target audio segments from the initial audio corresponding to the target character. Then, it scores the multiple target audio segments according to the standard pitch of the target character to obtain the score result of the target character. Finally, it displays the score result of the target character in the user interface. This application, by combining the timestamp of accurately locating each character with the standard pitch of that character, compares the user's audio data with the standard pitch at the character level to obtain the score result of a single character and provides feedback on the evaluation results of one or more characters. Compared to the scheme that provides feedback on the score result of the entire lyric, the technical solution of this application can achieve real-time feedback of the score result and improve the accuracy of the score result. Since the score result is obtained after each character is finished, the feedback latency can be greatly reduced, allowing the user to receive timely feedback during the singing process to correct their pronunciation problems of a certain word, thereby improving the user experience.

[0140] According to one aspect of this application, an embodiment of this application also provides a computer product including a computer program stored in a computer-readable storage medium. A processor 801 reads the computer program from the computer-readable storage medium and executes the computer program, causing a computer device 800 to perform... Figure 2 , Figure 5 as well as Figure 6 The relevant methods are shown.

[0141] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0143] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A singing scoring method, characterized in that, The method includes: In response to a singing instruction for a target song, during the user's singing of the target song, the extraction time window and standard pitch of the target character corresponding to the current singing progress are obtained; Based on the extraction time window of the target character, multiple target audio segments are sampled and obtained from the initial audio corresponding to the target character; Determine the first reference pitch and the second reference pitch based on the standard pitch of the target character; The minimum pitch difference for each target audio segment is determined based on the standard pitch, the first reference pitch, and the second reference pitch. The target pitch difference is determined from the minimum pitch difference of each target audio segment, and the score of the target character is obtained based on the target pitch difference; wherein, the target audio segment corresponding to the target pitch difference is taken as the score audio segment; the pitch corresponding to the target pitch difference of the score audio segment is determined as the reference pitch, and the pitch corresponding to the target pitch difference of the score audio segment is one of the standard pitch, the first reference pitch, and the second reference pitch; the score of the target character is determined based on the reference pitch and the pitch of each audio frame included in the score audio segment; Display the rating results for the target character.

2. The method according to claim 1, characterized in that, The extraction time window for obtaining the target character corresponding to the current singing progress includes: Obtain the timestamp of the target character corresponding to the current singing progress, and determine the target reserved time based on the timestamp of the target character; The extraction time window for the target character is determined based on the timestamp of the target character and the target reserved time.

3. The method according to claim 1, characterized in that, The step of sampling and acquiring multiple target audio segments from the initial audio corresponding to the target character according to the extraction time window of the target character includes: The number of audio frames corresponding to the initial audio is determined based on the extraction time window of the target character and the preset frame length. The number of audio frames is sampled according to a preset sampling method to obtain multiple target audio segments.

4. The method according to claim 1, characterized in that, The step of determining the scoring result of the target character based on the reference pitch and the pitch of each audio frame included in the scoring audio segment includes: Determine the number of audio frames included in the scoring audio segment; The pitch of each audio frame in the scoring audio segment is compared with the reference pitch in turn to obtain the comparison result of each audio frame. The score for the target character is determined based on the comparison results and the number of audio frames.

5. The method according to claim 4, characterized in that, The scoring results include normal, high, and low; determining the scoring result of the target character based on the comparison results and the number of audio frames includes: Based on the comparison results and the number of audio frames, a first proportion of audio frames in the scoring audio segment with a pitch higher than the reference pitch is determined, and a second proportion of audio frames in the scoring audio segment with a pitch lower than the reference pitch is determined. If the first ratio is greater than a set threshold and the second ratio is less than the set threshold, then the score of the target character is determined to be too high. If both the first ratio and the second ratio are less than or equal to the set threshold, then the scoring result of the target character is determined to be normal. If the second ratio is greater than the set threshold and the first ratio is less than the set threshold, then the score of the target character is determined to be too low.

6. The method according to any one of claims 1-5, characterized in that, The display of the rating results for the target character includes: Obtain the scoring results of a preset number of consecutive target characters in the target song; If a preset number of consecutive target characters have the same score, then the same score will be displayed at the position of the last character of the consecutive target characters.

7. A computer device, characterized in that, The computer device includes: A processor, suitable for implementing one or more computer programs; and, A computer storage medium storing one or more computer programs, said one or more computer programs being adapted to be loaded by the processor and executed as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more computer programs, which are adapted to be loaded by a processor and executed by the singing scoring method as described in any one of claims 1-6.

Images