Real-time singing evaluation method, system, electronic device and medium

Abstract

The present disclosure relates to a real-time singing evaluation method, system, electronic device and medium. The method comprises: pulling a user audio stream in real time, removing the accompaniment of the audio stream; judging whether the audio stream after removing the accompaniment is noise frame by frame, extracting the user pitch of the current frame when the current frame is not noise; adaptively adjusting the user pitch according to the original singing pitch line and the user historical pitch information to obtain the adjusted user pitch; determining whether the difference between the adjusted user pitch and the original singing pitch is within a preset interval to judge the accuracy of the pitch; judging whether the current phrase is ended, displaying the user pitch value of the current frame in real time when it is not ended, and calculating and displaying the current phrase score according to the proportion of the accurate frame number in the total frame number when it is ended; judging whether the song is ended, continuing to pull the user audio stream for the next phrase score calculation when it is not ended, and calculating the final score of the whole song by accumulating the scores of each phrase and displaying the final score when it is ended.

Description

Technical Field

[0001] This disclosure relates to the field of audio processing, and more specifically to real-time singing evaluation methods, systems, electronic devices, and media. Background Technology

[0002] Music is an area with strong user personalization needs. Singing favorite songs in karaoke is an effective way for users to express themselves, convey emotions, and achieve social goals. Providing real-time feedback on users' performance can improve their experience and increase their engagement time. However, current popular online karaoke products (such as Changba and Quanmin K Ge) are prone to problems with real-time feedback, such as noise and the inability to dynamically adjust pitch based on user performance, resulting in poor robustness of the feedback. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] According to one aspect of this disclosure, a real-time singing evaluation method is provided, comprising: real-time acquisition of a user audio stream and removal of the accompaniment from the audio stream through echo cancellation processing; frame-by-frame determination of whether the audio stream after removing the accompaniment is noise, and if the current frame is not noise, extracting the user pitch of the current frame; adaptively adjusting the user pitch of the current frame according to the original singer's pitch line and the user's historical pitch information to obtain the adjusted user pitch; determining the accuracy of the pitch of the current frame by determining whether the difference between the adjusted user pitch and the original singer's pitch of the current frame is within a preset range; determining whether the current musical phrase has ended, and if the current musical phrase has not ended, displaying the user pitch value of the current frame in real time, and if the current musical phrase has ended, calculating the score of the current musical phrase based on the proportion of the cumulative accurate frame count in the current musical phrase to the total number of frames and displaying the score of the current musical phrase in real time; determining whether the song has ended, and if the song has not ended, continuing to acquire the user audio stream for the next musical phrase score calculation, and if the song has ended, obtaining the final score of the entire song by accumulating the scores of each musical phrase and displaying the final score of the entire song in real time.

[0005] According to some embodiments of this disclosure, the user pitch of the frame is adaptively adjusted based on the original vocal pitch line and the user's historical pitch information to obtain the adjusted user pitch, which includes: calculating the weighted average of the user's historical pitch information; calculating the weighted average of the original vocal pitch information; and multiplying the ratio of the user pitch of the current frame to the weighted average of the user's historical pitch information by the weighted average of the original vocal pitch information to obtain the adjusted user pitch.

[0006] According to some embodiments of this disclosure, calculating the weighted average of user historical pitch information includes: obtaining the weighted average of user historical pitch information of the current frame by adding the product of the user pitch of the current frame and a first weight to the product of the weighted average of user historical pitch of the previous frame and a second weight.

[0007] According to some embodiments of this disclosure, the weighted average of the original pitch information includes: adding the product of the original pitch of the current frame and a first weight to the product of the weighted average of the original pitch of the previous frame and a second weight to obtain the weighted average of the original pitch information of the current frame.

[0008] According to some embodiments of this disclosure, the real-time singing evaluation method further includes: writing the adjusted user pitch into the user's historical pitch information.

[0009] According to some embodiments of this disclosure, the real-time singing evaluation method further includes: extracting the original melody by separating the vocal accompaniment and aligning the lyrics to obtain the original pitch line.

[0010] According to some embodiments of this disclosure, the real-time singing evaluation method further includes: displaying the original singer's pitch and lyrics in real time, and recording the user's singing to obtain a user audio stream.

[0011] According to another aspect of this disclosure, a real-time singing evaluation system is provided, including a unit configured to perform a real-time singing evaluation method according to any embodiment of this disclosure.

[0012] According to some embodiments of the present disclosure, an electronic device is provided, including: a memory; and a processor coupled to the memory, the processor being configured to execute a real-time singing evaluation method of any embodiment of the present disclosure based on instructions stored in the memory.

[0013] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, performs the real-time singing evaluation method of any embodiment described in this disclosure.

[0014] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, performs the real-time singing evaluation method of any embodiment described in this disclosure.

[0015] Other features, aspects, and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0016] Preferred embodiments of the present disclosure are described below with reference to the accompanying drawings. The accompanying drawings, which are included to provide a further understanding of the present disclosure, and which, together with the following detailed description, are incorporated in and form a part of this specification and are used to explain the present disclosure. It should be understood that the drawings described below only relate to some embodiments of the present disclosure and are not intended to limit the present disclosure. In the drawings:

[0017] Figure 1 An example flowchart of a real-time singing evaluation method according to an exemplary embodiment of the present disclosure is shown.

[0018] Figure 2 A schematic block diagram of a real-time singing evaluation system according to an exemplary embodiment of the present disclosure is shown.

[0019] Figure 3 A schematic block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0020] Figure 4 A block diagram illustrating an example structure of a computer system that may be employed in an exemplary embodiment of the present disclosure is shown.

[0021] It should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not necessarily drawn to actual scale. The same or similar reference numerals are used in the various drawings to denote the same or similar parts. Therefore, once an item is defined in one drawing, it may not be discussed further in subsequent drawings. Detailed Implementation

[0022] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. However, it is obvious that the described embodiments are only some embodiments of this disclosure, and not all embodiments. The following description of the embodiments is merely illustrative and is in no way intended to limit this disclosure or its application or use. It should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein.

[0023] It should be understood that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect. Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of components and steps set forth in these embodiments should be interpreted as merely exemplary and do not limit the scope of this disclosure.

[0024] As used in this disclosure, the term "comprising" and its variations are open-ended terms that include at least the following elements / features but do not exclude other elements / features, i.e., "including but not limited to". Furthermore, as used in this disclosure, the term "including" and its variations are open-ended terms that include at least the following elements / features but do not exclude other elements / features, i.e., "including but not limited to". Therefore, "comprising" and "including" are synonymous. The term "based on" means "at least partially based on".

[0025] Throughout this specification, the terms "one embodiment," "some embodiments," or "embodiment" mean that a specific feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this disclosure. For example, the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; and the term "some embodiments" means "at least some embodiments." Furthermore, the appearance of the phrases "in one embodiment," "in some embodiments," or "in an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment, but may refer to the same embodiment.

[0026] It should be noted that the concepts of "first," "second," etc., used in this disclosure are used only to distinguish different devices, modules, or units, and are not intended to define the order of functions performed by these devices, modules, or units or their interdependencies. Unless otherwise specified, the concepts of "first," "second," etc., are not intended to imply that the objects described herein must be in a given temporal, spatial, rank, or any other given order.

[0027] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0028] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0029] The embodiments of this disclosure are described in detail below with reference to the accompanying drawings; however, this disclosure is not limited to these specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. Furthermore, in one or more embodiments, specific features, structures, or characteristics can be combined in any suitable manner that will be apparent to those skilled in the art from this disclosure.

[0030] Traditional singing scoring methods typically influence scores based on the absolute major and minor of pitch. However, from a music theory perspective, a performance only needs to conform to the relative intervals of the melody. In other words, the overall key changes in a song should not affect the score.

[0031] To address the overall pitch variations in songs, this invention proposes an adaptive real-time scoring method. Specifically, noise is detected before pitch extraction; if the current frame contains noise, it is not included in the subsequent scoring process. During the user's performance, adaptive adjustments are made based on the original singer's pitch and the user's historical pitch information, dynamically adjusting the user's pitch display to match the original singer's pitch. Furthermore, the state of each frame is counted, rather than directly including the pitch difference in the score, reducing errors caused by significant differences in individual pronunciations or inaccurate real-time pitch extraction.

[0032] The following is for reference. Figures 1 to 4 This paper will describe in detail the real-time singing evaluation method and system according to exemplary embodiments of the present invention.

[0033] Figure 1 An example flowchart 100 of a real-time singing evaluation method according to an exemplary embodiment of the present invention is shown.

[0034] According to an exemplary embodiment of the present invention, when singing in an online or offline karaoke room, it is necessary to extract the original melody, for example, through vocal accompaniment separation and lyric alignment, to obtain the original pitch line. Simultaneously, the original pitch line and lyrics are displayed in real time on the front-end screen, and the user's singing is recorded to obtain the audio stream of the user's performance.

[0035] like Figure 1 As shown, in step S101, the user's audio stream is pulled in real time, and the accompaniment of the audio stream is removed by echo cancellation (AEC) processing.

[0036] Those skilled in the art should understand that echo cancellation (AEC) refers to the simultaneous occupation of the line in both directions of two-wire transmission with the same frequency. The signals transmitted in both directions of the line are completely mixed together, and the echo of the signal transmitted at this end becomes the interference signal of the signal at this end. The echo is canceled by using an adaptive filter or by using deep learning technology to remove the echo in order to achieve better received signal quality.

[0037] In step S102, it is determined frame by frame whether the audio stream after removing the accompaniment is noise.

[0038] If the current frame is noisy, the user pitch is not extracted to reduce misjudgment of the user pitch. Therefore, we return to step S101 to continue pulling the user audio stream and determine frame by frame whether the audio stream after removing the accompaniment is noisy. If the current frame is not noisy, we proceed to step S103 to extract the user pitch of the current frame.

[0039] Those skilled in the art should understand that noise can be determined using conventional methods in the field, such as, but not limited to, determining whether the current frame is noise by detecting the energy level of the current frame.

[0040] In step S104, the user pitch of the current frame is adaptively adjusted based on the original vocal pitch line and the user's historical pitch information to obtain the adjusted user pitch.

[0041] According to an exemplary embodiment of the present invention, the user pitch of the frame is adaptively adjusted based on the original pitch line and the user's historical pitch information to obtain the adjusted user pitch. This includes calculating the weighted average of the user's historical pitch information, calculating the weighted average of the original pitch information, and multiplying the ratio of the user pitch of the current frame to the weighted average of the user's historical pitch information by the weighted average of the original pitch information to obtain the adjusted user pitch.

[0042] According to an embodiment of the present invention, by introducing the concept of weight, the user's historical pitch information is continuously updated. Therefore, theoretically, the user's historical pitch information contains information from all past times, but the weight decreases as time goes further back.

[0043] According to an exemplary embodiment of the present invention, the weighted average value of the user historical pitch information of the current frame is obtained by adding the product of the user pitch of the current frame and the first weight to the product of the weighted average value of the user historical pitch of the previous frame and the second weight.

[0044] According to a specific embodiment of the present invention, the user historical pitch information hm(t) of the current frame is given by: hm(t) = Q1 × x(t) + Q2 × hm(t-1). Here, hm(t) is the updated user historical pitch information of the current frame, i.e., the weighted average of the user historical pitch information of the current frame. hm(t-1) is the weighted average of the user historical pitch information of the previous frame. x(t) is the user pitch value of the current frame. Q1 and Q2 are the weights of the user pitch value of the current frame and the weighted average of the user historical pitch information of the previous frame, respectively, and Q1 + Q2 = 1. According to a preferred embodiment of the present invention, Q1 can be 0.1 and Q2 can be 0.9, but the present invention is not limited thereto, and those skilled in the art can set Q1 and Q2 as needed.

[0045] According to an embodiment of the present invention, the original pitch information is continuously updated by introducing the concept of weight. Therefore, theoretically, the original pitch information also contains information from all past times, but the weight decreases as time goes further back.

[0046] According to an exemplary embodiment of the present invention, the weighted average value of the original pitch information of the current frame is obtained by adding the product of the original pitch of the current frame and the first weight to the product of the weighted average value of the original pitch of the previous frame and the second weight.

[0047] According to a specific embodiment of the present invention, the original pitch information of the current frame is gm(t) = q1 × y(t) + q2 × gm(t-1). Where gm(t) is the original pitch information updated at the current frame time, i.e., the weighted average of the original pitch information of the current frame. gm(t-1) is the weighted average of the original pitch information of the previous frame. y(t) is the original pitch value of the current frame. q1 and q2 are the weights of the original pitch value of the current frame and the weighted average of the original pitch information of the previous frame, respectively, and q1 + q2 = 1. According to a preferred embodiment of the present invention, q1 can be 0.1 and q2 can be 0.9, but the present invention is not limited thereto, and those skilled in the art can set q1 and q2 as needed.

[0048] According to a specific embodiment of the present invention, the adaptively adjusted pitch xa(t) = x(t) / hm(t-1)×gm(t-1). Where xa(t) is the adaptively adjusted user pitch of the current frame, x(t) is the user pitch of the current frame, hm(t-1) is the weighted average of the user's historical pitch information of the previous frame, and gm(t-1) is the weighted average of the original vocal pitch information of the previous frame.

[0049] In step S105, the accuracy of the pitch of the current frame is determined by whether the difference between the adjusted user pitch and the original pitch of the current frame is within a preset range.

[0050] According to an embodiment of the present invention, a preset range for the difference can be set according to the difficulty level of different songs. If the difference between the adjusted user pitch and the original pitch of the current frame is within the preset range, the pitch of the current frame is determined to be accurate; otherwise, the pitch of the current frame is determined to be inaccurate.

[0051] According to an exemplary embodiment of the present invention, the real-time singing evaluation method further includes writing the adjusted user pitch into the user's historical pitch information.

[0052] At step S106, it is determined whether the current musical phrase has ended. If the current musical phrase has not ended, proceed to step S107, display the user's pitch value of the current frame in real time, and return to step S101 to continue fetching the user's audio stream, and determine frame by frame whether the audio stream after removing the accompaniment is noise. If the current musical phrase has ended, proceed to step S108, calculate the score of the current musical phrase based on the proportion of the cumulative accurate frame count in the current musical phrase to the total number of frames, and display the score of the current musical phrase in real time.

[0053] At step S109, it is determined whether the song has ended. If the song has not ended, the process returns to step S101, the user's audio stream is retrieved in real time, and the accompaniment in the audio stream is removed through echo cancellation processing before calculating the score for the next musical phrase. If the current musical phrase has ended, the process proceeds to step S110, where the final score of the entire song is obtained by accumulating the scores of each musical phrase, and the final score of the entire song is displayed in real time.

[0054] The real-time singing evaluation method proposed in this disclosure reduces misjudgments of user pitch by detecting noise before extracting pitch and preventing subsequent scoring processes if the current frame contains noise. When processing the user's audio stream, adaptive adjustments are made based on the original singer's pitch and the user's historical pitch information, dynamically adjusting the user's pitch display to match the original singer's pitch, effectively eliminating the negative impact of overall pitch shifts on the scoring. Furthermore, by counting the state of each frame instead of directly including pitch differences in the score, errors caused by significant differences in individual pronunciations or inaccurate real-time pitch extraction are reduced.

[0055] Figure 2 A schematic block diagram of a real-time singing evaluation system 200 according to an exemplary embodiment of the present disclosure is shown. The real-time singing evaluation system 200 according to the present invention can achieve... Figure 1 The real-time singing evaluation method shown is 100.

[0056] like Figure 2 As shown, the real-time singing evaluation system 20 includes an audio stream processing unit 201, configured to pull the user's audio stream in real time and remove the accompaniment from the audio stream through echo cancellation processing.

[0057] According to an exemplary embodiment of the present invention, when singing in an online or offline karaoke room, it is necessary to extract the original melody, for example, through vocal accompaniment separation and lyric alignment, to obtain the original pitch line. Simultaneously, the original pitch line and lyrics are displayed in real time on the front-end screen, and the user's singing is recorded to obtain the audio stream of the user's performance.

[0058] Those skilled in the art should understand that echo cancellation refers to the simultaneous occupation of the line in both directions of two-wire transmission with the same frequency and duration. The signals transmitted in both directions of the line are completely mixed together, and the echo of the signal transmitted at this end becomes the interference signal of the signal at this end. The echo is canceled by using an adaptive filter or by using deep learning technology to remove the echo in order to achieve better received signal quality.

[0059] like Figure 2 As shown, the real-time singing evaluation system 200 includes a noise judgment unit 202, which is configured to judge frame by frame whether the audio stream after removing the accompaniment is noise.

[0060] If the current frame is noisy, the user's pitch is not extracted to reduce misjudgment of the user's pitch. Therefore, the user's audio stream is continued to be pulled, and the audio stream after removing the accompaniment is judged frame by frame to determine whether it is noise.

[0061] like Figure 2 As shown, the real-time singing evaluation system 200 includes a user pitch extraction unit 203, configured to extract the user pitch of the current frame when the current frame is not noise.

[0062] Those skilled in the art will understand that noise can be determined using conventional methods in the field, including but not limited to determining whether the current frame is noise by detecting the energy level of the current frame. If the current frame is noise, the user's pitch is not extracted to reduce misjudgment of the user's pitch.

[0063] like Figure 2 As shown, the real-time singing evaluation system 200 includes an adaptive adjustment unit 204, which is configured to adaptively adjust the user pitch of the current frame based on the original pitch line and the user's historical pitch information to obtain the adjusted user pitch.

[0064] Refer to the above about Figure 1 According to an exemplary embodiment of the present invention, the adaptive adjustment unit 203 is configured to calculate the weighted average of the user's historical pitch information, calculate the weighted average of the original vocal pitch information, and multiply the ratio of the user's pitch in the current frame to the weighted average of the user's historical pitch information by the weighted average of the original vocal pitch information to obtain the adjusted user pitch.

[0065] According to an embodiment of the present invention, by introducing the concept of weight, the user's historical pitch information is continuously updated. Therefore, theoretically, the user's historical pitch information contains information from all past times, but the weight decreases as time goes further back.

[0066] According to an exemplary embodiment of the present invention, the weighted average value of the user historical pitch information of the current frame is obtained by adding the product of the user pitch of the current frame and the first weight to the product of the weighted average value of the user historical pitch of the previous frame and the second weight.

[0067] According to a specific embodiment of the present invention, the user historical pitch information hm(t) of the current frame is given by: hm(t) = Q1 × x(t) + Q2 × hm(t-1). Here, hm(t) is the updated user historical pitch information of the current frame, i.e., the weighted average of the user historical pitch information of the current frame. hm(t-1) is the weighted average of the user historical pitch information of the previous frame. x(t) is the user pitch value of the current frame. Q1 and Q2 are the weights of the user pitch value of the current frame and the weighted average of the user historical pitch information of the previous frame, respectively, and Q1 + Q2 = 1. According to a preferred embodiment of the present invention, Q1 can be 0.1 and Q2 can be 0.9, but the present invention is not limited thereto, and those skilled in the art can set Q1 and Q2 as needed.

[0068] According to an embodiment of the present invention, the original pitch information is continuously updated by introducing the concept of weight. Therefore, theoretically, the original pitch information also contains information from all past times, but the weight decreases as time goes further back.

[0069] According to an exemplary embodiment of the present invention, the weighted average value of the original pitch information of the current frame is obtained by adding the product of the original pitch of the current frame and the first weight to the product of the weighted average value of the original pitch of the previous frame and the second weight.

[0070] According to a specific embodiment of the present invention, the original pitch information of the current frame is gm(t) = q1 × y(t) + q2 × gm(t-1). Where gm(t) is the original pitch information updated at the current frame time, i.e., the weighted average of the original pitch information of the current frame. gm(t-1) is the weighted average of the original pitch information of the previous frame. y(t) is the original pitch value of the current frame. q1 and q2 are the weights of the original pitch value of the current frame and the weighted average of the original pitch information of the previous frame, respectively, and q1 + q2 = 1. According to a preferred embodiment of the present invention, q1 can be 0.1 and q2 can be 0.9, but the present invention is not limited thereto, and those skilled in the art can set q1 and q2 as needed.

[0071] According to a specific embodiment of the present invention, the adaptively adjusted pitch xa(t) = x(t) / hm(t-1)×gm(t-1). Where xa(t) is the adaptively adjusted user pitch of the current frame, x(t) is the user pitch of the current frame, hm(t-1) is the weighted average of the user's historical pitch information of the previous frame, and gm(t-1) is the weighted average of the original vocal pitch information of the previous frame.

[0072] like Figure 2 As shown, the real-time singing evaluation system 200 includes a pitch accuracy judgment unit 205, which is configured to judge the accuracy of the pitch of the current frame by determining whether the difference between the adjusted user pitch and the original pitch of the current frame is within a preset range.

[0073] According to an embodiment of the present invention, a preset range for the difference can be set according to the difficulty level of different songs. If the difference between the adjusted user pitch and the original pitch of the current frame is within the preset range, the pitch of the current frame is determined to be accurate; otherwise, the pitch of the current frame is determined to be inaccurate.

[0074] According to an exemplary embodiment of the present invention, the real-time singing evaluation system 200 further includes a pitch writing unit 206, configured to write the adjusted user pitch into the user's historical pitch information.

[0075] like Figure 2 As shown, the real-time singing evaluation system 200 includes a phrase end judgment unit 207, configured to judge whether the current phrase has ended.

[0076] like Figure 2 As shown, the real-time singing evaluation system 200 includes a display unit 208, which is configured to display the user's pitch value of the current frame in real time before the current musical phrase ends.

[0077] like Figure 2 As shown, the real-time singing evaluation system 200 includes a musical phrase score calculation unit 209, which is configured to calculate the score of the current musical phrase based on the proportion of the cumulative accurate frame count in the current musical phrase to the total number of frames when the current musical phrase has ended.

[0078] According to an embodiment of the present invention, the display unit 208 is further configured to display the score of the current musical phrase in real time when the current musical phrase has ended.

[0079] like Figure 2 As shown, the real-time singing evaluation system 200 includes a song end judgment unit 210, configured to judge whether the song has ended.

[0080] Those skilled in the art will understand that if the song end judgment unit 210 determines that the song has not ended, the audio stream processing unit 201 continues to pull the user's audio stream in real time and removes the accompaniment from the audio stream through echo cancellation processing. The units continue to collaborate to calculate the score for the next musical phrase.

[0081] like Figure 2 As shown, the real-time singing evaluation system 200 includes a song score calculation unit 211, which is configured to obtain the final score of the entire song by accumulating the scores of each musical phrase when the song end judgment unit 210 determines that the song has ended.

[0082] According to an embodiment of the present invention, the display unit 207 is also configured to display the final score of the entire song in real time when the song ends.

[0083] The real-time singing evaluation system provided in the embodiments of this disclosure can implement the real-time singing evaluation method provided in any of the embodiments herein.

[0084] This application proposes an adaptive real-time singing scoring method and system. By detecting noise before extracting pitch, and preventing subsequent scoring processes when the current frame contains noise, misjudgments of user pitch are reduced. When processing the user's audio stream, adaptive adjustments are made by referencing the original singer's pitch line and the user's historical pitch information, dynamically adjusting the user's pitch display to match the original singer's pitch line, essentially eliminating the negative impact of overall pitch shifts on scoring. Furthermore, by counting the state of each frame instead of directly including pitch differences in the score, errors caused by excessively large differences in individual pronunciations or inaccurate real-time pitch extraction are reduced. Therefore, the impact of overall pitch shifts on scoring is reduced, improving the user experience.

[0085] Some embodiments of this disclosure also provide an electronic device. Figure 3 Block diagrams of some embodiments of the electronic device 3 of this disclosure are shown. This electronic device can be used to implement the methods described according to any embodiment of this disclosure.

[0086] For example, in some embodiments, the electronic device 3 can be various types of devices, including but not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. For example, the electronic device 3 may include a display panel for displaying data and / or execution results utilized according to the scheme of this disclosure. For example, the display panel can be of various shapes, such as a rectangular panel, an elliptical panel, or a polygonal panel. Furthermore, the display panel can be not only a planar panel but also a curved panel, or even a spherical panel.

[0087] like Figure 3 As shown, the electronic device 3 of this embodiment includes a memory 31 and a processor 32 coupled to the memory 31. It should be noted that... Figure 3 The components of the electronic device 3 shown are merely exemplary and not limiting; the electronic device 3 may have other components as needed for actual applications. The processor 32 can control other components in the electronic device 3 to perform desired functions.

[0088] In some embodiments, memory 31 is used to store one or more computer-readable instructions. When processor 32 executes the computer-readable instructions, the computer-readable instructions are executed by processor 32 to implement the method according to any of the above embodiments. For specific implementations and related explanations of the various steps of the method, please refer to the above embodiments; repeated details will not be elaborated here.

[0089] For example, processor 32 and memory 31 can communicate with each other directly or indirectly. For example, processor 32 and memory 31 can communicate via a network. The network can include wireless networks, wired networks, and / or any combination of wireless and wired networks. Processor 32 and memory 31 can also communicate with each other via a system bus, which is not limited in this disclosure.

[0090] For example, processor 32 can be embodied in various suitable processors, processing devices, such as central processing unit (CPU), graphics processing unit (GPU), network processor (NP), etc.; it can also be digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The central processing unit (CPU) can be an x86 or ARM architecture, etc. For example, memory 31 can include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Memory 31 can include, for example, system memory, which stores, for example, the operating system, application programs, boot loader, database, and other programs. Various application programs and various data can also be stored in the storage medium.

[0091] Furthermore, according to some embodiments of this disclosure, various operations / processes according to this disclosure, when implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 4 The computer system 400 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including functions such as those described above. Figure 4 A block diagram of an example structure of a computer system that may be employed according to embodiments of the present disclosure is shown.

[0092] exist Figure 4In this configuration, the Central Processing Unit (CPU) 401 performs various processes based on a program stored in the Read-Only Memory (ROM) 402 or a program loaded from the Storage Section 408 into the Random Access Memory (RAM) 403. The RAM 403 also stores data required as needed when the CPU 401 performs various processes. The CPU is merely an example; it can also be other types of processors, such as the various processors described above. The ROM 402, RAM 403, and Storage Section 408 can be various forms of computer-readable storage media, as described below. It should be noted that although... Figure 4 The image shows ROM 402, RAM 403 and storage device 408, but one or more of them may be combined or located in the same or different memory or storage modules.

[0093] CPU 401, ROM 402 and RAM 403 are interconnected via bus 404. Input / output interface 405 is also connected to bus 404.

[0094] The following components are connected to the input / output interface 405: input section 406, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output section 407, including displays such as cathode ray tube (CRT), liquid crystal display (LCD), speakers, vibrators, etc.; storage section 408, including hard disks, magnetic tapes, etc.; and communication section 409, including network interface cards such as LAN cards, modems, etc. Communication section 409 allows communication processing via a network such as the Internet. It is easy to understand that, although... Figure 4 The various devices or modules in the computer system 400 shown communicate via bus 404, but they may also communicate via a network or other means, wherein the network may include a wireless network, a wired network, and / or any combination of wireless and wired networks.

[0095] As needed, drive 410 is also connected to input / output interface 405. Removable media 411, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 410 as needed, so that computer programs read from them can be installed into storage section 408 as needed.

[0096] When the above series of processes are implemented through software, the program constituting the software can be installed from a network such as the Internet or from a storage medium such as removable media 411.

[0097] According to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the CPU 401, it performs the functions defined in the methods of embodiments of this disclosure.

[0098] It should be noted that, in the context of this disclosure, a computer-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0099] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0100] In some embodiments, a computer program is also provided, comprising: instructions that, when executed by a processor, cause the processor to perform the method of any of the above embodiments. For example, the instructions may be embodied in computer program code.

[0101] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0103] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0104] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0105] The above description is merely an embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0106] Many specific details are set forth in the description provided herein. However, it is understood that embodiments of this disclosure may be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of the description.

[0107] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0108] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A real-time singing evaluation method, comprising: It pulls the user's audio stream in real time and removes the accompaniment from the audio stream through echo cancellation processing; Frame by frame, determine whether the audio stream after removing the accompaniment is noise, and if the current frame is not noise, extract the user pitch of the current frame; The user pitch in the current frame is adaptively adjusted based on the original vocal pitch and the user's historical pitch information to obtain the adjusted user pitch, including: Calculate the weighted average of the user's historical pitch information; Calculate the weighted average of the original vocal pitch information; The adjusted user pitch is obtained by multiplying the ratio of the current frame's user pitch to the weighted average of the user's historical pitch information by the weighted average of the original pitch information. The accuracy of the pitch of the current frame is determined by whether the difference between the adjusted user pitch and the original pitch of the current frame is within a preset range. Determine whether the current musical phrase has ended. If the current musical phrase has not ended, display the user's pitch value of the current frame in real time. If the current musical phrase has ended, calculate the score of the current musical phrase based on the proportion of the cumulative accurate frame count in the current musical phrase to the total number of frames and display the score of the current musical phrase in real time. If the song has not ended, the system continues to fetch the user's audio stream to calculate the score for the next musical phrase. If the song has ended, the system accumulates the scores of each musical phrase to obtain the final score of the entire song and displays the final score in real time.

2. The real-time singing evaluation method as described in claim 1, wherein calculating the weighted average of the user's historical pitch information includes: The weighted average of the user's historical pitch information in the current frame is obtained by adding the product of the user's pitch in the current frame and the first weight to the product of the weighted average of the user's historical pitch in the previous frame and the second weight.

3. The real-time singing evaluation method as described in claim 1, wherein calculating the weighted average of the original vocal pitch information includes: The weighted average of the original pitch information of the current frame is obtained by adding the product of the original pitch of the current frame and the first weight to the product of the weighted average of the original pitch of the previous frame and the second weight.

4. The real-time singing evaluation method as described in claim 1 further includes: Write the adjusted user pitch into the user's historical pitch information.

5. The real-time singing evaluation method as described in claim 1, further comprising: The original melody was extracted by separating the vocal accompaniment and aligning the lyrics to obtain the original pitch line.

6. The real-time singing evaluation method as described in claim 5 further includes: It displays the original singer's pitch and lyrics in real time, and records the user's singing to obtain the user's audio stream.

7. A real-time singing evaluation system, comprising a unit configured to perform the method as described in any one of claims 1-6.

8. An electronic device, comprising: Memory; and A processor coupled to the memory, the memory storing instructions that, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, the computer program implementing the method according to any one of claims 1-6 when executed by a processor.

Images