Method, device and storage medium for authenticating singing ability

Abstract

The application discloses a singing ability authentication method and device and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: receiving a to-be-evaluated audio sent by a client, wherein the to-be-evaluated audio is obtained by recording a host account singing a target song; determining a singing score of the host account by comparing differences between the to-be-evaluated audio and a reference audio in multiple dimensions, wherein the reference audio comprises at least one of original singing audio and instrument audio corresponding to the target song; and performing singing ability authentication on the host account according to the singing score. The application can improve the efficiency of singing ability authentication on the host account.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, device and storage medium for certifying singing ability. Background Technology

[0002] By certifying the singing abilities of streamers on live streaming or content platforms, it is possible to clearly identify the singing abilities of different streamers on the platform.

[0003] In related technologies, the process of certifying a streamer's singing ability requires the streamer to record and upload an audio or video recording of themselves singing a song. Then, platform staff or professional musicians act as judges, listening to the submitted audio or video and evaluating the streamer's singing ability based on their professional knowledge and subjective feelings, thus achieving singing ability certification.

[0004] The above method for certifying a streamer's singing ability is inefficient. Summary of the Invention

[0005] This application provides a method, device, and storage medium for certifying singing ability, which can improve the efficiency of certifying the singing ability of live streamer accounts. The technical solution is as follows: According to one aspect of this application, a method for certifying singing ability is provided, the method being applied to a server, the method comprising: Receive audio to be evaluated sent by the client, which is obtained by recording the target song sung by the broadcaster's account; The singing score of the broadcaster's account is determined by comparing the differences between the audio to be evaluated and the benchmark audio in multiple dimensions. The benchmark audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song. The singing ability of the streamer account is certified based on the singing score.

[0006] According to another aspect of this application, a method for certifying singing ability is provided, the method comprising: The singing ability certification interface is displayed, which shows multiple candidate songs; By recording the streamer's account singing the target song, an audio file is obtained to be evaluated. The target song is the song selected by the streamer's account from a plurality of candidate songs. The audio to be evaluated is sent to the server. The audio to be evaluated is used to compare the differences with the benchmark audio in multiple dimensions to determine the singing score of the broadcaster account. The singing score is used to certify the singing ability of the broadcaster account. The benchmark audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song.

[0007] According to another aspect of this application, a singing ability authentication device is provided, the device comprising: The receiving module is used to receive the audio to be evaluated sent by the client. The audio to be evaluated is obtained by recording the target song sung by the broadcaster's account. The determination module is used to determine the singing score of the broadcaster account by comparing the difference between the audio to be evaluated and the benchmark audio in multiple dimensions. The benchmark audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song. The authentication module is used to authenticate the singing ability of the streamer account based on the singing score.

[0008] In an optional design, the determining module is used for: The singing score of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio in at least two of the dimensions of pitch, rhythm, and stability.

[0009] In an optional design, the determining module is used for: The first score of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio in the pitch dimension. The second score of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio in the rhythm dimension. The third rating of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio under the stability dimension. The singing score of the streamer's account is determined based on the first score, the second score, and the third score.

[0010] In an optional design, the determining module is used for: Based on the audio to be evaluated and the benchmark audio, the first score of the broadcaster's account is determined through the first output branch of the singing ability model; The first output branch is used to extract the first fundamental frequency sequence of the audio to be evaluated and the second fundamental frequency sequence of the reference audio, and calculate the pitch difference between the first fundamental frequency sequence and the second fundamental frequency sequence to determine the first score of the broadcaster account. The singing ability model is constructed based on a deep neural network.

[0011] In an optional design, the determining module is used for: Based on the audio to be evaluated and the benchmark audio, the second score of the broadcaster's account is determined through the second output branch of the singing ability model; The second output branch is used to extract the first start and end points of the notes in the audio to be evaluated and the second start and end points of the notes in the reference audio, and to calculate the deviation between the first start and end points and the second start and end points to determine the second score of the broadcaster's account. The singing ability model is constructed based on a deep neural network.

[0012] In an optional design, the determining module is used for: Based on the audio to be evaluated and the benchmark audio, the third score of the broadcaster's account is determined through the third output branch of the singing ability model; The third output branch is used to extract the first fundamental frequency sequence and the first amplitude sequence of the audio to be evaluated, as well as the second fundamental frequency sequence and the second amplitude sequence of the reference audio, and to analyze the jitter of the first fundamental frequency sequence and the second fundamental frequency sequence and the tremor of the first amplitude sequence and the second amplitude sequence to determine the third score of the broadcaster account. The singing ability model is constructed based on a deep neural network.

[0013] In an optional design, the determining module is used for: The singing score of the streamer's account is determined by a weighted sum of the first score, the second score, and the third score.

[0014] In an optional design, the determining module is used for: The singing score of the broadcaster's account is determined by comparing the differences between the audio to be evaluated and the benchmark audio across multiple dimensions, and by analyzing at least one of the timbre and singing techniques corresponding to the audio to be evaluated.

[0015] In an optional design, the device further includes: The processing module is used to perform audio alignment between the audio to be evaluated and the reference audio.

[0016] In an optional design, the authentication module is used for: If the singing score is higher than the score threshold, an authentication label will be granted to the streamer's account; The authentication label is used to reflect the singing ability of the streamer account.

[0017] According to another aspect of this application, a singing ability authentication device is provided, the device comprising: The display module is used to display the singing ability certification interface, which displays multiple candidate songs; The recording module is used to record the host's account singing the target song to obtain the audio to be evaluated. The target song is the song selected by the host's account from the multiple candidate songs. The sending module is used to send the audio to be evaluated to the server. The audio to be evaluated is used to compare the differences with the benchmark audio in multiple dimensions to determine the singing score of the broadcaster account. The singing score is used to certify the singing ability of the broadcaster account. The benchmark audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song.

[0018] According to another aspect of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one program, the at least one program being loaded and executed by the processor to implement the singing ability certification method as described above.

[0019] According to another aspect of this application, a computer-readable storage medium is provided, wherein at least one program is stored therein, the at least one program being loaded and executed by a processor to implement the singing ability authentication method as described above.

[0020] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the singing ability authentication method provided in various alternative implementations of the above aspects.

[0021] The beneficial effects of the technical solution provided in this application include at least the following: By comparing the audio samples obtained from a streamer's performance of a target song with the baseline audio sample across multiple dimensions, the streamer's singing score is determined, thus certifying the streamer's singing ability. Since the entire certification process is automated and requires no human intervention, it significantly improves the efficiency of certifying a streamer's singing ability. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a structural block diagram of a computer system provided in an exemplary embodiment of this application; Figure 2This is a schematic diagram of a singing ability certification process provided in an exemplary embodiment of this application; Figure 3 This is a flowchart illustrating a singing ability authentication method provided in an exemplary embodiment of this application; Figure 4 This is a flowchart illustrating a singing ability authentication method provided in an exemplary embodiment of this application; Figure 5 This is a flowchart illustrating a singing ability authentication method provided in an exemplary embodiment of this application; Figure 6 This is a schematic diagram of a singing ability authentication interface provided in an exemplary embodiment of this application; Figure 7 This is a schematic diagram of a personal homepage provided in an exemplary embodiment of this application; Figure 8 This is a schematic diagram of a singer authentication process provided in an exemplary embodiment of this application; Figure 9 This is a schematic diagram of a hardware environment provided in an exemplary embodiment of this application; Figure 10 This is a schematic diagram of a singing ability certification process provided in an exemplary embodiment of this application; Figure 11 This is a schematic diagram of the structure of a singing ability authentication device provided in an exemplary embodiment of this application; Figure 12 This is a schematic diagram of the structure of a singing ability authentication device provided in an exemplary embodiment of this application; Figure 13 This is a schematic diagram of the structure of a singing ability authentication device provided in an exemplary embodiment of this application; Figure 14 This is a schematic diagram of the structure of a terminal provided in an exemplary embodiment of this application; Figure 15 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application.

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0026] Figure 1 This is a structural block diagram of a computer system provided in an exemplary embodiment of this application. The computer system 100 includes: a terminal 110 and a server 120.

[0027] Terminal 110 has an application 111 installed and running that supports audio recording. This application 111 includes any one of the following: music streaming applications, song streaming applications, singing applications, live streaming applications, social networking applications, office applications, game applications, food delivery applications, online shopping applications, video-on-demand applications, short video applications, financial applications, lifestyle service applications, navigation applications, medical applications, learning applications, and mini-programs. In some embodiments, application 111 is a client. Terminal 110 is the terminal used by user 112, and user account 112 can be logged into application 111. Terminal 110 can refer to one of multiple terminals. Optionally, the device type of terminal 110 includes at least one of the following: smartphone, tablet, smartwatch, e-book reader, MP3 player, MP4 player, laptop, and desktop computer.

[0028] Terminal 110 is connected to server 120 via wireless or wired network.

[0029] Server 120 includes at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. Server 120 provides background services for application 111 that supports audio recording. Optionally, server 120 undertakes the primary computing task, and terminal 110 undertakes the secondary computing task; or, server 120 undertakes the secondary computing task, and terminal 110 undertakes the primary computing task; or, server 120 and terminal 110 collaborate on computing using a distributed computing architecture.

[0030] In an illustrative example, server 120 includes a processor 121, a user account database 122, a service management module 123, and a user-facing input / output interface (I / O interface) 124. The processor 121 loads instructions stored in server 120 and processes data in the user account database 122 and service management module 123. The user account database 122 stores data about user accounts used by terminal 110 and other terminals, such as user avatars, nicknames, and group memberships. The service management module 123 provides services to functions in application 111. The user-facing I / O interface 124 establishes communication and exchanges data with terminal 110 via a wireless or wired network.

[0031] Based on the above description of the implementation environment involved in this application, the method provided in the embodiments of this application will be described below.

[0032] Figure 2 This is a schematic diagram illustrating a singing ability certification process provided in an exemplary embodiment of this application. Figure 2 As shown, when the singing ability certification is triggered by the broadcaster account 201, the client records the broadcaster account 201 singing the target song 202, obtains the audio to be evaluated 203, and sends the audio to be evaluated 203 to the server. The target song 202 is the song selected by the broadcaster account 201 from multiple candidate songs, including songs specifically designed for the singing ability certification process.

[0033] The server determines the singing score 206 of the broadcaster account 201 by comparing the differences between the audio to be evaluated 203 and the benchmark audio 204 across multiple dimensions 205. The singing score 206 reflects the singing ability of the broadcaster account 201. The benchmark audio 204 includes at least one of the original vocal audio and instrumental audio corresponding to the target song 202. In some embodiments, the server determines the singing score 206 of the broadcaster account 201 by comparing the differences between the audio to be evaluated 203 and the benchmark audio 204 across at least two dimensions: pitch, rhythm, and stability. For example, the server determines a first score for the broadcaster account 201 by comparing the differences between the audio to be evaluated 203 and the benchmark audio 204 in the pitch dimension. A second score is determined by comparing the differences between the audio to be evaluated 203 and the benchmark audio 204 in the rhythm dimension. A third score is determined by comparing the differences between the audio to be evaluated 203 and the benchmark audio 204 in the stability dimension. Then, based on the first, second, and third scores, the singing score 206 for broadcaster account 201 is determined. In addition to comparing the differences between the audio to be evaluated 203 and the benchmark audio 204 across multiple dimensions 205, the server can also combine other information to jointly determine the singing score 206 for broadcaster account 201. For example, the server can determine the singing score 206 for broadcaster account 201 by comparing the differences between the audio to be evaluated 203 and the benchmark audio 204 across multiple dimensions 205, and by analyzing at least one of the timbre and singing techniques corresponding to the audio to be evaluated 203.

[0034] After receiving a singing score of 206 from streamer account 201, the server verifies the singing ability of streamer account 201 based on this score. Optionally, if the singing score 206 of streamer account 201 is higher than a scoring threshold, the server will grant streamer account 201 a certification tag. This certification tag reflects the singing ability of streamer account 201. When streamer account 201 is granted a certification tag, the certification tag can be displayed in the user interface related to streamer account 201 on the client side.

[0035] By comparing the audio samples obtained from a streamer's performance of a target song with the baseline audio sample across multiple dimensions, the streamer's singing score is determined, thus certifying the streamer's singing ability. Since the entire certification process is automated and requires no human intervention, it significantly improves the efficiency of certifying a streamer's singing ability.

[0036] Figure 3 This is a flowchart illustrating a singing ability authentication method provided in an exemplary embodiment of this application. The method can be used with a computer device, such as... Figure 1 The server-side component shown on the shown server. For example... Figure 3 As shown, the method includes: Step 302: Receive the audio to be evaluated sent by the client.

[0037] The audio to be evaluated is used to certify the singing ability of a broadcaster's account. This audio is obtained by recording the broadcaster singing a target song, which can be a song chosen by the broadcaster. The singing ability certification of a broadcaster's account is used to assess or determine the level of singing ability of that account, that is, to assess or determine the singing ability of the broadcaster associated with that account.

[0038] In some embodiments, the process of authenticating the singing ability of a broadcaster's account is triggered by a client logged into that account. Correspondingly, the audio to be evaluated is also recorded by the client logged into the broadcaster's account. When the client logged into the broadcaster's account triggers the singing ability authentication process and records the audio to be evaluated, it will send the audio to the server.

[0039] The client application contains logged-in broadcaster accounts, which are user accounts used to provide or publish at least one of the following types of content: video, audio, images, and text. Corresponding to the broadcaster accounts are viewer accounts, which are user accounts used to watch the content provided or published by the broadcaster accounts.

[0040] Step 304: Determine the singing score of the broadcaster's account by comparing the differences between the audio to be evaluated and the benchmark audio in multiple dimensions.

[0041] The reference audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song. Optionally, the original vocal audio includes the audio corresponding to the original vocal track, which is the audio track corresponding to the target song that only contains the original vocals. The instrumental audio includes the audio corresponding to the instrumental track, which is the audio track corresponding to the target song that only contains instrumental sounds, such as a Musical Instrument Digital Interface (MIDI) track.

[0042] The singing score reflects the singing ability of a live streamer account, that is, the singing ability of the streamer corresponding to the account. For example, the higher the singing score, the higher the singing ability of the live streamer account; the lower the singing score, the lower the singing ability of the live streamer account. Optionally, the server determines the singing score of the live streamer account by comparing the difference between the audio to be evaluated and the benchmark audio in at least two of the dimensions of pitch, rhythm, and stability.

[0043] In some embodiments, in addition to comparing the differences between the audio to be evaluated and the benchmark audio across multiple dimensions, the server can also combine other information to jointly determine the singing score of the broadcaster's account. For example, the server determines the singing score of the broadcaster's account by comparing the differences between the audio to be evaluated and the benchmark audio across multiple dimensions, and by analyzing at least one of the timbre and singing techniques corresponding to the audio to be evaluated.

[0044] Step 306: Verify the singing ability of the streamer's account based on the singing score.

[0045] Singing ability certification is used to assess or determine the singing ability level of a streamer's account, that is, to evaluate or determine the singing ability level of the streamer associated with the account. By conducting singing ability certification on a streamer's account, the certification result can be obtained.

[0046] Optionally, the server determines the target range within multiple preset scoring ranges for the singing score, and sets the singing ability level corresponding to the target range as the singing ability level of the streamer account, thereby certifying the streamer account's singing ability. In other words, the result of certifying the streamer account's singing ability is its singing ability level, and different preset scoring ranges correspond to different singing ability levels.

[0047] Optionally, if a streamer's singing score exceeds a certain threshold, the server will grant the streamer a certification tag. This certification tag reflects the streamer's singing ability. In other words, the result of certifying the streamer's singing ability determines whether a certification tag is granted. In some embodiments, when a streamer is granted a certification tag, the tag can be displayed in the user interface related to the streamer's account on the client side. For example, the user interface displaying the streamer's account information may show the certification tag.

[0048] In summary, the method provided in this embodiment determines the singing score of the broadcaster's account by comparing the audio to be evaluated obtained from the broadcaster's account singing the target song with the corresponding baseline audio in multiple dimensions, thereby certifying the broadcaster's singing ability. Since the entire process of certifying the broadcaster's singing ability is automated and requires no manual intervention, it significantly improves the efficiency of certifying the broadcaster's singing ability.

[0049] Figure 4 This is a flowchart illustrating a singing ability authentication method provided in an exemplary embodiment of this application. The method can be used with a computer device, such as... Figure 1 The client on the terminal shown. (For example...) Figure 4 As shown, the method includes: Step 402: Display the singing ability certification interface.

[0050] The singing ability certification interface is the user interface in the client used to trigger the singing ability certification of the streamer's account. The singing ability certification interface displays multiple candidate songs, which are preset for the singing ability certification process. The singing ability certification of the streamer's account is used to evaluate or determine the level of the streamer's singing ability, that is, to evaluate or determine the level of the streamer's singing ability corresponding to the account.

[0051] The client application contains logged-in broadcaster accounts, which are user accounts used to provide or publish at least one of the following types of content: video, audio, images, and text. Corresponding to the broadcaster accounts are viewer accounts, which are user accounts used to watch the content provided or published by the broadcaster accounts.

[0052] Step 404: Obtain the audio to be evaluated by recording the streamer's account singing the target song.

[0053] The audio to be evaluated is used to verify the singing ability of a broadcaster's account. The target song is the song selected by the broadcaster's account from multiple candidate songs. When the singing ability verification of the broadcaster's account is triggered, the client will record the broadcaster's account singing the target song, resulting in the audio to be evaluated.

[0054] In some embodiments, the client triggers a recording of the streamer's account singing the target song based on a singing ability authentication operation triggered in the singing ability authentication interface. This singing ability authentication operation can be a selection operation for the target song among multiple candidate songs, or it can be an operation different from the selection operation; this application embodiment does not limit this.

[0055] Optionally, the client can record the streamer's account singing the target song using the microphone inside the computer device, or it can record the streamer's account singing the target song using an external microphone on the computer device. This application embodiment does not limit this.

[0056] Step 406: Send the audio to be evaluated to the server.

[0057] The audio to be evaluated is used to compare differences with a benchmark audio across multiple dimensions to determine the singer's account's singing score. The singing score is used to certify the singer's ability, and the benchmark audio includes at least one of the original vocal recording and instrumental recordings corresponding to the target song. The singing ability certification is used to assess or determine the singer's level of ability, that is, to assess or determine the singing ability of the singer corresponding to the account.

[0058] It should be noted that the process of determining the singing score of the streamer's account, and the process of certifying the singing ability of the streamer's account based on the singing score, can be referred to the relevant content in other embodiments of this application, and will not be repeated here.

[0059] In summary, the method provided in this embodiment determines the singing score of the broadcaster's account by comparing the audio to be evaluated obtained from the broadcaster's account singing the target song with the corresponding baseline audio in multiple dimensions, thereby certifying the broadcaster's singing ability. Since the entire process of certifying the broadcaster's singing ability is automated and requires no manual intervention, it significantly improves the efficiency of certifying the broadcaster's singing ability.

[0060] Figure 5 This is a flowchart illustrating a singing ability authentication method provided in an exemplary embodiment of this application. This method can be used for... Figure 1 The system shown. (As shown in the image) Figure 5 As shown, the method includes: Step 502: The client displays the singing ability certification interface.

[0061] The singing ability certification interface is the user interface in the client used to trigger the singing ability certification of the streamer's account. The singing ability certification interface displays multiple candidate songs, which are preset for the singing ability certification process. The singing ability certification of the streamer's account is used to evaluate or determine the level of the streamer's singing ability, that is, to evaluate or determine the level of the streamer's singing ability corresponding to the account.

[0062] For example, Figure 6 This is a schematic diagram of a singing ability authentication interface provided in an exemplary embodiment of this application. Figure 6 As shown, the singing ability certification interface 601 displays multiple candidate songs 602, each with its own song title and cover image.

[0063] Step 504: The client obtains the audio to be evaluated by recording the streamer's account singing the target song.

[0064] The audio to be evaluated is used to verify the singing ability of the broadcaster's account. The target song is the song selected by the broadcaster from multiple candidate songs displayed on the singing ability verification interface. When the singing ability verification of the broadcaster's account is triggered, the client will record the broadcaster's account singing the target song to obtain the audio to be evaluated.

[0065] Optionally, the audio to be evaluated is audio containing only human voices, i.e., pure human voice audio or a cappella audio. The client processes the audio recorded by the broadcaster's account singing the target song using a vocal-accompaniment separation algorithm to obtain the audio to be evaluated. Alternatively, the client records the broadcaster's account singing the target song a cappella to obtain the audio to be evaluated.

[0066] In some embodiments, the client triggers a recording of the streamer's account singing the target song based on a singing ability authentication operation triggered in the singing ability authentication interface. This singing ability authentication operation can be a selection operation for the target song among multiple candidate songs, or it can be an operation different from the selection operation; this application embodiment does not limit this.

[0067] For example, continue to refer to Figure 6 The singing ability certification interface 601 also displays a certification button 603. The streamer's account selects the target song from multiple candidate songs 602 (e.g., Figure 6 If the target song is sung by the streamer's account and the authentication button 603 is triggered, the client will record the streamer's account singing the target song, thus obtaining the audio to be evaluated.

[0068] Step 506: The client sends the audio to be evaluated to the server.

[0069] During the singing ability verification process for a live streamer's account, if the client obtains the audio to be evaluated, the client will send the audio to be evaluated to the server. Alternatively, the client may send the audio to be evaluated to the server upon request. This application embodiment does not impose any limitations on this.

[0070] Step 508: The server determines the singer's score by comparing the audio to be evaluated with the benchmark audio across multiple dimensions.

[0071] The reference audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song. Optionally, the original vocal audio includes the audio corresponding to the original vocal track, which is the audio track corresponding to the target song that only contains the original vocals. The instrumental audio includes the audio corresponding to the instrumental track, which is the audio track corresponding to the target song that only contains instrumental sounds, such as a MIDI track.

[0072] The singing score reflects the singing ability of the broadcaster's account, that is, the singing ability of the broadcaster corresponding to the account. In some embodiments, the greater the difference between the audio to be evaluated and the benchmark audio in different dimensions, the lower the singing score of the broadcaster's account; the smaller the difference between the audio to be evaluated and the benchmark audio in different dimensions, the higher the singing score of the broadcaster's account.

[0073] In some embodiments, the server determines the singer's performance score by comparing the audio to be evaluated with a reference audio in at least two of the following dimensions: pitch, rhythm, and stability. Optionally, the singer's performance score is negatively correlated with the difference in pitch between the audio to be evaluated and the reference audio, negatively correlated with the difference in rhythm between the audio to be evaluated and the reference audio, and negatively correlated with the difference in stability between the audio to be evaluated and the reference audio.

[0074] Optionally, in determining the singing score of the broadcaster's account, the server determines the broadcaster's account's first score by comparing the difference between the audio to be evaluated and the benchmark audio in the pitch dimension. The first score can be negatively correlated with the magnitude of the difference between the audio to be evaluated and the benchmark audio in the pitch dimension. The server determines the broadcaster's account's second score by comparing the difference between the audio to be evaluated and the benchmark audio in the rhythm dimension. The second score can be negatively correlated with the magnitude of the difference between the audio to be evaluated and the benchmark audio in the rhythm dimension. The server determines the broadcaster's account's third score by comparing the difference between the audio to be evaluated and the benchmark audio in the stability dimension. The third score can be negatively correlated with the magnitude of the difference between the audio to be evaluated and the benchmark audio in the stability dimension. The server then determines the broadcaster's account's singing score based on the first, second, and third scores.

[0075] In some embodiments, the server determines the first, second, and third scores of the broadcaster's account using a singing ability model. This singing ability model is an Artificial Intelligence (AI) model built upon a deep neural network. For example, it could be a deep neural network model based on multi-task learning, employing a Convolutional Neural Network (CNN) + Long Short-Term Memory (LSTM) structure. The singing ability model includes a first output branch, a second output branch, and a third output branch. The first output branch determines the broadcaster's account's first score, the second output branch determines the second score, and the third output branch determines the third score. The singing ability model is trained using sample audio, benchmark audio, and the first, second, and third sample scores. For example, it can be trained using the errors between the first sample score and the first predicted score, the second sample score and the second predicted score, and the third sample score and the third predicted score. The first predicted score is determined by the sample audio and the baseline audio through the first output branch; the second predicted score is determined by the sample audio and the baseline audio through the second output branch; and the third predicted score is determined by the sample audio and the baseline audio through the third output branch. The sample audio includes audio recordings of the target song performed by individuals other than the broadcaster associated with the broadcaster's account. The first, second, and third sample scores are manually set.

[0076] Regarding the first rating: The server determines the first score of the broadcaster's account based on the audio to be evaluated and the benchmark audio, using the first output branch of the singing ability model. The first output branch extracts the first fundamental frequency sequence of the audio to be evaluated and the second fundamental frequency sequence of the benchmark audio, and calculates the pitch difference between the two sequences to determine the broadcaster's first score. The fundamental frequency sequence is a series of fundamental frequency values ​​extracted from the audio that change over time, and can be used to quantify the pitch information of the audio. For example, the first output branch extracts the first fundamental frequency sequence of the audio to be evaluated and the second fundamental frequency sequence of the benchmark audio, and calculates the pitch difference of the corresponding parts of the same song content in the target song within the first and second fundamental frequency sequences to determine the broadcaster's first score. Optionally, the first score is negatively correlated with the magnitude of the pitch difference.

[0077] Regarding the second rating: The server determines the second score of the broadcaster's account based on the audio to be evaluated and the benchmark audio, using the second output branch of the singing ability model. The second output branch extracts the first start and end points of notes in the audio to be evaluated and the second start and end points of notes in the benchmark audio, and calculates the deviation between the first and second start and end points to determine the second score of the broadcaster's account. The start and end points of a note include the start and end times of the note. For example, the second output branch extracts the first energy envelope of the audio to be evaluated and the second energy envelope of the benchmark audio, determines the first start and end points of notes in the audio to be evaluated based on the first energy envelope, and determines the second start and end points of notes in the benchmark audio based on the second energy envelope. Then, it calculates the deviation between the first and second start and end points of the same note to determine the second score of the broadcaster's account. Optionally, the second score is negatively correlated with the magnitude of the aforementioned deviation.

[0078] Regarding the third rating: The server determines the third score of the broadcaster's account based on the audio to be evaluated and the benchmark audio, using the third output branch of the singing ability model. The third output branch extracts the first fundamental frequency sequence and the first amplitude sequence of the audio to be evaluated, as well as the second fundamental frequency sequence and the second amplitude sequence of the benchmark audio. It then analyzes the jitter of the first and second fundamental frequency sequences and the tremolo of the first and second amplitude sequences to determine the third score of the broadcaster's account. The jitter of the fundamental frequency sequence reflects the instability or fluctuation of pitch. The amplitude sequence is a series of amplitude (signal strength) values ​​extracted from the audio that change over time, reflecting the volume changes of the audio. The tremolo of the amplitude sequence reflects the fluctuation or instability of the volume (amplitude). For example, the third output branch determines the third score of the broadcaster's account by analyzing the jitter of the first fundamental frequency sequence compared to the second fundamental frequency sequence, and the tremolo of the first amplitude sequence compared to the second amplitude sequence. In some embodiments, the third output branch determines the third score of the broadcaster's account by analyzing the jitter of the first fundamental frequency sequence itself and the tremolo of the first amplitude sequence itself. For example, the more severe the shaking, the lower the third score; the more severe the tremor, the lower the third score.

[0079] After obtaining the first, second, and third scores, the server determines the streamer's performance score based on the sum or weighted sum of the first, second, and third scores. When determining the streamer's performance score based on the weighted sum of the first, second, and third scores, the weights corresponding to each score can be pre-set or learned by a fully connected layer trained through machine learning based on training data; this embodiment does not impose such limitations.

[0080] In some embodiments, before comparing the differences between the audio to be evaluated and the reference audio across multiple dimensions, the server performs audio alignment between the two audio sets. Audio alignment is used to align the portions of the same song content corresponding to the target song in the audio set to be evaluated and the reference audio in the temporal domain. Optionally, the server performs audio alignment between the audio set to be evaluated and the reference audio in the temporal domain using a Dynamic Time Warping (DTW) algorithm or an alignment model based on deep learning.

[0081] It should be noted that, in addition to comparing the differences between the audio to be evaluated and the benchmark audio across multiple dimensions, the server can also combine other information to jointly determine the singer's account's performance score. For example, the server can determine the singer's account's performance score by comparing the differences between the audio to be evaluated and the benchmark audio across multiple dimensions, and by analyzing at least one of the timbre and singing techniques corresponding to the audio to be evaluated. Optionally, the server can analyze the spectral characteristics of the audio to be evaluated, such as analyzing the Mel Frequency Cepstral Coefficients (MFCCs), to assess the quality of the timbre and whether singing techniques (e.g., the presence of vibrato) are used, thereby obtaining the singer's account's fourth score. Then, based on the sum or weighted sum of the first, second, third, and fourth scores, the singer's account's performance score is determined. The process of analyzing the timbre and singing techniques corresponding to the audio to be evaluated can be implemented through the fourth output branch of the singing ability model.

[0082] In some embodiments, when a viewer account performs an interactive operation on the target audio content, the server determines recommended audio content based on the target audio content and recommends the recommended audio content to the viewer account. The target audio content is the audio content published by a first broadcaster account, and the recommended audio content is the audio content published by a second broadcaster account. The difference between the first rating corresponding to the first broadcaster account and the first rating corresponding to the second broadcaster account is less than a first threshold, and the difference between the second rating corresponding to the first broadcaster account and the second rating corresponding to the second broadcaster account is less than a second threshold, and the difference between the third rating corresponding to the first broadcaster account and the third rating corresponding to the second broadcaster account is less than a third threshold. That is, the server first determines the first broadcaster account corresponding to the target audio content, then determines the second broadcaster account based on the above conditions, and then determines the recommended audio content from the audio content published by the second broadcaster account. The aforementioned interactive operation includes at least one of the following: commenting, liking, collecting, and forwarding.

[0083] Step 510: The server verifies the singing ability of the streamer's account based on the singing score.

[0084] Singing ability certification is used to assess or determine the singing ability level of a streamer's account, that is, to evaluate or determine the singing ability level of the streamer associated with the account. By conducting singing ability certification on a streamer's account, the certification result can be obtained.

[0085] Optionally, the server determines the target range within multiple preset scoring ranges for the singing score, and sets the singing ability level corresponding to the target range as the singing ability level of the streamer account, thereby certifying the streamer account's singing ability. In other words, the result of certifying the streamer account's singing ability is its singing ability level, and different preset scoring ranges correspond to different singing ability levels.

[0086] Optionally, if a streamer's singing score exceeds a certain threshold, the server will grant the streamer a certification tag. This certification tag reflects the streamer's singing ability. In other words, the result of certifying the streamer's singing ability determines whether a certification tag is granted. In some embodiments, when a streamer is granted a certification tag, the tag can be displayed in the user interface related to the streamer's account on the client side. For example, the user interface displaying the streamer's account information may show the certification tag.

[0087] For example, Figure 7 This is a schematic diagram of a personal homepage provided in an exemplary embodiment of this application. For example... Figure 7 As shown, on the streamer's account profile page 701, the streamer's nickname and avatar are displayed, as well as the cover image 702 of the content posted by the streamer. If the streamer's singing ability is verified, a verification tag 703 will be displayed on the streamer's account profile page 701.

[0088] In some embodiments, the server recommends content posted by the streamer account to user accounts other than the streamer account based on the streamer account's singing ability certification result. For example, the higher the singing ability certification result indicates that the streamer account has a higher level of singing ability, the more likely the singing content posted by the streamer account is to be recommended to user accounts other than the streamer account; conversely, the lower the singing ability certification result indicates that the streamer account has a lower level of singing ability, the less likely the singing content posted by the streamer account is to be recommended to user accounts other than the streamer account.

[0089] In summary, the method provided in this embodiment determines the singing score of the broadcaster's account by comparing the audio to be evaluated obtained from the broadcaster's account singing the target song with the corresponding baseline audio in multiple dimensions, thereby certifying the broadcaster's singing ability. Since the entire process of certifying the broadcaster's singing ability is automated and requires no manual intervention, it significantly improves the efficiency of certifying the broadcaster's singing ability.

[0090] The method provided in this embodiment also determines the singing score by comparing the differences in at least two of the three dimensions of pitch, rhythm, and stability, constructing a multi-dimensional evaluation system. This overcomes the one-sidedness of single-dimensional evaluation and significantly improves the comprehensiveness and objectivity of the score. By calculating the first, second, and third scores separately and then synthesizing them to obtain the singing score, the interpretability of the evaluation is achieved, facilitating the identification of singing weaknesses. Determining the first score through the first output branch helps improve the accuracy and robustness of pitch evaluation. Determining the second score through the second output branch enables accurate evaluation of rhythm. Determining the third score through the third output branch enables objective quantification of singing abilities such as breath stability. Determining the singing score based on the weighted sum of the first, second, and third scores avoids a single indicator dominating the score. Combining the analysis of timbre and singing technique to determine the singing score expands the evaluation dimensions. By aligning the audio to be evaluated with the reference audio, the interference of differences in singing speed and start delay can be eliminated. By assigning certification labels, the user experience can be improved through visual labels.

[0091] The method provided in this application embodiment is used to perform multi-dimensional analysis and scoring of a streamer's singing ability, replacing traditional manual review. It can be applied to products that require certification or rating of users' singing abilities, such as live streaming platforms, online karaoke applications, and short video platforms. Taking the method provided in this application embodiment for singer certification as an example... Figure 8 This is a schematic diagram illustrating a singer authentication process provided in an exemplary embodiment of this application. For example... Figure 8 As shown, the method provided in this application mainly includes the following functions: 1. Automated authentication entry: The streamer can find the "Singer Authentication" entry in the application (APP) and initiate authentication at any time without prior appointment. 2. Standardized authentication process: The system provides a library of songs with different styles and difficulties. The streamer selects one to sing. During the performance, the interface provides accompaniment and scrolling lyrics, and records the streamer's voice in real time. 3. Instant feedback report: After the performance, the system generates a detailed analysis report within seconds. The report includes not only the total score and rating (such as S / A / B / C or "Pass / Fail"), but also scores and evaluations for multiple dimensions such as pitch, rhythm, and stability. It can also highlight sections with flaws in the performance. 4. Authentication badge: Streamers who reach the specified score or rating will automatically receive the "Certified Singer" badge on their personal homepage and live stream.

[0092] The hardware environment involved in the method provided in the embodiments of this application is described below. Figure 9 This is a schematic diagram of a hardware environment provided in an exemplary embodiment of this application. For example... Figure 9As shown, the hardware environment is divided into a front-end (client) 901 and a back-end (server cluster) 902. The front-end 901 runs on the broadcaster's smartphone, personal computer (PC), and other terminal devices. It is responsible for providing the user interface (UI) interaction, playing accompaniment, recording the audio to be evaluated, and uploading the audio data to the server. The back-end 902 includes an application server, an AI computing server, and a database. The application server is responsible for handling business logic, such as user requests, task distribution, and storing authentication results. The AI ​​computing server deploys the core singing ability model and is equipped with accelerated hardware such as a graphics processing unit (GPU) to receive audio data and perform complex analysis calculations. The database is used to store user information, a song library, benchmark audio data, authentication results, and analysis reports.

[0093] For example, Figure 10 This is a schematic diagram illustrating a singing ability certification process provided in an exemplary embodiment of this application. Figure 10As shown, in step S1, data preparation and preprocessing are performed. The client records the host's voice while they sing, forming the audio to be evaluated. To eliminate accompaniment interference, a voice / accompaniment separation algorithm can be used, or the host can be recorded a cappella. The client uploads the audio to be evaluated to the application server, which then sends it along with the corresponding benchmark audio to the AI ​​computing server. In step S2, audio alignment is performed. Since the host's singing speed may differ from the original recording, alignment is necessary. The AI ​​server uses the DTW algorithm or a deep learning-based alignment model to precisely align the audio to be evaluated with the benchmark audio in the time domain, ensuring that subsequent comparisons use the same word or note. Besides DTW, a neural network model based on an attention mechanism can be used for soft alignment, which may be more robust when dealing with nonlinear scaling. In step S3, multi-dimensional feature extraction is performed. The AI ​​server inputs the aligned audio to be evaluated and the benchmark audio into the singing ability model. This model is a multi-task learning deep neural network (such as a CNN+LSTM structure) containing multiple parallel output branches, each used to extract and quantize different singing dimensions. Besides CNN+LSTM, a Transformer-based model architecture can also be used, which may be more advantageous in capturing long-term audio dependencies. Pitch Analysis Branch: Extracts the fundamental frequency sequence of two audio streams and calculates the pitch difference (in cents) at corresponding time points to obtain a pitch score (first score). This branch can identify problems such as out-of-tune, insufficient pitch, or excessive pitch. Pitch extraction is not limited to fundamental frequency estimation algorithms; end-to-end deep learning models can also be used to directly predict pitch from the waveform. Rhythm Analysis Branch: Extracts the energy envelope and note start and end points of the audio. By comparing the deviation of the audio to be evaluated from the reference audio in note start and end times, a rhythm score (second score) is obtained. This branch can identify problems such as rushing or dragging. Stability Analysis Branch: Analyzes the jitter of the pitch sequence and the tremolo of the amplitude sequence to evaluate the broadcaster's breath control and vocal smoothness when singing long notes, obtaining a stability score (third score). (Optional) Timbre / Technique Analysis Branch: By analyzing spectral features (such as MFCC), the quality of timbre can be evaluated, or the use of advanced techniques such as vibrato can be identified. In step S4, a comprehensive score is generated. The quantified scores from multiple dimensions obtained in step S3 are input into a pre-defined weighted layer or a fully connected layer trained through machine learning. The scores are weighted and summed according to the importance of each dimension to calculate the final comprehensive score (performance score). Besides using fixed weights or a simple fully connected layer, a more complex model can be designed; for example, a judge preference model can be introduced to simulate judges with different style preferences, providing more diverse evaluations. Based on the comprehensive score, it is mapped to a pre-defined rating system (e.g., >90 points is grade S, 80-90 points is grade A), generating the final certification result and analysis report. In step S5, the results are returned and stored.The AI ​​computing server returns data including the overall score, scores for each dimension, and a rating report to the application server. The application server stores the results in a database and sends them to the client, which then parses the data and displays it to the broadcaster in a visually appealing format.

[0094] Furthermore, for terminals with good network conditions and strong computing power, some lightweight models can be deployed on the client side to enable real-time scoring and feedback for the broadcaster during the performance. The aforementioned model can also be extended to duet scenarios, achieving more complex effects by evaluating each individual's performance and the degree of coordination between the two (such as harmonic harmony and rhythmic synchronization). Moreover, the core ideas of the above evaluation framework can also be applied to other talent certifications, such as instrumental performance (compared to standard sheet music) and dance (compared to standard motion capture data).

[0095] The method provided in this application has the following beneficial effects: 1. Objectivity and Standardization: By constructing a multi-dimensional, quantifiable AI evaluation model for singing ability. This model is based on unified algorithms and data standards for evaluation, eliminating the subjectivity and inconsistency of manual review, and ensuring that all broadcasters receive fair and objective evaluations under the same criteria.

[0096] 2. Improved Processing Efficiency: The AI-powered authentication process can handle hundreds or even thousands of authentication requests in parallel, reducing the manual review process, which previously took hours or even days, to within seconds. This high-concurrency processing capability solves the efficiency bottleneck of manual review.

[0097] 3. Provides precise technical feedback: This solution not only provides an overall score but also offers quantitative analysis across multiple dimensions, including pitch and rhythm, and can pinpoint specific problematic musical phrases. This refined technical feedback is unavailable through manual review and can effectively guide broadcasters in targeted training, improving their singing skills and ultimately enhancing the overall content quality.

[0098] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user data. These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without user confirmation), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of related user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0099] It should be noted that the order of the method steps provided in the embodiments of this application can be appropriately adjusted, and the steps can also be added or removed as appropriate. Any method variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further.

[0100] Figure 11 This is a schematic diagram of the structure of a singing ability authentication device provided in an exemplary embodiment of this application. Figure 11 As shown, the device includes: The receiving module 1101 is used to receive the audio to be evaluated sent by the client, which is obtained by recording the target song sung by the anchor account; The determination module 1102 is used to determine the singing score of the broadcaster account by comparing the difference between the audio to be evaluated and the benchmark audio in multiple dimensions. The benchmark audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song. The authentication module 1103 is used to authenticate the singing ability of the live streamer account based on the singing score.

[0101] In an optional design, the determining module 1102 is used for: The singing score of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio in at least two of the dimensions of pitch, rhythm, and stability.

[0102] In an optional design, the determining module 1102 is used for: The first score of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio in the pitch dimension. The second score of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio in the rhythm dimension. The third rating of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio under the stability dimension. The singing score of the streamer's account is determined based on the first score, the second score, and the third score.

[0103] In an optional design, the determining module 1102 is used for: Based on the audio to be evaluated and the benchmark audio, the first score of the broadcaster's account is determined through the first output branch of the singing ability model; The first output branch is used to extract the first fundamental frequency sequence of the audio to be evaluated and the second fundamental frequency sequence of the reference audio, and calculate the pitch difference between the first fundamental frequency sequence and the second fundamental frequency sequence to determine the first score of the broadcaster account. The singing ability model is constructed based on a deep neural network.

[0104] In an optional design, the determining module 1102 is used for: Based on the audio to be evaluated and the benchmark audio, the second score of the broadcaster's account is determined through the second output branch of the singing ability model; The second output branch is used to extract the first start and end points of the notes in the audio to be evaluated and the second start and end points of the notes in the reference audio, and to calculate the deviation between the first start and end points and the second start and end points to determine the second score of the broadcaster's account. The singing ability model is constructed based on a deep neural network.

[0105] In an optional design, the determining module 1102 is used for: Based on the audio to be evaluated and the benchmark audio, the third score of the broadcaster's account is determined through the third output branch of the singing ability model; The third output branch is used to extract the first fundamental frequency sequence and the first amplitude sequence of the audio to be evaluated, as well as the second fundamental frequency sequence and the second amplitude sequence of the reference audio, and to analyze the jitter of the first fundamental frequency sequence and the second fundamental frequency sequence and the tremor of the first amplitude sequence and the second amplitude sequence to determine the third score of the broadcaster account. The singing ability model is constructed based on a deep neural network.

[0106] In an optional design, the determining module 1102 is used for: The singing score of the streamer's account is determined by a weighted sum of the first score, the second score, and the third score.

[0107] In an optional design, the determining module 1102 is used for: The singing score of the broadcaster's account is determined by comparing the differences between the audio to be evaluated and the benchmark audio across multiple dimensions, and by analyzing at least one of the timbre and singing techniques corresponding to the audio to be evaluated.

[0108] In an optional design, such as Figure 12 As shown, the device further includes: Processing module 1104 is used to perform audio alignment between the audio to be evaluated and the reference audio.

[0109] In an optional design, the authentication module 1103 is used for: If the singing score is higher than the score threshold, an authentication label will be granted to the streamer's account; The authentication label is used to reflect the singing ability of the streamer account.

[0110] Figure 13 This is a schematic diagram of the structure of a singing ability authentication device provided in an exemplary embodiment of this application. Figure 13 As shown, the device includes: Display module 1301 is used to display a singing ability certification interface, which displays multiple candidate songs; The recording module 1302 is used to record the host account singing the target song to obtain the audio to be evaluated, wherein the target song is the song selected by the host account from the plurality of candidate songs; The sending module 1303 is used to send the audio to be evaluated to the server. The audio to be evaluated is used to compare the differences with the benchmark audio in multiple dimensions to determine the singing score of the broadcaster account. The singing score is used to certify the singing ability of the broadcaster account. The benchmark audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song.

[0111] It should be noted that the singing ability certification device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the singing ability certification device and the singing ability certification method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0112] Embodiments of this application also provide a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the singing ability authentication method provided in the above-described method embodiments.

[0113] In some embodiments, the computer device described above is a terminal. For example, Figure 14 This is a schematic diagram of the structure of a terminal provided in an exemplary embodiment of this application.

[0114] Typically, terminal 1400 includes a processor 1401 and a memory 1402.

[0115] Processor 1401 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0116] Memory 1402 may include one or more computer-readable storage media, which may be non-transitory. Memory 1402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 1402 is used to store at least one instruction, which is executed by processor 1401 to implement the singing ability authentication method provided in the method embodiments of this application.

[0117] In some embodiments, the terminal 1400 may also optionally include a peripheral device interface 1403 and at least one peripheral device. The processor 1401, memory 1402, and peripheral device interface 1403 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1403 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 1404, a display screen 1405, a camera assembly 1406, an audio circuit 1407, and a power supply 1408.

[0118] Peripheral device interface 1403 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1401 and memory 1402. In some embodiments, processor 1401, memory 1402 and peripheral device interface 1403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1401, memory 1402 and peripheral device interface 1403 can be implemented on separate chips or circuit boards, and this application embodiment is not limited in this respect.

[0119] The radio frequency (RF) circuit 1404 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1404 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1404 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1404 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1404 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0120] Display screen 1405 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1405 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1401 for processing. In this case, display screen 1405 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1405, which serves as the front panel of terminal 1400; in other embodiments, there may be at least two display screens 1405, respectively disposed on different surfaces of terminal 1400 or in a folded design; in still other embodiments, display screen 1405 may be a flexible display screen, disposed on a curved or folded surface of terminal 1400. Furthermore, display screen 1405 may also be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1405 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0121] The camera assembly 1406 is used to acquire images or videos. Optionally, the camera assembly 1406 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal 1400, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1406 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0122] The audio circuit 1407 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1401 for processing, or input to the radio frequency circuit 1404 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 1400. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1401 or the radio frequency circuit 1404 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1407 may also include a headphone jack.

[0123] Power supply 1408 is used to power the various components in terminal 1400. Power supply 1408 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1408 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0124] In some embodiments, the terminal 1400 further includes one or more sensors 1409. The one or more sensors 1409 include, but are not limited to: an accelerometer 1410, a gyroscope 1411, a pressure sensor 1412, an optical sensor 1413, and a proximity sensor 1414.

[0125] Accelerometer 1410 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal 1400. For example, accelerometer 1410 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1401 can control touchscreen 1405 to display the user interface in landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1410. Accelerometer 1410 can also be used for games or for acquiring user motion data.

[0126] The gyroscope sensor 1411 can detect the orientation and rotation angle of the terminal 1400. The gyroscope sensor 1411 can work in conjunction with the accelerometer sensor 1410 to acquire the user's 3D movements on the terminal 1400. Based on the data acquired by the gyroscope sensor 1411, the processor 1401 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0127] The pressure sensor 1412 can be disposed on the side bezel of the terminal 1400 and / or on the lower layer of the touch display screen 1405. When the pressure sensor 1412 is disposed on the side bezel of the terminal 1400, it can detect the user's grip signal on the terminal 1400, and the processor 1401 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1412. When the pressure sensor 1412 is disposed on the lower layer of the touch display screen 1405, the processor 1401 can control the operable controls on the UI interface based on the user's pressure operation on the touch display screen 1405. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0128] Optical sensor 1413 is used to collect ambient light intensity. In one embodiment, processor 1401 can control the display brightness of touch display screen 1405 based on the ambient light intensity collected by optical sensor 1413. Specifically, when the ambient light intensity is high, the display brightness of touch display screen 1405 is increased; when the ambient light intensity is low, the display brightness of touch display screen 1405 is decreased. In another embodiment, processor 1401 can also dynamically adjust the shooting parameters of camera assembly 1406 based on the ambient light intensity collected by optical sensor 1413.

[0129] The proximity sensor 1414, also known as a distance sensor, is typically located on the front panel of the terminal 1400. The proximity sensor 1414 is used to detect the distance between the user and the front of the terminal 1400. In one embodiment, when the proximity sensor 1414 detects that the distance between the user and the front of the terminal 1400 is gradually decreasing, the processor 1401 controls the touchscreen display 1405 to switch from a screen-on state to a screen-off state; when the proximity sensor 1414 detects that the distance between the user and the front of the terminal 1400 is gradually increasing, the processor 1401 controls the touchscreen display 1405 to switch from a screen-off state to a screen-on state.

[0130] Those skilled in the art will understand that Figure 14 The structure shown does not constitute a limitation on terminal 1400 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0131] In some embodiments, the computer device described above is a server. For example, Figure 15 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application, which can be implemented as a server.

[0132] The computer device 1500 includes a central processing unit (CPU) 1501, a system memory 1504 including random access memory (RAM) 1502 and read-only memory (ROM) 1503, and a system bus 1505 connecting the system memory 1504 and the CPU 1501. The computer device 1500 also includes a basic input / output system (I / O system) 1506 to facilitate information transfer between various components within the computer device, and a mass storage device 1507 for storing the operating system 1513, application programs 1514, and other program modules 1515.

[0133] The basic input / output system 1506 includes a display 1508 for displaying information and an input device 1509 for user input, such as a mouse or keyboard. Both the display 1508 and the input device 1509 are connected to the central processing unit 1501 via an input / output controller 1510 connected to the system bus 1505. The basic input / output system 1506 may also include the input / output controller 1510 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1510 also provides output to a display screen, printer, or other types of output devices.

[0134] The mass storage device 1507 is connected to the central processing unit 1501 via a mass storage controller (not shown) connected to the system bus 1505. The mass storage device 1507 and its associated computer-readable storage media provide non-volatile storage for the computer device 1500. That is, the mass storage device 1507 may include computer-readable storage media (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0135] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable storage instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage devices, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 1504 and mass storage device 1507 described above can be collectively referred to as memory.

[0136] The memory stores one or more programs, which are configured to be executed by one or more central processing units 1501. The one or more programs contain instructions for implementing the above method embodiments, and the central processing unit 1501 executes the one or more programs to implement the methods provided by the various method embodiments described above.

[0137] According to various embodiments of this application, the computer device 1500 can also be connected to a remote computer device on a network, such as the Internet. That is, the computer device 1500 can be connected to a network 1512 via a network interface unit 1511 connected to the system bus 1505, or the network interface unit 1511 can be used to connect to other types of networks or remote computer device systems (not shown).

[0138] The memory further includes one or more programs stored in the memory, and the one or more programs include steps performed by a computer device in the methods provided in the embodiments of this application.

[0139] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set. When the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor of a computer device, the singing ability authentication method provided in the above-described method embodiments is implemented.

[0140] This application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the singing ability authentication method provided in the above-described method embodiments.

[0141] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0142] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent switching, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for certifying singing ability, characterized in that, The method is applied to the server side, and the method includes: Receive audio to be evaluated sent by the client, which is obtained by recording the target song sung by the broadcaster's account; The singing score of the broadcaster's account is determined by comparing the differences between the audio to be evaluated and the benchmark audio in multiple dimensions. The benchmark audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song. The singing ability of the streamer's account is certified based on the singing score.

2. The method according to claim 1, characterized in that, The process of determining the singing score of the broadcaster's account by comparing the audio to be evaluated with the benchmark audio across multiple dimensions includes: The singing score of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio in at least two of the dimensions of pitch, rhythm, and stability.

3. The method according to claim 2, characterized in that, The process of determining the singing score of the broadcaster's account by comparing the difference between the audio to be evaluated and the benchmark audio in at least two dimensions of pitch, rhythm, and stability includes: The first score of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio in the pitch dimension. The second score of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio in the rhythm dimension. The third rating of the broadcaster's account is determined by comparing the difference between the audio to be evaluated and the benchmark audio under the stability dimension. The singing score of the streamer's account is determined based on the first score, the second score, and the third score.

4. The method according to claim 3, characterized in that, The process of determining the first score of the broadcaster's account by comparing the difference between the audio to be evaluated and the benchmark audio in the pitch dimension includes: Based on the audio to be evaluated and the benchmark audio, the first score of the broadcaster's account is determined through the first output branch of the singing ability model; The first output branch is used to extract the first fundamental frequency sequence of the audio to be evaluated and the second fundamental frequency sequence of the reference audio, and calculate the pitch difference between the first fundamental frequency sequence and the second fundamental frequency sequence to determine the first score of the broadcaster account. The singing ability model is constructed based on a deep neural network.

5. The method according to claim 3, characterized in that, The step of determining the second score of the broadcaster's account by comparing the difference between the audio to be evaluated and the benchmark audio in the rhythm dimension includes: Based on the audio to be evaluated and the benchmark audio, the second score of the broadcaster's account is determined through the second output branch of the singing ability model; The second output branch is used to extract the first start and end points of the notes in the audio to be evaluated and the second start and end points of the notes in the reference audio, and to calculate the deviation between the first start and end points and the second start and end points to determine the second score of the broadcaster's account. The singing ability model is constructed based on a deep neural network.

6. The method according to claim 3, characterized in that, The process of determining the third rating of the broadcaster's account by comparing the difference between the audio to be evaluated and the benchmark audio under the stability dimension includes: Based on the audio to be evaluated and the benchmark audio, the third score of the broadcaster's account is determined through the third output branch of the singing ability model; The third output branch is used to extract the first fundamental frequency sequence and the first amplitude sequence of the audio to be evaluated, as well as the second fundamental frequency sequence and the second amplitude sequence of the reference audio, and to analyze the jitter of the first fundamental frequency sequence and the second fundamental frequency sequence and the tremor of the first amplitude sequence and the second amplitude sequence to determine the third score of the broadcaster account. The singing ability model is constructed based on a deep neural network.

7. The method according to claim 3, characterized in that, Determining the singing score of the streamer's account based on the first score, the second score, and the third score includes: The singing score of the streamer's account is determined by a weighted sum of the first score, the second score, and the third score.

8. The method according to any one of claims 1 to 7, characterized in that, The process of determining the singing score of the broadcaster's account by comparing the audio to be evaluated with the benchmark audio across multiple dimensions includes: The singing score of the broadcaster's account is determined by comparing the differences between the audio to be evaluated and the benchmark audio across multiple dimensions, and by analyzing at least one of the timbre and singing techniques corresponding to the audio to be evaluated.

9. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The audio to be evaluated is aligned with the reference audio.

10. The method according to any one of claims 1 to 7, characterized in that, The process of certifying the singing ability of the streamer account based on the singing score includes: If the singing score is higher than the score threshold, an authentication label will be granted to the streamer's account; The certification label is used to reflect the singing ability of the streamer account.

11. A method for certifying singing ability, characterized in that, The method is applied to a client, and the method includes: The singing ability certification interface is displayed, which shows multiple candidate songs; By recording the streamer's account singing the target song, an audio file is obtained to be evaluated. The target song is the song selected by the streamer's account from a plurality of candidate songs. The audio to be evaluated is sent to the server. The audio to be evaluated is used to compare the differences with the benchmark audio in multiple dimensions to determine the singing score of the broadcaster account. The singing score is used to certify the singing ability of the broadcaster account. The benchmark audio includes at least one of the original vocal audio and instrumental audio corresponding to the target song.

12. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the singing ability certification method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one program, which is loaded and executed by a processor to implement the singing ability authentication method as described in any one of claims 1 to 11.

14. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the computer device to perform the singing ability certification method as described in any one of claims 1 to 11.

Images