An audio evaluation method and apparatus

By grading audio information in multiple dimensions, the problem of low audio evaluation accuracy in the existing technology is solved, and the evaluation accuracy and user experience are improved. It is especially suitable for audio scoring systems such as karaoke and recitation.

CN115394317BActive Publication Date: 2024-11-12GUANGZHOU QUWAN NETWORK TECH CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202211020857.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-24
Publication Date
2024-11-12
Estimated Expiration
2042-08-24

AI Technical Summary

Technical Problem

The existing audio evaluation methods only consider a single dimension, resulting in low evaluation accuracy and affecting users' innovative gameplay experience in the audio scoring system.

Method used

By obtaining the original audio information input by the user and the preset reference audio information, the content score, tone score, rhythm score and beat score are calculated separately, and the final score is determined by combining these scores to consider the original audio information from multiple dimensions. Real rating.

Benefits of technology

Improves the accuracy of audio reviews, enhances the user experience, and can help original music get more impartial ratings and popularity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115394317B_ABST
    Figure CN115394317B_ABST
Patent Text Reader

Abstract

The present application discloses an audio evaluation method and device, which obtains the original audio information input by the user and pre-processes the original audio information and the preset reference audio information respectively, obtains the first audio information corresponding to the original audio information and the first reference information corresponding to the reference audio information, then respectively performs the first comprehensive calculation, the second comprehensive calculation, the third comprehensive calculation and the fourth comprehensive calculation on the first audio information and the first reference information, obtains the content score, the pitch score, the rhythm score and the beat score of the original audio information, and finally determines the final score of the original audio information according to the content score, the pitch score, the rhythm score and the beat score of the original audio information. The scheme uses the reference audio information to score the content, pitch, rhythm and beat of the original audio information respectively, that is, to consider the real score of the original audio information from multiple dimensions, thereby improving the accuracy of the audio evaluation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an audio evaluation method and device. Background Art

[0002] With the development of 5G and artificial intelligence, pan-entertainment products such as live streaming and short videos are gradually emerging, and various social interaction methods are emerging in an endless stream. More and more users like to compete in karaoke, recitation, dubbing, etc. in audio scoring systems, and compete based on the scores. Therefore, more and more audio evaluation or scoring methods have emerged to meet the different needs of users.

[0003] However, existing audio evaluation methods often only consider a single dimension, so the evaluation accuracy is not high, which affects users' experience of innovative gameplay in the audio rating system. Summary of the invention

[0004] In view of this, the present application provides an audio evaluation method and device to solve the problem that existing audio evaluation methods often only consider a single dimension, so the evaluation accuracy is not high, which affects the user's experience of innovative gameplay in the audio scoring system.

[0005] To achieve the above objectives, the proposed solution is as follows:

[0006] In a first aspect, an audio evaluation method includes:

[0007] Get the original audio information input by the user;

[0008] Preprocessing the original audio information and the preset reference audio information respectively to obtain first audio information corresponding to the original audio information and first reference information corresponding to the reference audio information;

[0009] Performing a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information;

[0010] Performing a second comprehensive calculation on the first audio information and the first reference information to obtain a pitch score of the original audio information;

[0011] Performing a third comprehensive calculation on the first audio information and the first reference information to obtain a rhythm score of the original audio information;

[0012] Performing a fourth comprehensive calculation on the first audio information and the first reference information to obtain a beat score of the original audio information;

[0013] The final score of the original audio information is determined based on the content score, pitch score, rhythm score and beat score of the original audio information.

[0014] Preferably, performing a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information includes:

[0015] Performing feature calculations on the first audio information and the first reference information respectively to obtain second audio information corresponding to the first audio information and second reference information corresponding to the first reference information;

[0016] Calculate the target distance between the second audio information and the second reference information to obtain the target distance between the first audio information and the first reference information;

[0017] The target distance is quantized according to a preset quantization rule to obtain a content score of the original audio information.

[0018] Preferably, performing a second comprehensive calculation on the first audio information and the first reference information to obtain the pitch score of the original audio information includes:

[0019] Extracting fundamental frequencies of the first audio information and the first reference information respectively to obtain fundamental frequency information of the first audio information and fundamental frequency information of the first reference information;

[0020] The fundamental frequency information of the first audio information and the fundamental frequency information of the first reference information are subjected to similarity calculation to obtain a pitch score of the original audio information.

[0021] Preferably, performing a third comprehensive calculation on the first audio information and the first reference information to obtain the rhythm score of the original audio information includes:

[0022] respectively determining time information of each note in the first audio information and time information of each note in the first reference information;

[0023] For each note in the first audio information, based on the time information of the note and the time information of the corresponding note in the first reference information, calculate the time difference between the note and the corresponding note in the first reference information;

[0024] The time difference between each note in the first audio information and the corresponding note in the first reference information is integrated and calculated to obtain the rhythm score of the original audio information.

[0025] Preferably, performing a fourth comprehensive calculation on the first audio information and the first reference information to obtain the beat score of the original audio information includes:

[0026] respectively determining a target tempo of the first audio information and a target tempo of the first reference information;

[0027] A beat score of the original audio information is determined based on the target beat of the first audio information and the target beat of the first reference information.

[0028] Preferably, determining the final score of the original audio information according to the content score, the pitch score, the rhythm score and the beat score of the original audio information comprises:

[0029] Performing audio rhythm calculation on the first audio information and the second audio information respectively to obtain a target rhythm array length corresponding to the first audio information and a reference rhythm array length corresponding to the first reference information;

[0030] Obtaining a first score of the original audio information based on the target rhythm array length, the reference rhythm array length, the content score, the pitch score, the rhythm score, and the beat score;

[0031] The first score is constrained by using a preset constraint rule to obtain a final score of the original audio information.

[0032] Preferably, obtaining a first score of the original audio information based on the target rhythm array length, the reference rhythm array length, the content score, the pitch score, the rhythm score and the beat score comprises:

[0033] Obtaining a first length, a second length, and a third length corresponding to the reference rhythm array length;

[0034] If the target rhythm array length is less than the first length, and the rhythm score of the original audio information is greater than a first preset threshold, the first score of the original audio information is a second preset threshold;

[0035] If the target rhythm array length is smaller than the first length and the rhythm score of the original audio information is not greater than the first preset threshold, the first score of the original audio information is the rhythm score of the original audio information.

[0036] Preferably, it also includes:

[0037] If the target rhythm array length is not less than the second length and not greater than the third length, and the beat score is a third preset threshold, then respectively calculating the product of the content score and the first preset coefficient, the product of the pitch score and the second preset coefficient, and the product of the rhythm score and the third preset coefficient, and adding the product of the content score and the first preset coefficient, the product of the pitch score and the second preset coefficient, and the product of the rhythm score and the third preset coefficient to obtain a first score of the original audio information;

[0038] If the target rhythm array length is not less than the second length and not greater than the third length, and the beat score is the fourth preset threshold or the fifth preset threshold, the product of the content score and the fourth preset coefficient, and the product of the tone score and the first preset coefficient are calculated respectively, and the product of the content score and the fourth preset coefficient, and the product of the tone score and the first preset coefficient are added to obtain a first score of the original audio information.

[0039] Preferably, it also includes:

[0040] If the target rhythm array length is not less than the first length and not greater than the second length, and the beat score is a third preset threshold, respectively calculating the product of the content score and the second preset coefficient, the product of the tone score and the fifth preset coefficient, and the product of the beat score and the third preset coefficient, and adding the product of the content score and the second preset coefficient, the product of the tone score and the fifth preset coefficient, and the product of the beat score and the third preset coefficient to obtain a first score of the original audio information;

[0041] If the target rhythm array length is not less than the first length and not greater than the second length, and the beat score is the fourth preset threshold or the fifth preset threshold, then the product of the content score and the fifth preset coefficient, the product of the pitch score and the second preset coefficient, the product of the rhythm score and the third preset coefficient, and the product of the beat score and the third preset coefficient are calculated respectively, and the product of the content score and the fifth preset coefficient, the product of the pitch score and the second preset coefficient, the product of the rhythm score and the third preset coefficient, and the product of the beat score and the third preset coefficient are added to obtain a first score for the original audio information.

[0042] In a second aspect, an audio evaluation device includes:

[0043] An acquisition module is used to acquire original audio information input by the user;

[0044] A preprocessing module, used to preprocess the original audio information and the preset reference audio information respectively to obtain first audio information corresponding to the original audio information and first reference information corresponding to the reference audio information;

[0045] a content score calculation module, configured to perform a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information;

[0046] a tone score calculation module, configured to perform a second comprehensive calculation on the first audio information and the first reference information to obtain a tone score of the original audio information;

[0047] a rhythm score calculation module, configured to perform a third comprehensive calculation on the first audio information and the first reference information to obtain a rhythm score of the original audio information;

[0048] a beat score calculation module, configured to perform a third comprehensive calculation on the first audio information and the first reference information to obtain a beat score of the original audio information;

[0049] The final score determination module is used to determine the final score of the original audio information according to the content score, pitch score, rhythm score and beat score of the original audio information.

[0050] It can be seen from the above technical scheme that the present application obtains the original audio information input by the user and pre-processes the original audio information and the preset reference audio information respectively, obtains the first audio information corresponding to the original audio information and the first reference information corresponding to the reference audio information, then performs a first comprehensive calculation on the first audio information and the first reference information to obtain the content score of the original audio information; performs a second comprehensive calculation on the first audio information and the first reference information to obtain the pitch score of the original audio information; performs a third comprehensive calculation on the first audio information and the first reference information to obtain the rhythm score of the original audio information; performs a fourth comprehensive calculation on the first audio information and the first reference information to obtain the beat score of the original audio information; finally, according to the content score, pitch score, rhythm score and beat score of the original audio information, the final score of the original audio information is determined. This scheme uses the reference audio information to score the content, pitch, rhythm and beat of the original audio information respectively, that is, to consider the real score of the original audio information from multiple dimensions, thereby improving the accuracy of audio evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0052] Figure 1 An optional flow chart of an audio evaluation method provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of the application system structure of an audio evaluation method provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of the structure of an audio evaluation device provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of the structure of an audio evaluation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] With the development of 5G and artificial intelligence, pan-entertainment products such as live streaming and short videos are gradually emerging, and various social interaction methods are emerging in an endless stream. More and more users like to compete in karaoke, recitation, dubbing, etc. in audio scoring systems, and compete based on the scores. Therefore, more and more audio evaluation or scoring methods have emerged to meet the different needs of users.

[0058] However, existing audio evaluation methods often only consider a single dimension, so the evaluation accuracy is not high, which affects users' experience of innovative gameplay in the audio rating system.

[0059] Based on the above defects, an embodiment of the present invention provides an audio evaluation method, which is applicable to various music evaluation scenarios, can also be applied to various voice or audio scoring systems, and can also be applied to various computer terminals or smart terminals. The execution subject can be a processor or server of the computer terminal or smart terminal. The method flow of the method is as follows: Figure 1 As shown, specifically including:

[0060] S1: Obtain the original audio information input by the user.

[0061] In the embodiments provided in the present application, the original audio information input by the user can be the audio data of the user in entertainment software such as karaoke rooms and voice chat rooms. Here is an example. In a voice chat product, the user can create a singing competition room as the room owner, and can invite friends to join the room for singing competition. The users can sing the same song in sequence, or multiple users in the same room can compete for the microphone. The user who successfully competes for the microphone will sing. The evaluation method provided in the present application will evaluate the audio information of the user's singing.

[0062] S2: Preprocessing the original audio information and the preset reference audio information respectively to obtain first audio information corresponding to the original audio information and first reference information corresponding to the reference audio information.

[0063] Optionally, for a song, the preset reference audio information is the original song, which is not limited in this embodiment.

[0064] Since the original audio information input by the user and the preset reference audio information have different formats, sampling rates and numbers of channels, the first audio information and the first reference information are preprocessed respectively. Taking the first audio information as an example, the preprocessing method includes but is not limited to: checking the sampling rate, number of channels and quantization bit width of the first audio information in WAV format, so as to ensure that the first audio information meets the preset target sampling rate, target channel and target bit width; wherein the target sampling rate can be set to 16KHz, the target channel can be set to single channel (Mono), and the target bit width can be set to 16bits.

[0065] It is understandable that when the first audio information and the first reference information both meet preset requirements such as target sampling rate, target channel and target bit width, the evaluation can be performed more accurately to improve the scoring accuracy.

[0066] S3: Perform a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information.

[0067] The process of comprehensive calculation of the first audio information and the first reference information may include scoring the similarity of the specific audio content of the two to obtain the content score (die_score) of the original audio information. The higher the similarity, the closer the specific audio content of the first audio information is to the first reference information, which means that the score of the original audio information in the specific audio content dimension is higher.

[0068] S4: Perform a second comprehensive calculation on the first audio information and the first reference information to obtain a pitch score of the original audio information.

[0069] The process of performing the second comprehensive calculation on the first audio information and the first reference information may include calculating the similarity of the two in terms of pitch type such as sound melody or sound emotion, to obtain a pitch score (pitch_score) of the original audio information.

[0070] S5: Perform a third comprehensive calculation on the first audio information and the first reference information to obtain a rhythm score of the original audio information.

[0071] The process of performing the third comprehensive calculation on the first audio information and the first reference information may include comparing the time information of each note of the two, that is, scoring the similarity in rhythm, and obtaining the rhythm score (onset_score) of the original audio information.

[0072] S6: Perform a fourth comprehensive calculation on the first audio information and the first reference information to obtain a beat score of the original audio information.

[0073] The process of performing the fourth comprehensive calculation on the first audio information and the first reference information may include calculating the similarity of the beats of the two, thereby obtaining the beat score (beat_score) of the original audio information.

[0074] S7: Determine a final score of the original audio information based on the content score, pitch score, rhythm score and beat score of the original audio information.

[0075] In this step, the content score, pitch score, rhythm score and beat score of the original audio information obtained previously are comprehensively considered, and corresponding weight parameters can be set according to the characteristics of these four scores, and then these four scores are integrated to obtain the final score (all_score) of the original audio information.

[0076] Optionally, in the above scheme, the processes of performing the first comprehensive calculation, the second comprehensive calculation, the third comprehensive calculation and the fourth comprehensive calculation on the first audio information and the first reference information can be performed simultaneously, in no particular order, and are not limited here.

[0077] It can be seen from the above technical scheme that the present application obtains the original audio information input by the user and pre-processes the original audio information and the preset reference audio information respectively, obtains the first audio information corresponding to the original audio information and the first reference information corresponding to the reference audio information, then performs a first comprehensive calculation on the first audio information and the first reference information to obtain the content score of the original audio information; performs a second comprehensive calculation on the first audio information and the first reference information to obtain the pitch score of the original audio information; performs a third comprehensive calculation on the first audio information and the first reference information to obtain the rhythm score of the original audio information; performs a fourth comprehensive calculation on the first audio information and the first reference information to obtain the beat score of the original audio information; finally, according to the content score, pitch score, rhythm score and beat score of the original audio information, the final score of the original audio information is determined. This scheme uses the reference audio information to score the content, pitch, rhythm and beat of the original audio information respectively, that is, to consider the real score of the original audio information from multiple dimensions, thereby improving the accuracy of audio evaluation.

[0078] In addition, the original audio information input by the user is scored based on different dimensions and comprehensively considered, which improves the accuracy of audio evaluation and provides a better user experience. At the same time, original music can also be used as reference audio information to help original singers increase their popularity.

[0079] In the method provided by the embodiment of the present invention, the process of performing a first comprehensive calculation on the first audio information and the first reference information to obtain the content score of the original audio information may include:

[0080] S31: performing feature calculations on the first audio information and the first reference information respectively to obtain second audio information corresponding to the first audio information and second reference information corresponding to the first reference information.

[0081] Before calculating the features of the first audio information and the first reference information, the first audio information and the first reference information may be respectively initialized by framing, windowing, pre-emphasis, etc., which can facilitate the subsequent feature extraction process.

[0082] Taking the first audio information as an example, this step may include the following process:

[0083] Perform Mel-cepstral coefficient feature extraction on the first audio information to obtain Mel-cepstral coefficient feature information of the first audio information, then perform first-order difference calculation on the Mel-cepstral coefficient feature information of the first audio information to obtain first-order information of the first audio information, and then perform second-order difference calculation on the first-order information of the first audio information to obtain second audio information corresponding to the first audio information.

[0084] S32: Calculate the target distance between the second audio information and the second reference information to obtain the target distance between the first audio information and the first reference information.

[0085] Optionally, this step can use a dynamic time warping (DTW) algorithm to calculate the target distance between the second audio information and the second reference information to obtain the target distance between the first audio information and the first reference information. The smaller the target distance, the closer the audio content of the original audio information is to the audio content of the reference audio information, that is, the more accurate the audio content of the original audio information is.

[0086] S33: quantizing the target distance according to a preset quantization rule to obtain a content score of the original audio information.

[0087] The content score of the original audio information is expressed as die_score, and the content score of the original audio information can be calculated by the following formula:

[0088] die_score=-0.08*d(T(i n ),R(i m ))*d(T(i n ),R(i m ))+100,

[0089] Wherein, T represents the second audio information, R represents the second reference information, d(T(i n ),R(i m )) indicates the i-th n The frame feature and the i-th m The target distance between frame features is usually expressed as Euclidean distance, which is then quantized and converted into a percentage according to a preset quantization rule. After multiple tests, an exponential fitting curve function is obtained to finally determine the content score of the original audio information.

[0090] The above-mentioned embodiment describes how the present application performs a first comprehensive calculation on the first audio information and the first reference information to obtain the content score of the original audio information. The following describes in detail how the present application performs a second comprehensive calculation on the first audio information and the first reference information to obtain the tone score of the original audio information.

[0091] The fundamental frequencies of the first audio information and the first reference information are extracted respectively to obtain the fundamental frequency information of the first audio information and the fundamental frequency information of the first reference information. Then, a dynamic time normalization algorithm can be used to calculate the similarity between the fundamental frequency information of the first audio information and the fundamental frequency information of the first reference information to obtain the pitch score of the original audio information.

[0092] Specifically, in the process of extracting the baseband, since the baseband information of the audio information often changes with time, the signal is usually divided into frames in the baseband extraction, and the frame length can be set to 20ms, which is not limited in this embodiment, and then the baseband is extracted frame by frame. The process of extracting the baseband of each frame may include:

[0093] 1) Time domain method.

[0094] The waveform of the first audio information is used as input to find the minimum positive period of the waveform, wherein the periodicity of the first audio information and the first reference information is similar.

[0095] 2) Frequency domain method.

[0096] First, the first audio information is subjected to Fourier transform to obtain the spectrum of the first audio information. In the process of Fourier transform, only the amplitude spectrum can be taken, that is, the phase spectrum is discarded. The spectrum will have peaks at integer multiples of the base frequency, so the greatest common divisor of these peak frequencies can be calculated.

[0097] The following is a specific description of how to perform the third comprehensive calculation on the first audio information and the first reference information in the present application to obtain the rhythm score of the original audio information.

[0098] S51: respectively determining time information of each note in the first audio information and time information of each note in the first reference information.

[0099] S52: For each note in the first audio information, based on the time information of the note and the time information of the corresponding note in the first reference information, calculate the time difference between the note and the corresponding note in the first reference information.

[0100] S53: Integrate and calculate the time difference between each note in the first audio information and the corresponding note in the first reference information to obtain the rhythm score of the original audio information.

[0101] Optionally, a dynamic time normalization algorithm may be used to calculate the rhythm score of the original audio information.

[0102] Specifically, the rhythm score of the original audio information can be obtained by using the ONSET calculation method. There are two variables in the calculation method, one variable is the threshold value, the value range can be set to 0.01-0.99 (mainly used for peak picking), and the other variable is the onset value (for the detection part in the variable, it can include high-frequency content, complex domain, energy and spectral difference). In one example, the complex domain method is used to solve the onset value of the note corresponding to the first audio information and the first reference information, so as to obtain the rhythm score of the original audio information, including: using fast Fourier transform for calculation, the FFT size can be set to 1024, the increment size is set to 512, the peak threshold threshold is set to 0.5, the silence threshold is set to -50dB, and the internal minimum onset interval is set to 40ms. Because this complex domain method has a phase sound encoder, the size of the window window can be set to 1024, and the number of hops hope is set to 512. Therefore, the threshold of the ONSET calculation method is selected by changing the peak value, and the size of the threshold affects the size of the onset value.

[0103] The following specifically describes the process of performing the fourth comprehensive calculation on the first audio information and the first reference information to obtain the beat score of the original audio information.

[0104] A target tempo of the first audio information and a target tempo of the first reference information are determined respectively.

[0105] In one example, taking the first audio information as an example, the process of determining the target beat of the first audio information using a dynamic time normalization algorithm may include: dividing the first audio information into sub-bands according to a preset division rule; obtaining the impact starting point signal of each sub-band; calculating the period of the impact starting point signal of each sub-band based on the impact starting point signal of each sub-band; determining the harmonic beat group of the first audio information based on the period of the impact starting point signal of each sub-band; and obtaining the target beat of the first audio information based on the harmonic beat group.

[0106] The same method can be used to determine the target beat of the first reference information. If the target beat of the first audio information is the same as the target beat of the first reference information, the beat score of the original audio information is set to 100 points; if the target beat of the first audio information is relatively small compared to the target beat of the first reference information, the beat score of the original audio information is set to 50 points; if the target beat of the first audio information is very different from the target beat of the first reference information, the beat score of the original audio information is set to 0 points.

[0107] In one embodiment provided in the present application, for step S7, the process of determining the final score of the original audio information according to the content score, pitch score, rhythm score and beat score of the original audio information may include the following steps:

[0108] S71: performing audio rhythm calculation on the first audio information and the second audio information respectively to obtain a target rhythm array length corresponding to the first audio information and a reference rhythm array length corresponding to the first reference information.

[0109] S72: Obtain a first score of the original audio information based on the target rhythm array length, the reference rhythm array length, the content score, the pitch score, the rhythm score and the beat score.

[0110] S73: Constraining the first score by using a preset constraint rule to obtain a final score of the original audio information.

[0111] Optionally, for step S72, the process of obtaining a first score of the original audio information based on the target rhythm array length, the reference rhythm array length, the content score, the pitch score, the rhythm score and the beat score may include:

[0112] S721: Obtain a first length, a second length, and a third length corresponding to the reference rhythm array length.

[0113] Specifically, the first length corresponding to the reference rhythm array length can be set to half of the reference rhythm array length, the second length can be set to 70% of the reference rhythm array length, and the third length can be set to 100% of the reference rhythm array length. This embodiment does not impose any restrictions on this.

[0114] S722: If the target rhythm array length is smaller than the first length, and the rhythm score of the original audio information is greater than a first preset threshold, the first score of the original audio information is a second preset threshold.

[0115] Optionally, the first preset threshold may be 60 points, and the second preset threshold may be 59 points.

[0116] S723: If the target rhythm array length is smaller than the first length, and the rhythm score of the original audio information is not greater than the first preset threshold, the first score of the original audio information is the rhythm score of the original audio information.

[0117] S724: If the target rhythm array length is not less than the second length and not greater than the third length, and the beat score is the third preset threshold, then the product of the content score and the first preset coefficient, the product of the pitch score and the second preset coefficient, and the product of the rhythm score and the third preset coefficient are calculated respectively, and the product of the content score and the first preset coefficient, the product of the pitch score and the second preset coefficient, and the product of the rhythm score and the third preset coefficient are added together to obtain a first score for the original audio information.

[0118] S725: If the target rhythm array length is not less than the second length and not greater than the third length, and the beat score is the fourth preset threshold or the fifth preset threshold, then the product of the content score and the fourth preset coefficient, and the product of the pitch score and the first preset coefficient are calculated respectively, and the product of the content score and the fourth preset coefficient, and the product of the pitch score and the first preset coefficient are added together to obtain a first score for the original audio information.

[0119] S726: If the length of the target rhythm array is not less than the first length and not greater than the second length, and the beat score is the third preset threshold, then the product of the content score and the second preset coefficient, the product of the pitch score and the fifth preset coefficient, and the product of the beat score and the third preset coefficient are calculated respectively, and the product of the content score and the second preset coefficient, the product of the pitch score and the fifth preset coefficient, and the product of the beat score and the third preset coefficient are added together to obtain a first score for the original audio information.

[0120] S727: If the target rhythm array length is not less than the first length and not greater than the second length, and the beat score is the fourth preset threshold or the fifth preset threshold, then the product of the content score and the fifth preset coefficient, the product of the pitch score and the second preset coefficient, the product of the rhythm score and the third preset coefficient, and the product of the beat score and the third preset coefficient are calculated respectively, and the product of the content score and the fifth preset coefficient, the product of the pitch score and the second preset coefficient, the product of the rhythm score and the third preset coefficient, and the product of the beat score and the third preset coefficient are added together to obtain a first score for the original audio information.

[0121] In the above steps, the first preset coefficient, the second preset coefficient, the third preset coefficient, the fourth preset coefficient and the fifth preset coefficient can be set to 0.3, 0.6, 0.1, 0.7 and 0.2 respectively. The specific setting values ​​can be set according to different audio types and scenes, and this embodiment does not limit this.

[0122] In the above step S73, the constraint rules may include:

[0123] If the first score is not less than 100 points, the final score is 100 points; if the first score is not greater than 0 points, the final score is 0 points; if the first score is between 0 and 100 points, the final score is the first score.

[0124] The audio evaluation method provided in this application can be applied to various system platforms, including Android and iOS, etc. The method can be packaged and compiled into a complete so library and interface file by ndk, wherein the actual calling method follows the integrated calling process.

[0125] In an example, the method is applied to a structural block diagram of an Android end voice chat system as shown in Figure 2 Please refer to Figure 2. The user creates a "You can sing" room in the voice chat system and invites friends to enter the room to play the singing or snatch singing game together. When a song is selected, the system will provide the corresponding template audio, and then the user can start singing. Among them, the first two sentences of the song can be played as the introduction content so that the user is familiar with the song. The following three or four sentences belong to the singing content, and only the lyrics are provided. The user needs to sing the corresponding content while looking at the lyrics. After the singing content is completed, the system will pre-process the singing voice and template audio input by the user at the same time. Next, the init interface is called to implement the pre-processing of the singing voice and template audio, that is, the sampling rate, number of channels and bit width of the singing voice and template audio are checked, and then the processing interface is called to implement the comprehensive scoring of the singing voice and template audio. Next, the evaluation results are output through the stop interface and displayed to the user. If the user is not satisfied with the evaluation results, he can sing again. This process can only loop the processing interface and the stop interface; finally, if the user is satisfied with the evaluation results, the destory interface is called to end the evaluation.

[0126] and Figure 1 Corresponding to the method described above, the embodiment of the present invention further provides an audio evaluation device for Figure 1 In the specific implementation of the method, the audio evaluation device provided in the embodiment of the present invention can be used in a computer terminal or various mobile devices, combined with Figure 3 , the audio evaluation device is introduced, such as Figure 3 As shown, the device may include:

[0127] The acquisition module 10 is used to acquire original audio information input by the user.

[0128] The preprocessing module 20 is used to preprocess the original audio information and the preset reference audio information respectively to obtain the first audio information corresponding to the original audio information and the first reference information corresponding to the reference audio information.

[0129] The content score calculation module 30 is used to perform a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information.

[0130] The tone score calculation module 40 is used to perform a second comprehensive calculation on the first audio information and the first reference information to obtain the tone score of the original audio information.

[0131] The rhythm score calculation module 50 is used to perform a third comprehensive calculation on the first audio information and the first reference information to obtain the rhythm score of the original audio information.

[0132] The beat score calculation module 60 is used to perform a third comprehensive calculation on the first audio information and the first reference information to obtain the beat score of the original audio information.

[0133] The final score determination module 70 is used to determine the final score of the original audio information according to the content score, pitch score, rhythm score and beat score of the original audio information.

[0134] It can be seen from the above technical scheme that the present application obtains the original audio information input by the user and pre-processes the original audio information and the preset reference audio information respectively, obtains the first audio information corresponding to the original audio information and the first reference information corresponding to the reference audio information, then performs a first comprehensive calculation on the first audio information and the first reference information to obtain the content score of the original audio information; performs a second comprehensive calculation on the first audio information and the first reference information to obtain the pitch score of the original audio information; performs a third comprehensive calculation on the first audio information and the first reference information to obtain the rhythm score of the original audio information; performs a fourth comprehensive calculation on the first audio information and the first reference information to obtain the beat score of the original audio information; finally, according to the content score, pitch score, rhythm score and beat score of the original audio information, the final score of the original audio information is determined. This scheme uses the reference audio information to score the content, pitch, rhythm and beat of the original audio information respectively, that is, to consider the real score of the original audio information from multiple dimensions, thereby improving the accuracy of audio evaluation.

[0135] In one example, the content score calculation module 30 may include:

[0136] The feature calculation module is used to perform feature calculation on the first audio information and the first reference information respectively to obtain the second audio information corresponding to the first audio information and the second reference information corresponding to the first reference information.

[0137] The target distance calculation module is used to perform target distance calculation on the second audio information and the second reference information to obtain a target distance between the first audio information and the first reference information.

[0138] The quantization module is used to quantize the target distance according to a preset quantization rule to obtain a content score of the original audio information.

[0139] In one example, the pitch score calculation module 40 may include:

[0140] The baseband extraction module is used to extract the baseband of the first audio information and the first reference information respectively to obtain the baseband information of the first audio information and the baseband information of the first reference information.

[0141] The similarity calculation module is used to calculate the similarity between the fundamental frequency information of the first audio information and the fundamental frequency information of the first reference information to obtain the pitch score of the original audio information.

[0142] In one example, the rhythm score calculation module 50 may include:

[0143] The time information determination module is used to respectively determine the time information of each note in the first audio information and the time information of each note in the first reference information.

[0144] The time difference determination module is used to calculate the time difference between each note in the first audio information and the corresponding note in the first reference information based on the time information of the note and the time information of the corresponding note in the first reference information.

[0145] The integration calculation module is used to integrate and calculate the time difference between each note in the first audio information and the corresponding note in the first reference information to obtain the rhythm score of the original audio information.

[0146] In one example, the beat score calculation module 60 may include:

[0147] The target beat determination module is used to determine the target beat of the first audio information and the target beat of the first reference information respectively.

[0148] A beat score determination module is used to determine the beat score of the original audio information based on the target beat of the first audio information and the target beat of the first reference information.

[0149] In one example, the final score determination module 70 may include:

[0150] The rhythm array length calculation module is used to perform audio rhythm calculation on the first audio information and the first audio information respectively to obtain a target rhythm array length corresponding to the first audio information and a reference rhythm array length corresponding to the first reference information.

[0151] The first score obtaining module is used to obtain a first score of the original audio information based on the target rhythm array length, the reference rhythm array length, the content score, the pitch score, the rhythm score and the beat score.

[0152] The constraint module is used to constrain the first score by using a preset constraint rule to obtain a final score of the original audio information.

[0153] In one example, the first scoring module may include:

[0154] The length acquisition module is used to acquire the first length, the second length and the third length corresponding to the reference rhythm array length.

[0155] The first calculation module is used for, if the target rhythm array length is less than the first length and the rhythm score of the original audio information is greater than a first preset threshold, then the first score of the original audio information is a second preset threshold.

[0156] The second calculation module is used for, if the target rhythm array length is less than the first length and the rhythm score of the original audio information is not greater than the first preset threshold, then the first score of the original audio information is the rhythm score of the original audio information.

[0157] The third calculation module is used to calculate the product of the content score and the first preset coefficient, the product of the pitch score and the second preset coefficient, and the product of the rhythm score and the third preset coefficient respectively if the length of the target rhythm array is not less than the second length and not greater than the third length, and the beat score is a third preset threshold, and add the product of the content score and the first preset coefficient, the product of the pitch score and the second preset coefficient, and the product of the rhythm score and the third preset coefficient to obtain a first score for the original audio information.

[0158] The fourth calculation module is used to calculate the product of the content score and the fourth preset coefficient, and the product of the tone score and the first preset coefficient respectively if the length of the target rhythm array is not less than the second length and not greater than the third length, and the beat score is the fourth preset threshold or the fifth preset threshold, and add the product of the content score and the fourth preset coefficient, and the product of the tone score and the first preset coefficient to obtain a first score for the original audio information.

[0159] The fifth calculation module, if the length of the target rhythm array is not less than the first length and not greater than the second length, and the beat score is the third preset threshold, then calculate the product of the content score and the second preset coefficient, the product of the pitch score and the fifth preset coefficient, and the product of the beat score and the third preset coefficient respectively, and add the product of the content score and the second preset coefficient, the product of the pitch score and the fifth preset coefficient, and the product of the beat score and the third preset coefficient to obtain the first score of the original audio information.

[0160] The sixth calculation module is used to calculate the product of the content score and the fifth preset coefficient, the product of the pitch score and the second preset coefficient, the product of the rhythm score and the third preset coefficient, and the product of the beat score and the third preset coefficient respectively if the length of the target rhythm array is not less than the first length and not greater than the second length, and the beat score is the fourth preset threshold or the fifth preset threshold, and add the product of the content score and the fifth preset coefficient, the product of the pitch score and the second preset coefficient, the product of the rhythm score and the third preset coefficient, and the product of the beat score and the third preset coefficient to obtain a first score for the original audio information.

[0161] Furthermore, the embodiment of the present application provides an audio evaluation device. Optionally, Figure 4 The hardware structure diagram of the audio evaluation equipment is shown in FIG. Figure 4 The hardware structure of the audio evaluation device may include: at least one processor 01 , at least one communication interface 02 , at least one memory 03 and at least one communication bus 04 .

[0162] In the embodiment of the present application, the number of the processor 01 , the communication interface 02 , the memory 03 , and the communication bus 04 is at least one, and the processor 01 , the communication interface 02 , and the memory 03 communicate with each other through the communication bus 04 .

[0163] The processor 01 may be a central processing unit CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0164] The memory 03 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory.

[0165] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to execute the following audio evaluation method, including:

[0166] Get the original audio information input by the user;

[0167] Preprocessing the original audio information and the preset reference audio information respectively to obtain first audio information corresponding to the original audio information and first reference information corresponding to the reference audio information;

[0168] Performing a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information;

[0169] performing a second comprehensive calculation on the first audio information and the first reference information to obtain a pitch score of the original audio information;

[0170] Performing a third comprehensive calculation on the first audio information and the first reference information to obtain a rhythm score of the original audio information;

[0171] Performing a fourth comprehensive calculation on the first audio information and the first reference information to obtain a beat score of the original audio information;

[0172] The final score of the original audio information is determined according to the content score, pitch score, rhythm score and beat score of the original audio information. Optionally, the detailed functions and extended functions of the program can refer to the description of the audio evaluation method in the method embodiment.

[0173] The embodiment of the present application further provides a storage medium, which can store a program suitable for execution by a processor, and when the program is running, controls the device where the storage medium is located to execute the following audio evaluation method, including:

[0174] Get the original audio information input by the user;

[0175] Preprocessing the original audio information and the preset reference audio information respectively to obtain first audio information corresponding to the original audio information and first reference information corresponding to the reference audio information;

[0176] Performing a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information;

[0177] performing a second comprehensive calculation on the first audio information and the first reference information to obtain a pitch score of the original audio information;

[0178] Performing a third comprehensive calculation on the first audio information and the first reference information to obtain a rhythm score of the original audio information;

[0179] Performing a fourth comprehensive calculation on the first audio information and the first reference information to obtain a beat score of the original audio information;

[0180] The final score of the original audio information is determined based on the content score, pitch score, rhythm score and beat score of the original audio information.

[0181] Specifically, the storage medium may be a computer-readable storage medium, and the computer-readable storage medium may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk or a ROM.

[0182] Optionally, the detailed functions and extended functions of the program may refer to the description of the audio evaluation method in the method embodiment.

[0183] In addition, the functional modules in the various embodiments of the present disclosure can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a live broadcast device, or a network device, etc.) to perform all or part of the steps of the methods of the various embodiments of the present disclosure.

[0184] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0185] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0186] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio evaluation method, characterized in that: include: Get the original audio information input by the user; Preprocessing the original audio information and the preset reference audio information respectively to obtain first audio information corresponding to the original audio information and first reference information corresponding to the reference audio information; Performing a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information; performing a second comprehensive calculation on the first audio information and the first reference information to obtain a pitch score of the original audio information; Performing a third comprehensive calculation on the first audio information and the first reference information to obtain a rhythm score of the original audio information; Performing a fourth comprehensive calculation on the first audio information and the first reference information to obtain a beat score of the original audio information; Determine the final score of the original audio information according to the content score, pitch score, rhythm score and beat score of the original audio information; including: respectively calculating the audio rhythm of the first audio information and the first audio information to obtain the target rhythm array length corresponding to the first audio information and the reference rhythm array length corresponding to the first reference information; obtaining the first length, the second length and the third length corresponding to the reference rhythm array length; if the target rhythm array length is less than the first length and the rhythm score of the original audio information is greater than a first preset threshold, the first score of the original audio information is the second preset threshold; if the target rhythm array length is less than the first length and the rhythm score of the original audio information is not greater than the first preset threshold, the first score of the original audio information is the rhythm score of the original audio information; constrain the first score using a preset constraint rule to obtain the final score of the original audio information.

2. The method according to claim 1, characterized in that The performing a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information includes: Performing feature calculations on the first audio information and the first reference information respectively to obtain second audio information corresponding to the first audio information and second reference information corresponding to the first reference information; The second audio information and the second reference information are used to calculate the target distance, and the target distance is obtained. a target distance between the first audio information and the first reference information; The target distance is quantized according to a preset quantization rule to obtain a content score of the original audio information.

3. The method according to claim 1, characterized in that The performing a second comprehensive calculation on the first audio information and the first reference information to obtain the pitch score of the original audio information includes: Extracting fundamental frequencies of the first audio information and the first reference information respectively to obtain fundamental frequency information of the first audio information and fundamental frequency information of the first reference information; The fundamental frequency information of the first audio information and the fundamental frequency information of the first reference information are subjected to similarity calculation to obtain a pitch score of the original audio information.

4. The method according to claim 1, characterized in that: The performing a third comprehensive calculation on the first audio information and the first reference information to obtain the rhythm score of the original audio information includes: respectively determining time information of each note in the first audio information and time information of each note in the first reference information; For each note in the first audio information, based on the time information of the note and the time information of the corresponding note in the first reference information, calculate the time difference between the note and the corresponding note in the first reference information; The time difference between each note in the first audio information and the corresponding note in the first reference information is integrated and calculated to obtain the rhythm score of the original audio information.

5. The method according to claim 1, characterized in that The performing a fourth comprehensive calculation on the first audio information and the first reference information to obtain the beat score of the original audio information includes: respectively determining a target tempo of the first audio information and a target tempo of the first reference information; A beat score of the original audio information is determined based on the target beat of the first audio information and the target beat of the first reference information.

6. The method according to claim 1, characterized in that Also includes: If the target rhythm array length is not less than the second length and not greater than the third length, and the beat score is a third preset threshold, then respectively calculating the product of the content score and the first preset coefficient, the product of the pitch score and the second preset coefficient, and the product of the rhythm score and the third preset coefficient, and adding the product of the content score and the first preset coefficient, the product of the pitch score and the second preset coefficient, and the product of the rhythm score and the third preset coefficient to obtain a first score of the original audio information; If the target rhythm array length is not less than the second length and not greater than the third length, and the beat score is the fourth preset threshold or the fifth preset threshold, the product of the content score and the fourth preset coefficient, and the product of the tone score and the first preset coefficient are calculated respectively, and the product of the content score and the fourth preset coefficient, and the product of the tone score and the first preset coefficient are added to obtain a first score of the original audio information.

7. The method according to claim 1, characterized in that Also includes: If the target rhythm array length is not less than the first length and not greater than the second length, and the beat score is a third preset threshold, respectively calculating the product of the content score and the second preset coefficient, the product of the tone score and the fifth preset coefficient, and the product of the beat score and the third preset coefficient, and adding the product of the content score and the second preset coefficient, the product of the tone score and the fifth preset coefficient, and the product of the beat score and the third preset coefficient to obtain a first score of the original audio information; If the target rhythm array length is not less than the first length and not greater than the second length, and the beat score is the fourth preset threshold or the fifth preset threshold, then the product of the content score and the fifth preset coefficient, the product of the pitch score and the second preset coefficient, the product of the rhythm score and the third preset coefficient, and the product of the beat score and the third preset coefficient are calculated respectively, and the product of the content score and the fifth preset coefficient, the product of the pitch score and the second preset coefficient, the product of the rhythm score and the third preset coefficient, and the product of the beat score and the third preset coefficient are added to obtain a first score for the original audio information.

8. An audio evaluation device, characterized in that: include: An acquisition module is used to acquire original audio information input by the user; A preprocessing module, used to preprocess the original audio information and the preset reference audio information respectively to obtain first audio information corresponding to the original audio information and first reference information corresponding to the reference audio information; a content score calculation module, configured to perform a first comprehensive calculation on the first audio information and the first reference information to obtain a content score of the original audio information; a tone score calculation module, configured to perform a second comprehensive calculation on the first audio information and the first reference information to obtain a tone score of the original audio information; a rhythm score calculation module, configured to perform a third comprehensive calculation on the first audio information and the first reference information to obtain a rhythm score of the original audio information; a beat score calculation module, configured to perform a third comprehensive calculation on the first audio information and the first reference information to obtain a beat score of the original audio information; A final score determination module is used to determine the final score of the original audio information according to the content score, pitch score, rhythm score and beat score of the original audio information; including: respectively calculating the audio rhythm of the first audio information and the first audio information to obtain the target rhythm array length corresponding to the first audio information and the reference rhythm array length corresponding to the first reference information; obtaining the first length, the second length and the third length corresponding to the reference rhythm array length; if the target rhythm array length is less than the first length and the rhythm score of the original audio information is greater than a first preset threshold, the first score of the original audio information is the second preset threshold; if the target rhythm array length is less than the first length and the rhythm score of the original audio information is not greater than the first preset threshold, the first score of the original audio information is the rhythm score of the original audio information; and constraining the first score using a preset constraint rule to obtain the final score of the original audio information.

Citation Information

Patent Citations

  • Method and device for determining K sing score

    CN108008930A

  • Singing scoring method and device, electronic equipment and storage medium

    CN109524025A

  • Percussion instrument, percussion practice data processing method and device, and electronic equipment

    CN110796998A

  • Voice evaluation method and device

    CN112349300A

  • Audio data teaching evaluation method and device thereof, equipment, medium and product

    CN113870897A