Singing skill detection method, singing skill display method and computer device

Abstract

The present application relates to a singing skill detection method, a singing skill display method, a computer device, and a computer program product. By obtaining the audio to be detected and the fundamental frequency sequence corresponding to the audio to be detected, the pitch change amplitude of each fundamental frequency in the fundamental frequency sequence is obtained, and the pitch change trend corresponding to each fundamental frequency is determined according to the pitch change amplitude, and then the singing skill corresponding to the fundamental frequency sequence interval is determined according to the duration of each pitch change trend and the pitch change value of the fundamental frequency sequence within the duration. Compared with the traditional deep learning modeling-based method of identifying singing skill information in audio, this solution can improve the efficiency of singing skill detection by utilizing the pitch change information of the fundamental frequency of the audio to determine the singing skill present in the audio.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a singing skill detection method, a singing skill display method, an apparatus, a computer device, a storage medium, and a computer program product. Background Art

[0002] With the rapid development of digital and mobile music, users now have access to a variety of singing apps. After recording a song using the app on their device, their singing skills need to be tested to provide a clear understanding of their performance. Currently, this method typically relies on deep learning-based recognition. However, deep learning-based recognition requires extensive training data, making it difficult to use in real time.

[0003] Therefore, the current singing skill detection method has the defect of low detection efficiency. Summary of the Invention

[0004] Based on this, it is necessary to provide a singing skill detection method, singing skill display method, device, computer equipment, computer-readable storage medium and computer program product that can improve detection efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a singing skill detection method, the method comprising:

[0006] Acquire the audio to be detected and a fundamental frequency sequence of the audio to be detected, where the fundamental frequencies of different time points in the audio to be detected are sorted in chronological order;

[0007] Obtaining the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence, and determining the pitch variation trend of each fundamental frequency according to the pitch variation amplitude of each audio;

[0008] Determining at least one fundamental frequency sequence interval in the fundamental frequency sequence according to the pitch change trend corresponding to the fundamental frequency at each time point in the fundamental frequency sequence, wherein the pitch change trend of each fundamental frequency in the fundamental frequency sequence interval is the same and the time points are continuous;

[0009] The singing skills included in the fundamental frequency sequence interval are determined according to the duration of the fundamental frequency sequence interval and the pitch change within the fundamental frequency sequence interval.

[0010] In one embodiment, the step of obtaining the audio to be detected and the fundamental frequency sequence of the audio to be detected includes:

[0011] Acquire the audio to be detected and perform human voice detection on the audio to be detected to obtain a fundamental frequency sequence corresponding to the human voice in the audio to be detected.

[0012] In one embodiment, obtaining the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence includes:

[0013] Obtaining the slope of each fundamental frequency in the fundamental frequency sequence according to the pitch amplitudes at adjacent time points in the fundamental frequency sequence;

[0014] The pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence is determined according to the slope of each fundamental frequency.

[0015] In one embodiment, obtaining slope information of each fundamental frequency in the fundamental frequency sequence according to the pitch amplitudes at adjacent time points in the fundamental frequency sequence includes:

[0016] Obtaining a difference between a pitch amplitude corresponding to a first time point in the fundamental frequency sequence and a pitch amplitude corresponding to a second time point, and obtaining a ratio of the difference to the pitch amplitude corresponding to the second time point; the first time point and the second time point are adjacent time points in the fundamental frequency sequence, and the first time point is greater than the second time point;

[0017] obtaining a slope of the fundamental frequency corresponding to the second time point according to the ratio;

[0018] Slope information of the fundamental frequencies corresponding to the plurality of second time points is obtained as the slope information of each fundamental frequency in the fundamental frequency sequence.

[0019] In one embodiment, determining the pitch change trend of each fundamental frequency according to the pitch change amplitude of each fundamental frequency includes:

[0020] Obtaining a comparison result between the slope of each fundamental frequency and a preset slope; the preset slope is used to indicate a trend that the pitch value of the fundamental frequency does not change;

[0021] The pitch change trends corresponding to the respective fundamental frequencies are determined according to the comparison results.

[0022] In one embodiment, determining the pitch change trend corresponding to each fundamental frequency according to the comparison result includes:

[0023] If the slope of the fundamental frequency is greater than the preset slope, determining that the pitch change trend of the fundamental frequency is an increasing trend;

[0024] If the slope of the fundamental frequency is less than the preset slope, determining that the pitch change trend of the fundamental frequency is a decreasing trend;

[0025] If the slope of the fundamental frequency is equal to the preset slope, it is determined that the pitch change trend of the fundamental frequency is a constant trend.

[0026] In one embodiment, determining the singing skills included in the fundamental frequency sequence interval corresponding to the duration according to the duration of the fundamental frequency sequence interval and the pitch change value within the fundamental frequency sequence interval includes:

[0027] For each fundamental frequency sequence interval, if the duration of the fundamental frequency sequence interval is greater than the preset time, and the difference between the pitch value corresponding to the start time of the fundamental frequency sequence interval and the pitch value corresponding to the end time of the fundamental frequency sequence interval is greater than the preset interval difference threshold, it is determined that the fundamental frequency sequence interval contains glissando technique.

[0028] In a second aspect, the present application provides a method for demonstrating singing skills, the method comprising:

[0029] Obtain the user's audio to be tested according to the recording command input by the user;

[0030] Detecting the singing skills contained in the audio to be detected by the above-mentioned singing skills detection method;

[0031] Demonstrate the singing skills contained in the audio to be detected.

[0032] In a third aspect, the present application provides a singing skill detection device, the device comprising:

[0033] A first acquisition module is configured to acquire an audio to be detected and a fundamental frequency sequence of the audio to be detected, wherein the fundamental frequency sequence is a sequence obtained by sorting the fundamental frequencies of the audio to be detected at different time points in chronological order;

[0034] a second acquisition module, configured to acquire a pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence, and determine a pitch variation trend of each fundamental frequency according to the pitch variation amplitude of each fundamental frequency;

[0035] An interval determination module is configured to determine at least one fundamental frequency sequence interval in the fundamental frequency sequence according to the pitch change trend corresponding to the fundamental frequency at each time point in the fundamental frequency sequence, wherein the pitch change trend of each fundamental frequency in the fundamental frequency sequence interval is the same and the time points are consecutive.

[0036] The detection module is used to determine the singing skills contained in the fundamental frequency sequence interval according to the duration of the fundamental frequency sequence interval and the pitch change within the fundamental frequency sequence interval.

[0037] In a fourth aspect, the present application provides a singing skill demonstration device, comprising:

[0038] The command acquisition module is used to obtain the user's audio to be detected according to the recording command input by the user;

[0039] A skill detection module, configured to detect the singing skills contained in the audio to be detected by using the above-mentioned singing skill detection method;

[0040] The output module is used to display the singing skills contained in the audio to be detected.

[0041] In a fifth aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0042] In a sixth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0043] In a seventh aspect, the present application provides a computer program product, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0044] The above-mentioned singing skill detection method, singing skill display method, device, computer equipment, storage medium, and computer program product obtain the audio to be detected and the fundamental frequency sequence corresponding to the audio to be detected, obtain the pitch change amplitude of each fundamental frequency in the fundamental frequency sequence, determine the pitch change trend corresponding to each fundamental frequency based on the pitch change amplitude, and then determine the singing skill corresponding to the fundamental frequency sequence interval based on the duration of each pitch change trend and the pitch change value of the fundamental frequency sequence within the duration. Compared with traditional deep learning modeling-based methods for identifying singing skill information in audio, this solution can improve the efficiency of singing skill detection by utilizing the pitch change information of the audio fundamental frequency to determine the singing skill present in the audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A diagram showing an application environment of a singing skill detection method according to an embodiment;

[0046] Figure 2 1 is a flow chart of a singing skill detection method according to an embodiment;

[0047] Figure 3 A schematic diagram of an interface for determining a pitch change trend in one embodiment;

[0048] Figure 4 A schematic diagram of an interface for determining singing technique steps in one embodiment;

[0049] Figure 5 is a flow chart of a singing skill detection method according to another embodiment;

[0050] Figure 61. A flowchart of a method for demonstrating singing skills according to an embodiment of the present invention;

[0051] Figure 7 is a structural block diagram of a singing skill detection device in one embodiment;

[0052] Figure 8 is a structural block diagram of a singing skill display device in one embodiment;

[0053] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0055] The singing skill detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 can communicate with the user's recording device. The terminal 102 can obtain the user's dry voice input by the user through the recording device as the audio to be detected. The terminal 102 can obtain the pitch change information of each fundamental frequency in the fundamental frequency sequence based on the audio to be detected and its corresponding fundamental frequency sequence, and determine the singing skills in the audio to be detected according to the trend represented by each pitch change information. Among them, the terminal 102 can be but not limited to various personal computers, laptops, smart phones and tablets. The recording device can be but not limited to various microphones.

[0056] In one embodiment, Figure 2 As shown, a singing skill detection method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0057] Step S202: Acquire the audio to be detected and its fundamental frequency sequence. The fundamental frequency sequence is a sequence obtained by sorting the fundamental frequencies of the audio to be detected at different time points in a chronological order.

[0058] The audio to be detected may be the audio input by the user through a recording device. For example, the user may record the singing audio as the audio to be detected in the singing software of his handheld terminal. After recording the user's singing audio, the singing software may detect the singing skills in the user's singing audio. For example, the terminal 102 may display buttons such as skill detection or singing analysis after the user records the singing audio. When the terminal 102 detects the user's trigger signal for these buttons, it may determine to start the singing skill detection in the user's audio to be detected. When analyzing the singing skills in the audio to be detected, the terminal 102 may first obtain the fundamental frequency sequence corresponding to the audio to be detected. The terminal 102 may obtain the fundamental frequency sequence corresponding to the audio to be detected by the fundamental frequency extraction technology. The fundamental frequency extraction technology refers to extracting the fundamental frequency (fundamental frequency: F0) curve of the human voice in the user's dry voice.

[0059] For example, in one embodiment, obtaining the audio to be detected and the fundamental frequency sequence of the audio to be detected includes: obtaining the audio to be detected and performing human voice detection on the audio to be detected to obtain the fundamental frequency sequence corresponding to the human voice in the audio to be detected. In this embodiment, when obtaining the fundamental frequency sequence of the fundamental frequency to be detected, the terminal 102 may perform human voice detection on the audio to be detected to obtain the fundamental frequency sequence corresponding to the human voice in the audio to be detected.

[0060] The fundamental frequency sequence corresponding to the audio to be tested can be a frame-level fundamental frequency sequence. The fundamental frequency sequence can include multiple frames of fundamental frequency information, each of which can represent the pitch information of the user singing the audio to be tested. The fundamental frequency sequence can be a time-series sequence, i.e., the fundamental frequency information in the fundamental frequency sequence can be sorted in chronological order. Thus, after receiving the audio sequence to be tested, terminal 102 can detect the singing technique therein based on the fundamental frequency sequence.

[0061] Step S204 : obtaining the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence, and determining the pitch variation trend of each fundamental frequency according to the pitch variation amplitude of each fundamental frequency.

[0062] The fundamental frequency sequence may include multiple fundamental frequency information, each of which may have corresponding pitch information. Furthermore, because the pitch of the audio to be detected may vary when the user sings, the pitch of each fundamental frequency information in the fundamental frequency sequence may be inconsistent. That is, the fundamental frequency sequence may be a curve composed of multiple fundamental frequency information. Within the fundamental frequency sequence curve, corresponding pitch variation information may exist within a certain time period, and each pitch variation within the fundamental frequency sequence may have a different variation amplitude. Terminal 102 may obtain the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence.

[0063] The terminal 102 may determine the pitch variation amplitude through the slope information of a curve formed by the pitch information of multiple fundamental frequencies within a certain time period. For example, in one embodiment, obtaining the pitch variation amplitude of each fundamental frequency in a fundamental frequency sequence includes: obtaining the slope of each fundamental frequency in the fundamental frequency sequence based on the pitch amplitude at adjacent time points in the fundamental frequency sequence; and determining the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence based on the slope. In this embodiment, the fundamental frequency sequence may be a time-based sequence. The terminal 102 may obtain the slope information of each fundamental frequency in the fundamental frequency sequence through the pitch amplitude information at adjacent time points in the fundamental frequency sequence, and further obtain the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence.

[0064] Specifically, the terminal 102 can perform a slope operation on the pitch amplitude information of two adjacent fundamental frequencies at time points in the fundamental frequency sequence to obtain the slope information of the fundamental frequency at the earlier time point among the fundamental frequencies at the adjacent time points. The terminal 102 can perform the above operation on the fundamental frequencies at every two adjacent time points to obtain the slope information corresponding to each fundamental frequency in the fundamental frequency sequence. Thus, the terminal 102 can obtain the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence based on the multiple slope information. For the fundamental frequency information at the last time point in the fundamental frequency sequence, since there is no other fundamental frequency information after the fundamental frequency information, the terminal 102 can assume that the slope of the fundamental frequency information at the last time point is zero.

[0065] After obtaining the above-mentioned pitch variation amplitude, the terminal 102 can determine the pitch variation trend corresponding to the above-mentioned each fundamental frequency according to the pitch variation amplitude. Among them, the pitch variation trend can include trends such as growth, decrease and no change. Different pitch variation amplitudes can represent different pitch variation trends. For example, the terminal 102 can determine the pitch variation trend corresponding to the pitch variation amplitude by comparing the above-mentioned pitch variation amplitude with a set value. Among them, the above-mentioned pitch variation trend can also be expressed using a numerical value. For example, each pitch variation trend is represented by a different numerical value, such as 1 for growth, -1 for decrease, and 0 for no change.

[0066] Step S206 , determining at least one fundamental frequency sequence interval in the fundamental frequency sequence according to the pitch change trend corresponding to the fundamental frequency at each time point in the fundamental frequency sequence, wherein the pitch change trend of each fundamental frequency in the fundamental frequency sequence interval is the same and the time points are continuous.

[0067] Among them, each fundamental frequency in the above fundamental frequency sequence may have corresponding pitch information, and the terminal 102 may determine the pitch change trend corresponding to each fundamental frequency based on the pitch information corresponding to each fundamental frequency, and the terminal 102 may also numerically express the above pitch change trend, thereby obtaining the following: Figure 3The pitch change trend diagram shown by the curve 304 in the middle. The terminal 102 can determine multiple base frequency sequence intervals in the base frequency sequence based on the pitch change trend corresponding to the base frequency at each time point in the base frequency sequence. Among them, the pitch change trend in the above base frequency sequence intervals is the same and the time points are continuous. For example, Figure 3 As shown, if the value of the increasing trend is 1, the value of the constant trend is 0, and the value of the decreasing trend is -1, if the terminal 102 needs to find the base frequency sequence interval in the increasing trend, the terminal 102 can Figure 3 The fundamental frequencies with multiple values ​​of 1 and continuous time points form a fundamental frequency sequence interval with an increasing trend; if the terminal 102 needs to find a fundamental frequency sequence interval with a constant trend, the terminal 102 can Figure 3 The fundamental frequencies with multiple values ​​of 0 and continuous time points form a fundamental frequency sequence interval with a constant trend; if the terminal 102 needs to find a fundamental frequency sequence interval with a decreasing trend, the terminal 102 can Figure 3 The terminal 102 can obtain multiple base frequency sequence intervals in the base frequency sequence based on the above method.

[0068] Since each fundamental frequency sequence interval can be composed of multiple consecutive time points, each fundamental frequency sequence interval has a certain duration. Terminal 102 can determine the singing technique contained in the corresponding fundamental frequency sequence within the duration based on the duration of the multiple fundamental frequency sequence intervals and the pitch change value within each fundamental frequency sequence interval. If each fundamental frequency within the fundamental frequency sequence interval has the same pitch change trend, then the pitch at the starting time point and the pitch at the ending time point of the fundamental frequency sequence interval can be different within the duration of the fundamental frequency sequence interval. For example, for a fundamental frequency sequence interval with an increasing trend, the pitch at the ending time point will be greater than the pitch at the starting time point; for a fundamental frequency sequence interval with a decreasing trend, the pitch at the ending time point will be less than the pitch at the starting time point; and for a fundamental frequency sequence interval with a constant trend, the pitch at the ending time point will be equal to the pitch at the starting time point. Terminal 102 can then determine the singing technique within the fundamental frequency sequence interval based on the degree of pitch change within the fundamental frequency sequence interval.

[0069] Step S206 : determining the singing skills included in the fundamental frequency sequence interval according to the duration of the fundamental frequency sequence interval and the pitch variation within the fundamental frequency sequence interval.

[0070] After determining each pitch change trend in the fundamental frequency sequence, terminal 102 can map the pitch change trend to the fundamental frequency sequence, thereby obtaining a pitch change trend sequence corresponding to the fundamental frequency sequence. Since the pitch change trends corresponding to certain time points can be the same, each trend in the pitch change trend sequence can have a certain duration. Terminal 102 can determine the singing skill information contained in the fundamental frequency sequence interval corresponding to the duration based on the duration of each pitch change trend and the pitch change value of the fundamental frequency sequence within the duration. The fundamental frequency sequence can include multiple fundamental frequency sequence intervals, and terminal 102 can detect singing skill information for the fundamental frequency sequence intervals using a presence detection method. Specifically, terminal 102 can detect whether the fundamental frequency sequence interval contains singing skill information that can be detected based on the duration of the pitch change trend and the pitch change value. Specifically, the singing skill information can be specific singing skill information, and terminal 102 can detect whether each fundamental frequency sequence interval contains the specific singing skill information. It should be noted that the terminal can determine multiple singing skills based on the duration of the fundamental frequency sequence interval and the pitch change within the fundamental frequency sequence interval.

[0071] In the above-mentioned singing technique detection method, the audio to be detected is obtained, along with the fundamental frequency sequence corresponding to the audio to be detected, and the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence is obtained. The pitch variation trend corresponding to each fundamental frequency is determined based on the pitch variation amplitude. The singing technique corresponding to the fundamental frequency sequence interval is then determined based on the duration of each pitch variation trend and the pitch variation value of the fundamental frequency sequence within that duration. Compared to traditional deep learning modeling-based approaches for identifying singing technique information in audio, this solution improves the efficiency of singing technique detection by utilizing the pitch variation information of the audio's fundamental frequencies to identify the singing technique present in the audio.

[0072] In one embodiment, the slope of each fundamental frequency in the fundamental frequency sequence is obtained based on the pitch amplitudes at adjacent time points in the fundamental frequency sequence, including: obtaining the difference between the pitch amplitude corresponding to a first time point in the fundamental frequency sequence and the pitch amplitude corresponding to a second time point, and obtaining the ratio of the difference to the pitch amplitude corresponding to the second time point; obtaining the slope information of the fundamental frequency corresponding to the second time point based on the ratio; the first time point and the second time point are adjacent time points in the fundamental frequency sequence and the first time point is greater than the second time point; and obtaining the slopes of the fundamental frequencies corresponding to multiple second time points as the slopes of each fundamental frequency in the fundamental frequency sequence.

[0073] In this embodiment, terminal 102 can determine the slope information of each piece of fundamental frequency information in the fundamental frequency sequence by calculating the pitch change information of every two pieces of fundamental frequency information in the fundamental frequency sequence. Each time point in the fundamental frequency sequence may have a corresponding pitch amplitude. Terminal 102 can obtain the difference between the pitch amplitude information corresponding to a first time point in the fundamental frequency sequence and the pitch amplitude corresponding to a second time point, and then obtain the ratio of this difference to the pitch amplitude corresponding to the second time point. The first and second time points are adjacent time points in the fundamental frequency sequence, and the time value of the first time point is greater than that of the second time point. Terminal 102 can obtain the slope information of the fundamental frequency corresponding to the second time point based on the ratio. That is, terminal 102 can obtain the slope information of the fundamental frequency information at the earlier time point of the two pieces of fundamental frequency information at the adjacent time points based on the ratio. Terminal 102 can perform the above slope calculation on the fundamental frequency information of each group of adjacent time points in the fundamental frequency sequence, thereby obtaining the slopes of the fundamental frequencies at multiple second time points. Terminal 102 can use the multiple slopes as the slopes of each fundamental frequency in the fundamental frequency sequence.

[0074] Specifically, the fundamental frequency sequence can be a time series sequence, and the terminal 102 can calculate the slope information corresponding to the fundamental frequency information at the earlier time point in a group of fundamental frequency information at adjacent time points in the fundamental frequency sequence through a specific formula. The slope calculation formula is as follows: slope = (x(n+1)-x(n)) / x(n). Among them, n is a time point in the fundamental frequency sequence, n+1 is a time point adjacent to time point n and with a time value greater than n, slope is the slope corresponding to the fundamental frequency information at time point n, x(n) can be the pitch information at time point n, and x(n+1) is the pitch information at time point n+1. The terminal 102 can calculate the fundamental frequency slope at the current time point n through the above formula, thereby obtaining the pitch change trend sequence corresponding to the fundamental frequency sequence. After the pitch change trend sequence is expressed in the form of a trend line, a pitch change trend curve can be obtained.

[0075] Through this embodiment, the terminal 102 can determine the slope corresponding to each fundamental frequency information based on the pitch amplitude information corresponding to each adjacent time point, so that the terminal 102 can determine the pitch change trend corresponding to each fundamental frequency based on the slope information, and detect the singing skills in the audio to be detected based on the pitch change trend, thereby improving the efficiency of singing skill detection.

[0076] In one embodiment, the pitch change trend corresponding to each fundamental frequency is determined based on the pitch change amplitude of each fundamental frequency, including: obtaining the comparison result of the slope information of each fundamental frequency and the preset slope value; the preset slope value is used to represent the trend of no change in the pitch value of the fundamental frequency; and determining the pitch change trend corresponding to each fundamental frequency based on the comparison result.

[0077] In this embodiment, the fundamental frequency sequence may include multiple fundamental frequency information, and each fundamental frequency information may have a corresponding pitch amplitude. The terminal 102 may determine the pitch change trend corresponding to each fundamental frequency based on the pitch change amplitude between the multiple fundamental frequency information. The pitch change amplitude may be represented by slope information. The terminal 102 may obtain a comparison result between the slope information corresponding to each fundamental frequency information and a preset slope value, and determine the pitch change trend corresponding to each fundamental frequency based on the comparison result. The preset slope value represents the slope value corresponding to when the pitch value of the fundamental frequency does not change. That is, the terminal 102 may compare the slope information corresponding to each fundamental frequency information with the slope value corresponding when there is no change, thereby determining whether the pitch change trend of each fundamental frequency is a change, and the specific trend information of the change when the pitch change trend changes.

[0078] Through this embodiment, the terminal 102 can determine the pitch change trend of each fundamental frequency in the fundamental frequency sequence corresponding to the audio to be detected by comparing the slope information with the preset slope value, so that the terminal 102 can detect the singing skill information in the fundamental frequency sequence based on the pitch change trend, thereby improving the efficiency of singing skill detection.

[0079] In one embodiment, the pitch change trend corresponding to each fundamental frequency is determined based on the comparison result, including: if the slope information is greater than the preset slope, determining that the pitch change trend of the fundamental frequency corresponding to the slope is an increasing trend; if the slope is less than the preset slope, determining that the pitch change trend of the fundamental frequency corresponding to the slope is a decreasing trend; if the slope is equal to the preset slope, determining that the pitch change trend of the fundamental frequency corresponding to the slope is a constant trend.

[0080] In this embodiment, terminal 102 can determine the pitch change trend corresponding to each fundamental frequency in the fundamental frequency sequence based on a comparison result of the slope information with a preset slope value. The comparison result of the slope information with the preset slope value can have multiple results, and each comparison result can correspond to a trend. For example, the comparison results can include greater than, less than, and equal to. When terminal 102 detects that the slope information of the fundamental frequency is greater than the preset slope value, terminal 102 can determine that the pitch change trend of the fundamental frequency is an increasing trend; when terminal 102 detects that the slope information of the fundamental frequency is less than the preset slope value, terminal 102 can determine that the pitch change trend of the fundamental frequency is a decreasing trend; and when terminal 102 detects that the slope information of the fundamental frequency is equal to the preset slope value, terminal 102 can determine that the pitch change trend of the fundamental frequency is a constant trend. Each of the above pitch change trends can also be numerically represented for better visualization. For example, an increasing trend can be represented as 1, a constant trend as 0, and a decreasing trend as -1. It should be noted that the numerical expression corresponding to each pitch change trend may also be other numerical values. Thus, the terminal 102 may determine the pitch change trend of the fundamental frequency corresponding to each slope information according to the following formula.

[0081] Among them, slpoe(n) is the slope at time point n in the fundamental frequency sequence, and value represents the numerical value of the pitch change trend corresponding to the fundamental frequency.

[0082] After the terminal 102 determines the pitch change trend of each fundamental frequency in the fundamental frequency sequence based on the slope information, the following can be obtained: Figure 3 The image shown, Figure 3 The terminal 102 can classify the pitch change trend based on -1, 0, and 1 based on the numerical values ​​corresponding to the above pitch change trends. Figure 3 Curve 302 represents a fundamental frequency sequence, containing pitch information corresponding to multiple fundamental frequencies. Curve 304 represents a pitch change trend curve. Terminal 102 classifies the fundamental frequencies based on their slope information, resulting in a curve such as curve 304. Curve 304 contains certain line segments parallel to the horizontal axis, indicating a stable pitch change trend within that time period, such as a stable increase, a stable decrease, or a stable constant pitch. Terminal 102 can thus perform singing skill detection based on these curves.

[0083] Through this embodiment, the terminal 102 can determine the different types of pitch change trends corresponding to each fundamental frequency in the fundamental frequency sequence based on the comparison result of the slope information and the preset slope value, so that the terminal 102 can determine the singing skill information in the fundamental frequency sequence based on these pitch change trends, thereby improving the efficiency of singing skill detection.

[0084] Singing skills may include but are not limited to glissando. Therefore, in one embodiment, based on the duration of the fundamental frequency sequence interval and the pitch changes within the fundamental frequency sequence interval, the singing skills contained in the corresponding fundamental frequency sequence interval within the duration are determined, including: for each fundamental frequency sequence interval, if the duration of the fundamental frequency sequence interval is greater than a preset time value, and the difference between the pitch value corresponding to the start time of the fundamental frequency sequence interval and the pitch value corresponding to the end time of the fundamental frequency sequence interval is greater than a preset interval difference threshold, it is determined that the fundamental frequency sequence interval contains the glissando skill.

[0085] In this embodiment, the terminal 102 can obtain multiple baseband sequence intervals from the above-mentioned baseband sequence, and each baseband sequence interval has the same pitch change trend. For each baseband sequence interval in the baseband sequence, the terminal 102 can detect the duration of the baseband sequence interval and the pitch change value within the baseband sequence interval to determine the singing skills in the baseband sequence interval. Among them, the singing skills contained in the above-mentioned baseband sequence interval can be glissando skills. If the terminal 102 detects that the duration of a baseband sequence interval is greater than a preset time value, and the difference between the pitch value corresponding to the start time of the baseband sequence interval and the pitch value corresponding to the end time of the baseband sequence interval is greater than a preset interval difference threshold, then the terminal 102 can determine that there is a glissando skill in the baseband sequence interval. That is, the terminal 102 can detect whether there is a glissando skill in the audio to be detected.

[0086] Specifically, the terminal 102 can determine whether there is portamento in the audio by setting two thresholds, including a portamento duration threshold and a portamento amplitude threshold. Figure 4 As shown, Figure 4 Schematic diagram of an interface for determining singing technique steps in one embodiment. Gliding is the process of changing from one pitch to another in a gliding-like manner, such as Figure 4 Therefore, the terminal 102 needs to detect whether there is a certain value of interval difference between the pitch at the start time and the end time in the above-mentioned fundamental frequency sequence interval, for example, an interval difference greater than one semitone, and the duration of the fundamental frequency sequence interval needs to be greater than a preset time value, for example, greater than 100ms. When both of the above two conditions are met, the terminal 102 can determine that there is a glissando technique in the fundamental frequency sequence interval.

[0087] In addition, the terminal can also determine other singing skills based on the duration of the above-mentioned fundamental frequency sequence interval and the pitch change within the fundamental frequency sequence interval. For example, the terminal can detect whether the duration of the fundamental frequency sequence interval is greater than the time threshold, and detect whether the pitch after the pitch change reaches the first preset pitch threshold. When the duration of the fundamental frequency sequence interval is greater than the time threshold and the pitch after the change in the fundamental frequency sequence interval is greater than the first preset pitch threshold, it can be determined that there are high-pitched skills in the fundamental frequency sequence interval. Similarly, when the terminal detects that the duration of the fundamental frequency sequence interval is greater than the time threshold and the pitch after the change in the fundamental frequency sequence interval is less than or equal to the second preset pitch threshold, it can be determined that there are bass skills in the fundamental frequency sequence interval. The first preset pitch threshold can be greater than the second preset pitch threshold. It should be noted that the singing skills detected by the terminal through the duration of the above-mentioned fundamental frequency sequence interval and the pitch change within the fundamental frequency sequence interval are not limited to the above-mentioned singing skills.

[0088] Through this embodiment, the terminal 102 can determine the glissando technique and other included singing techniques in the audio to be detected based on the duration of the fundamental frequency sequence interval and the pitch change value in the interval, thereby improving the efficiency of singing technique detection.

[0089] In one embodiment, Figure 5 As shown, Figure 5 The figure is a flow chart of a singing skill detection method in another embodiment. In this embodiment, the terminal 102 can first obtain the audio to be detected input by the user through a recording device, such as the singing audio input by the user. The terminal 102 can perform fundamental frequency detection on the audio to be detected, thereby extracting the fundamental frequency sequence corresponding to the human voice. The fundamental frequency sequence can be a time series sequence. The terminal 102 can calculate the slope information of each fundamental frequency in the fundamental frequency sequence using the above-mentioned slope formula, and classify the slope information of the fundamental frequency according to the size of the slope information of each fundamental frequency based on the above-mentioned classification formula to obtain a pitch change trend curve. The terminal 102 can use the pitch change trend curve to obtain multiple fundamental frequency sequence intervals formed by fundamental frequencies with the same pitch change trend and continuous time points. The terminal 102 can determine whether there is a glissando technique in the fundamental frequency sequence interval by judging whether the duration of the fundamental frequency sequence interval is greater than a preset time threshold and whether the pitch change amplitude in the fundamental frequency sequence interval is greater than a preset interval difference. The above-mentioned method for detecting glissando skills can be used in the user's singing practice stage, and positive feedback can be given to the user by returning the glissando detection results in real time; singing skill detection can also be applied in intelligent tuning scenarios.

[0090] Through this embodiment, the terminal 102 can determine the singing skills present in the audio by utilizing the pitch change information of the fundamental frequency of the audio, without having to train the detection model using labeled data, thereby improving the efficiency of singing skill detection.

[0091] In one embodiment, Figure 6 As shown, a singing skill display method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0092] Step S302: Obtain the user's audio to be detected according to the recording instruction input by the user.

[0093] The recording command can be used in a variety of scenarios. For example, if a user uses the singing practice function in the terminal, the recording command can be the recording command in the singing practice function; if a user uses the intelligent audio adjustment function in the terminal, the recording command can be the recording command in the intelligent audio adjustment function. After receiving the recording command input by the user, the terminal can obtain the user's audio to be tested, such as obtaining the user's singing audio.

[0094] Step S304: Detect the singing skills contained in the audio to be detected by using the above-mentioned singing skills detection method.

[0095] The audio to be detected may include singing techniques, such as glissando or other singing techniques. The terminal may detect the singing techniques in the audio to be detected using the singing technique detection method described above, thereby detecting the singing techniques contained in the audio to be detected. For example, if the audio to be detected includes glissando, the terminal may detect the glissando in the audio to be detected using the singing technique detection method described above.

[0096] Step S306: display the singing skills contained in the audio to be detected.

[0097] Among them, users can detect singing skills in various functions. For example, users can detect singing skills such as glissando in recorded audio during the song practice stage; users can also detect singing skills such as glissando in recorded audio in the smart tuning function. After detecting the singing skills contained in the above-mentioned audio to be tested, the terminal can display the detected singing skills. For example, when the glissando skill is detected in the audio to be tested, if the singing practice function is used, the glissando skill detection result will be returned in real time. If the smart tuning function is used, the glissando skill detection result will be displayed in the singing skill detection interface.

[0098] In the above-mentioned singing technique display method, when the user wants to record, the audio to be detected is obtained, and the fundamental frequency sequence corresponding to the audio to be detected is obtained. The pitch change amplitude of each fundamental frequency in the fundamental frequency sequence is obtained, and the pitch change trend corresponding to each fundamental frequency is determined based on the pitch change amplitude. Then, based on the duration of each pitch change trend and the pitch change value of the fundamental frequency sequence within the duration, the corresponding singing technique in the fundamental frequency sequence interval is determined, and the singing technique is output and displayed. Compared with traditional deep learning modeling-based methods for identifying singing technique information in audio, this solution can improve the efficiency of singing technique detection by utilizing the pitch change information of the audio fundamental frequency to determine the singing technique present in the audio.

[0099] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0100] Based on the same inventive concept, embodiments of the present application further provide a singing skill detection device for implementing the aforementioned singing skill detection method, and a singing skill demonstration device for implementing the aforementioned singing skill demonstration method. The implementation solutions provided by these devices are similar to those described in the aforementioned methods. Therefore, the specific limitations of one or more of the following embodiments of the singing skill detection device and the singing skill demonstration device can be found in the above-described limitations of the singing skill detection method and the singing skill demonstration method, and will not be further elaborated here.

[0101] In one embodiment, Figure 7 As shown, a singing skill detection device is provided, comprising: a first acquisition module 500, a second acquisition module 502 and a detection module 504, wherein:

[0102] The first acquisition module 500 is configured to acquire the audio to be detected and a fundamental frequency sequence of the audio to be detected. The fundamental frequency sequence is a sequence obtained by sorting the fundamental frequencies of the audio to be detected at different time points in chronological order.

[0103] The second acquisition module 502 is configured to acquire the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence, and determine the pitch variation trend of each fundamental frequency according to the pitch variation amplitude of each fundamental frequency.

[0104] The interval determination module 504 is configured to determine at least one fundamental frequency sequence interval in the fundamental frequency sequence according to the pitch change trend corresponding to the fundamental frequency at each time point in the fundamental frequency sequence, wherein the pitch change trend of each fundamental frequency in the fundamental frequency sequence interval is the same and the time points are continuous.

[0105] The detection module 506 is configured to determine the singing skills included in the fundamental frequency sequence interval according to the duration of the fundamental frequency sequence interval and the pitch variation within the fundamental frequency sequence interval.

[0106] In one embodiment, the first acquisition module 500 is specifically configured to acquire the audio to be detected and perform human voice detection on the audio to be detected to obtain a fundamental frequency sequence corresponding to the human voice in the audio to be detected.

[0107] In one embodiment, the second acquisition module 502 is specifically configured to obtain a slope of each fundamental frequency in the fundamental frequency sequence based on the pitch amplitudes at adjacent time points in the fundamental frequency sequence; and determine a pitch variation amplitude of each fundamental frequency based on the slope of each fundamental frequency.

[0108] In one embodiment, the above-mentioned second acquisition module 502 is specifically used to obtain the difference between the pitch amplitude corresponding to the first time point in the fundamental frequency sequence and the pitch amplitude corresponding to the second time point, and obtain the ratio of the difference to the pitch amplitude corresponding to the second time point; the first time point and the second time point are adjacent time points in the fundamental frequency sequence and the first time point is greater than the second time point; the slope of the fundamental frequency corresponding to the second time point is obtained according to the ratio; the slopes of the fundamental frequencies corresponding to multiple second time points are obtained as the slope information of each fundamental frequency in the fundamental frequency sequence.

[0109] In one embodiment, the second acquisition module 502 is specifically used to obtain a comparison result between the slope of each fundamental frequency and a preset slope; the preset slope is used to characterize the trend of no change in the pitch value of the fundamental frequency; and the pitch change trend corresponding to each fundamental frequency is determined based on the comparison result.

[0110] In one embodiment, the second acquisition module 502 is specifically configured to determine that the pitch change trend of the fundamental frequency is an increasing trend if the slope of the fundamental frequency is greater than the preset slope; determine that the pitch change trend of the fundamental frequency is a decreasing trend if the slope of the fundamental frequency is less than the preset slope; and determine that the pitch change trend of the fundamental frequency is a constant trend if the slope of the fundamental frequency is equal to the preset slope.

[0111] In one embodiment, the above-mentioned detection module 504 is specifically used to determine the singing skills contained in the fundamental frequency sequence interval based on the duration of the fundamental frequency sequence interval and the pitch changes within the fundamental frequency sequence interval, including: for each fundamental frequency sequence interval, if the duration of the fundamental frequency sequence interval is greater than a preset time, and the difference between the pitch value corresponding to the start time of the fundamental frequency sequence interval and the pitch value corresponding to the end time of the fundamental frequency sequence interval is greater than a preset interval difference threshold, it is determined that the fundamental frequency sequence interval contains the glissando technique.

[0112] In one embodiment, Figure 8 As shown, a singing skill display device is provided, comprising: an instruction acquisition module 600, a skill detection module 602 and an output module 604, wherein:

[0113] The instruction acquisition module 600 is used to acquire the user's audio to be detected according to the recording instruction input by the user.

[0114] The skill detection module 602 is used to detect the singing skills contained in the audio to be detected by using the above-mentioned singing skill detection method.

[0115] The output module 604 is used to display the singing skills contained in the audio to be detected.

[0116] Each module in the aforementioned singing skill detection device and singing skill demonstration device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in the form of hardware, or may be stored in a memory within the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0117] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown. The computer device includes a processor, memory, communication interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a singing skill detection method and a singing skill display method are implemented. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0118] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0119] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned singing skill detection method and singing skill display method when executing the computer program.

[0120] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned singing skill detection method and singing skill display method are implemented.

[0121] In one embodiment, a computer program product is provided, including a computer program, which implements the above-mentioned singing skill detection method and singing skill display method when executed by a processor.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0123] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0124] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A singing skill detection method, characterized in that: The method comprises: Acquire the audio to be detected and a fundamental frequency sequence of the audio to be detected, where the fundamental frequencies of different time points in the audio to be detected are sorted in chronological order; Obtaining the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence, and determining the pitch variation trend of each fundamental frequency according to the pitch variation amplitude of each fundamental frequency; Determining at least one fundamental frequency sequence interval in the fundamental frequency sequence according to the pitch change trend corresponding to the fundamental frequency at each time point in the fundamental frequency sequence, wherein the pitch change trend of each fundamental frequency in the fundamental frequency sequence interval is the same and the time points are continuous; The singing technique contained in the fundamental frequency sequence interval is determined based on the duration of the fundamental frequency sequence interval and the pitch change within the fundamental frequency sequence interval, including: detecting whether the duration is greater than a preset time and whether the amplitude of the pitch change is greater than a preset interval difference threshold, to determine whether a glissando technique exists; detecting whether the duration is greater than a time threshold and whether the pitch after the pitch change is greater than a first preset pitch threshold or less than or equal to a second preset pitch threshold, to determine whether a high pitch technique or a low pitch technique exists.

2. The method according to claim 1, characterized in that The obtaining of the audio to be detected and the fundamental frequency sequence of the audio to be detected includes: Acquire the audio to be detected and perform human voice detection on the audio to be detected to obtain a fundamental frequency sequence corresponding to the human voice in the audio to be detected.

3. The method according to claim 1, characterized in that The obtaining of the pitch variation amplitude of each fundamental frequency in the fundamental frequency sequence includes: Obtaining the slope of each fundamental frequency in the fundamental frequency sequence according to the pitch amplitudes at adjacent time points in the fundamental frequency sequence; The pitch variation amplitude of each fundamental frequency is determined according to the slope of each fundamental frequency.

4. The method according to claim 3, characterized in that Obtaining the slope of each fundamental frequency in the fundamental frequency sequence according to the pitch amplitudes at adjacent time points in the fundamental frequency sequence includes: Obtaining a difference between a pitch amplitude corresponding to a first time point in the fundamental frequency sequence and a pitch amplitude corresponding to a second time point, and obtaining a ratio of the difference to the pitch amplitude corresponding to the second time point; the first time point and the second time point are adjacent time points in the fundamental frequency sequence, and the first time point is greater than the second time point; obtaining a slope of the fundamental frequency corresponding to the second time point according to the ratio; The slopes of the fundamental frequencies corresponding to the multiple second time points are obtained as the slopes of the fundamental frequencies in the fundamental frequency sequence.

5. The method according to claim 3, characterized in that Determining the pitch change trend of each fundamental frequency according to the pitch change amplitude of each fundamental frequency includes: Obtaining a comparison result between the slope of each fundamental frequency and a preset slope; the preset slope is used to indicate a trend that the pitch value of the fundamental frequency does not change; The pitch change trends corresponding to the respective fundamental frequencies are determined according to the comparison results.

6. The method according to claim 5, characterized in that Determining the pitch change trends corresponding to the respective fundamental frequencies according to the comparison results includes: If the slope of the fundamental frequency is greater than the preset slope, determining that the pitch change trend of the fundamental frequency is an increasing trend; If the slope of the fundamental frequency is less than the preset slope, determining that the pitch change trend of the fundamental frequency is a decreasing trend; If the slope of the fundamental frequency is equal to the preset slope, it is determined that the pitch change trend of the fundamental frequency is a constant trend.

7. The method according to claim 1, characterized in that The step of determining the singing skills included in the fundamental frequency sequence interval according to the duration of the fundamental frequency sequence interval and the pitch change within the fundamental frequency sequence interval includes: For each fundamental frequency sequence interval, if the duration of the fundamental frequency sequence interval is greater than the preset time, and the difference between the pitch value corresponding to the start time of the fundamental frequency sequence interval and the pitch value corresponding to the end time of the fundamental frequency sequence interval is greater than the preset interval difference threshold, it is determined that the fundamental frequency sequence interval contains glissando technique.

8. A method for demonstrating singing skills, characterized in that: The method comprises: Obtain the user's audio to be tested according to the recording command input by the user; Detecting the singing skills contained in the audio to be detected by the singing skill detection method according to any one of claims 1 to 7; Demonstrate the singing skills contained in the audio to be detected.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images