Singing processing method, device, equipment, medium, program product and vehicle

Abstract

The invention relates to a singing processing method, device and equipment, a medium, a program product and a vehicle, and the method comprises the steps: determining the current fundamental frequency of a current frame voice signal in a target singing voice stream; and determining the maximum fundamental frequency and the minimum fundamental frequency of the target singing voice stream according to the current fundamental frequency and the historical fundamental frequency of each historical frame voice signal. And then determining whether the current fundamental frequency is in a target frequency interval, and if the current fundamental frequency is determined to be in the target frequency interval, correcting the current frame voice signal according to a preset equalization effect value corresponding to the target frequency interval to obtain a current frame target voice signal. Wherein the target frequency interval comprises a first frequency interval corresponding to the maximum fundamental frequency or a second frequency interval corresponding to the minimum fundamental frequency. According to the technical scheme, the flexibility of beautifying the song and the naturalness of the beautified song can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of speech processing, and in particular to a singing voice processing method, device, equipment, medium, program product and vehicle. Background Art

[0002] With the continuous advancement of intelligent technology, the scene of Karaoke has expanded from the traditional KTV box to a more diverse and personalized environment, such as car KTV. These new KTV scenes not only provide a more convenient singing experience, but also put forward higher requirements for singing voice beautification technology to meet users' pursuit of sound quality and expressiveness.

[0003] At present, the singing voice beautification technology mainly prepares an original template for the singing voice in advance, and adjusts the pitch of the singer's singing to be consistent with the original template, so as to achieve the effect of beautifying the singing voice.

[0004] However, the existing technology has low flexibility in beautifying singing voices and low naturalness of the singing voices after beautification. Summary of the invention

[0005] One of the objects of the present invention is to provide a singing voice processing method to solve the technical problems in the prior art of flexibility in singing voice beautification and low naturalness of the singing voice after beautification; the second object is to provide a singing voice processing device; the third object is to provide an electronic device; the fourth object is to provide a computer-readable storage medium; the fifth object is to provide a computer program product; the sixth object is to provide a vehicle.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A singing voice processing method, comprising:

[0008] Determine the current fundamental frequency of the current frame speech signal in the target singing voice stream;

[0009] Determine the maximum fundamental frequency and the minimum fundamental frequency of the target singing voice stream according to the current fundamental frequency and the historical fundamental frequencies of each historical frame voice signal;

[0010] Determine whether the current fundamental frequency is in a target frequency interval, where the target frequency interval includes a first frequency interval corresponding to the maximum fundamental frequency or a second frequency interval corresponding to the minimum fundamental frequency;

[0011] If it is determined that the current base frequency is in the target frequency interval, the current frame speech signal is modified according to a preset equalization effect value corresponding to the target frequency interval to obtain a current frame target speech signal.

[0012] According to the above technical means, when the pitch of the singing voice is in the high and low notes, the current frame voice signal is beautified and enhanced. When the pitch is in the normal range, no processing is done on the current frame voice signal. And there is no need to perform a large amount of calculation analysis on the rhythm of the singing content to ensure real-time performance. In addition, this technical solution does not require the use of native templates, thereby ensuring the singer's tonality flexibility in different pitch ranges, and will not introduce distortion due to pitch correction, thereby improving the flexibility of beautifying the singing voice and the naturalness of the beautified singing voice.

[0013] Further, determining whether the current base frequency is within a target frequency interval includes:

[0014] Determining whether the current fundamental frequency is equal to the maximum fundamental frequency or the minimum fundamental frequency;

[0015] If the current fundamental frequency is equal to the maximum fundamental frequency or the minimum fundamental frequency, it is determined that the current fundamental frequency is in the target frequency interval.

[0016] According to the above technical means, considering that it takes a certain amount of processing time to update the first frequency interval, the second frequency interval, and the third frequency interval, when the current base frequency is equal to the maximum base frequency, it can be determined that the current frame voice signal is definitely a high pitch, and when the current base frequency is equal to the minimum base frequency, it can be determined that the current frame voice signal is definitely a low pitch. Therefore, it is possible to first determine whether the current base frequency is equal to the maximum base frequency or the minimum base frequency, and when the current base frequency is equal to the maximum base frequency or the minimum base frequency, directly determine that the current frame voice signal is a high pitch or a low pitch, saving subsequent processing processes and improving processing efficiency.

[0017] Further, determining whether the current base frequency is within a target frequency interval includes:

[0018] If the current fundamental frequency is less than the maximum fundamental frequency and greater than the minimum fundamental frequency, determining a first frequency interval corresponding to the maximum fundamental frequency and a second frequency interval corresponding to the minimum fundamental frequency from a plurality of preset frequency intervals;

[0019] If the current base frequency is in the first frequency interval or the second frequency interval, it is determined that the current base frequency is in the target frequency interval.

[0020] According to the above technical means, when the current base frequency is neither the maximum base frequency nor the minimum base frequency, it is necessary to further determine whether the current base frequency is in the first frequency interval corresponding to the maximum base frequency and the second frequency interval corresponding to the minimum base frequency to determine whether the current base frequency needs to be beautified and corrected.

[0021] Further, determining the first frequency interval corresponding to the maximum fundamental frequency and the second frequency interval corresponding to the minimum fundamental frequency from a plurality of preset frequency intervals includes:

[0022] Calculating a first distance between the maximum fundamental frequency and each preset frequency interval, and a second distance between the minimum fundamental frequency and each preset frequency interval;

[0023] Determine a preset frequency interval in which the first distance is the smallest as the first frequency interval;

[0024] The preset frequency interval in which the second distance is the smallest is determined as the second frequency interval.

[0025] According to the above technical means, since the distance between the frequency and the frequency interval can represent the similarity between the two, the first distance between the maximum fundamental frequency and each preset frequency interval and the second distance between the minimum fundamental frequency and each preset frequency interval can be calculated to determine the first frequency interval corresponding to the maximum fundamental frequency and the second frequency interval corresponding to the minimum fundamental frequency, thereby improving processing efficiency.

[0026] Furthermore, the method further comprises:

[0027] Calculating a first frequency difference between the maximum fundamental frequency and the minimum fundamental frequency;

[0028] If the first frequency difference is greater than a first preset frequency difference, determining the first frequency interval and a preset frequency interval adjacent to the first frequency interval and smaller than the first frequency interval as a new first frequency interval;

[0029] If the first frequency difference is greater than a first preset frequency difference, the second frequency interval and a preset frequency interval adjacent to the second frequency interval and smaller than the second frequency interval are determined as new second frequency intervals.

[0030] According to the above technical means, in the case of large pitch changes, the treble interval (first frequency interval) and the bass interval (second frequency interval) are dynamically relaxed, thereby improving the accuracy of subsequent determination of whether the current frame speech signal is bass or treble, thereby improving the accuracy of beautification processing of the current frame speech signal.

[0031] Further, the determining of the current fundamental frequency of the current frame speech signal in the target singing voice stream comprises:

[0032] Calculating a differential function of the current frame speech signal;

[0033] According to the speech existence probability of the current frame speech signal, the preset absolute threshold is modified to obtain a modified absolute threshold;

[0034] The current fundamental frequency is determined according to the corrected absolute threshold, the differential function of the current frame speech signal and the sampling rate of the target singing speech stream.

[0035] According to the above technical means, the preset absolute threshold is corrected using the speech existence probability of the current frame speech signal, so that the current fundamental frequency is calculated according to the dynamically changing absolute threshold (corrected absolute threshold), which effectively improves the robustness of the fundamental frequency estimation.

[0036] Further, the calculating the differential function of the current frame speech signal includes:

[0037] Performing Fourier transform on the current frame speech signal to obtain the current frame frequency domain speech signal;

[0038] Calculating the energy of the current frame frequency domain speech signal according to the current frame frequency domain speech signal;

[0039] If the energy is greater than the preset energy, a differential function of the current frame speech signal is calculated.

[0040] According to the above technical means, before calculating the current fundamental frequency of the current frame speech signal, it is also possible to detect whether there is speech in the current frame speech signal. When it is determined that there is speech in the current frame speech signal, subsequent beautification and correction processing is performed. If it is determined that there is no speech in the current frame speech signal, subsequent beautification and correction processing is not performed, thereby avoiding a lot of redundant processing and operations, as well as the problem of erroneous correction of the pitch of noise, and improving processing efficiency and accuracy.

[0041] Further, if it is determined that the current fundamental frequency is in the target frequency interval, the current frame speech signal is corrected according to a preset equalization effect value corresponding to the target frequency interval to obtain the current frame target speech signal, including:

[0042] If the target frequency interval is the first frequency interval, the current frame frequency domain speech signal is corrected according to the first equalization effect curve and the preset harmonic ratio to obtain the current frame target speech signal;

[0043] If the target frequency interval is the second frequency interval, the current frame frequency domain speech signal is corrected according to the second equalization effect curve to obtain the current frame target speech signal;

[0044] The equalization effect curve is used to represent the mapping relationship between the frequency and the preset equalization effect value.

[0045] According to the above technical means, the nonlinear function can make the current frame target speech signal produce rich harmonic components, including odd and even harmonics. After the current frame speech signal passes through the nonlinear function and is added to the current frame speech signal, the amplitude of the odd and even harmonics contained in the current frame target speech signal can be enhanced, and the mid-high frequency harmonics of the current frame target speech signal can be introduced to improve the fullness of the treble.

[0046] Further, the determining of the current fundamental frequency of the current frame speech signal in the target singing voice stream comprises:

[0047] Determine the current initial fundamental frequency of the current frame speech signal;

[0048] If a second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame of speech signal is less than a second preset frequency difference, determining the current initial fundamental frequency as the current fundamental frequency of the current frame of speech signal;

[0049] If the second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame speech signal is greater than or equal to the second preset frequency difference, the historical fundamental frequency of the previous frame speech signal is determined as the current fundamental frequency of the current frame speech signal.

[0050] According to the above technical means, by limiting the jumping range of the fundamental frequency, only the initial fundamental frequency estimated within the limited range (the second preset frequency difference) will be used, which can effectively resist the fundamental frequency estimation jump when the speech starts and ends, and improve the accuracy of processing.

[0051] Further, the determining of the current fundamental frequency of the current frame speech signal in the target singing voice stream comprises:

[0052] Determine the current initial fundamental frequency of the current frame speech signal;

[0053] If a second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame of speech signal is less than a second preset frequency difference, determining the current initial fundamental frequency as the current fundamental frequency of the current frame of speech signal;

[0054] If the second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame speech signal is greater than or equal to the second preset frequency difference, the historical fundamental frequency of the previous frame speech signal is determined as the current fundamental frequency of the current frame speech signal.

[0055] According to the above technical means, dynamic energy control of the target voice signal of the current frame is performed through the target gain, which helps to adjust the dynamic range of the target singing voice stream, that is, adjust the difference between the strongest and weakest parts in the target singing voice stream. By adjusting the target gain, the dynamic range of the target singing voice stream can be made more consistent, reducing abrupt energy changes, and making the overall listening experience smoother and more pleasant.

[0056] A singing voice processing device, comprising:

[0057] A first determination module is used to determine the current fundamental frequency of the current frame speech signal in the target singing voice stream;

[0058] A second determination module is used to determine the maximum fundamental frequency and the minimum fundamental frequency of the target singing voice stream according to the current fundamental frequency and the historical fundamental frequencies of the voice signals of each historical frame;

[0059] A third determination module is used to determine whether the current fundamental frequency is in a target frequency interval, where the target frequency interval includes a first frequency interval corresponding to the maximum fundamental frequency or a second frequency interval corresponding to the minimum fundamental frequency;

[0060] The correction module is used to correct the current frame speech signal according to the preset equalization effect value corresponding to the target frequency interval if it is determined that the current base frequency is in the target frequency interval, so as to obtain the current frame target speech signal.

[0061] Further, the third determination module is specifically used to:

[0062] Determining whether the current fundamental frequency is equal to the maximum fundamental frequency or the minimum fundamental frequency;

[0063] If the current fundamental frequency is equal to the maximum fundamental frequency or the minimum fundamental frequency, determining that the current fundamental frequency is in the target frequency interval;

[0064] If the current fundamental frequency is less than the maximum fundamental frequency and greater than the minimum fundamental frequency, a first frequency interval corresponding to the maximum fundamental frequency and a second frequency interval corresponding to the minimum fundamental frequency are determined from a plurality of preset frequency intervals.

[0065] Further, the third determination module is specifically used to:

[0066] If the current base frequency is in the first frequency interval or the second frequency interval, it is determined that the current base frequency is in the target frequency interval.

[0067] Further, the third determination module is specifically used to:

[0068] Calculating a first distance between the maximum fundamental frequency and each preset frequency interval, and a second distance between the minimum fundamental frequency and each preset frequency interval;

[0069] Determine a preset frequency interval in which the first distance is the smallest as the first frequency interval;

[0070] The preset frequency interval in which the second distance is the smallest is determined as the second frequency interval.

[0071] Furthermore, the singing voice processing device further includes a fourth determining module, which is used to:

[0072] Calculating a first frequency difference between the maximum fundamental frequency and the minimum fundamental frequency;

[0073] If the first frequency difference is greater than a first preset frequency difference, determining the first frequency interval and a preset frequency interval adjacent to the first frequency interval and smaller than the first frequency interval as a new first frequency interval;

[0074] If the first frequency difference is greater than a first preset frequency difference, the second frequency interval and a preset frequency interval adjacent to the second frequency interval and smaller than the second frequency interval are determined as new second frequency intervals.

[0075] Further, the first determining module is specifically configured to:

[0076] Calculating a differential function of the current frame speech signal;

[0077] According to the speech existence probability of the current frame speech signal, the preset absolute threshold is modified to obtain a modified absolute threshold;

[0078] The current fundamental frequency is determined according to the corrected absolute threshold, the differential function of the current frame speech signal and the sampling rate of the target singing speech stream.

[0079] Further, the first determining module is specifically configured to:

[0080] Performing Fourier transform on the current frame speech signal to obtain the current frame frequency domain speech signal;

[0081] Calculating the energy of the current frame frequency domain speech signal according to the current frame frequency domain speech signal;

[0082] If the energy is greater than the preset energy, a differential function of the current frame speech signal is calculated.

[0083] Furthermore, the correction module is specifically used for:

[0084] If the target frequency interval is the first frequency interval, the current frame frequency domain speech signal is corrected according to the first equalization effect curve and the preset harmonic ratio to obtain the current frame target speech signal;

[0085] If the target frequency interval is the second frequency interval, the current frame frequency domain speech signal is corrected according to the second equalization effect curve to obtain the current frame target speech signal;

[0086] The equalization effect curve is used to represent the mapping relationship between the frequency and the preset equalization effect value.

[0087] Further, the first determining module is specifically configured to:

[0088] The current base frequency of the current frame speech signal, including:

[0089] Determine the current initial fundamental frequency of the current frame speech signal;

[0090] If a second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame of speech signal is less than a second preset frequency difference, determining the current initial fundamental frequency as the current fundamental frequency of the current frame of speech signal;

[0091] If the second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame speech signal is greater than or equal to the second preset frequency difference, the historical fundamental frequency of the previous frame speech signal is determined as the current fundamental frequency of the current frame speech signal.

[0092] Furthermore, the singing voice processing device further includes a gain module, which is used to:

[0093] Determining a target gain of the current frame speech signal according to the energy corresponding to the current frame speech signal and the historical energy corresponding to each historical frame speech signal;

[0094] According to the target gain and the current frame target speech signal, a current frame target speech signal after gain is obtained.

[0095] An electronic device comprises: a processor, a memory and computer execution instructions stored in the memory and executable on the processor, wherein the processor is used to implement the above-mentioned singing voice processing method when executing the computer execution instructions.

[0096] A computer-readable storage medium stores computer-executable instructions, which are used to implement the above-mentioned singing voice processing method when executed by a processor.

[0097] A computer program product includes a computer program, and when the computer program is executed by a processor, it is used to implement the above-mentioned singing voice processing method.

[0098] A vehicle comprises the above-mentioned singing voice processing method.

[0099] Beneficial effects of the present invention:

[0100] (1) The present invention does not need to use native templates, thereby ensuring the singer's tonality flexibility in different voice zones, and will not introduce distortion due to pitch correction, thereby improving the flexibility of singing beautification and the naturalness of the singing after beautification; (2) When the singing pitch is in the high and low notes, the current frame voice signal is beautified and enhanced. When the pitch is in the normal range, the current frame voice signal is not processed. In addition, there is no need to perform a large amount of calculation analysis on the rhythm of the singing content, ensuring real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] Figure 1 A schematic diagram of a flow chart of a first embodiment of a singing voice processing method provided by an embodiment of the present invention;

[0102] Figure 2 A schematic diagram of a flow chart of a second embodiment of a singing voice processing method provided by an embodiment of the present invention;

[0103] Figure 3 A schematic diagram of a flow chart of a third embodiment of a singing voice processing method provided by an embodiment of the present invention;

[0104] Figure 4 A schematic diagram of a differential function provided by an embodiment of the present invention;

[0105] Figure 5 A flowchart of a fourth embodiment of a singing voice processing method provided by an embodiment of the present invention;

[0106] Figure 6 A flowchart of a fifth embodiment of a singing voice processing method provided by an embodiment of the present invention;

[0107] Figure 7 A flowchart of a sixth embodiment of a singing voice processing method provided by an embodiment of the present invention;

[0108] Figure 8 A flow chart of Embodiment 7 of a singing voice processing method provided by an embodiment of the present invention;

[0109] Fig. 9 A structural schematic diagram of a singing voice processing device provided by an embodiment of the present invention;

[0110] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0111] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0112] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and corresponding operation entrances shall be provided for users to choose to authorize or refuse.

[0113] With the continuous advancement of intelligent technology, karaoke scenes have expanded from traditional KTV boxes to more diverse and personalized environments. For example, people can sing karaoke in real time on live broadcast platforms and interact with the audience; they can also enjoy the fun of music through the car KTV system while driving; they can also use the karaoke software installed on their mobile phones to record their singing. These new karaoke scenes not only provide a more convenient singing experience, but also put forward higher requirements for singing beautification technology to meet users' pursuit of sound quality and expressiveness.

[0114] In the current karaoke scene, vocal beautification technology is mainly divided into two directions: real-time processing and non-real-time processing, which mainly corrects the singing in terms of pitch, timbre, reverberation, rhythm and so on.

[0115] Among them, non-real-time processing technology is suitable for users to perform post-processing after singing. After the singing, users can save the singing voice through the recording function in the home KTV system and K song software, and then beautify the singing voice. Real-time processing technology is suitable for scenarios that require instant feedback, such as live KTV and car KTV. In these scenarios, the sound must be processed in real time while the singer is singing.

[0116] Whether it is real-time processing or non-real-time processing, for the vocal pitch, it is mainly to prepare a native template for the vocals in advance and adjust the pitch of the singer's singing to be consistent with the native template. However, since the pitch in the native template is fixed, when the singer sings in different tonality ranges, the pitch will be forced to be adjusted to the tonality range of the template, which often causes the pitch-adjusted singing to sound unnatural, and even lose the singer's original style and emotional expression.

[0117] In summary, the existing technology has the problem of limited flexibility in beautifying singing voices and low naturalness of the beautified singing voices.

[0118] Based on the above technical problems, the technical concept of the present invention is as follows: for most singers, it is difficult to sing high and low notes, and the demand for beautification of singing voice is high; when the pitch is in the middle range (the interval except high and low notes in singing voice), the singing difficulty is low, and the demand for beautification of singing voice is also low. Therefore, for each frame of voice signal in the singing voice stream, it can be determined based on the singing voice stream whether the pitch of the frame voice signal is high or low, and then the high and low notes are corrected, and the middle range is not corrected. In this way, the purpose of improving the beauty of singing voice can be achieved without using a stereotyped native template to beautify the singing voice, avoiding the distortion problem caused by pitch correction, retaining the naturalness and authenticity of the singing voice in the middle range, as well as the voice characteristics and personality of the singer, making the corrected singing voice more recognizable and attractive, and improving the singer's tonality flexibility in different ranges and the naturalness of the beautified singing voice.

[0119] The technical solution of the present invention is described in detail below through specific embodiments.

[0120] It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0121] Figure 1 Schematic diagram of the process flow of the first embodiment of the singing voice processing method provided by the embodiment of the present invention. Figure 1 As shown, the singing voice processing method may include the following steps:

[0122] S11, determining the current fundamental frequency of the current frame speech signal in the target singing voice stream.

[0123] It is understandable that the execution subject of the embodiment of the present invention is an electronic device, which may be a terminal device, such as a computer, a tablet computer, a vehicle-mounted terminal, etc., or a server, such as a background processing platform, etc. Therefore, this embodiment is explained by collectively referring to the terminal device and the server as an electronic device, and whether the electronic device is specifically a terminal device or a server can be determined according to actual conditions.

[0124] The target song voice stream may be an audio stream acquired in real time while the user is singing the target song; or it may be a complete audio stream acquired after the user has finished singing the target song.

[0125] Taking the in-car KTV scene as an example, a karaoke system is set up inside the vehicle, and users can order songs based on the karaoke system. The karaoke system responds to the user's song ordering operation, plays the accompaniment audio of the target song through the audio system, displays the lyrics of the target song and corresponding control controls (such as song switching controls, original singing controls, accompaniment volume increase controls, etc.) through the display, and collects the user's voice through the audio acquisition device. Afterwards, during the user's singing, each frame of the user's voice signal can be collected in real time, so that each frame of the voice signal can be beautified in real time, so that the user and the audience can immediately hear the processed sound effect. It is also possible to beautify the singing according to the complete target singing voice stream obtained after the singing is completed after the user finishes singing, so that the beautified singing can be published and disseminated.

[0126] In a possible implementation, the differential function of the current frame speech signal can be calculated first, and then the preset absolute threshold is corrected according to the speech existence probability of the current frame speech signal to obtain the corrected absolute threshold. Finally, the current fundamental frequency is determined according to the corrected absolute threshold, the differential function of the current frame speech signal and the sampling rate of the target singing voice stream.

[0127] It should be understood that the implementation process and principle of this implementation method can refer to Figure 3 The embodiments shown are not described in detail here.

[0128] S12. Determine the maximum fundamental frequency and the minimum fundamental frequency of the target singing voice stream according to the current fundamental frequency and the historical fundamental frequencies of the voice signals of each historical frame.

[0129] The historical frame voice signal refers to the voice signal before the current frame voice signal in the target singing voice stream. Exemplarily, assuming that the current frame voice signal is the 10th frame in the target singing voice stream, the historical frame voice signal is the 1st to 9th frame voice signal of the target singing voice stream.

[0130] In a possible implementation, the maximum value among the current fundamental frequency and the historical fundamental frequencies of the speech signals of each historical frame is determined as the maximum fundamental frequency, and the minimum value is determined as the minimum fundamental frequency.

[0131] In another possible implementation, the current fundamental frequency can be compared with the maximum fundamental frequency and the minimum fundamental frequency determined when processing the previous frame of speech signal, and the maximum value of the three is determined as the maximum fundamental frequency, and the minimum value of the three is determined as the minimum fundamental frequency.

[0132] Exemplarily, the current frame speech signal is the 10th frame in the target singing voice stream, then the maximum fundamental frequency (f1_max) and the minimum fundamental frequency (f1_min) determined when processing the 9th frame are obtained, and the current fundamental frequency (f0) is compared with f1_max and f1_min, and the maximum fundamental frequency (f0_max) and the minimum fundamental frequency (f0_min) are determined by the following formula:

[0133] f0_max=max(f1_max,f0)

[0134] f0_min=min(f1_min,f0)

[0135] S13: Determine whether the current base frequency is within the target frequency range.

[0136] The target frequency interval includes a first frequency interval corresponding to a maximum fundamental frequency or a second frequency interval corresponding to a minimum fundamental frequency.

[0137] In a possible implementation, it can be determined whether the current base frequency is the maximum base frequency or the minimum base frequency. If the current base frequency is the maximum base frequency or the minimum base frequency, it means that the current base frequency is in the target frequency interval, which is a treble or a bass, and subsequent correction processing is required. If the current base frequency is not the maximum base frequency or the minimum base frequency, it is necessary to further determine whether the current base frequency is in the first frequency interval corresponding to the maximum base frequency or the second frequency interval corresponding to the minimum base frequency. If the current base frequency is in the first frequency interval corresponding to the maximum base frequency or the second frequency interval corresponding to the minimum base frequency, it also means that the current base frequency is a treble or a bass, and subsequent correction processing is required. If the current base frequency is not in the first frequency interval corresponding to the maximum base frequency or the second frequency interval corresponding to the minimum base frequency, it means that the current base frequency is in the normal interval and no subsequent correction processing is required.

[0138] It should be understood that this implementation will be described later. Figure 2 The detailed description is given in the illustrated embodiment and will not be repeated here.

[0139] S14: If it is determined that the current base frequency is in the target frequency interval, the current frame speech signal is corrected according to a preset equalization effect value corresponding to the target frequency interval to obtain the current frame target speech signal.

[0140] In practical applications, the preset equalizer (EQ) corresponding to the treble and the preset equalizer corresponding to the bass can be designed based on the actual scene and the psychoacoustic hearing effect, which are used to suppress or compensate the energy of different frequency ranges of the treble and bass. In music, the two sets of preset equalizer values ​​enhance the breath of the singer singing the treble or bass respectively. Based on the psychoacoustic hearing of the human ear, the two sets of preset equalizer values ​​make the treble sound brighter and the bass sound deeper.

[0141] For high notes, a first equalization effect curve can be pre-designed to enhance the voice signal in the frequency range of 2K-4K Hz. After experimental verification, the current frame voice signal is raised in the frequency range of 2K-4K Hz, so that the human ear can feel that the high notes are brighter. Specifically, the frequency point of 2K-4K Hz can be enhanced by 1.7 times according to the first equalization effect curve, and the target voice signal of the current frame obtained after enhancement is about 6 decibels (dB).

[0142] For bass, a second equalization effect curve can be pre-designed to enhance the 200-400 Hz voice signal. After experimental verification, the current frame voice signal is raised within the 200-400 Hz frequency range, so that the human ear can feel that the bass is thicker. Specifically, the 200-400 Hz frequency point can be enhanced by 2 times according to the second equalization effect curve.

[0143] Considering that the human ear is less sensitive to loudness changes of low frequencies than to loudness changes of high frequencies, the preset equalization effect value for bass can be stronger than the preset equalization effect value for treble.

[0144] The equalization effect curve is used to represent the mapping relationship between the frequency and the preset equalization effect value.

[0145] In a possible implementation, the current frame speech signal may be multiplied by the corresponding equalization effect curve, and the product of the two may be determined as the current frame target speech signal. That is, assuming that the target frequency interval is the first frequency interval, the current frame speech signal is multiplied by the first equalization effect curve to obtain the current frame target speech signal; assuming that the target frequency interval is the second frequency interval, the current frame speech signal is multiplied by the second equalization effect curve to obtain the current frame target speech signal.

[0146] In a possible implementation, if the target frequency interval is the first frequency interval, the current frame frequency domain voice signal is corrected according to the first equalization effect curve and the preset harmonic ratio to obtain the current frame target voice signal. If the target frequency interval is the second frequency interval, the current frame frequency domain voice signal is corrected according to the second equalization effect curve to obtain the current frame target voice signal.

[0147] Specifically, if the target frequency interval is the first frequency interval, the target speech signal of the current frame can be calculated by the following formula:

[0148] Y=X_fft*EQ1+ɑ*P(X_fft*EQ1)

[0149] Where Y is the target speech signal of the current frame, X_fft is the frequency domain speech signal of the current frame, which is obtained by Fourier transforming the speech signal of the current frame, EQ1 is the vector corresponding to the first equalization effect curve, ɑ is the preset harmonic ratio, P(x) is a nonlinear function, p(x) = |x| + x 2 .

[0150] Furthermore, if the target frequency interval is the second frequency interval, the target speech signal of the current frame can be calculated by the following formula:

[0151] Y = X_fft*EQ2

[0152] Wherein, EQ2 is a vector corresponding to the second equalization effect curve.

[0153] In this way, the nonlinear function can make the current frame target speech signal produce rich harmonic components, including odd and even harmonics. After the current frame speech signal passes through the nonlinear function and is added to the current frame speech signal, the amplitude of the odd and even harmonics contained in the current frame target speech signal can be enhanced, and the mid-high frequency harmonics of the current frame target speech signal can be introduced to improve the fullness of the treble.

[0154] The singing voice processing method provided by the embodiment of the present invention determines the current fundamental frequency of the current frame voice signal in the target singing voice stream. Then, according to the current fundamental frequency and the historical fundamental frequencies of each historical frame voice signal, the maximum fundamental frequency and the minimum fundamental frequency of the target singing voice stream are determined. Then, it is determined whether the current fundamental frequency is in the target frequency interval. If it is determined that the current fundamental frequency is in the target frequency interval, the current frame voice signal is corrected according to the preset equalization effect value corresponding to the target frequency interval to obtain the current frame target voice signal. Among them, the target frequency interval includes the first frequency interval corresponding to the maximum fundamental frequency or the second frequency interval corresponding to the minimum fundamental frequency. In this technical scheme, when each frame language signal of the target singing voice stream is subjected to singing voice beautification processing, the maximum fundamental frequency and the minimum fundamental frequency of the target singing voice stream can be determined according to the frame voice signal and the historical frame voice signal before the frame voice signal in the target singing voice stream, and then it is determined whether the current frame voice signal is high pitch and low pitch, so that when the current frame voice signal is low pitch or high pitch, it is enhanced by using the pre-configured preset equalization effect value equalization effect value.

[0155] Compared with the prior art, this technical solution beautifies and enhances the current frame voice signal when the pitch of the singing voice is in the high and low notes. When the pitch is in the normal range, no processing is done on the current frame voice signal. And there is no need to perform a large amount of calculation analysis on the rhythm of the singing content to ensure real-time performance. In addition, this technical solution does not require the use of native templates, thereby ensuring the singer's tonality flexibility in different pitch ranges, and will not introduce distortion due to pitch correction, thereby improving the flexibility of beautifying the singing voice and the naturalness of the beautified singing voice.

[0156] Optionally, in some embodiments, the target gain of the current frame speech signal can also be determined according to the energy corresponding to the current frame speech signal and the historical energy corresponding to each historical frame speech signal. Then, the current frame target speech signal after gain is obtained according to the target gain and the current frame target speech signal.

[0157] Specifically, the energy corresponding to the current frame speech signal and the historical energy corresponding to each historical frame speech signal can be averaged to determine the average energy. Then, the energy corresponding to the current frame speech signal is divided by the average energy, and the quotient obtained by the division is determined as the target gain of the current frame speech signal. Finally, the target gain is multiplied by the current frame target speech signal, and the product is determined as the target gain and the current frame target speech signal.

[0158] In this embodiment, the target gain is used to dynamically control the energy of the target speech signal of the current frame, which helps to adjust the dynamic range of the target singing voice stream, that is, to adjust the difference between the strongest and weakest parts in the target singing voice stream. By adjusting the target gain, the dynamic range of the target singing voice stream can be made more consistent, the abrupt energy changes can be reduced, and the overall listening experience can be made smoother and more pleasant.

[0159] Furthermore, after determining the target gain of the current frame speech signal, it is also possible to determine whether the previous frame speech signal is in the same target frequency interval as the current frame speech signal. If the previous frame speech signal is not in the same target frequency interval as the current frame speech signal, the switching of the target gain is smoothed, and then the target gain of the smoothed processing is multiplied by the current frame target speech signal to obtain the current frame target speech signal after gain.

[0160] Exemplarily, assuming that the current frame speech signal is in the first frequency interval and the previous frame speech signal is in the second frequency interval or the normal interval (the frequency interval between the first frequency interval and the second frequency interval), the target gain of the current frame speech signal needs to be switched for smoothing.

[0161] By smoothing the switching of the target gain, the naturalness of the target speech signal of the current frame after gain can be effectively increased and distortion can be reduced.

[0162] Next, based on Figure 1 In the embodiment shown, by Figure 2 The specific implementation process and principle of S23 are explained.

[0163] Figure 2 Schematic diagram of the flow chart of the second embodiment of the singing voice processing method provided by the embodiment of the present invention. Figure 2 As shown, S13 may include the following steps:

[0164] S21. Determine whether the current base frequency is equal to the maximum base frequency or the minimum base frequency.

[0165] S22: If the current base frequency is equal to the maximum base frequency or the minimum base frequency, determine that the current base frequency is in the target frequency range.

[0166] In a possible implementation, if the current base frequency is equal to the maximum base frequency, it is determined that the current base frequency is in the first frequency interval; if the current base frequency is equal to the minimum base frequency, it is determined that the current base frequency is in the second frequency interval.

[0167] S23: If the current fundamental frequency is less than the maximum fundamental frequency and greater than the minimum fundamental frequency, determine a first frequency interval corresponding to the maximum fundamental frequency and a second frequency interval corresponding to the minimum fundamental frequency from a plurality of preset frequency intervals.

[0168] In a possible implementation, a first distance between the maximum fundamental frequency and each preset frequency interval and a second distance between the minimum fundamental frequency and each preset frequency interval are calculated. The preset frequency interval with the smallest first distance is determined as the first frequency interval. The preset frequency interval with the smallest second distance is determined as the second frequency interval.

[0169] Specifically, each scale in the key of C major may be divided into a plurality of preset frequency intervals (eg, 10). It should be understood that some semitone frequencies may also be taken into account. During the division process, the frequency may be logarithmized and the preset frequency intervals may be divided in the logarithmic domain.

[0170] Furthermore, since the distance between the frequency and the frequency interval can represent the similarity between the two, the first distance between the maximum fundamental frequency and each preset frequency interval and the second distance between the minimum fundamental frequency and each preset frequency interval can be calculated to determine the first frequency interval corresponding to the maximum fundamental frequency and the second frequency interval corresponding to the minimum fundamental frequency, thereby improving processing efficiency.

[0171] Furthermore, considering the situation where the pitch of the singing voice varies greatly, the first frequency interval where the treble is located and the second frequency interval where the bass is located need to be appropriately relaxed.

[0172] In one implementation, a first frequency difference between the maximum fundamental frequency and the minimum fundamental frequency may be calculated. If the first frequency difference is greater than the first preset frequency difference, the first frequency interval and the preset frequency interval adjacent to the first frequency interval and smaller than the first frequency interval are determined as the new first frequency interval. If the first frequency difference is greater than the first preset frequency difference, the second frequency interval and the preset frequency interval adjacent to the second frequency interval and smaller than the second frequency interval are determined as the new second frequency interval.

[0173] Optionally, the first number of preset frequency intervals adjacent to the first frequency interval and smaller than the first frequency interval can be one or more. Similarly, the second number of preset frequency intervals adjacent to the second frequency interval and smaller than the second frequency interval can be one or more, and the first number and the second number can be the same or different. The first number and the second number can be pre-configured, and the embodiment of the present invention does not impose specific restrictions on this.

[0174] In the above implementation, when the pitch changes greatly, the treble interval (first frequency interval) and the bass interval (second frequency interval) are dynamically relaxed, thereby improving the accuracy of subsequent determination of whether the current frame speech signal is bass or treble, thereby improving the accuracy of beautification processing of the current frame speech signal.

[0175] S24: If the current base frequency is in the first frequency interval or the second frequency interval, determine that the current base frequency is in the target frequency interval.

[0176] In one possible manner, in the same manner as determining the first frequency interval and the second frequency interval, the third distance between the current base frequency and each preset frequency interval may be calculated, and the preset frequency interval with the smallest third distance may be determined as the third frequency interval. Further, it is determined whether the third frequency interval is the first frequency interval or the second frequency interval, and if the third frequency interval is the first frequency interval or the second frequency interval, it is determined that the current base frequency is in the target frequency interval.

[0177] Optionally, after the first frequency interval and the second frequency interval are updated, the process of determining whether the current baseband is in the new first frequency interval or the new second frequency interval may be implemented as follows:

[0178] f0_idx>=f0_max_idx-K

[0179] f0_idx<=f0_min_idx+K

[0180] Among them, f0_idx is the index of the third frequency interval, f0_max_idx is the index of the first frequency interval, f0_min_idx is the index of the second frequency interval, and K is the first number (the second number).

[0181] In the above technical scheme, since C major is one of the most basic modes. By dividing a plurality of preset frequency intervals according to the scale of C major, these preset frequency intervals can contain the treble and bass of the singing voice. Subsequently, by mapping the maximum fundamental frequency, the minimum fundamental frequency and the current fundamental frequency to the preset frequency interval, it is determined whether the current frame voice signal is a treble and a bass, thereby improving the accuracy of pitch recognition. Meanwhile, considering that it takes a certain processing time to update the first frequency interval, the second frequency interval and the third frequency interval, when the current fundamental frequency is equal to the maximum fundamental frequency, it is possible to determine that the current frame voice signal is definitely a treble, and when the current fundamental frequency is equal to the minimum fundamental frequency, it is possible to determine that the current frame voice signal is definitely a bass. Therefore, it is possible to first determine whether the current fundamental frequency is equal to the maximum fundamental frequency or the minimum fundamental frequency, and when the current fundamental frequency is equal to the maximum fundamental frequency or the minimum fundamental frequency, it is directly determined that the current frame voice signal is a treble or a bass, thereby saving the subsequent processing process and improving the processing efficiency.

[0182] Next, based on Figure 1 and Figure 2 In the embodiment shown, by Figure 3 The specific implementation process and principle of S11 are explained.

[0183] Figure 3 Schematic diagram of the flow chart of the third embodiment of the singing voice processing method provided by the embodiment of the present invention. Figure 3 As shown, S11 may include the following steps:

[0184] S31. Calculate the differential function of the current frame speech signal.

[0185] The difference formula can be expressed by the following formula:

[0186] d_ind(l+1)=∑(x(n+l)-x(n+l-1)) 2

[0187] Where n is the time step, l is the offset, x(n) is the current frame speech signal, and d_ind(l+1) is the differential function of the current frame speech signal.

[0188] S32. According to the speech existence probability of the current frame speech signal, the preset absolute threshold is modified to obtain a modified absolute threshold.

[0189] Optionally, the corrected absolute threshold may be determined according to the following formula:

[0190] threshold=threshold0+P_speech 2 *0.1

[0191] Wherein, threshold is the corrected absolute threshold, threshold0 is the preset absolute threshold, and P_speech is the speech existence probability of the current frame speech signal.

[0192] Among them, the probability of speech existence in the current frame speech signal can be determined by the mask value (English: mask) of the front-end noise reduction algorithm module. Specifically, when the current frame frequency domain speech signal is subjected to noise reduction processing, the mask is used in the front-end module to multiply with X_fft and complete the noise reduction. For each frequency point, mask is a value in the range of 0-1, which can be a decimal. Among them, the value of mask is 1, indicating that there is no suppression at all and speech exists; the value of mask is 0, indicating that there is complete suppression and noise exists. Therefore, the probability of speech existence in the current frame speech signal can be determined based on the value of mask contained in the current frame frequency domain speech signal.

[0193] S33, determining the current fundamental frequency according to the corrected absolute threshold, the differential function of the current frame speech signal and the sampling rate of the target singing speech stream.

[0194] In a possible implementation, the fundamental frequency period of the current frame speech signal is determined according to the modified absolute threshold and the differential function of the current frame speech signal. Then, the ratio of the sampling rate to the fundamental frequency period is determined as the current fundamental frequency.

[0195] Figure 4 Schematic diagram of the differential function provided by the embodiment of the present invention. Figure 4 As shown, the horizontal axis of the differential function is the frequency (unit Hz), and the vertical axis is the differential function value. It should be understood that the horizontal coordinate of the first local minimum point of the differential function is 180 Hz, and the vertical coordinate is 0.25. If 0.25 is greater than the corrected absolute threshold, the horizontal coordinate corresponding to the second local minimum point is determined as the frequency cycle; if 0.25 is less than or equal to the corrected absolute threshold, the horizontal coordinate corresponding to the first local minimum point is determined as the frequency cycle.

[0196] In the above embodiment, the fundamental frequency period is calculated by finding the minimum value of the differential function according to the absolute threshold, and then the fundamental frequency period is calculated by the determined minimum value, thereby determining the current fundamental frequency. However, most of the absolute thresholds in the prior art are pre-set and cannot be adapted to all application scenarios, resulting in low accuracy of the calculated current fundamental frequency. Considering that the physical meaning corresponding to the absolute threshold is the degree of signal aperiodicity, the technical solution uses the speech existence probability of the current frame speech signal to correct the preset absolute threshold, thereby calculating the current fundamental frequency according to the dynamically changing absolute threshold (corrected absolute threshold), which effectively improves the robustness of the fundamental frequency estimation.

[0197] Figure 5Schematic diagram of the flow chart of the fourth embodiment of the singing voice processing method provided by the embodiment of the present invention. Figure 5 As shown, S31 may include the following steps:

[0198] S51. Perform Fourier transform on the current frame speech signal to obtain the current frame frequency domain speech signal.

[0199] S52. Calculate the energy of the current frame frequency domain speech signal according to the current frame frequency domain speech signal.

[0200] In a possible implementation, the energy of the current frame frequency domain speech signal can be calculated by the following formula:

[0201] Power_X=∑ w |X_fft| 2

[0202] Among them, Power_X is the energy of the frequency domain speech signal of the current frame, X_fft is the frequency domain speech signal of the current frame, and w is the frequency point.

[0203] S53: If the energy is greater than the preset energy, a differential function of the current frame speech signal is calculated.

[0204] In practical applications, if the energy is greater than the preset energy, it is determined that there is speech in the current frame speech signal, and the differential function of the current frame speech is calculated for subsequent processing; if the energy is less than or equal to the preset energy, it is determined that there is no speech in the current frame speech signal, and the current fundamental frequency of the current frame speech is determined to be 0, that is, no subsequent processing is performed.

[0205] In one possible implementation, if the energy is greater than the preset energy and the number of mask values ​​1 is greater than a third preset number, it is determined that speech exists in the current frame speech signal, and the differential function of the current frame speech is calculated for subsequent processing.

[0206] In the above embodiment, before calculating the current fundamental frequency of the current frame speech signal, it is also possible to detect whether the current frame speech signal has speech. When it is determined that the current frame speech signal has speech, the subsequent beautification correction processing is performed. If it is determined that the current frame speech signal does not have speech, the subsequent beautification correction processing is not performed, thereby avoiding a lot of redundant processing and operations, as well as the problem of erroneous correction of the pitch of noise, and improving processing efficiency and accuracy.

[0207] Optionally, the current initial fundamental frequency of the current frame voice signal may be determined first. If the second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame voice signal is less than the second preset frequency difference, the current initial fundamental frequency is determined as the current fundamental frequency of the current frame voice signal. If the second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame voice signal is greater than or equal to the second preset frequency difference, the historical fundamental frequency of the previous frame voice signal is determined as the current fundamental frequency of the current frame voice signal.

[0208] Exemplarily, the second preset frequency difference may be set within a range of 80 Hz-150 Hz, such as 120 Hz, which may be determined based on actual conditions, and the embodiment of the present invention does not impose any specific limitation on this.

[0209] It should be understood that the current initial fundamental frequency of the current frame speech signal can be determined by referring to the processing method of determining the initial fundamental frequency of the current frame speech signal in the above embodiment, which will not be described in detail here.

[0210] In the above method, by limiting the jumping range of the fundamental frequency, only the initial fundamental frequency estimated within the limited range (the second preset frequency difference) will be used, which can effectively resist the fundamental frequency estimation jump when the speech starts and ends, and improve the accuracy of processing.

[0211] Next, in the scenario of real-time beautification and correction of the singing voice of the user in the car KTV, the scheme designed by the present invention is specifically explained through several specific examples.

[0212] Figure 6 Schematic diagram of the flow chart of the fifth embodiment of the singing voice processing method provided by the embodiment of the present invention. Figure 6 As shown, the singing voice processing method may include the following steps:

[0213] S61. Arrange a microphone array in the car to pick up the singer's singing signal.

[0214] S62, performing howling suppression and noise suppression on the picked-up singing voice signal of the singer to obtain the current frame voice signal.

[0215] S63. Estimating the fundamental frequency of the current frame speech signal to determine the current fundamental frequency of the current frame speech signal.

[0216] S64: Update the maximum fundamental frequency and the minimum fundamental frequency, and map to obtain a first frequency interval and a second frequency interval.

[0217] S65. Detect whether the current base frequency is in the first frequency interval or the second frequency interval.

[0218] S66. When the current base frequency is in the first frequency interval or the second frequency interval, perform corresponding enhancement and beautification processing on the current frame speech signal to obtain the current frame target speech signal.

[0219] Figure 7 Schematic diagram of the flow chart of the sixth embodiment of the singing voice processing method provided by the embodiment of the present invention. Figure 7 As shown, the singing voice processing method may include the following steps:

[0220] S71. Input the singer's singing voice signal into a howling suppression algorithm to obtain the singer's singing voice signal after suppression.

[0221] S72. Input the suppressed vocal signal of the singer into the noise reduction model to obtain the current frame speech signal.

[0222] S73. Calculate the current fundamental frequency of the current frame speech signal.

[0223] S74. Determine a first frequency interval, a second frequency interval, and a third frequency interval.

[0224] S75: Determine whether the third frequency is in the first frequency interval or the second frequency interval.

[0225] If the third frequency is in the first frequency interval, execute S76; if the third frequency is in the second frequency interval, execute S79; if the third frequency is not in the first frequency interval and is not in the second frequency interval, directly determine the current frame speech signal as the current frame target speech signal and output it.

[0226] S76. Enhance the current frame speech signal according to the first equalization effect curve to obtain an enhanced current frame speech signal.

[0227] S77. Enhance the enhanced current frame speech signal again through a nonlinear function to obtain a current frame target speech signal.

[0228] S78. Perform energy dynamic control processing on the target speech signal of the current frame according to the target gain of the speech signal of the current frame, generate the target speech signal of the current frame after gain, and output it.

[0229] S79. Enhance the current frame speech signal according to the second equalization effect curve to obtain an enhanced current frame speech signal.

[0230] It should be understood that S78 is executed after S79 is executed.

[0231] Figure 8 Schematic diagram of the flow chart of the seventh embodiment of the singing voice processing method provided by the embodiment of the present invention. Figure 8 As shown, the singing voice processing method may include the following steps:

[0232] S81. Determine the current fundamental frequency, the maximum fundamental frequency, and the minimum fundamental frequency.

[0233] S82. According to the current base frequency, the maximum base frequency and the minimum base frequency, search, from a plurality of preset frequency intervals, a first frequency interval closest to the current base frequency, a second frequency interval closest to the maximum base frequency and a third frequency interval closest to the minimum base frequency.

[0234] S83. Obtain a first frequency interval, a second frequency interval, and a third frequency interval.

[0235] S84. Update the first frequency interval and the second frequency interval according to the frequency difference between the maximum fundamental frequency and the minimum fundamental frequency to obtain a new first frequency interval and a new second frequency interval.

[0236] S85. If the third frequency interval is the new first frequency interval, the current frame speech signal is corrected according to the first equalization effect curve and the nonlinear function to obtain the current frame target speech signal.

[0237] S86: If the third frequency interval is a new second frequency interval, the current frame speech signal is corrected according to the second equalization effect curve to obtain the current frame target speech signal.

[0238] S87: If the third frequency interval is neither the new first frequency interval nor the new second frequency interval, the current frame speech signal is determined as the current frame target speech signal.

[0239] Combined with the above embodiments, the singing voice processing method can enhance the high and low notes in real time, so that when the singer sings karaoke, the high notes sound brighter and the low notes sound richer, and the singer can sing the high and low notes more easily. The solution has low latency and can be run on the car computer.

[0240] Fig. 9 Schematic diagram of the structure of the singing voice processing device provided by the embodiment of the present invention. Fig. 9 As shown, the singing voice processing device 90 comprises:

[0241] The first determination module 91 is used to determine the current fundamental frequency of the current frame speech signal in the target singing voice stream.

[0242] The second determination module 92 is used to determine the maximum fundamental frequency and the minimum fundamental frequency of the target singing voice stream according to the current fundamental frequency and the historical fundamental frequencies of the voice signals of each historical frame.

[0243] The third determination module 93 is used to determine whether the current fundamental frequency is in a target frequency interval, where the target frequency interval includes a first frequency interval corresponding to a maximum fundamental frequency or a second frequency interval corresponding to a minimum fundamental frequency.

[0244] The correction module 94 is used to correct the current frame speech signal according to the preset equalization effect value corresponding to the target frequency interval if it is determined that the current fundamental frequency is in the target frequency interval, so as to obtain the current frame target speech signal.

[0245] Further, the third determining module 93 is specifically configured to:

[0246] Determine whether the current base frequency is equal to the maximum base frequency or the minimum base frequency.

[0247] If the current fundamental frequency is equal to the maximum fundamental frequency or the minimum fundamental frequency, it is determined that the current fundamental frequency is in the target frequency interval.

[0248] Further, the third determining module 93 is specifically configured to:

[0249] If the current fundamental frequency is less than the maximum fundamental frequency and greater than the minimum fundamental frequency, a first frequency interval corresponding to the maximum fundamental frequency and a second frequency interval corresponding to the minimum fundamental frequency are determined from a plurality of preset frequency intervals.

[0250] If the current base frequency is in the first frequency interval or the second frequency interval, it is determined that the current base frequency is in the target frequency interval.

[0251] Further, the third determining module 93 is specifically configured to:

[0252] A first distance between the maximum fundamental frequency and each preset frequency interval and a second distance between the minimum fundamental frequency and each preset frequency interval are calculated.

[0253] The preset frequency interval with the smallest first distance is determined as the first frequency interval.

[0254] The preset frequency interval with the smallest second distance is determined as the second frequency interval.

[0255] Furthermore, the singing voice processing device 90 further includes a fourth determining module, which is used to:

[0256] A first frequency difference between the maximum fundamental frequency and the minimum fundamental frequency is calculated.

[0257] If the first frequency difference is greater than the first preset frequency difference, the first frequency interval and a preset frequency interval adjacent to the first frequency interval and smaller than the first frequency interval are determined as a new first frequency interval.

[0258] If the first frequency difference is greater than the first preset frequency difference, the second frequency interval and a preset frequency interval adjacent to the second frequency interval and smaller than the second frequency interval are determined as a new second frequency interval.

[0259] Further, the first determining module 91 is specifically configured to:

[0260] Calculate the difference function of the current frame speech signal.

[0261] According to the speech existence probability of the current frame speech signal, the preset absolute threshold is modified to obtain the modified absolute threshold.

[0262] The current fundamental frequency is determined according to the corrected absolute threshold, the differential function of the current frame speech signal and the sampling rate of the target singing speech stream.

[0263] Further, the first determining module 91 is specifically configured to:

[0264] Perform Fourier transform on the current frame speech signal to obtain the current frame frequency domain speech signal.

[0265] According to the current frame frequency domain speech signal, the energy of the current frame frequency domain speech signal is calculated.

[0266] If the energy is greater than the preset energy, the differential function of the current frame speech signal is calculated.

[0267] Further, the correction module 94 is specifically used for:

[0268] If the target frequency interval is the first frequency interval, the current frame frequency domain speech signal is corrected according to the first equalization effect curve and the preset harmonic ratio to obtain the current frame target speech signal.

[0269] If the target frequency interval is the second frequency interval, the current frame frequency domain speech signal is corrected according to the second equalization effect curve to obtain the current frame target speech signal.

[0270] The equalization effect curve is used to represent the mapping relationship between the frequency and the preset equalization effect value.

[0271] Further, the first determining module 91 is specifically configured to:

[0272] The current base frequency of the current frame speech signal, including:

[0273] Determine the current initial fundamental frequency of the current frame speech signal.

[0274] If the second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame of speech signal is less than the second preset frequency difference, the current initial fundamental frequency is determined as the current fundamental frequency of the current frame of speech signal.

[0275] If the second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame of speech signal is greater than or equal to the second preset frequency difference, the historical fundamental frequency of the previous frame of speech signal is determined as the current fundamental frequency of the current frame of speech signal.

[0276] Furthermore, the singing voice processing device 90 further includes a gain module, which is used to:

[0277] The target gain of the current frame speech signal is determined according to the energy corresponding to the current frame speech signal and the historical energy corresponding to each historical frame speech signal.

[0278] According to the target gain and the target speech signal of the current frame, the target speech signal of the current frame after gain is obtained.

[0279] The singing voice processing device provided in the embodiment of the present invention can be used to execute the singing voice processing method in any of the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0280] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, all or part of them can be integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements. They can also be implemented in the form of hardware. Some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. In addition, all or part of these modules can be integrated together or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0281] Fig.10 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Fig.10 As shown, the electronic device 100 may include: a processor 101, a memory 102, and computer execution instructions stored in the memory 102 and executable on the processor 101. When the processor 101 executes the computer execution instructions, the singing processing method provided in any of the aforementioned embodiments is implemented.

[0282] Optionally, the above-mentioned components of the electronic device 100 may be connected via a system bus.

[0283] The memory 102 may be a separate storage unit or a storage unit integrated in a processor. The number of processors may be one or more.

[0284] Optionally, the electronic device 100 may also include a communication interface for interacting with other devices.

[0285] It should be understood that the processor 101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in the present invention may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0286] The system bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0287] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The above-mentioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed. The above-mentioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc and any combination thereof.

[0288] The electronic device provided in the embodiment of the present invention can be used to execute the singing voice processing method provided in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0289] An embodiment of the present invention provides a vehicle, including the above-mentioned electronic device 100, and its implementation principle and technical effect are similar, which will not be repeated here.

[0290] An embodiment of the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are run on a computer, the computer executes the above-mentioned singing voice processing method.

[0291] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0292] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0293] An embodiment of the present invention also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when at least one processor executes the computer program, the above-mentioned singing processing method can be implemented.

[0294] The above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present invention is within the protection scope of the present invention.

Claims

1. A singing voice processing method, characterized in that: include: Determine the current fundamental frequency of the current frame speech signal in the target singing voice stream; Determine the maximum fundamental frequency and the minimum fundamental frequency of the target singing voice stream according to the current fundamental frequency and the historical fundamental frequencies of each historical frame voice signal; Determine whether the current fundamental frequency is in a target frequency interval, where the target frequency interval includes a first frequency interval corresponding to the maximum fundamental frequency or a second frequency interval corresponding to the minimum fundamental frequency; If it is determined that the current base frequency is in the target frequency interval, the current frame speech signal is modified according to a preset equalization effect value corresponding to the target frequency interval to obtain a current frame target speech signal.

2. The method according to claim 1, characterized in that: The determining whether the current base frequency is within a target frequency interval includes: Determining whether the current fundamental frequency is equal to the maximum fundamental frequency or the minimum fundamental frequency; If the current fundamental frequency is equal to the maximum fundamental frequency or the minimum fundamental frequency, it is determined that the current fundamental frequency is in the target frequency interval.

3. The method according to claim 2, characterized in that The determining whether the current base frequency is within a target frequency interval includes: If the current fundamental frequency is less than the maximum fundamental frequency and greater than the minimum fundamental frequency, determining a first frequency interval corresponding to the maximum fundamental frequency and a second frequency interval corresponding to the minimum fundamental frequency from a plurality of preset frequency intervals; If the current base frequency is in the first frequency interval or the second frequency interval, it is determined that the current base frequency is in the target frequency interval.

4. The method according to claim 3, characterized in that The step of determining the first frequency interval corresponding to the maximum fundamental frequency and the second frequency interval corresponding to the minimum fundamental frequency from a plurality of preset frequency intervals comprises: calculating a first distance between the maximum fundamental frequency and each preset frequency interval, and a second distance between the minimum fundamental frequency and each preset frequency interval; Determine a preset frequency interval in which the first distance is the smallest as the first frequency interval; The preset frequency interval in which the second distance is the smallest is determined as the second frequency interval.

5. The method according to claim 4, characterized in that The method further comprises: Calculating a first frequency difference between the maximum fundamental frequency and the minimum fundamental frequency; If the first frequency difference is greater than a first preset frequency difference, determining the first frequency interval and a preset frequency interval adjacent to the first frequency interval and smaller than the first frequency interval as a new first frequency interval; If the first frequency difference is greater than a first preset frequency difference, the second frequency interval and a preset frequency interval adjacent to the second frequency interval and smaller than the second frequency interval are determined as new second frequency intervals.

6. The method according to any one of claims 1 to 5, characterized in that: The method of determining the current fundamental frequency of the current frame speech signal in the target singing voice stream comprises: Calculating a differential function of the current frame speech signal; According to the speech existence probability of the current frame speech signal, the preset absolute threshold is modified to obtain a modified absolute threshold; The current fundamental frequency is determined according to the corrected absolute threshold, the differential function of the current frame speech signal and the sampling rate of the target singing speech stream.

7. The method according to claim 6, characterized in that The calculating the differential function of the current frame speech signal comprises: Performing Fourier transform on the current frame speech signal to obtain the current frame frequency domain speech signal; Calculating the energy of the current frame frequency domain speech signal according to the current frame frequency domain speech signal; If the energy is greater than the preset energy, a differential function of the current frame speech signal is calculated.

8. The method according to claim 7, characterized in that If it is determined that the current fundamental frequency is in the target frequency interval, the current frame speech signal is modified according to a preset equalization effect value corresponding to the target frequency interval to obtain the current frame target speech signal, including: If the target frequency interval is the first frequency interval, the current frame frequency domain voice signal is corrected according to the first equalization effect curve and the preset harmonic ratio to obtain the current frame target voice signal; if the target frequency interval is the second frequency interval, the current frame frequency domain voice signal is corrected according to the second equalization effect curve to obtain the current frame target voice signal; The equalization effect curve is used to represent the mapping relationship between the frequency and the preset equalization effect value.

9. The method according to any one of claims 1 to 5, characterized in that: The method of determining the current fundamental frequency of the current frame speech signal in the target singing voice stream comprises: Determine the current initial fundamental frequency of the current frame speech signal; If the second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame voice signal is less than the second preset frequency difference, the current initial fundamental frequency is determined as the current fundamental frequency of the current frame voice signal; if the second frequency difference between the current initial fundamental frequency and the historical fundamental frequency of the previous frame voice signal is greater than or equal to the second preset frequency difference, the historical fundamental frequency of the previous frame voice signal is determined as the current fundamental frequency of the current frame voice signal.

10. The method according to claim 7 or 8, characterized in that: The method further comprises: Determining a target gain of the current frame speech signal according to the energy corresponding to the current frame speech signal and the historical energy corresponding to each historical frame speech signal; According to the target gain and the current frame target speech signal, a current frame target speech signal after gain is obtained.

11. A singing voice processing device, characterized in that: include: A first determination module is used to determine the current fundamental frequency of the current frame speech signal in the target singing voice stream; A second determination module is used to determine the maximum fundamental frequency and the minimum fundamental frequency of the target singing voice stream according to the current fundamental frequency and the historical fundamental frequencies of the voice signals of each historical frame; A third determination module is used to determine whether the current fundamental frequency is in a target frequency interval, where the target frequency interval includes a first frequency interval corresponding to the maximum fundamental frequency or a second frequency interval corresponding to the minimum fundamental frequency; The correction module is used to correct the current frame speech signal according to the preset equalization effect value corresponding to the target frequency interval if it is determined that the current base frequency is in the target frequency interval, so as to obtain the current frame target speech signal.

12. An electronic device comprising: A processor, a memory, and computer-executable instructions stored in the memory and executable on the processor, wherein the processor is used to implement the singing processing method as described in any one of claims 1 to 10 when executing the computer-executable instructions.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the singing voice processing method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it is used to implement the singing voice processing method as described in any one of claims 1 to 10.

15. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 12.

Images