Tone library generation method, sound synthesis method and system, and audio processing chip

Abstract

The present invention relates to a method for generating a timbre library, a sound synthesis method and system, an audio processing chip, a computer-readable storage medium, and an electronic device. In the method for generating the timbre library, a first amplitude spectrum is first calculated for a windowed sound source audio frame, and the first amplitude spectrum is used to update an average amplitude spectrum, and similarity is calculated for each peak greater than an amplitude threshold in the first amplitude spectrum and a low-pass kernel, and then the largest value in the result and its corresponding low-pass kernel and local maximum peak are selected to form a similar data group, and then the similar data group is used to compensate the probability value in the probability table, and then the center frequency, bandwidth, and maximum amplitude of the resonance amplitude characteristics of the sound source signal are found from the compensated probability table and the average amplitude spectrum. Obviously, the method for generating the timbre library of the present invention greatly simplifies the algorithm for generating the timbre library, improves the generation efficiency of the timbre library, and can save resources.

Description

Technical Field

[0001] The present invention relates to the field of audio processing technology, and in particular to a method for generating a timbre library, a sound synthesis method and system, an audio processing chip, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the development of audio technology, more and more kinds of sounds are synthesized, such as live sound cards, voice-changing headphones, voice-changing karaoke machines and other products. These synthesis devices require the software to store the timbre files of the corresponding instruments. These timbre files often use the cepstrum method and the spectrum envelope method to generate the timbre library, which increases the consumption of flash and requires a large storage space. In some applications, such as real-time sound synthesis systems, the sound source signal is often directly recorded and needs to be processed in real time. If the timbre information is generated in the existing way, the processing algorithm will be complex, the efficiency will be low, and the system space requirement will be too large, especially for small systems, it is difficult to apply. Summary of the invention

[0003] Based on the above situation, the main purpose of the present invention is to provide a method for generating a timbre library, a sound synthesis method and system, an audio processing chip, a computer-readable storage medium, and an electronic device to solve the problem that existing sound synthesis equipment takes up too much space.

[0004] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0005] A first aspect of the present invention provides a method for generating a timbre library, comprising the steps of:

[0006] S110: Acquire a sound source signal, and divide the sound source signal into frames to obtain a plurality of sound source audio frames;

[0007] After executing S120 to S160 for each of the sound source audio frames, execute S170;

[0008] S120: performing windowing processing on the sound source audio frame;

[0009] S130: Calculate the amplitude spectrum of the signal after the windowing process, recorded as the first amplitude spectrum;

[0010] S140: updating an average amplitude spectrum according to the first amplitude spectrum, wherein an initial value of the average amplitude spectrum is 0;

[0011] S150: In the first amplitude spectrum, find a peak whose amplitude is greater than a preset amplitude threshold, record it as a local maximum peak, and determine which low-pass core has the highest waveform distribution similarity with each local maximum peak among multiple preset low-pass cores, record the sum of the products of the local maximum peak and the low-pass core with the highest similarity as the resonance peak occurrence probability, and take the frequency point corresponding to the position of the local maximum peak, the low-pass core with the highest similarity, and the resonance peak occurrence probability corresponding to the local maximum peak as a similarity data group;

[0012] Wherein, the peak value and bandwidth of each low-pass core are different;

[0013] S160: selecting similar data groups corresponding to the first preset number of similar data groups with the largest probability of occurrence of the resonance peaks in each similar data group, and compensating the corresponding probability values ​​of the selected similar data groups in the probability table, and the greater the probability of occurrence of the resonance peak, the greater the compensation value of the corresponding probability value;

[0014] The probability table includes a correspondence between frequency points, low-pass kernels and probability values, the frequency points traverse the frequency points in the frequency domain signal, and the initial probability values ​​are 0;

[0015] S170: Determine, according to the probability table, the corresponding frequency points and low-pass kernels of the first second preset number with the highest probability of occurrence, and record them as the center frequency point and the maximum low-pass kernel, respectively;

[0016] S180: Find the amplitude value corresponding to each of the center frequencies in the average amplitude spectrum and record it as the maximum amplitude. The corresponding center frequency, the maximum low-pass kernel and the maximum amplitude form a resonance peak information. Each resonance peak information forms the timbre corresponding to the sound source signal in the timbre library.

[0017] Preferably, the step S120 includes the steps of:

[0018] S121: converting the sound source audio frame into a frequency domain signal, and calculating the amplitude spectrum, which is recorded as a second amplitude spectrum;

[0019] S122: Determine a windowing length according to the frequency points corresponding to the peak positions in the second amplitude spectrum;

[0020] S123: Perform windowing processing on the sound source audio frame according to the windowing length.

[0021] Preferably, the step S122 comprises the steps of:

[0022] S122a: Find the position of the first peak in the second amplitude spectrum that is greater than a preset peak threshold, and calculate the corresponding period according to the frequency point corresponding to the position, which is recorded as a preset period; wherein the preset peak threshold is determined according to the frequency points corresponding to each peak position in the first amplitude spectrum;

[0023] S122b: The windowing length is 1.5 to 2.5 times the preset period.

[0024] Preferably, the preset peak threshold in step S122a is smaller than the maximum amplitude value in the second amplitude spectrum.

[0025] Preferably, the step S140 specifically includes: adding a product obtained by multiplying the average amplitude spectrum by 0.99 and a product obtained by multiplying the second amplitude spectrum by 0.01 to obtain a new average amplitude spectrum.

[0026] Preferably, the preset amplitude threshold in step S150 is determined according to the amplitude average value of the first amplitude spectrum.

[0027] Preferably, the step S150 includes:

[0028] S151: In the first amplitude spectrum, find a peak whose amplitude is greater than a preset amplitude threshold, record it as a local maximum peak, and execute S152 and S153 for each local maximum peak;

[0029] S152: performing dot product sum operations on the local maximum peak and each of the preset low-pass kernels to obtain a plurality of dot product values;

[0030] S153: Find the maximum value among multiple point product values, which is the probability of occurrence of the resonance peak in the similar data group. The low-pass kernel used in calculating the probability of occurrence of the resonance peak is the low-pass kernel in the similar data group. The frequency point corresponding to the position of the corresponding local maximum peak, the low-pass kernel with the highest similarity, and the probability of occurrence of the resonance peak corresponding to the local maximum peak are taken as the similar data group.

[0031] Preferably, the low-pass kernel is determined by the following formula:

[0032]

[0033]

[0034] Where x is the position of the low-pass kernel, c is the coefficient, and different low-pass kernels have different c values; Y[x] is the value of the low-pass kernel at the x-th position; and D is the number of low-pass kernel positions.

[0035] Preferably, in step S160, the probability value is compensated in such a manner that the compensation value of each probability value is increased by 1 in the order of the corresponding probability of occurrence of the resonance peak from small to large.

[0036] Preferably, the second preset number is less than or equal to the first preset number, and each is selected from 4 to 6.

[0037] Preferably, the step S170 includes the steps of:

[0038] S171: Calculate the sum of each probability value corresponding to each frequency point in the probability table to obtain the probability sum corresponding to each frequency point, and select the frequency point corresponding to the second preset number with the largest probability sum as the central frequency point;

[0039] S172: For each of the central frequency points, find the corresponding maximum probability value in the probability table, and the low-pass kernel corresponding to the maximum probability value is the maximum low-pass kernel.

[0040] Preferably, executing S120 to S160 on each of the sound source audio frames includes:

[0041] Determine whether the energy of each of the sound source audio frames is greater than a preset energy value, and if so, execute S120 to S160.

[0042] A second aspect of the present invention provides a sound synthesis method, comprising the steps of:

[0043] S100: Generate a timbre library using any of the above methods;

[0044] S200: Select a timbre from the timbre library, and generate filters according to each formant information of the selected timbre;

[0045] S300: Acquire a sound signal frame;

[0046] S400: generating a pulse sequence according to the sound signal frame, and filtering the pulse sequence using each of the filters respectively, and then adding the results to obtain a synthetic signal; wherein the period of the pulse sequence is determined according to the fundamental frequency of the sound signal frame;

[0047] S500: Output the synthesized signal as an output signal.

[0048] Preferably, the step 400 comprises the steps of:

[0049] S410: Using the sound signal frame to update the sound energy, the initial value of the sound energy is 0;

[0050] S420: Performing fundamental frequency detection on the sound signal frame to obtain the sound fundamental frequency;

[0051] S430: determining a period of the pulse sequence according to the fundamental frequency of the sound, and generating the pulse sequence;

[0052] S440: Using each of the filters to filter the pulse sequence respectively, and then adding the results to obtain an intermediate signal;

[0053] S450: Use the sound energy to perform amplitude modulation on the intermediate signal to obtain a synthesized signal.

[0054] Preferably, the step S500 specifically includes the steps of:

[0055] S510: Processing the synthesized signal through an effector to obtain the output signal;

[0056] S520: Output the output signal.

[0057] Preferably, the effector in step S510 includes at least one of a delay device, a reverberator and a wah pedal effector.

[0058] Preferably, the sound source signal comes from a musical instrument;

[0059] The sound signal frame includes a human voice from a microphone.

[0060] A third aspect of the present invention provides a timbre library module, comprising:

[0061] A sound source acquisition unit acquires a sound source signal and divides the sound source signal into frames to obtain a plurality of sound source audio frames;

[0062] The probability table unit is connected to the sound source acquisition unit and is used to perform the following processing on each of the sound source audio frames:

[0063] The sound source audio frame is subjected to windowing processing; the signal after the windowing processing is converted into a frequency domain signal, which is recorded as a first amplitude spectrum; the average amplitude spectrum is updated according to the first amplitude spectrum, wherein the initial value of the average amplitude spectrum is 0; in the first amplitude spectrum, a peak whose amplitude is greater than a preset amplitude threshold is found, which is recorded as a local maximum peak, and it is determined which low-pass core has the highest waveform distribution similarity with each local maximum peak among multiple preset low-pass cores, and the sum of the products of the local maximum peak and the low-pass core with the highest similarity is recorded as the probability of resonance peak occurrence, and the frequency point corresponding to the position of the local maximum peak is recorded as , the probability of occurrence of the resonance peak corresponding to the low-pass kernel with the highest similarity and the local maximum peak as a similar data group; wherein the peak value and bandwidth of each low-pass kernel are different; and the similar data group corresponding to the first preset number of the resonance peaks with the largest probability of occurrence is selected, and the probability value corresponding to the selected similar data group in the probability table is compensated, and the greater the probability of occurrence of the resonance peak, the greater the compensation value of the corresponding probability value; wherein the probability table includes the corresponding relationship between frequency points, low-pass kernels and probability values, and the frequency points traverse the frequency points in the frequency domain signal, and the initial probability values ​​are 0;

[0064] The timbre library generating unit is connected to the probability table unit, and is used to determine the corresponding frequency points and low-pass kernels of the first second preset number with the highest probability of occurrence according to the probability table, which are recorded as the center frequency point and the maximum low-pass kernel respectively; find the amplitude value corresponding to each of the center frequency points in the average amplitude spectrum, which is recorded as the maximum amplitude, and the corresponding center frequency point, the maximum low-pass kernel and the maximum amplitude form a resonance peak information, and each resonance peak information forms the timbre corresponding to the sound source signal in the timbre library.

[0065] A fourth aspect of the present invention provides a sound synthesis system, comprising:

[0066] The Sound Library module as described above;

[0067] A filter module, connected to the timbre library module, for selecting a timbre from the timbre library and generating filters according to each formant information of the selected timbre;

[0068] A synthesis module is connected to the filter module, and is used to obtain a sound signal frame, generate a pulse sequence according to the sound signal frame, and use each of the filters to filter the pulse sequence respectively, and then add the results to obtain a synthesized signal; and is used to output the synthesized signal as an output signal.

[0069] A fifth aspect of the present invention provides an audio processing chip, in which a circuit module is integrated, and the circuit module is configured to implement the timbre library generation method as described in any one of the above items or the sound synthesis method as described in any one of the above items.

[0070] The sixth aspect of the present invention provides an electronic device, characterized in that it includes the audio processing chip, input module and output module as described above, the audio processing chip has an input interface and an output interface, the input module is connected to the first input interface, and is used to obtain the sound source signal and / or the sound signal frame; the output module is connected to the output interface, and is used to output the output signal.

[0071] Preferably, the input module includes a microphone, and the output module includes a speaker or an earphone.

[0072] Preferably, the electronic device includes a speaker, a karaoke player, a voice toy or a device with a host sound card.

[0073] A seventh aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, the method for generating a timbre library as described in any one of the above items or the method for synthesizing sound as described in any one of the above items is implemented.

[0074] The method for generating a timbre library of the present invention sets multiple low-pass kernels, uses the low-pass kernels to process the data near the larger peak in the amplitude spectrum, and compensates the probability value in the probability table according to the processing result, and then finds the corresponding center frequency point, low-pass kernel and maximum amplitude in the probability table and the average amplitude spectrum according to the size of the probability value. These three together characterize a resonance peak information. In this way, the acquired sound source signal will generate multiple resonance peak information to form a resonance peak group, which can characterize the timbre information of the sound source signal. It can be seen that this method for generating a timbre library can more accurately understand the data distribution of each local maximum peak in the sound source audio frame through calculation with the low-pass kernel, thereby improving the compensation accuracy of the probability value in the probability table, thereby improving the accuracy of resonance peak information extraction, and the algorithm is simple, which reduces the occupation of resources, and is particularly suitable for the application of small systems.

[0075] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

[0077] Figure 1 A flowchart of a preferred implementation of the method for generating a timbre library provided by the present invention;

[0078] Figure 2 A flowchart of a preferred implementation of the sound synthesis method provided by the present invention;

[0079] Figure 3 A preferred implementation system diagram of the timbre library module provided by the present invention;

[0080] Figure 4 A system diagram of a preferred embodiment of the sound synthesis system provided by the present invention;

[0081] Figure 5 A waveform diagram of a preferred implementation of a low-pass kernel in the method for generating a timbre library provided by the present invention. DETAILED DESCRIPTION

[0082] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the following detailed description of the present invention, some specific details are described in detail. In order to avoid confusing the essence of the present invention, known methods, processes, procedures, and components are not described in detail.

[0083] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0084] Unless the context clearly requires otherwise, throughout the specification and claims, the words "include", "comprising" and similar words should be interpreted in an inclusive sense rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to".

[0085] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0086] It should be noted that the use of step numbers (letters or numbers) to refer to certain specific method steps in the present invention is only for the purpose of convenience and brevity of description, and is by no means intended to limit the order of these method steps. Those skilled in the art will appreciate that the order of the relevant method steps should be determined by the technology itself and should not be inappropriately limited by the existence of step numbers.

[0087] This application provides a method for generating a timbre library, such as Figure 1 As shown, the steps include:

[0088] S110: Acquire a sound source signal, and divide the sound source signal into frames to obtain a plurality of sound source audio frames; wherein the sound source signal can be acquired through a device such as a microphone, and one sound source signal comes from one musical instrument.

[0089] After executing S120 to S160 for each sound source audio frame, execute S170. That is, obtain the current sound source audio frame, then execute S120 to S160 for the current sound source audio frame, then obtain the next sound source audio frame as the current sound source audio frame, return to S120 for execution, and continue to execute to S160. This cycle is repeated until all sound source audio frames have executed S120 to S160, and then execute S170.

[0090] S120: Perform windowing processing on the sound source audio frame;

[0091] S130: Calculate the amplitude spectrum of the signal after the windowing process, recorded as a first amplitude spectrum. Specifically, perform Fourier transform on the signal after the windowing process to obtain a frequency domain signal, and obtain the first amplitude spectrum by calculating the absolute value of the frequency domain signal;

[0092] S140: updating the average amplitude spectrum according to the first amplitude spectrum, wherein the initial value of the average amplitude spectrum is 0; that is, each sound source audio frame is updated on the average amplitude spectrum obtained in the previous frame, so that the average amplitude spectrum after the current sound source signal is processed can contain information of each sound source audio frame.

[0093] S150: In the first amplitude spectrum, find the peak whose amplitude is greater than the preset amplitude threshold, record it as the local maximum peak, and determine which low-pass core has the highest waveform distribution similarity with each local maximum peak among multiple preset low-pass cores, record the sum of the products of the local maximum peak and the low-pass core with the highest similarity as the resonance peak occurrence probability, and take the frequency point corresponding to the position of the local maximum peak, the low-pass core with the highest similarity, and the resonance peak occurrence probability local maximum peak corresponding to the local maximum peak as the similarity data group, that is, for each local maximum peak, The peak information will be selected to perform similarity calculation with multiple preset low-pass cores, so as to select the low-pass core with the highest similarity to the local maximum peak, and the sum of the products of the low-pass core and the local maximum peak is recorded as the probability of resonance peak occurrence. In this way, each local maximum peak can obtain a group of similar data groups, which includes the frequency point where the local maximum peak is located, the low-pass core with the highest similarity thereto, and the corresponding probability of resonance peak occurrence. In this group of similarity data groups, the probability of resonance peak occurrence is the probability of resonance peak occurrence in the bandwidth represented by the low-pass core at the frequency point. It can be understood that when the number of peaks greater than the preset amplitude threshold is N in the first amplitude spectrum, N groups of similar data groups can be obtained after executing S150. Among them, the low-pass core is a number of pre-set data distribution curves to show different data distribution situations. The peak value and bandwidth of each low-pass core are different. Each low-pass core has a peak, and some low-pass cores have a larger peak value, some have a smaller peak value, and some have a longer bandwidth, and some have a shorter bandwidth. Through S150, it can be determined which preset data distribution the data distribution of each local maximum peak is closer to, and the probability of a resonance peak appearing at the frequency point corresponding to the local maximum peak with the low-pass kernel having the highest similarity.

[0094] S160: Select the similar data groups corresponding to the first preset number L of the largest probability of occurrence of each resonance peak, and compensate the probability value corresponding to the selected similar data group in the probability table, and the greater the probability of occurrence of the resonance peak, the greater the compensation value of the corresponding probability value. That is, sort the similar data groups in descending order of the probability of occurrence of the resonance peak, select the first L similar data groups, and then find the probability values ​​corresponding to these L similar data groups in the probability table, and compensate these L probability values;

[0095] Among them, the probability table includes the correspondence between frequency points, low-pass kernels and probability values. The frequency points traverse the frequency points in the frequency domain signal, the low-pass kernel traverses the preset low-pass kernels, the frequency points corresponding to the peak positions in the similar data group correspond to the frequency points in the probability table, and the low-pass kernel corresponds to the low-pass kernel in the probability table. And the initial probability values ​​are 0. In this way, according to the frequency points and low-pass kernels of each group of similar data groups, the correspondence in the probability table that satisfies the frequency points and low-pass kernels is found, and then the probability values ​​in the corresponding relationship are compensated. Specifically, the compensation method is that when the probability of the resonance peak in the similar data group is greater, the compensation value of the probability value in the corresponding relationship is greater. It should be noted that the probability table of the present invention is only used to indicate the correspondence between frequency points, low-pass kernels and probability values, and is not limited to its specific form of expression. Specifically, the probability table may be in the form of a table, where the first column represents the frequency points, the second column represents the low-pass kernels, and the third column represents the probability values. The first column and the second column form various combinations of frequency points in the frequency domain signal and preset low-pass kernels. The probability table may also be in the form of a two-dimensional chart, where the horizontal axis represents the frequency points in the frequency domain signal, and the vertical axis represents the low-pass kernels. The intersection of the two represents the corresponding probability value. Of course, the probability table may also be in other forms.

[0096] S170: Determine the corresponding frequency points and low-pass kernels of the first second preset number with the highest probability of occurrence according to the probability table, and record them as the center frequency point and the maximum low-pass kernel respectively. The maximum low-pass kernel represents the bandwidth corresponding to the center frequency point; that is, according to the analysis of each data in the first probability table, find out the positions with the highest possibility of forming resonance peaks. These positions can more accurately reflect the resonance peak situation of the sound source signal, and record the frequency points and low-pass kernels corresponding to the positions, which are recorded as the center frequency point and the maximum low-pass kernel respectively. In this way, the second preset number of center frequency points and the maximum low-pass kernel corresponding to each center frequency point can be found.

[0097] S180: Find the amplitude value corresponding to each center frequency point in the average amplitude spectrum and record it as the maximum amplitude. The corresponding center frequency point, maximum low-pass kernel and maximum amplitude form a resonance peak information. Each resonance peak information forms the timbre information corresponding to the sound source signal in the timbre library. In this way, find the maximum amplitude of the second preset number in the average amplitude spectrum, and then generate resonance peak information including the second preset number.

[0098] In the above method, when executing steps S120 to S160 for each sound source audio frame of the current sound source signal, the values ​​of the average amplitude spectrum and the probability table are updated. After each sound source audio frame completes steps S120 to S160, a probability table and an average amplitude spectrum corresponding to the sound source signal are obtained, and then a resonance peak group that can characterize the sound source signal is found according to the probability table and the average amplitude spectrum. The above method for generating the timbre library executes steps S110 to S180 for each sound source signal to obtain a timbre information corresponding to the sound source signal. Each time a sound source signal is obtained, steps S110 to S180 are executed once to store the timbre information corresponding to the sound source signal. In this way, the timbre information of different sound source signals constitutes a timbre library to be used by processes such as sound synthesis. It should be noted that the generation process of the timbre information can be real-time, or it can be executed in advance before sound synthesis and then stored.

[0099] The method for generating a timbre library of the present invention sets multiple low-pass kernels, uses the low-pass kernels to process the data near the larger peak in the amplitude spectrum to determine the distribution of the local maximum peak, and compensates the probability value in the probability table according to the processing result, and then finds the corresponding center frequency point, low-pass kernel and maximum amplitude in the probability table and the average amplitude spectrum according to the size of the probability of occurrence. These three together characterize a resonance peak information. In this way, the acquired sound source signal will generate multiple resonance peak information to form a resonance peak group, which can characterize the timbre information of the sound source signal. It can be seen that this method for generating a timbre library can more accurately understand the data distribution of each local maximum peak in the sound source audio frame through calculation with the low-pass kernel, thereby improving the compensation accuracy of the probability value in the probability table, thereby improving the accuracy of resonance peak information extraction, and the algorithm is simple, which reduces the occupation of resources, and is particularly suitable for the application of small systems.

[0100] Among them, in step S110, the sound source signal is framed, and the framing method in the prior art can be used for framing, which will not be described in detail here. For each sound source audio frame obtained by framing, some sound source audio frames are empty frames. If S120 to S160 are to be executed, unnecessary resource waste is increased and the timeliness of the sound color library generation is also affected. In order to solve these problems, in a preferred embodiment of the present invention, the energy of the sound source audio frame is first judged. If it is an empty frame, the next frame is directly obtained. If it is not an empty frame, steps S120 to S160 are executed, that is, executing S120 to S160 for each sound source audio frame includes:

[0101] Determine whether the energy of each sound source audio frame is greater than a preset energy value. If so, execute S120 to S160 for the sound source audio frame; if not, reacquire the sound source audio frame, that is, acquire the next sound source audio frame.

[0102] The window length during the windowing process in step S120 can be set in advance to a fixed window length, or can be determined according to the period of each frequency point of the current sound source audio frame. Specifically, step S120 includes the following steps:

[0103] S121: converting the sound source audio frame into a frequency domain signal, and calculating the amplitude spectrum, which is recorded as a second amplitude spectrum, that is, first performing Fourier transform on the sound source audio frame to convert it into a frequency domain signal, and then calculating the absolute value of the frequency domain signal to obtain the second amplitude spectrum;

[0104] S122: Determine the window length according to the frequency points corresponding to the peak positions in the second amplitude spectrum;

[0105] S123: Perform windowing processing on the sound source audio frame according to the windowing length.

[0106] By using the second amplitude spectrum to calculate the window length, the Fourier transform algorithm of the entire algorithm (also used in step S130) can be utilized, thereby reducing the amount of code, reducing the occupation of system resources, and improving the efficiency of timbre library generation. Especially in embodiments of real-time generation of timbre libraries and in small systems (systems with limited storage space, etc.), the use of this method is particularly effective in improving system resources.

[0107] In one embodiment, the above step S122 includes the steps of:

[0108] S122a: Find the position of the first peak in the second amplitude spectrum that is greater than a preset peak threshold, and calculate the corresponding period according to the frequency point corresponding to the position, which is recorded as a preset period T0; wherein the preset peak threshold is related to the frequency point corresponding to each peak position in the first amplitude spectrum;

[0109] S122b: The windowing length is greater than the preset period T0, specifically 1.5 to 2.5 times the preset period T0, such as 1.5 times T0, 2.0 times T0, 2.2 times T0, 2.5 times T0, etc. Preferably, the windowing length is 1.5 times T0.

[0110] The window length is determined by using the frequency point at which the first peak in the second amplitude spectrum is greater than the peak threshold, which simplifies the calculation amount of the window length and further reduces the resource occupation of the system.

[0111] Further preferably, the preset peak threshold in step S122a is less than or equal to the maximum amplitude value in the second amplitude spectrum, such as 0.5 times, 0.7 times, 0.4 times, etc. of the maximum amplitude value. By associating the preset peak threshold with the maximum amplitude value of the second amplitude spectrum, the purpose is to normalize the amplitude for subsequent processing. The preset peak threshold can also directly set an amplitude value, or select a certain amplitude value in the second amplitude spectrum.

[0112] Of course, the above step S122 can also use a correlation method to determine the windowing length, such as calculating the windowing length according to the frequency points corresponding to multiple peak positions at the same time. In this way, since some peak positions correspond to the same frequency points, when using this method to determine the windowing length, it is necessary to first exclude the peak positions with the same frequency points, and then select some or all of the peak positions with different frequency points for calculation. The specific correlation algorithm can be selected differently according to needs, which will not be described in detail here.

[0113] In the above step S140, the average amplitude spectrum is updated by adding the product obtained by multiplying the average amplitude spectrum by the first low-pass coefficient and the product obtained by multiplying the first amplitude spectrum by the second low-pass coefficient as the new average amplitude spectrum, and the sum of the first low-pass coefficient and the second low-pass coefficient is equal to 1. In another embodiment, it is preferred that the first low-pass coefficient is greater than the second low-pass coefficient so that the first amplitude spectrum gradually affects the entire average amplitude spectrum. Further preferably, the first low-pass coefficient is 0.99 and the second low-pass coefficient is 0.01, that is, step S140 specifically includes: adding the average amplitude spectrum multiplied by 0.99 and the second amplitude spectrum multiplied by 0.01 as the new average amplitude spectrum. It should be noted that other values ​​can also be selected for the first low-pass coefficient and the second low-pass coefficient.

[0114] In step S150, the preset amplitude threshold is preferably determined according to the amplitude average value of the first amplitude spectrum, such as 2 times, 2.5 times, 2.8 times, etc. of the amplitude average value. Further preferably, the preset amplitude threshold can be 2.5 times the amplitude average value of the first amplitude spectrum. Specifically, the amplitude average value in the first amplitude spectrum is first calculated, that is, the amplitude values ​​in the first amplitude spectrum are added and averaged. By using this method to determine the preset amplitude threshold, the information of the effective signal (i.e., non-noise) can be more accurately extracted, and the interference of the noise signal can be avoided as much as possible. In another embodiment, the preset amplitude threshold can also be set to be related to the maximum amplitude value in the first amplitude spectrum.

[0115] In the above step S150, the method of determining the low-pass kernel with the highest similarity to the local maximum peak can be determined by image comparison. In a preferred embodiment of the present invention, it is implemented by a dot product sum operation. Specifically, when a peak with an amplitude greater than a preset amplitude threshold is found in the first amplitude spectrum and recorded as a local maximum peak (N as mentioned above), each local maximum peak is processed as follows. Specifically, steps S152 to S154 are executed for each local maximum peak and then S160 is executed. That is to say, step S150 includes the steps of:

[0116] S151: In the first amplitude spectrum, find a peak whose amplitude is greater than a preset amplitude threshold, record it as a local maximum peak, and execute S152 and S153 for each local maximum peak;

[0117] S152: performing a dot product sum operation on the local maximum peak and each preset low-pass kernel respectively to obtain a plurality of dot product values. Specifically, selecting data within a third preset number centered at the position where the local maximum peak is located in the first amplitude spectrum as a calculation data interval, the third preset number being equal to the number of data of the low-pass kernel, such as including data of D positions, and then performing a dot product sum operation on the calculation data interval and each low-pass kernel respectively to obtain a plurality of dot product values, that is, as many dot product values ​​as there are low-pass kernels;

[0118] S153: Find the maximum value among these dot product values, record it as the dot product maximum value, that is, the probability of occurrence of the resonance peak in the similar data group, and record the low-pass kernel when calculating the probability of occurrence of the resonance peak as the low-pass kernel in the similar data group. The frequency point corresponding to the position of the corresponding local maximum peak, the low-pass kernel with the highest similarity and the probability of occurrence of the resonance peak are taken as the similar data group.

[0119] Specifically, if the number of peaks greater than the preset amplitude threshold found in step S151 is N, then steps S152 and S153 are executed N times, and the number of low-pass kernels is M, and the number of each low-pass kernel is D. Then for the i-th local maximum peak, step S152 calculates the dot product sum operation with each low-pass kernel according to the following formula (1).

[0120]

[0121] Where i is the position of the i-th local maximum peak in the first amplitude spectrum, from 1 to N; P[ik] is the amplitude value of the [ik]th position in the first amplitude spectrum, Q j [wk] is the amplitude value of the [wk]th position of the jth low-pass kernel; j ranges from 1 to M, k ranges from -w to +w; w is (D+1) / 2. In this way, step S152 obtains M V i，j , and each V i，j With P[i], Q j Corresponding; in step S153, compare these M V i，j The size of V is the largest. i，j , which is denoted as V imax , and the low-pass kernel used in the calculation is recorded as Q imax , the frequency point corresponding to the i-th local maximum peak position is recorded as S i , so that the similar data set corresponding to the i-th local maximum peak position is (V imax , Q imax , S i ). After executing steps S152 and S153 for each local maximum peak position, N similar data groups are obtained, and then S160 is executed.

[0122] In the above embodiments, there are many ways to generate the low-pass kernel, as long as it can reflect the various distribution laws of the data, such as a triangular curve, a sine curve, etc. A low-pass kernel of the present invention adopts a Gaussian kernel, and further, it is generated according to the following formulas (2) and (3). The curve of this method is smoother and more in line with the distribution of natural probability. In subsequent processing, especially in probability value compensation, it is closer to the actual situation:

[0123]

[0124]

[0125] Where x is the position of the low-pass kernel, and its maximum value can be set according to the sampling frequency; c is the coefficient, and different low-pass kernels have different c values, reflecting the variance of the data in the low-pass kernel; Y[x] is the value of the low-pass kernel at the x-th position; and D is the number of low-pass kernel positions.

[0126] In this low-pass kernel generation method, c represents the degree of data dispersion, which can be specifically set by considering the acquisition frequency. In order to better reflect the degree of data concentration, the present invention preferably uses c of 0.1 to 5, such as 0.1, 0.2, 0.8, 1, 2, 3, 4, 5, etc., so that the value will not be too large. After setting the number of low-pass kernels, you can choose a middle value from this range. For example, when the number of low-pass kernels M is five, you can choose 0.1, 0.3, 1, 3, 5. When the number of low-pass kernels is 11, correspondingly, Figure 5 Curves for the five low-pass kernels in .

[0127] In step S160, for multiple probability values ​​that need to be compensated, in the same sound source audio frame, each compensation value is generally set in a certain order, such as setting each compensation value of the probability value corresponding to the similar data group in the order of the probability of occurrence of the resonance peak (which is the correlation in the similar data group) from small to large. If the difference between the compensation values ​​is too large (such as the difference between two adjacent compensation values ​​is 3, 4 or more), the impact caused by a certain misalignment will be too large, and it will be difficult to correct it in the subsequent accumulation; and if the difference between the compensation values ​​is too small (such as the difference between two adjacent compensation values ​​is 0.2, 0.5, etc.), it is still difficult to distinguish the high and low probability values ​​after compensation. In order to better solve the above problem, in a preferred embodiment of the present invention, the compensation method for the probability value is: the compensation value formula of each probability value is increased by 1 in the order of the corresponding resonance peak occurrence probability from small to large. Specifically, in this step, the V values ​​in the N similar data groups can be firstly calculated. imax Sort by size, select the first preset number, and each V imax Each of them corresponds to a similar data group, and then according to the similar data group (Q imax , S i) Find the probability value of the corresponding coordinate in the probability table and compensate for the probability value, that is, find the coordinate S in the probability table. i , vertical coordinate Q imax is the corresponding probability value, and this value is compensated. Taking the first preset number of five in step S160 as an example, the resonance peak occurrence probability V is selected imax Sort the five largest similar data groups and sort them according to their respective resonance peak occurrence probability V imax Arranged in order from small to large, the probability values ​​in the probability table corresponding to each similar data group are recorded as p1, p2, p3, p4, p5, and the compensation values ​​of p1, p2, p3, p4, p5 are added by 1 in sequence, such as 1, 2, 3, 4, 5, or 2, 3, 4, 5, 6, etc. Further preferably, these compensation values ​​are added by 1 in sequence from small to large starting from 1, that is, in the embodiment where the first preset number is five, p1, p2, p3, p4, p5 are 1, 2, 3, 4, 5, respectively, that is, each time S120-S160 is executed, there will be five probability values ​​in the probability table that are increased by 1, 2, 3, 4, 5, respectively.

[0128] Of course, the compensation values ​​are not limited to increasing in an arithmetic progression, and the difference between two adjacent compensation values ​​may also be unequal.

[0129] Among them, if the second preset number is greater than the first preset number, that is, the number of resonance peaks finally determined is greater than the number of probability value compensation each time, then when the probability value is compensated, it will always accumulate at several fixed frequency points, reducing the accuracy of resonance peak selection. Preferably, the second preset number is less than or equal to the first preset number. Further preferably, the values ​​of the two are equal.

[0130] For the number of low-pass kernels, too many can be more accurate in determining the similarity of the local maximum peak, but it will also increase the amount of calculation. Too few can not accurately reflect the data distribution of the local maximum peak. Considering that the spectrum of sound signals such as speech or music is not very flat, but a series of peaks, the number of effective energy concentration is often mainly at 3 to 7 peaks. Therefore, using the above number will not increase the amount of calculation for the execution of the entire method too much, and can accurately characterize the resonance peak information of the sound. Preferably, the number of low-pass kernels, the first preset number and the second preset number are all selected from 4 to 6, and more preferably, the three are equal, all five, to facilitate the calculation in the entire method.

[0131] The first method of step S170 includes the following steps:

[0132] S171: Calculate the sum of each probability value corresponding to each frequency point in the probability table to obtain the probability sum corresponding to each frequency point, and select the frequency point corresponding to the second preset number with the largest probability sum as the center frequency point; in the probability table, each frequency point corresponds to multiple probability values ​​(the probability values ​​of the first preset number), add these probability values ​​to obtain the probability sum corresponding to the frequency point, obtain the corresponding frequency point and probability sum, then compare these probability sums, select the largest probability sum of the second preset number, and use the frequency points corresponding to these probability sums as the center frequency points;

[0133] S172: For each center frequency point, find the low-pass kernel corresponding to the maximum probability value in the probability table, record it as the maximum low-pass kernel, and the maximum low-pass kernel represents the bandwidth corresponding to the center frequency point, representing the bandwidth; each center frequency point corresponds to multiple low-pass kernels (a first preset number), and then select the low-pass kernel with the largest corresponding probability value as the maximum low-pass kernel, that is, the bandwidth corresponding to the center frequency point, so as to obtain the center frequency points and maximum low-pass kernels corresponding to the second preset number.

[0134] Specifically, in step S171, the probability values ​​of each frequency point are first summed, that is, the probability values ​​corresponding to the same frequency point are added together to obtain the probability sum corresponding to the frequency point, and then the corresponding sequence of the frequency points and the probability sums are obtained. Then, the probability sums of these frequency points are compared, and the frequency points corresponding to the largest probability sums are selected as the center frequencies representing the resonance peak information. Since the probability sums are used for comparison, the probability of occurrence of these frequency points can be more accurately reflected, so that the center frequency of the selected resonance peak is more accurate. For each center frequency point, in the probability table, there are multiple low-pass cores corresponding, that is, they correspond to different bandwidths. In step S172, the low-pass core with the largest probability of occurrence of this center frequency point is selected as the bandwidth at this frequency point, that is, the bandwidth is still selected by the maximum probability value. Therefore, the selection of the bandwidth can also more accurately characterize the bandwidth of the sound source signal at the center frequency point.

[0135] A second method can also be used to select the center frequency and the maximum low-pass kernel. First, select the bandwidth (i.e., the low-pass kernel), that is, first calculate the sum of the probability values ​​corresponding to each low-pass kernel, obtain the probability sum corresponding to each low-pass kernel, and find the second preset number of low-pass kernels with the largest probability sum as the maximum low-pass kernel. Then, the frequency point corresponding to the point with the largest probability value corresponding to each maximum low-pass kernel is used as the center frequency point, so that the center frequency point and the maximum low-pass kernel corresponding to the second preset number are obtained. Alternatively, directly find the probability value with the largest second preset number from the probability table, and the frequency points corresponding to these probability values ​​are recorded as the center frequency points, and the low-pass kernel is recorded as the maximum low-pass kernel.

[0136] Considering that if the number of low-pass kernels is too large, the amount of calculation will be greatly increased, and when this value is large to a certain extent, the information change of the final resonance peak group will not be very large, that is, the effect is not obvious, therefore, the number of low-pass kernels is generally not too large. If the bandwidth is determined by the probability sum method first, it is very likely that all low-pass kernels will be selected in the end, that is, the second method. Undoubtedly, this method reduces the accuracy of the selection of the center frequency point. The number of frequency points is usually much larger than the number of low-pass kernels. The center frequency point is first determined by the probability sum method (that is, the first method), and then the corresponding bandwidth (that is, the low-pass kernel) is determined. This will not only reduce the amount of calculation, but also more accurately reflect the center frequency and its corresponding bandwidth.

[0137] The present invention also provides a sound synthesis method, which can be used for the synthesis of human voice and musical instrument sound, and can also be used for the synthesis of other sounds and timbre in a timbre library, such as Figure 2 As shown, the synthesis comprises the steps of:

[0138] S100: Generate a timbre library using any of the above methods;

[0139] S200: Select a timbre from the timbre library, and generate filters according to each resonance peak information of the selected timbre. Specifically, which timbre is selected can be known when obtaining the sound synthesis instruction, that is, the target instrument to be synthesized, and then the timbre to be synthesized is selected from the timbre library, and the timbre includes a second preset number of resonance peak information, each resonance peak information includes a corresponding center frequency, a maximum low-pass kernel, and a maximum amplitude. A filter can be generated according to this information. Since there is a second preset number of resonance peak information, a second preset number of filters will be generated.

[0140] S300: Acquire a sound signal frame, which can be obtained through a microphone. For example, a human voice can be obtained through a microphone, and the synthesis method is used for synthesizing the human voice.

[0141] S400: Generate a pulse sequence according to the sound signal frame, and use each filter to filter the pulse sequence respectively, and then add each result to obtain a synthetic signal; wherein the period of the pulse sequence is determined according to the fundamental frequency of the sound signal frame, and the pulse sequence is a single pulse sequence;

[0142] S500: Output the synthesized signal as an output signal.

[0143] During sound synthesis, the production methods of the above-mentioned timbre libraries can be used in advance to generate the timbre corresponding to each sound source signal. When sound synthesis is required, the information of the timbre library can be directly called. It is also possible to generate each timbre in real time during sound synthesis. The sound synthesis method of the present invention generates a filter group of a target instrument according to the resonance peak information of the same timbre in the timbre library, generates a pulse sequence for a sound signal frame by extracting the base frequency, and then uses the filter group for filtering to obtain a synthesized signal. This method has a very low storage space requirement for flash, and can process each point of the sound signal frame independently, rather than a block operation. Therefore, it has good real-time performance and is particularly suitable for low-cost embedded sound synthesis processing.

[0144] It should be noted that the above steps S100 and S200 only need to be performed before the step of filtering using a filter in S400, and are not limited to being performed before S300.

[0145] In the above step S400, the sound energy can be directly used as the amplitude of the pulse sequence, and then the synthesized signal is obtained after filtering through a filter. In a preferred embodiment of the present invention, a pulse signal with an amplitude of 1 is first generated, and then the sound energy generated by the sound signal frame is used to perform amplitude modulation on the filtered pulse sequence. Specifically, step 400 includes the steps of:

[0146] S410: using the sound signal frame to update the sound energy, the initial value of the sound energy is 0;

[0147] S420: Performing fundamental frequency detection on the sound signal frame, and obtaining the fundamental frequency of the sound, specifically, extracting fundamental frequency features through a correlation method or other methods well known to those skilled in the art, such as an extraction model trained by a neural network;

[0148] S430: determining the period of the pulse sequence according to the fundamental frequency of the sound, and generating the pulse sequence, that is, the interval between each pulse in the pulse sequence can be obtained according to the fundamental frequency of the sound;

[0149] S440: Using each filter to filter the pulse sequence respectively, and then adding each result to obtain an intermediate signal;

[0150] S450: Use the sound energy to perform amplitude modulation on the intermediate signal to obtain a synthesized signal, that is, multiply the value of each position of the intermediate signal by the value of the corresponding position of the sound energy obtained in step S410 as the value of the corresponding position of the synthesized signal.

[0151] Among them, in the above step S410, for the update of the sound energy, specifically, the product of the sound energy and the third low-pass coefficient can be added to the product of the energy of the current sound signal frame and the fourth low-pass coefficient, and the sum obtained can be used as the sound energy of the current sound signal frame. The sum of the third low-pass coefficient and the fourth low-pass coefficient is 1, and the third low-pass coefficient is much larger than the fourth low-pass coefficient, such as the third low-pass coefficient is 0.99 and the fourth low-pass coefficient is 0.01. Of course, the two can also use other values ​​respectively. The above fundamental frequency detection can be implemented by the fundamental frequency detection method of the prior art, such as the correlation method.

[0152] By adopting the above method, the energy of the current sound signal frame is correlated with the energy of the previous sound signal frame, thereby making the sound of the output signal smoother; and after some sound processing, sound effect processing is required in the later stage, and some sound effect processing also requires volume adjustment. When the amplitude modulation is placed after the filtering in the above manner, the subsequent volume adjustment can be combined with step S450 for execution, thereby improving the efficiency of sound synthesis; and if the amplitude modulation is placed in the front, since many sound signal frames are silent frames, a lot of volume change information will be lost.

[0153] The filter in step S200 is preferably a second-order wireless impulse response filter. Specifically, the method of generating a filter according to each resonance peak information can be generated by Butterworth or other filter algorithms, and each resonance peak information will be converted into a corresponding set of coefficients: a1, a2, b0, b1, b2. Then the pulse sequence filtered by one of the filters can be calculated according to the following formula:

[0154] u[m]=b0×v[m]+b1×v[m-1]+b2×v[m-2]+a1×u[m-1]+a2×u[m-2];

[0155] Among them, u[m] is the value of the m-th position of the pulse sequence after filtering by a filter; v[m] is the value of the m-th position of the pulse sequence before filtering; v[m-1] is the value of the m-1-th position of the pulse sequence before filtering; v[m-2] is the value of the m-2-th position of the pulse sequence before filtering; u[m-1] is the value of the m-1-th position of the pulse sequence after filtering by a filter; u[m-2] is the value of the m-2-th position of the pulse sequence after filtering by a filter. In this way, after filtering by the second preset number of filters respectively, u[m] at the same position are added to obtain the pulse sequence after filtering.

[0156] It should be noted that as long as the above S410 is executed before S450, it is not limited to that S410 must be executed first before S420 can be executed.

[0157] In order to improve the sound quality of the output signal, the above step S500 specifically includes:

[0158] S510: Processing the synthesized signal through an effector to obtain an output signal;

[0159] S520: Output the output signal.

[0160] Specifically, the effector in step S510 includes at least one of a delay device, a reverberator and a wah pedal effector.

[0161] Among them, in the above-mentioned embodiments, the Fourier transform at each location can be a radix-2 Fourier transform. In a preferred embodiment of the present invention, a split-radix fast Fourier transform is used to improve the time-frequency domain conversion rate.

[0162] In a specific embodiment, in the above-mentioned synthesis method, the sound source signal comes from a musical instrument; the sound signal frame includes a human voice from a microphone, so that after being processed by the synthesis method, the human voice can be converted into the sound of a musical instrument, such as in a karaoke device, the singing performed by the human voice can be played as the singing of a musical instrument (i.e., a timbre in the timbre library).

[0163] The present invention also provides a timbre library module, which is used to execute the timbre library generation method of any of the above embodiments, such as Figure 3 As shown, the timbre library module 10 includes:

[0164] A sound source acquisition unit 11 acquires a sound source signal and divides the sound source signal into frames to obtain a plurality of sound source audio frames;

[0165] The probability table unit 12 is connected to the sound source acquisition unit 11, and is used to perform the following processing on each sound source audio frame: perform windowing processing on the sound source audio frame; convert the signal after windowing processing into a frequency domain signal, which is recorded as a first amplitude spectrum; update the average amplitude spectrum according to the first amplitude spectrum, wherein the initial value of the average amplitude spectrum is 0; in the first amplitude spectrum, find the peak whose amplitude is greater than the preset amplitude threshold, which is recorded as the local maximum peak, and determine which low-pass core has the highest waveform distribution similarity with each local maximum peak among multiple preset low-pass cores, record the sum of the products of the local maximum peak and the low-pass core with the highest similarity as the probability of resonance peak occurrence, and take the frequency point corresponding to the position of the local maximum peak, the low-pass core with the highest similarity, and the resonance peak occurrence probability corresponding to the local maximum peak as a similar data group; wherein the peak value and bandwidth of each low-pass core are different; and select the similar data group corresponding to the first preset number with the highest probability of each resonance peak occurrence, and compensate the probability value corresponding to the selected similar data group in the probability table, and the greater the probability of resonance peak occurrence, the greater the compensation value of the corresponding probability value;

[0166] The probability table includes a correspondence between frequency points, low-pass kernels and probability values, the frequency points traverse the frequency points in the frequency domain signal, and the initial probability values ​​are 0;

[0167] The timbre library generating unit 13 is connected to the probability table unit 12, and is used to determine the corresponding frequency points and low-pass kernels of the first second preset number with the highest probability of occurrence according to the probability table, which are respectively recorded as the center frequency point and the low-pass kernel; wherein the second preset number is less than or equal to the first preset number; and then find the amplitude value corresponding to each center frequency point in the average amplitude spectrum, which is recorded as the maximum amplitude, and the corresponding center frequency point, maximum low-pass kernel and maximum amplitude form a resonance peak information, and each resonance peak information forms the timbre corresponding to the sound source signal in the timbre library.

[0168] Furthermore, the present invention also provides a sound synthesis system, such as Figure 4 As shown, including:

[0169] The above-mentioned sound library module 10;

[0170] The filter module 20 is connected to the timbre library module 10 and is used to select a timbre in the timbre library and generate filters according to each formant information of the selected timbre;

[0171] The synthesis module 30 is connected to the filter module 20, and is used to obtain a sound signal frame, generate a pulse sequence according to the sound signal frame, and use each filter to filter the pulse sequence respectively, and then add each result to obtain a synthesized signal; and output the synthesized signal as an output signal.

[0172] The present invention further provides an audio processing chip, in which a circuit module is integrated, and the circuit module is configured to implement the method for generating a timbre library as described in any of the above embodiments or the method for synthesizing sound as described in any of the above embodiments.

[0173] In addition, the present invention also provides an electronic device, including a speaker, a karaoke, a voice toy or a device with a host sound card, the electronic device includes the above-mentioned audio processing chip, an input module and an output module, the audio processing chip has an input interface and an output interface, the input module is connected to the input interface, and is used to obtain the sound source signal and / or sound signal frame, that is, the input interface can obtain the sound source signal required by the timbre library module 10 through the input module to generate timbre, and can also obtain the sound signal frame required by the synthesis module 30 through the input module, so that the synthesis module performs sound synthesis. The output module is connected to the output interface, and is used to output the output signal.

[0174] Specifically, the input module includes a microphone; the output module includes a speaker or an earphone.

[0175] In addition, the present invention also provides a computer-readable storage medium, such as a CD, a USB flash drive, a hard disk, etc., on which a computer program is stored, and when the computer program is executed, the method for generating a timbre library as described in any of the above embodiments or the method for synthesizing a sound as described in any of the above embodiments is implemented. When the computer program is executed, it can be presented as a demo visual dialog box, or it can be directly an executable exe file.

[0176] Those skilled in the art will appreciate that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0177] It should be understood that the above-mentioned embodiments are merely illustrative and not restrictive. Without departing from the basic principles of the present invention, various obvious or equivalent modifications or substitutions that can be made by those skilled in the art to the above-mentioned details will all be included in the scope of the claims of the present invention.

Claims

1. A method for generating a timbre library, characterized in that: Includes steps: S110: Acquire a sound source signal, and divide the sound source signal into frames to obtain a plurality of sound source audio frames; After executing S120 to S160 for each of the sound source audio frames, execute S170; S120: performing windowing processing on the sound source audio frame; S130: Calculate the amplitude spectrum of the signal after the windowing process, recorded as the first amplitude spectrum; S140: updating an average amplitude spectrum according to the first amplitude spectrum, wherein an initial value of the average amplitude spectrum is 0; S150: In the first amplitude spectrum, find a peak whose amplitude is greater than a preset amplitude threshold value, record it as a local maximum peak, and determine which low-pass core has the highest waveform distribution similarity with each local maximum peak among multiple preset low-pass cores, record the sum of the products of the local maximum peak and the low-pass core with the highest similarity as the resonance peak occurrence probability, and take the frequency point corresponding to the position of the local maximum peak, the low-pass core with the highest similarity, and the resonance peak occurrence probability corresponding to the local maximum peak as a similarity data group; Wherein, the peak value and bandwidth of each low-pass core are different; S160: selecting similar data groups corresponding to the first preset number of similar data groups with the largest probability of occurrence of the resonance peaks in each similar data group, and compensating the probability values ​​corresponding to the selected similar data groups in the probability table, and the greater the probability of occurrence of the resonance peaks, the greater the compensation value of the corresponding probability value; The probability table includes a correspondence between frequency points, low-pass kernels and probability values, the frequency points traverse the frequency points in the frequency domain signal, and the initial probability values ​​are 0; S170: Determine, according to the probability table, the corresponding frequency points and low-pass kernels of the first second preset number with the highest probability of occurrence, and record them as the center frequency point and the maximum low-pass kernel, respectively; S180: Find the amplitude value corresponding to each of the center frequencies in the average amplitude spectrum and record it as the maximum amplitude. The corresponding center frequency, the maximum low-pass kernel and the maximum amplitude form a resonance peak information. Each resonance peak information forms the timbre corresponding to the sound source signal in the timbre library.

2. The method for generating a timbre library according to claim 1, characterized in that: The step S120 comprises the following steps: S121: converting the sound source audio frame into a frequency domain signal, and calculating the amplitude spectrum, which is recorded as a second amplitude spectrum; S122: Determine a windowing length according to the frequency points corresponding to the peak positions in the second amplitude spectrum; S123: Perform windowing processing on the sound source audio frame according to the windowing length.

3. The method for generating a timbre library according to claim 2, characterized in that: The step S122 comprises the steps of: S122a: Find the position of the first peak in the second amplitude spectrum that is greater than a preset peak threshold, and calculate the corresponding period according to the frequency point corresponding to the position, which is recorded as a preset period; wherein the preset peak threshold is determined according to the frequency points corresponding to each peak position in the first amplitude spectrum; S122b: The windowing length is 1.5 to 2.5 times the preset period.

4. The method for generating a timbre library according to claim 3, characterized in that: The preset peak threshold in step S122a is smaller than the maximum amplitude value in the second amplitude spectrum.

5. The method for generating a timbre library according to claim 1, characterized in that: The step S140 specifically includes: adding a product obtained by multiplying the average amplitude spectrum by 0.99 and a product obtained by multiplying the second amplitude spectrum by 0.01 to obtain a new average amplitude spectrum.

6. The method for generating a timbre library according to claim 1, characterized in that: The preset amplitude threshold in step S150 is determined according to the amplitude average value of the first amplitude spectrum.

7. The method for generating a timbre library according to claim 1, characterized in that: The step S150 includes: S151: In the first amplitude spectrum, find a peak whose amplitude is greater than a preset amplitude threshold, record it as a local maximum peak, and execute S152 and S153 for each local maximum peak; S152: performing dot product sum operations on the local maximum peak and each of the preset low-pass kernels to obtain a plurality of dot product values; S153: Find the maximum value among multiple point product values, which is the probability of occurrence of the resonance peak in the similar data group. The low-pass kernel used in calculating the probability of occurrence of the resonance peak is the low-pass kernel in the similar data group. The frequency point corresponding to the position of the corresponding local maximum peak, the low-pass kernel with the highest similarity, and the probability of occurrence of the resonance peak corresponding to the local maximum peak are taken as the similar data group.

8. The method for generating a timbre library according to claim 1, characterized in that: The low-pass kernel is determined by the following formula: Where x is the position of the low-pass kernel, c is the coefficient, and different low-pass kernels have different c values; Y[x] is the value of the low-pass kernel at the x-th position; and D is the number of low-pass kernel positions.

9. The method for generating a timbre library according to claim 1, characterized in that: In the step S160, the probability value is compensated in such a way that the compensation value of each probability value is increased by 1 in the order of the corresponding resonance peak occurrence probability from small to large.

10. The method for generating a timbre library according to claim 1, characterized in that: The second preset number is less than or equal to the first preset number, and each is selected from 4 to 6.

11. The method for generating a timbre library according to any one of claims 1 to 10, characterized in that: The step S170 includes the following steps: S171: Calculate the sum of each probability value corresponding to each frequency point in the probability table to obtain the probability sum corresponding to each frequency point, and select the frequency point corresponding to the second preset number with the largest probability sum as the central frequency point; S172: For each of the central frequency points, find the corresponding maximum probability value in the probability table, and the low-pass kernel corresponding to the maximum probability value is the maximum low-pass kernel.

12. The method for generating a timbre library according to any one of claims 1 to 10, characterized in that: Executing S120 to S160 on each of the sound source audio frames includes: Determine whether the energy of each of the sound source audio frames is greater than a preset energy value, and if so, execute S120 to S160.

13. A sound synthesis method, characterized in that: Includes steps: S100: Generate a timbre library using the method described in any one of claims 1 to 12; S200: Select a timbre from the timbre library, and generate filters according to each formant information of the selected timbre; S300: Acquire a sound signal frame; S400: generating a pulse sequence according to the sound signal frame, and filtering the pulse sequence using each of the filters respectively, and then adding the results to obtain a synthetic signal; wherein the period of the pulse sequence is determined according to the fundamental frequency of the sound signal frame; S500: Output the synthesized signal as an output signal.

14. The synthesis method according to claim 13, characterized in that: The step 400 comprises the steps of: S410: using the sound signal frame to update the sound energy, wherein the initial value of the sound energy is 0; S420: Performing fundamental frequency detection on the sound signal frame to obtain the sound fundamental frequency; S430: determining a period of the pulse sequence according to the fundamental frequency of the sound, and generating the pulse sequence; S440: Using each of the filters to filter the pulse sequence respectively, and then adding the results to obtain an intermediate signal; S450: Use the sound energy to perform amplitude modulation on the intermediate signal to obtain a synthesized signal.

15. The synthesis method according to claim 13 or 14, characterized in that: The step S500 specifically includes the following steps: S510: Processing the synthesized signal through an effector to obtain the output signal; S520: Output the output signal.

16. The synthesis method according to claim 15, characterized in that The effector in step S510 includes at least one of a delay device, a reverberator and a wah pedal effector.

17. The synthesis method according to any one of claims 13 to 16, characterized in that: The sound source signal comes from a musical instrument; The sound signal frame includes a human voice from a microphone.

18. A timbre library module, characterized in that: include: A sound source acquisition unit acquires a sound source signal and divides the sound source signal into frames to obtain a plurality of sound source audio frames; The probability table unit is connected to the sound source acquisition unit and is used to perform the following processing on each of the sound source audio frames: The sound source audio frame is subjected to windowing processing; the signal after the windowing processing is converted into a frequency domain signal, which is recorded as a first amplitude spectrum; the average amplitude spectrum is updated according to the first amplitude spectrum, wherein the initial value of the average amplitude spectrum is 0; in the first amplitude spectrum, a peak whose amplitude is greater than a preset amplitude threshold is found, which is recorded as a local maximum peak, and it is determined which low-pass core has the highest waveform distribution similarity with each local maximum peak among multiple preset low-pass cores, and the sum of the products of the local maximum peak and the low-pass core with the highest similarity is recorded as the probability of resonance peak occurrence, and the frequency point corresponding to the position of the local maximum peak is recorded as , the probability of occurrence of the resonance peak corresponding to the low-pass kernel with the highest similarity and the local maximum peak as a similar data group; wherein the peak value and bandwidth of each low-pass kernel are different; and the similar data group corresponding to the first preset number of the resonance peaks with the largest probability of occurrence is selected, and the probability value corresponding to the selected similar data group in the probability table is compensated, and the greater the probability of occurrence of the resonance peak, the greater the compensation value of the corresponding probability value; wherein the probability table includes the corresponding relationship between frequency points, low-pass kernels and probability values, and the frequency points traverse the frequency points in the frequency domain signal, and the initial probability values ​​are 0; The timbre library generating unit is connected to the probability table unit, and is used to determine the corresponding frequency points and low-pass kernels of the first second preset number with the highest probability of occurrence according to the probability table, which are recorded as the center frequency point and the maximum low-pass kernel respectively; find the amplitude value corresponding to each of the center frequency points in the average amplitude spectrum, which is recorded as the maximum amplitude, and the corresponding center frequency point, the maximum low-pass kernel and the maximum amplitude form a resonance peak information, and each resonance peak information forms the timbre corresponding to the sound source signal in the timbre library.

19. A sound synthesis system, characterized in that: include: The sound library module of claim 18; A filter module, connected to the timbre library module, for selecting a timbre from the timbre library and generating filters according to each formant information of the selected timbre; A synthesis module, connected to the filter module, for acquiring a sound signal frame, generating a pulse sequence according to the sound signal frame, and filtering the pulse sequence using each of the filters respectively, and then adding the results to obtain a synthesized signal; and for outputting the synthesized signal as an output signal.

20. An audio processing chip, in which a circuit module is integrated, characterized in that: The circuit module is configured to implement the method for generating a timbre library as described in any one of claims 1 to 12 or the method for synthesizing a sound as described in any one of claims 13 to 17.

21. An electronic device, characterized in that: It comprises the audio processing chip, input module and output module as described in claim 20, the audio processing chip has an input interface and an output interface, the input module is connected to the first input interface, and is used to obtain the sound source signal and / or the sound signal frame; the output module is connected to the output interface, and is used to output the output signal.

22. The electronic device according to claim 21, characterized in that: The input module includes a microphone, and the output module includes a speaker or an earphone.

23. The electronic device according to claim 21 or 22, characterized in that: The electronic device includes a speaker, a karaoke box, a voice toy or a device with a host sound card.

24. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the method for generating a timbre library as described in any one of claims 1 to 12 or the method for synthesizing a sound as described in any one of claims 13 to 17 is implemented.

Images