A method and system for determining the fundamental frequency of a pipa string

Abstract

This invention provides a method and system for determining the fundamental frequency of a pipa (a traditional Chinese stringed instrument). The method involves performing time-frequency transformation on an audio signal to extract spectral peaks, weighting the peak energy according to a preset harmonic attenuation function, and setting a candidate fundamental frequency set. First, an initial harmonic cluster is constructed based on initial anharmonic coefficients. The cluster with the highest energy is selected to calculate real-time anharmonic coefficients, and then these coefficients are used to reconstruct the candidate harmonic clusters. Each harmonic cluster is evaluated from two aspects: harmonic energy continuity and time-domain correlation. The total harmonic energy is calculated and combined with a missing component penalty to obtain an energy continuity score. Simultaneously, the time correlation score is obtained by calculating the mean time cross-correlation between the fundamental frequency and the energy envelopes of each harmonic component. A weighted sum is then used to obtain a comprehensive score.

Description

Technical Field

[0001] This application belongs to the field of fundamental frequency determination, and in particular relates to a method and system for determining the fundamental frequency of a pipa string. Background Technology

[0002] As an important traditional Chinese plucked string instrument, the pipa experiences pitch drift due to the expansion and contraction of its soundboard and frets, as well as changes in the tension of its nylon-steel strings. This necessitates frequent pitch checks before performances. Current technology, primarily using external electronic tuners, cannot address the real-time, minute pitch shifts caused by environmental changes within the instrument. Achieving automatic tuning hinges on identifying the fundamental frequency, i.e., pitch.

[0003] The overtones of the pipa are not strictly integer multiples of the fundamental frequency, exhibiting anharmonicity, which degrades the performance of methods based on ideal harmonic models. During pipa playing, higher harmonic energy decays rapidly, and some harmonic components may even be missing, making algorithms relying on complete harmonic structures prone to misjudgments, such as octave or fifth errors. Furthermore, the impact noise generated at the moment of plucking the strings has high energy and a wide spectrum, easily interfering with the detection of stable harmonic components and affecting the accuracy and stability of the algorithm. Currently, mainstream fundamental frequency determination algorithms can be categorized into time-domain, frequency-domain, and time-frequency-domain methods. Time-domain methods, such as the autocorrelation function method, estimate the fundamental frequency based on the periodicity of the signal, but are sensitive to phase changes and susceptible to noise interference. Frequency-domain methods, such as the harmonic summation method and the harmonic product spectrum method, utilize the energy distribution characteristics of harmonics, but these methods struggle to address the anharmonicity issues present in instruments like the pipa and are prone to failure when fundamental frequency components are missing. Existing technologies, when processing instrument signals with complex harmonic structures and anharmonicity, rarely incorporate the continuity of the harmonic structure and the temporal variation information of each harmonic component. Therefore, how to design a method that can handle anharmonicity, suppress noise interference, and comprehensively utilize spectral and temporal characteristics to improve the accuracy of determining the fundamental frequency of the pipa string is a problem that urgently needs to be solved in the current technical field. Summary of the Invention

[0004] This invention proposes a method for determining the fundamental frequency of a pipa string, addressing the problems of insufficient accuracy in determining the fundamental frequency of a pipa string due to the difficulty in handling anharmonicity and suppressing noise interference in existing technologies, and the failure to comprehensively utilize spectral and time-domain characteristics. The method includes the following steps:

[0005] The audio signal of the pipa strings is acquired and time-frequency transformed to obtain a time-spectrum graph; in each spectral frame of the time-spectrum graph, the spectral peaks are extracted, and the energy of the spectral peaks is weighted according to a preset pipa timbre harmonic attenuation function to obtain a weighted peak set;

[0006] Set a candidate fundamental frequency set; based on the initial anharmonicity coefficients, search and construct preliminary candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set; select the preliminary candidate harmonic cluster with the highest total energy, and calculate the real-time anharmonicity coefficients based on the actual frequencies of the harmonics within the preliminary candidate harmonic clusters; use the real-time anharmonicity coefficients to re-search and construct candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set.

[0007] For each candidate harmonic cluster, the energy of all weighted peak values ​​within the harmonic cluster is accumulated, and a continuity penalty is applied based on the absence of harmonics within a predetermined order to obtain an energy continuity score. The energy envelopes of the fundamental frequency component and all other harmonic components within the harmonic cluster are extracted on a continuous spectral frame. The mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the other harmonic components is calculated to obtain a time correlation score. The energy continuity score and the time correlation score are weighted and summed to obtain a comprehensive score.

[0008] The candidate harmonic cluster with the highest comprehensive score is selected, and the candidate fundamental frequency corresponding to the harmonic cluster is determined as the fundamental frequency of the pipa string.

[0009] Optionally, the time-frequency transformation is a short-time Fourier transform, and a predetermined sampling rate, window function, window length, and frame shift are set.

[0010] Optionally, the step of weighting the energy of the spectral peaks according to a preset pipa timbre harmonic attenuation function includes:

[0011] Weighted energy The calculation formula is: ,in, The original energy of the spectral peak of harmonic order n. This is the preset attenuation factor.

[0012] Optionally, the set of candidate fundamental frequencies includes:

[0013] Within a preset frequency range, discrete candidate fundamental frequency points are generated with a preset frequency resolution to form the candidate fundamental frequency set.

[0014] Optionally, the step of using the real-time anharmonicity coefficients to re-search and construct candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set includes:

[0015] For each candidate fundamental frequency Based on the real-time anharmonicity coefficient B, the theoretical frequency of the k-th harmonic is calculated according to the following anharmonicity relation. : ;

[0016] At the theoretical frequency Within the preset frequency window above and below, the peak with the largest energy in the weighted peak set is searched as the k-th harmonic component.

[0017] Optionally, the step of accumulating the energy of all weighted peak values ​​within the harmonic cluster and applying a continuity penalty based on the absence of harmonics within a predetermined order to obtain an energy continuity score includes:

[0018] The total energy is obtained by summing the energies of all weighted peak values ​​within the candidate harmonic cluster;

[0019] The number of missing harmonics m within a predetermined order range is counted, and the total energy is multiplied by a preset penalty factor. The energy continuity score is obtained by raising the value of m to the power of m.

[0020] Optionally, the step of extracting the energy envelopes of the fundamental frequency component and other harmonic components within the harmonic cluster on a continuous spectral frame, and calculating the mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the other harmonic components to obtain a time correlation score includes:

[0021] Extract the energy envelope of the fundamental frequency component and the energy envelope of N harmonic components within a predetermined order range;

[0022] Calculate the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of each harmonic component;

[0023] The time correlation score is obtained by taking the arithmetic mean of all calculated cross-correlation coefficients.

[0024] Optionally, the weighted summation of the energy continuity score and the time correlation score to obtain a comprehensive score includes:

[0025] Weighting coefficients are assigned to the energy continuity score and the time correlation score, respectively. The comprehensive score is obtained by weighting and summing the energy continuity score and the time correlation score using the weighting coefficients.

[0026] Furthermore, the present invention also relates to a system for determining the fundamental frequency of a pipa string, comprising the following modules:

[0027] An extraction module is used to acquire the audio signal of the pipa strings and perform time-frequency transformation to obtain a time-frequency spectrum; in each spectrum frame of the time-frequency spectrum, the spectrum peaks are extracted, and the energy of the spectrum peaks is weighted according to a preset pipa timbre harmonic attenuation function to obtain a weighted peak set;

[0028] The module is used to set a set of candidate fundamental frequencies; based on the initial anharmonicity coefficients, it searches and constructs preliminary candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set; it selects the preliminary candidate harmonic cluster with the highest total energy, and calculates the real-time anharmonicity coefficients based on the actual frequencies of the harmonics within the preliminary candidate harmonic clusters; using the real-time anharmonicity coefficients, it re-searches and constructs candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set.

[0029] The calculation module is used to accumulate the energy of all weighted peak values ​​within each candidate harmonic cluster and apply a continuity penalty based on the absence of harmonics within a predetermined order to obtain an energy continuity score; extract the energy envelopes of the fundamental frequency component and other harmonic components within the harmonic cluster on a continuous spectrum frame, calculate the mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the other harmonic components to obtain a time correlation score; and perform a weighted summation of the energy continuity score and the time correlation score to obtain a comprehensive score.

[0030] The determination module is used to select the candidate harmonic cluster with the highest comprehensive score, and determine the candidate fundamental frequency corresponding to the harmonic cluster as the fundamental frequency of the pipa string.

[0031] Optionally, the time-frequency transformation is a short-time Fourier transform, and a predetermined sampling rate, window function, window length, and frame shift are set.

[0032] Optionally, the step of weighting the energy of the spectral peaks according to a preset pipa timbre harmonic attenuation function includes:

[0033] Weighted energy The calculation formula is: ,in, The original energy of the spectral peak of harmonic order n. This is the preset attenuation factor.

[0034] Optionally, the set of candidate fundamental frequencies includes:

[0035] Within a preset frequency range, discrete candidate fundamental frequency points are generated with a preset frequency resolution to form the candidate fundamental frequency set.

[0036] Optionally, the step of using the real-time anharmonicity coefficients to re-search and construct candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set includes:

[0037] For each candidate fundamental frequency Based on the real-time anharmonicity coefficient B, the theoretical frequency of the k-th harmonic is calculated according to the following anharmonicity relation. : ;

[0038] At the theoretical frequency Within the preset frequency window above and below, the peak with the largest energy in the weighted peak set is searched as the k-th harmonic component.

[0039] Optionally, the step of accumulating the energy of all weighted peak values ​​within the harmonic cluster and applying a continuity penalty based on the absence of harmonics within a predetermined order to obtain an energy continuity score includes:

[0040] The total energy is obtained by summing the energies of all weighted peak values ​​within the candidate harmonic cluster;

[0041] The number of missing harmonics m within a predetermined order range is counted, and the total energy is multiplied by a preset penalty factor. The energy continuity score is obtained by raising the value of m to the power of m.

[0042] Optionally, the step of extracting the energy envelopes of the fundamental frequency component and other harmonic components within the harmonic cluster on a continuous spectral frame, and calculating the mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the other harmonic components to obtain a time correlation score includes:

[0043] Extract the energy envelope of the fundamental frequency component and the energy envelope of N harmonic components within a predetermined order range;

[0044] Calculate the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of each harmonic component;

[0045] The time correlation score is obtained by taking the arithmetic mean of all calculated cross-correlation coefficients.

[0046] Optionally, the weighted summation of the energy continuity score and the time correlation score to obtain a comprehensive score includes:

[0047] Weighting coefficients are assigned to the energy continuity score and the time correlation score, respectively. The comprehensive score is obtained by weighting and summing the energy continuity score and the time correlation score using the weighting coefficients.

[0048] This invention utilizes a preset harmonic attenuation function for pipa timbre to weight the spectral peaks, highlighting harmonic components that match the pipa's timbre characteristics, suppressing noise interference, and initially improving the accuracy of harmonic identification. Through an initial search of initial anharmonic coefficients and a secondary precise search of real-time anharmonic coefficients calculated based on actual harmonic frequencies, it aligns anharmonic phenomena caused by the physical characteristics of the strings, improving the accuracy of harmonic cluster construction. Evaluation is conducted from the perspective of harmonic energy and continuity, penalizing missing harmonics to avoid misjudgments. Furthermore, it utilizes the time correlation analysis between the fundamental frequency and the energy envelope of each harmonic component, leveraging the strong temporal correlation of components from the same sound source to enhance the ability to distinguish target string tones from background noise in complex acoustic environments, achieving high-precision determination of the pipa string's fundamental frequency. Attached Figure Description

[0049] Figure 1 A flowchart of the first embodiment;

[0050] Figure 2 This is a schematic diagram of spectrum peak extraction and weighting;

[0051] Figure 3 A schematic diagram for constructing candidate harmonic clusters;

[0052] Figure 4 A schematic diagram of energy continuity score;

[0053] Figure 5 This is a schematic diagram illustrating the time correlation of the energy envelope.

[0054] Figure 6 A schematic diagram for determining the comprehensive score and fundamental frequency. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] In the first embodiment, the present invention proposes a method for determining the fundamental frequency of a pipa string, such as... Figure 1 This includes the following steps:

[0057] A) Obtain the audio signal of the pipa strings and perform time-frequency transformation to obtain a time-frequency spectrum; extract the spectral peaks in each spectral frame of the time-frequency spectrum, and weight the energy of the spectral peaks according to the preset pipa timbre harmonic attenuation function to obtain a weighted peak set;

[0058] Specifically, single notes played on the pipa are captured by a microphone and converted from analog to digital at a sampling rate of 44100Hz to obtain a digital audio signal. This digital audio signal is then windowed and framed using a Hamming window, with a frame length of 4096 sampling points and a frame shift of 1024 sampling points. A fast Fourier transform is performed on each frame, and the amplitude spectrum of the transform results is arranged in chronological order to generate a time-frequency spectrum. Within a stable sound segment of the time-frequency spectrum, all frequency points are traversed. If the energy of a frequency point is higher than the energy of the two adjacent frequency points and higher than a preset global noise threshold, then that frequency point and its energy are recorded as a spectral peak. In one embodiment, the pipa timbre harmonic attenuation function is A(k) = 1 / k, where k is the harmonic order. For each spectral peak, the possible harmonic order k of the spectral peak is estimated, and the energy is multiplied by k for weighted compensation to generate a weighted peak set.

[0059] In some embodiments, the time-frequency transformation is a short-time Fourier transform, and a predetermined sampling rate, window function, window length, and frame shift are set.

[0060] The one-dimensional pipa audio time-series signal is converted into a two-dimensional time-frequency representation, namely a spectrogram. The core parameters of the Short-Time Fourier Transform (SFT) are set. For example, for a 16-bit PCM format pipa solo audio sampled at 44100Hz, a Hanning window is set to reduce spectral leakage, and the window length is set to 2048 sampling points to obtain a frequency resolution of approximately 21.5Hz. Simultaneously, a frame shift of 512 sampling points is set, indicating a 75% overlap between frames, thus ensuring a smooth temporal transition. The audio signal is divided into multiple consecutive data frames of 2048 sampling points each, and each frame is multiplied point-by-point by the Hanning window function. A Fast Fourier Transform is performed on each windowed frame to obtain the complex spectrum of that time frame. The squared or absolute value of the modulus of each spectral component is taken to obtain the energy spectrum or amplitude spectrum. Arranging the spectra of all time frames along the time axis creates a spectrogram. The horizontal axis represents time, and the vertical axis represents frequency. The brightness or color of each point in the graph represents the energy level of the corresponding time and frequency point. This spectrogram serves as the data basis for all subsequent harmonic detection and analysis.

[0061] In order to adjust the peak energy in the spectrum according to the inherent timbre characteristics of the pipa, in some embodiments, the step of weighting the energy of the spectral peaks according to a preset pipa timbre harmonic attenuation function includes:

[0062] The preset harmonic decay function for the pipa tone is an exponential decay function;

[0063] If the original energy of the peak value of the spectrum with harmonic order n is The preset attenuation factor is The weighted energy The calculation formula is: .

[0064] Statistical analysis of the spectra of a large number of standard pipa pitch samples reveals the average trend of harmonic energy attenuation with increasing order. Based on this, a fixed attenuation factor is set, for example... This value reflects the characteristic of the pipa's tone that each subsequent harmonic typically retains about 85% of the energy of the preceding harmonic. Assuming that within a certain time frame, candidate harmonic peaks around a fundamental frequency are detected, the original energies of the first four harmonics of that fundamental frequency are respectively... , , and The weighted energies, calculated using the exponential decay formula, are as follows: , , , This weighting reduces the weight of spurious peaks or noise peaks that do not conform to the natural attenuation pattern of the pipa, thereby improving the accuracy of subsequent fundamental frequency estimation. Figure 2 As shown.

[0065] B) Set a candidate fundamental frequency set; based on the initial anharmonicity coefficients, search and construct preliminary candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set; select the preliminary candidate harmonic cluster with the highest total energy, and calculate the real-time anharmonicity coefficients based on the actual frequencies of the harmonics within the preliminary candidate harmonic clusters; use the real-time anharmonicity coefficients to re-search and construct candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set.

[0066] For each candidate fundamental frequency Based on the initial anharmonicity coefficient, the allowable frequency offset range is set, and the theoretical harmonic frequency is expressed as... Where k is the harmonic order, in the weighted peak set, with the theoretical harmonic frequency as the center, the actual spectral peaks are searched within the tolerance band determined by the initial anharmonicity coefficient. If one or more weighted peaks exist within the corresponding frequency window, the peak closest to the theoretical harmonic frequency or with the largest weighted energy is selected as the matching result of that harmonic order. Thus, the peaks matched for each order are collected to obtain the preliminary candidate harmonic clusters corresponding to the candidate fundamental frequency. For all preliminary candidate harmonic clusters corresponding to the candidate fundamental frequency, the weighted energy of each harmonic peak within them is accumulated, and the preliminary candidate harmonic cluster with the largest total energy is obtained. Using the deviation between the actual frequency of each harmonic in the harmonic cluster and its corresponding theoretical harmonic frequency, the relative offset or mean square deviation is calculated and normalized to obtain the real-time anharmonicity coefficient that reflects the actual anharmonicity of the current string vibration, representing the true anharmonicity characteristics under the fundamental frequency assumption. The calculated real-time anharmonicity coefficients are used as new frequency tolerance parameters to replace the initial anharmonicity coefficients, and harmonic searches are performed again for each candidate fundamental frequency. Specifically, for each candidate fundamental frequency, the search window width of each order harmonic is adaptively adjusted according to the updated anharmonicity coefficients to make the search range more closely match the actual anharmonic distribution. The corresponding spectral peaks of each order theoretical harmonic are matched again in the weighted peak set, thereby reconstructing a more accurate and stable candidate harmonic cluster.

[0067] More specifically, a frequency set ranging from 50Hz to 1500Hz with a step size of 0.1Hz is defined as the candidate fundamental frequency set; the initial anharmonicity coefficient B is set to 0.0001; for each candidate fundamental frequency... Using aharmonic relations Calculate the theoretical frequencies of the first 15 harmonics. For each theoretical frequency Search within the weighted peak set for the existence of an actual peak whose frequency is within a certain range. If the peak value exists within a ±2% error window, then the peak value is classified into... The corresponding initial candidate harmonic clusters; traverse all initial candidate harmonic clusters, calculate and compare the sum of the energies of all weighted peak values ​​within them, and find the harmonic cluster with the largest total energy; use the actual frequencies of all harmonics within this cluster. With order n, the anharmonic coefficients B in the anharmonicity relation are obtained by fitting the equation using the least squares method, thus obtaining the real-time anharmonic coefficients; for each candidate fundamental frequency Using this real-time anharmonicity coefficient B, the theoretical harmonic frequency is recalculated and a matching weighted peak value is searched to construct a candidate harmonic cluster.

[0068] In some embodiments, the setting of candidate base frequencies includes:

[0069] Within a preset frequency range, discrete candidate fundamental frequency points are generated with a preset frequency resolution to form the candidate fundamental frequency set.

[0070] The efficiency and accuracy of fundamental frequency searching are improved by narrowing the search range through a limited and reasonable list of potential fundamental frequencies. First, the frequency range of the instrument is determined. Based on the physical structure and conventional playing range of the pipa, a preset frequency range is set, for example, starting from the lowest note of the pipa. The corresponding frequency range from 110Hz to the highest frequency. That is, 1760Hz. This range covers all possible fundamental frequencies, avoiding unnecessary calculations in invalid frequency regions. To strike a balance between computational accuracy and computational cost, a frequency resolution, such as 1Hz, is set. Starting from the lower limit of the frequency range at 110Hz, discrete frequency points are generated sequentially in 1Hz increments until the upper limit of 1760Hz is reached. This constitutes a candidate fundamental frequency set containing multiple discrete frequency points {110, 111, 112, ..., 1759, 1760}. In subsequent analysis, each frequency point in this set will be evaluated one by one to determine the probability that the point is the true fundamental frequency.

[0071] In some embodiments, the step of re-searching and constructing candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set using the real-time anharmonicity coefficients includes:

[0072] For each candidate fundamental frequency Based on the real-time anharmonicity coefficient B, the theoretical frequency of the k-th harmonic is calculated according to the following anharmonicity relation. : ;

[0073] At the theoretical frequency Within the preset frequency window above and below, the peak with the largest energy in the weighted peak set is searched as the k-th harmonic component.

[0074] The anharmonicity caused by string stiffness is used to more accurately locate harmonics. For each candidate fundamental frequency in the candidate fundamental frequency set, for example... The theoretical frequencies of each harmonic are calculated using the anharmonicity coefficient (B = 0.0002) estimated in real time from the audio signal (Hz) and the anharmonicity coefficient (B = 0.0002). For example, the theoretical frequency of the second harmonic is... Hz, not the ideal 440Hz. Similarly, for all harmonic orders to be considered, such as 1 to 15, a list of theoretical harmonic frequencies is generated. For each calculated theoretical harmonic frequency... Set a small search window, for example, ±5Hz. Taking Hz as an example, within the frequency range of [435.176, 445.176] Hz, find the weighted spectrum peak obtained in the previous steps. If one or more peaks exist within this range, select the one with the highest weighted energy. The largest peak value, as a reference to the candidate fundamental frequency The second harmonic component corresponding to Hz. Repeat the above search process for all theoretical harmonic frequencies to construct a candidate harmonic cluster containing the actual spectral peaks for the candidate fundamental frequency, such as... Figure 3 .

[0075] C) For each candidate harmonic cluster, the energy of all weighted peak values ​​within the harmonic cluster is accumulated, and a continuity penalty is applied according to the absence of harmonics within a predetermined order to obtain an energy continuity score; the energy envelopes of the fundamental frequency component and the remaining harmonic components within the harmonic cluster are extracted on a continuous spectrum frame, and the mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the remaining harmonic components is calculated to obtain a time correlation score; the energy continuity score and the time correlation score are weighted and summed to obtain a comprehensive score;

[0076] For each candidate harmonic cluster, the energy of all weighted peak values ​​within the cluster is summed to obtain a basic energy score. The first 10 harmonics are checked for missing values; for each missing harmonic, the basic energy score is multiplied by a penalty factor of 0.9, resulting in an energy continuity score. For the harmonic cluster, the energy values ​​of the fundamental frequency peak and each harmonic peak within 20 consecutive spectral frames on the time-spectrum graph are extracted to obtain their respective energy envelope sequences. The Pearson correlation coefficient between the fundamental frequency energy envelope sequence and the energy envelope sequences of other harmonics is calculated, and the arithmetic mean of all calculated correlation coefficients is taken to obtain a time correlation score. Weights are then assigned. It is 0.7. The score is 0.3, and the overall score is = ×Energy continuity score+ × Time-related score.

[0077] In some embodiments, the step of accumulating the energy of all weighted peak values ​​within the harmonic cluster and applying a continuity penalty based on the absence of harmonics within a predetermined order to obtain an energy continuity score includes:

[0078] The energy continuity score is calculated by summing the energies of all weighted peak values ​​within the candidate harmonic cluster to obtain the total energy. ;

[0079] The number of missing harmonics m within a predetermined order range is counted, and the total energy is multiplied by a preset penalty factor. The energy continuity score is obtained by raising the value of m to the power of m. ,Right now: .

[0080] Accumulate harmonic energies and count missing harmonics. Assume a candidate harmonic cluster is constructed for a candidate fundamental frequency, and the intended harmonic range for analysis is the first 10 orders. After searching, the 1st, 2nd, 3rd, 5th, 6th, 7th, and 9th harmonics are successfully found, with a weighted total energy of [value missing]. The 4th, 8th, and 10th harmonics did not find any peaks within their respective search windows, therefore the number of missing harmonics is m=3. A preset penalty factor is further set, for example... A value less than 1 indicates that for each missing harmonic, the total score will be penalized proportionally. Based on the example data above, the score is... A high-energy harmonic cluster with a complete harmonic sequence will achieve higher [performance / quality]. Conversely, even harmonic clusters with high total energy but incomplete structure will have lower scores, such as... Figure 4 As shown.

[0081] In some embodiments, the step of extracting the energy envelopes of the fundamental frequency component and other harmonic components within the harmonic cluster on a continuous spectral frame, and calculating the mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the other harmonic components to obtain a time correlation score includes:

[0082] The time correlation score is calculated by extracting the energy envelope of the fundamental frequency component. and the energy envelope of N harmonic components within a predetermined order range ;

[0083] Calculate the time cross-correlation coefficient between the energy envelope of the fundamental frequency component and the energy envelopes of each harmonic component. ;

[0084] For all calculated cross-correlation coefficients The arithmetic mean is used to obtain the time correlation score. ,Right now: .

[0085] To evaluate the consistency of energy changes across the time dimension of components within a candidate harmonic cluster, for a candidate harmonic cluster, the energy changes of the fundamental frequency component and N harmonic components within a predetermined order range, such as the first 8 harmonic components, are tracked over multiple consecutive time frames, such as 20 time frames, forming a set of time-series data, i.e., the energy envelope. and ,in The envelope reflects the onset, duration, and decay of each frequency component over time. The fundamental frequency energy envelope is calculated. With each harmonic energy envelope Normalized cross-correlation coefficients between The coefficients represent the degree of synchronous change between the two envelope curves, with values ​​ranging from -1 to 1; the closer to 1, the better the synchronicity. For example, the first eight cross-correlation coefficients might be 0.99, 0.97, 0.98, 0.95, 0.96, 0.93, 0.94, and 0.91. The time correlation score is obtained by summing the N cross-correlation coefficients and calculating their arithmetic mean. For the example data above, The higher the score, the more likely the components of the harmonic cluster originate from the same source, such as... Figure 5 .

[0086] In some embodiments, the weighted summation of the energy continuity score and the time correlation score to obtain a comprehensive score includes:

[0087] Weighting coefficients are assigned to the energy continuity score and the time correlation score, respectively. The comprehensive score is obtained by weighting and summing the energy continuity score and the time correlation score using the weighting coefficients.

[0088] By weighted fusion of the evaluation metrics from the two dimensions mentioned above, a comprehensive score is derived to judge the merits of each candidate fundamental frequency. An energy continuity score is then assigned based on prior knowledge or experimental tuning. Time correlation score Assign weights. For example, if the integrity of the harmonic structure is considered slightly more important than temporal synchronization, weights can be set... and The sum of the two weighting coefficients is typically 1, used to balance the contribution of different features to the decision. For a candidate fundamental frequency, assume the calculated energy continuity score is... The time correlation score is The weighted summation formula was used to calculate the overall score, which was S = 0.94992. This process was repeated for each member of the candidate fundamental frequency set to obtain the candidate fundamental frequency with the highest overall score S, which was then determined as the most reliable pipa note fundamental frequency in the current audio frame.

[0089] D), select the candidate harmonic cluster with the highest comprehensive score, and determine the candidate fundamental frequency corresponding to the harmonic cluster as the fundamental frequency of the pipa string.

[0090] Sort all candidate harmonic clusters by their overall scores and select the cluster with the highest score. The candidate fundamental frequency corresponding to the highest-scoring harmonic cluster is the determined fundamental frequency of the pipa string. Figure 6 .

[0091] In a second embodiment, the present invention also provides a system for determining the fundamental frequency of a pipa string, comprising the following modules:

[0092] An extraction module is used to acquire the audio signal of the pipa strings and perform time-frequency transformation to obtain a time-frequency spectrum; in each spectrum frame of the time-frequency spectrum, the spectrum peaks are extracted, and the energy of the spectrum peaks is weighted according to a preset pipa timbre harmonic attenuation function to obtain a weighted peak set;

[0093] The module is used to set a set of candidate fundamental frequencies; based on the initial anharmonicity coefficients, it searches and constructs preliminary candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set; it selects the preliminary candidate harmonic cluster with the highest total energy, and calculates the real-time anharmonicity coefficients based on the actual frequencies of the harmonics within the preliminary candidate harmonic clusters; using the real-time anharmonicity coefficients, it re-searches and constructs candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set.

[0094] The calculation module is used to accumulate the energy of all weighted peak values ​​within each candidate harmonic cluster and apply a continuity penalty based on the absence of harmonics within a predetermined order to obtain an energy continuity score; extract the energy envelopes of the fundamental frequency component and other harmonic components within the harmonic cluster on a continuous spectrum frame, calculate the mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the other harmonic components to obtain a time correlation score; and perform a weighted summation of the energy continuity score and the time correlation score to obtain a comprehensive score.

[0095] The determination module is used to select the candidate harmonic cluster with the highest comprehensive score, and determine the candidate fundamental frequency corresponding to the harmonic cluster as the fundamental frequency of the pipa string.

[0096] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0097] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining the fundamental frequency of a pipa string, characterized in that, Includes the following steps: The audio signal of the pipa strings is acquired and time-frequency transformed to obtain a time-spectrum graph; in each spectral frame of the time-spectrum graph, the spectral peaks are extracted, and the energy of the spectral peaks is weighted according to a preset pipa timbre harmonic attenuation function to obtain a weighted peak set; Define a candidate fundamental frequency set; based on the initial anharmonicity coefficients, search for and construct preliminary candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set; The initial candidate harmonic cluster with the highest total energy is selected, and the real-time anharmonicity coefficient is calculated based on the actual frequency of the harmonics within the initial candidate harmonic cluster. Using the real-time anharmonicity coefficient, a new candidate harmonic cluster is constructed for each candidate fundamental frequency in the weighted peak set. For each candidate harmonic cluster, the energy of all weighted peak values ​​within the harmonic cluster is accumulated, and a continuity penalty is applied according to the absence of harmonics within a predetermined order to obtain an energy continuity score. Extract the energy envelopes of the fundamental frequency component and other harmonic components within the harmonic cluster on a continuous spectrum frame, calculate the mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the other harmonic components, and obtain the time correlation score. The energy continuity score and the time correlation score are weighted and summed to obtain a comprehensive score; The candidate harmonic cluster with the highest comprehensive score is selected, and the candidate fundamental frequency corresponding to the harmonic cluster is determined as the fundamental frequency of the pipa string. The step of weighting the energy of the spectral peaks according to a preset pipa timbre harmonic attenuation function includes: Weighted energy The calculation formula is: ,in, The original energy of the spectral peak of harmonic order n. The preset attenuation factor; The step of re-searching and constructing candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set using the real-time anharmonicity coefficients includes: For each candidate fundamental frequency Based on the real-time anharmonicity coefficient B, the theoretical frequency of the k-th harmonic is calculated according to the following anharmonicity relation. ; At the theoretical frequency Within the preset frequency window above and below, the peak with the largest energy in the weighted peak set is searched as the k-th harmonic component; The energy of all weighted peak values ​​within the harmonic cluster is accumulated, and a continuity penalty is applied based on the absence of harmonics within a predetermined order to obtain an energy continuity score, including: The total energy is obtained by summing the energies of all weighted peak values ​​within the candidate harmonic cluster; The number of missing harmonics m within a predetermined order range is counted, and the total energy is multiplied by a preset penalty factor. The energy continuity score is obtained by raising the value of m to the power of m. The process involves extracting the energy envelopes of the fundamental frequency component and all other harmonic components within the harmonic cluster on a continuous spectral frame, calculating the mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the other harmonic components, and obtaining a time correlation score, including: Extract the energy envelope of the fundamental frequency component and the energy envelope of N harmonic components within a predetermined order range; Calculate the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of each harmonic component; The time correlation score is obtained by taking the arithmetic mean of all calculated cross-correlation coefficients.

2. The method according to claim 1, characterized in that, The time-frequency transformation is a short-time Fourier transform, and a predetermined sampling rate, window function, window length, and frame shift are set.

3. The method according to claim 1, characterized in that, The set of candidate fundamental frequencies includes: Within a preset frequency range, discrete candidate fundamental frequency points are generated with a preset frequency resolution to form the candidate fundamental frequency set.

4. The method according to claim 1, characterized in that, The weighted summation of the energy continuity score and the time correlation score to obtain the comprehensive score includes: Weighting coefficients are assigned to the energy continuity score and the time correlation score, respectively. The comprehensive score is obtained by weighting and summing the energy continuity score and the time correlation score using the weighting coefficients.

5. A system for determining the fundamental frequency of a pipa string, used to implement the method as described in any one of claims 1 to 4, characterized in that, Includes the following modules: An extraction module is used to acquire the audio signal of the pipa strings and perform time-frequency transformation to obtain a time-frequency spectrum; in each spectrum frame of the time-frequency spectrum, the spectrum peaks are extracted, and the energy of the spectrum peaks is weighted according to a preset pipa timbre harmonic attenuation function to obtain a weighted peak set; The construction module is used to set the candidate fundamental frequency set; based on the initial anharmonicity coefficients, it searches for and constructs preliminary candidate harmonic clusters for each candidate fundamental frequency in the weighted peak set; The initial candidate harmonic cluster with the highest total energy is selected, and the real-time anharmonicity coefficient is calculated based on the actual frequency of the harmonics within the initial candidate harmonic cluster. Using the real-time anharmonicity coefficient, a new candidate harmonic cluster is constructed for each candidate fundamental frequency in the weighted peak set. The calculation module is used to accumulate the energy of all weighted peak values ​​within each candidate harmonic cluster and apply a continuity penalty based on the absence of harmonics within a predetermined order to obtain an energy continuity score. Extract the energy envelopes of the fundamental frequency component and other harmonic components within the harmonic cluster on a continuous spectrum frame, calculate the mean of the time cross-correlation coefficients between the energy envelope of the fundamental frequency component and the energy envelopes of the other harmonic components, and obtain the time correlation score. The energy continuity score and the time correlation score are weighted and summed to obtain a comprehensive score; The determination module is used to select the candidate harmonic cluster with the highest comprehensive score, and determine the candidate fundamental frequency corresponding to the harmonic cluster as the fundamental frequency of the pipa string.

Images