Method for cutting an audio signal, computing device and storage medium

Abstract

Embodiments of the present application relate to a method, device and medium for cutting an audio signal. The method comprises obtaining a plurality of continuous frames of audio signals of an audio signal to be cut; extracting a plurality of information of pitch information, volume information and speech information of each frame of audio signal in the obtained plurality of continuous frames of audio signals; generating an audio feature of each frame of audio signal in the plurality of continuous frames of audio signals based on the extracted plurality of information; predicting, via a first neural network model, a distance from each frame of audio signal in the plurality of continuous frames of audio signals to a nearest note transition point for the audio feature within a first sliding window range of each frame of audio signal; and performing a regression fitting for the predicted distance to determine a position of the note transition point for cutting the audio signal to be cut. Thus, the accuracy of cutting for a vocal melody audio can be effectively improved.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of audio processing technology, and more specifically to a method, computing device and storage medium for cutting audio signals. Background Technology

[0002] Traditional methods for segmenting audio signals determine the note transition points for segmentation by calculating the probability that a point is a note transition point, such as by identifying note transition points through frequency detection.

[0003] The aforementioned traditional methods for segmenting audio signals all have shortcomings. The former may cause other frames near the true note transition point to fall within the probability judgment threshold range, resulting in non-note transition points being identified as note transition points, or they may be affected by the characteristics of the audio signal (such as consonants and breathy sounds in human singing), causing some note transition points to be unrecognized. The latter may be due to the pitch instability of the audio signal (such as pitch jitter in human singing), which may cause interference to note transition points and result in insufficient accuracy.

[0004] In summary, the shortcomings of traditional methods for segmenting audio signals are that they are not accurate enough in identifying note transition points, which makes it impossible to accurately segment audio signals according to note changes. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a method, computing device, and storage medium for segmenting audio signals, which can effectively improve the accuracy of segmenting human vocal melody audio.

[0006] According to a first aspect of the present invention, a method for cutting an audio signal is provided, comprising: acquiring a series of consecutive frames of audio signals relating to an audio signal to be cut; extracting various information from pitch information, volume information, and speech information of each frame of the acquired series of consecutive frames of audio signals; generating audio features for each frame of the consecutive frames of audio signals based on the extracted various information; predicting the distance from each frame of the consecutive frames of audio signals to the nearest note transition point via a first neural network model for the audio features within a first sliding window range relating to each frame of the audio signals; and performing regression fitting on the predicted distance to determine the location of the note transition point for cutting the audio signal to be cut.

[0007] In some embodiments, the audio features for each frame of audio signal include one or more of the following: frequency-related features extracted based on pitch information, including frequency variation features, cumulative frequency variation features, frequency confidence features, and / or frequency confidence variation features; and energy-related features extracted based on volume information, including volume features, volume variation features, phase features, amplitude features, and / or amplitude variation features.

[0008] In some embodiments, the audio features for each frame of audio signal further include: Mel frequency cepstral coefficient variation features extracted based on speech information, the Mel frequency cepstral coefficient variation features and / or frequency confidence variation features used to determine speech features about note transition points, the speech features being related to at least one of the initial, final, consonant, vowel, textual attributes of the sound, and mode of sound production.

[0009] In some embodiments, before extracting the pitch information of each frame of audio signal in the acquired consecutive multi-frame audio signal, the method includes: predicting the frequency of each frame of audio signal in the multi-frame audio signal via a second neural network model, and obtaining frequency reliability information of the predicted frequency of the second neural network model for each frame of audio signal; calculating the pitch of each frame of audio signal based on the extracted frequency of each frame of audio signal; and smoothing the calculated pitch of each frame of audio signal to obtain the smoothed pitch of each frame of audio signal, so as to extract the pitch information of each frame of audio signal in the acquired consecutive multi-frame audio signal based on the smoothed pitch.

[0010] In some embodiments, smoothing the pitch of each calculated frame of audio signal includes: calculating the influence weights of the pitch of each frame of audio signal within the second sliding window relative to the current frame of audio signal when the current frame of audio signal is the center frame within the second sliding window, based on the analytic hierarchy process (AHP), including: setting each frame within the second sliding window as a scheme layer; setting the frequency confidence information of the predicted frequency of each frame of audio signal within the second sliding window, the distance of each frame of audio signal within the second sliding window to the current frame of audio signal, and the pitch difference between each frame of audio signal within the second sliding window and the current frame of audio signal as a criterion layer; setting the influence weights of the pitch of each frame of audio signal within the second sliding window relative to the current frame of audio signal as a decision layer; and smoothing the pitch of the current frame of audio signal based on the calculated influence weights and the pitch of each frame of audio signal within the second sliding window.

[0011] In some embodiments, predicting the distance of each frame of audio signal to the nearest note transition point includes: in response to determining that the current frame of audio signal is at a predetermined frame position within a first sliding window range, acquiring audio features of each frame of audio signal within the first sliding window range; based on the acquired audio features of each frame, predicting the distance of the current frame of audio signal to the nearest note transition point via a first neural network model; based on the time series of multiple consecutive frames of audio signal, shifting the first sliding window backward by one frame to predict the distance of the next frame of audio signal to the nearest note transition point; and sequentially predicting the distance of each frame of audio signal in the multiple consecutive frames of audio signal to the nearest note transition point when it is at a predetermined frame position within the first sliding window range, until the rear boundary of the first sliding window overlaps with the last frame of audio signal in the multiple consecutive frames of audio signal.

[0012] In some embodiments, predicting the distance from each frame of audio signal to the nearest note transition point further includes: in response to determining that the first sliding window includes multiple first sliding windows, wherein the distance from the center frame of any one of the multiple first sliding windows to a predetermined frame position is different from the distance from the center frame of the other first sliding windows to the predetermined frame position; based on each first sliding window, predicting the distance from each frame of audio signal to the nearest note transition point in a series of consecutive frames of audio signal for each first sliding window, respectively, for audio features within each first sliding window range for each frame of audio signal, via a first neural network model; aggregating the acquired distances from each frame of audio signal to the nearest note transition point in a series of consecutive frames of audio signal for each first sliding window; and performing regression fitting on the aggregated distances to determine the position of the note transition point for cutting the audio signal to be cut.

[0013] In some embodiments, regression fitting for the predicted distance includes: initializing a fitting curve for note transition points based on a time series of consecutive multi-frame audio signals; adding a note transition point starting from the first frame of the audio signal based on the time series of consecutive multi-frame audio signals, and gradually moving the note transition point backward with a step size of one frame of audio signal, until it is determined that the curve fitted by the added note transition point at the current position is closest to the distance from the nearest note transition point in each frame of audio signal in the region; and after determining the position of a note transition point, adding a new note transition point after a predetermined safe distance from the determined note transition point, and continuing regression fitting until no more new note transition points can be added.

[0014] According to a second aspect of the present invention, a computing device is provided, the computing device comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform steps according to the method of the first aspect.

[0015] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a machine, implements the method according to the first aspect.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0017] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0018] Figure 1 A schematic diagram of a system for implementing a method for cutting audio signals according to an embodiment of the present invention is shown.

[0019] Figure 2 A flowchart of a method for cutting audio signals according to an embodiment of the present invention is shown.

[0020] Figure 3 A flowchart of a method for smoothing the pitch of each frame of audio signal calculated according to an embodiment of the present invention is shown.

[0021] Figure 4 A flowchart of a method for predicting the distance to a note transition point according to an embodiment of the present invention is shown.

[0022] Figure 5 A flowchart of another method for predicting the distance of a note transition point according to an embodiment of the present invention is shown.

[0023] Figure 6 A flowchart of a method for regression fitting of a note transition point position curve according to an embodiment of the present invention is shown.

[0024] Figure 7 A schematic diagram of fitting the distance of note transition points by regression through multiple first sliding windows according to an embodiment of the present invention is shown.

[0025] Figure 8 A block diagram of an electronic device according to an embodiment of the present invention is shown.

[0026] Figure 9 This diagram illustrates a comparison between algorithmic and manual annotation results for segmenting a musical phrase recording. Detailed Implementation

[0027] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0029] As described earlier, traditional methods for segmenting audio signals, especially human voice audio signals, have shortcomings. For example, calculating the probability that a point is a note transition point may cause other frames near the true note transition point to also fall within the probability judgment threshold, resulting in non-note transition points being identified as note transition points. Or, due to the characteristics of the audio signal (such as consonants and breathy sounds in human singing), some note transition points may not be identified. For example, identifying note transition points by frequency detection may be affected by the pitch instability of the audio signal (such as pitch jitter in human singing), leading to interference with note transition points and insufficient accuracy.

[0030] For example, a method that calculates the probability that a point is a note transition point might yield a 40% probability for point 'a'. When the note's starting point is clear, this 40% probability usually indicates that the point is not actually a note transition point, but rather a point near one. However, when the note's starting point is unclear (e.g., due to glissando, legato, plosives, consonants, breathy sounds, etc.; and in the case of audio recordings of singing, the pronunciation and content of the lyrics can blur the note's starting point—for example, singing the triplets 'do-do-do' with 'ba' is more likely to blur the starting point than 'ai'), the 40% probability point is highly likely to be a true note transition point when the starting point is unclear. Furthermore, this method of identifying note transition points through probability calculation might also result in a situation where only one true note transition point should be identified more than once.

[0031] In summary, the shortcomings of traditional methods for segmenting audio signals are that they are not accurate enough in identifying note transition points, which makes it impossible to accurately segment audio signals according to note changes.

[0032] To at least partially address one or more of the aforementioned problems and other potential issues, an exemplary embodiment of the present invention proposes a scheme for segmenting audio signals. In this scheme, multiple consecutive frames of audio signals to be segmented are acquired; various information, including pitch, volume, and speech information, is extracted from each frame of the acquired multiple consecutive frames; based on the extracted information, audio features are generated for each frame; and for the audio features within a first sliding window range of each frame, a first neural network model is used to predict the distance from each frame to the nearest note transition point. The present invention can obtain a sequence of distances from each frame to the nearest transition point for the multiple consecutive frames of audio signals to be segmented, using the distance between each frame and the nearest transition point as the basis for predicting the transition point.

[0033] Furthermore, the present invention can perform regression fitting on the predicted distance to determine the position of the note transition point for cutting the audio signal to be cut. Thus, by performing regression fitting on the distance of each frame of audio signal to the nearest transition point to predict the position of the note transition point of the audio signal to be cut, the prediction accuracy can be improved, and the situation of the same note transition point being repeatedly identified or missing note transition points can be avoided.

[0034] Therefore, the present invention can accurately determine the note transition points of an audio signal and accurately segment the audio signal based on the determined note transition points.

[0035] Figure 1A schematic diagram of a system 100 for implementing a method for cutting audio signals according to an embodiment of the present invention is shown. Figure 1 As shown, system 100 includes a computing device 110, an audio acquisition device 130, and a network 140. The computing device 110 and the audio acquisition device 130 can be communicatively connected to exchange data, for example, through the network 140 (e.g., the Internet); for example, through Bluetooth; or through a data cable connection.

[0036] The audio acquisition device 130 may be, for example, a microphone, recorder, voice recorder, sound card, or other recording equipment. The audio acquisition device 130 can acquire sound, such as human voice in this solution. The type and number of the audio acquisition device 130 can be determined based on the actual equipment conditions and usage requirements of the scenario to pick up and acquire human voice, thereby generating audio segments to be labeled. The audio acquisition device 130 can also provide the acquired human voice to the computing device 110. In some embodiments, the computing device 110 can interact with the audio acquisition device 130 to control the audio acquisition device 130 to acquire human voice.

[0037] Regarding the computing device 110, it is used, for example, to acquire human voice audio collected by the audio acquisition device 130 to obtain an audio signal to be segmented, such as a recording file of Zhang San singing a song. After acquiring the human voice audio, the computing device 110 can perform preliminary segmentation on the acquired human voice audio to obtain an audio signal to be segmented. For example, in the aforementioned recording file of Zhang San singing, the first half minute is blank, which can be removed, and the audio signal to be segmented can be generated starting from where Zhang San begins singing. The computing device 110 can also segment the audio signal to be segmented in units of frames, dividing the audio signal to be segmented into multiple consecutive frames of audio signal, for example, segmenting the audio signal to be segmented into frames of 10ms (milliseconds) to generate multiple consecutive frames of audio signal about the audio signal to be segmented.

[0038] The computing device 110 may have one or more processing units, including dedicated processing units such as GPUs, FPGAs, and ASICs, and general-purpose processing units such as CPUs. Additionally, one or more virtual machines may run on each computing device 110. In some embodiments, the computing device 110 and the audio acquisition device 130 may be integrated together or separately configured. In some embodiments, the computing device 110 includes, for example, an audio signal acquisition module 112, an audio information extraction module 114, an audio feature generation module 116, a distance prediction module 118, and a note transition point location determination module 120.

[0039] The audio signal acquisition module 112 is used to acquire multiple consecutive frames of audio signals related to the audio signal to be cut.

[0040] The audio information extraction module 114 is used to extract pitch information, volume information, and various other information from the speech information of each frame of the acquired multi-frame audio signal.

[0041] Regarding the audio feature generation module 116, it is used to generate audio features for each frame of audio signal in a series of consecutive audio signals based on the extracted information.

[0042] Regarding the distance prediction module 118, it is used to predict the distance from each frame of audio signal to the nearest note transition point in a series of consecutive audio signals via a first neural network model, based on the audio features within a first sliding window range of each frame of audio signal.

[0043] Regarding the note transition point location determination module 120, it is used to perform regression fitting on the predicted distance to determine the location of the note transition point for cutting the audio signal to be cut.

[0044] Figure 2 A flowchart of a method 200 for cutting an audio signal according to an embodiment of the present invention is shown. Method 200 may be performed by, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 8 The method is performed at the illustrated electronic device 800. It should be understood that method 200 may also include additional steps not shown and / or the steps shown may be omitted, and the scope of the invention is not limited in this respect.

[0045] In step 202, the computing device 110 acquires a series of audio signals about the audio signal to be cut.

[0046] Regarding continuous multi-frame audio signals, for example, taking a frame of 10ms (milliseconds) as an example, for the audio signal S to be segmented, generate the continuous multi-frame audio signal F = {f1, f2, f3, f4, f5, ..., f...} 1000}, where f1 to f 1000 Each frame in the video is 10 milliseconds long.

[0047] In step 204, the computing device 110 extracts various information from the pitch information, volume information, and speech information of each frame of the acquired multi-frame audio signal.

[0048] In some embodiments, before extracting the pitch information of each frame of audio signal in the acquired consecutive multi-frame audio signal, the method includes: predicting the frequency of each frame of audio signal in the multi-frame audio signal via a second neural network model, and obtaining frequency reliability information of the predicted frequency of the second neural network model for each frame of audio signal; calculating the pitch of each frame of audio signal based on the extracted frequency of each frame of audio signal; so as to extract the pitch information of each frame of audio signal in the acquired consecutive multi-frame audio signal. Furthermore, the calculated pitch of each frame of audio signal may be smoothed to obtain the smoothed pitch of each frame of audio signal, so as to extract the pitch information of each frame of audio signal in the acquired consecutive multi-frame audio signal based on the smoothed pitch.

[0049] Regarding the second neural network model, an example is the CREPE model, CEPEE (Convolutional Rectified Phase Encoding Neural Network).

[0050] Regarding frequency reliability information, for example, in this scheme, a neural network model is used to detect the pitch of an audio frame. Frequency reliability information refers to the reliability of the output of the neural network. For example, if the second neural network model is the CREPE model, then the frequency reliability information represents the reliability of the frequency information predicted by the CREPE model for the current frame.

[0051] Therefore, before extracting the pitch information of each frame of audio signal from the acquired continuous multi-frame audio signal, each frame of audio signal is smoothed to eliminate the influence of human voice fluctuations on frequency and reduce the influence of low-frequency sound, so that each frame can more realistically and clearly reflect its pitch characteristics after smoothing.

[0052] The following will combine Figure 3 The detailed method for smoothing the pitch of each calculated frame of audio signal will not be elaborated here.

[0053] In step 206, the computing device 110 generates audio features for each frame of the audio signal in a series of consecutive frames of audio signals based on the extracted information.

[0054] In some embodiments, the audio features for each frame of audio signal include one or more of the following: frequency-related features extracted based on pitch information, such as frequency variation features, cumulative frequency variation features, frequency confidence features, and / or frequency confidence variation features; and energy-related features extracted based on volume information, such as volume features, volume variation features, phase features, amplitude features, and / or amplitude variation features.

[0055] Regarding the characteristics of cumulative frequency changes, this can be illustrated by comparing the cumulative frequency (pitch) changes of the audio signals of n consecutive frames preceding and following the target frame audio data. For example, using a series of consecutive audio signals F = {f1, f2, f3, f4, f5, ..., f...} 1000 The 100th frame audio signal f in} 100 For example, compare "f" 75 to f 100 "and "f 100 to f 125 The cumulative frequency change characteristics. For example, when a person sings, the duration of a note is usually no less than 150ms (milliseconds). The pitch change may be small every 10ms. By combining the frequency change characteristics with the cumulative frequency change characteristics over a longer range, we can better determine whether there is a note transition point and the location of the note transition point.

[0056] It should be understood that, based on actual needs, frequency-related features extracted from pitch information are not limited to the features listed above, and energy-related features extracted from volume information are not limited to the features listed above.

[0057] In some embodiments, the audio features for each frame of audio signal further include: Mel frequency cepstral coefficient variation features extracted based on speech information, the Mel frequency cepstral coefficient variation features and / or frequency confidence variation features used to determine speech features about note transition points, the speech features being related to at least one of the initial, final, consonant, vowel, textual attributes of the sound, and mode of sound production.

[0058] Regarding Mel-Frequency Cepstral Coefficients (MFCCs), in the field of sound processing, the Mel-Frequency Cepstrum is a linear transformation of the logarithmic energy spectrum based on the nonlinear Mel scale of sound frequencies.

[0059] Regarding the extraction of speech information, for example, the scikits.talkbox library in Python can be used to obtain the corresponding data of the Mel frequency cepstral spectrum. Extracting speech information, such as lyrics, makes it easier to identify note transition points.

[0060] In step 208, the computing device 110 predicts the distance from each frame of audio signal to the nearest note transition point via a first neural network model, based on the audio features within a first sliding window range for each frame of audio signal.

[0061] Regarding the first neural network, it is, for example, an LSTM (Long Short-Term Memory) neural network; the first neural network can also be, for example, a GRU (Gate Recurrent Unit), a Transformer model, or a TCN (Temporal Convolutional Network).

[0062] In step 210, the computing device 110 performs regression fitting on the predicted distance to determine the location of the note transition point for cutting the audio signal to be cut.

[0063] Regarding the location of note transition points, for example, a frame is 10ms (milliseconds), and the audio signal F = {f1, f2, f3, f4, f5, ..., f...} for multiple consecutive frames... 1000 The distance from f1 to the nearest note transition point is 2 frames (positive values ​​represent subsequent frames, negative values ​​represent earlier frames), the distance from f2 to the nearest note transition point is 1 frame, the distance from f4 to the nearest note transition point is -1 frame, and the distance from f5 to the nearest note transition point is -2 frames. At this time, the third frame f3 of the continuous multi-frame audio signal F is considered to be the note transition point. The position of the 10ms duration represented by frame f3 can be determined as a note transition point (abstracting a 10ms duration frame as a point), or the center point of frame f3 can be determined as a note transition point. There must be a note transition point in the position occupied by frame f3.

[0064] The note transition point refers to the position where a note changes from another note. The change in pitch is determined by the tuning system of the audio signal being segmented. Alternatively, a pitch change can be defined as a change occurring when the pitch exceeds a predetermined threshold. Tuning systems include equal temperament, Pythagorean tuning, and just intonation. Taking equal temperament as an example, the interval between any two adjacent notes is a semitone. Changes exceeding a semitone are considered changes from one note to another; for example, mi to fa, do to la, and do to sharp do are all considered changes from one note to another.

[0065] Furthermore, when applied to the recognition of human vocal melodies, if the sheet music corresponding to the audio signal to be cut can be determined, and if the corresponding sheet music does not show key changes (that is, it only contains notes of one key), the notes of the required note transition points in the audio signal to be cut can also be defined as the constituent notes corresponding to that key. For example, if the corresponding sheet music is in C major, in the audio signal to be cut, the conversion between the international notes CDEFGAB (corresponding to the solfège names do-re-mi-fa-so-la-si-do) is from one note to another.

[0066] It should be understood that, due to the inherent instability of human voices, in this method, when identifying which note a human voice falls into and thus determining whether a note transition point has occurred, the pitch fluctuation within a certain threshold range of that note can be considered as belonging to that note. For example, the International Pitch symbol C4 (do in C major) is considered to belong to the note C4 ± 20 cents (or, for example, the fluctuation within 25% of the frequency corresponding to C4). That is, it can be considered that the change from B4 + 20 cents to C4 - 20 cents is a change from one note to another.

[0067] Therefore, this solution can adapt to the note transition point recognition error caused by the instability of human voice pitch within a certain range, or solve the note transition point recognition problem caused by, for example, when a person sings out of tune or sings inaccurate notes.

[0068] The above scheme achieves the segmentation of a human voice audio into multiple consecutive frames of audio data. Pitch, volume, and speech information are extracted from each frame to generate audio features for the multiple frames. A neural network model then predicts the distance of each frame to the nearest transition point. Regression fitting is performed using these distances to predict the positions of all note transition points in the audio signal to be segmented. Based on this information, the audio signal is segmented. This significantly improves the accuracy of segmenting human voice audio, especially melodic human voice audio.

[0069] Figure 3 A flowchart of a method 300 for smoothing the pitch of each calculated frame of audio signal according to an embodiment of the present invention is shown. Method 300 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 8 The method is performed at the illustrated electronic device 800. It should be understood that method 300 may also include additional steps not shown and / or the steps shown may be omitted, and the scope of the invention is not limited in this respect.

[0070] In step 302, the computing device 110 calculates the influence weight of the pitch of each frame audio signal within the second sliding window range relative to the current frame audio signal when the current frame audio signal is used as the center frame within the second sliding window range, based on the hierarchical analysis method.

[0071] For example,

[0072] It is worth noting that when the second sliding window is slid to smooth the audio signals of each frame of a continuous multi-frame audio signal, the central frame audio signal is smoothed using the initial pitch of each frame audio signal, rather than using the pitch of each smoothed frame audio signal to smooth the central frame. This avoids the loss of pitch changes caused by over-smoothing.

[0073] Regarding the range of the second sliding window, the second sliding window slides along the temporal sequence of multiple consecutive audio frames, with a step size of one audio frame. At each step, the influence weight of the multiple consecutive audio frames is calculated. For example, if the range of the second sliding window is 11 frames, presented in the format of "first five frames + center frame + last five frames," the audio data of the five frames before and after the center frame, as well as their own influence on the pitch of the center frame, can be calculated. Therefore, the pitch of each frame of the multiple consecutive audio signals can be smoothed.

[0074] In some embodiments, the sound production method, distance relative to the center frame, position relative to other frames, and / or singing method of each frame of audio signal within the second sliding window range are related to the corresponding influence weights.

[0075] Regarding the manner of vocalization, such as consonants, vowels, plosives, etc.

[0076] Regarding singing and playing techniques, these include, for example, glissando, legato, sustained notes, and accents.

[0077] Regarding the position relative to other frames, such as the beginning, end, or middle of a note.

[0078] In step 304, the computing device 110 sets each frame in the second sliding window as a scheme layer.

[0079] In step 306, the computing device 110 sets the frequency confidence information of the predicted frequency of each frame audio signal in the second sliding window, the distance of each frame audio signal in the second sliding window to the current frame audio signal, and the pitch difference between each frame audio signal in the second sliding window and the current frame audio signal as the criterion layer.

[0080] Regarding frequency reliability information, factors such as the manner of vocalization, distance from the center frame, position relative to other frames, and / or singing / playing methods all affect the frequency reliability information of the audio signal in that frame; specifically, for example, consonants, breathy sounds, and glissando can lead to a decrease in frequency reliability.

[0081] In step 308, the computing device 110 sets the influence weight of the pitch of each frame audio signal within the second sliding window range relative to the current frame audio signal as the decision layer.

[0082] In step 310, the computing device 110 smooths the pitch of the current frame audio signal based on the calculated influence weight and the pitch of each frame audio signal within the second sliding window range.

[0083] In some embodiments, the smoothing granularity of each frame in a series of audio signals is different, and the smoothing granularity is related to the pitch drop within a second sliding window of each audio signal at the center frame position.

[0084] For example, a small pitch difference within the second sliding window indicates that the pitch of each frame of audio signal within the second sliding window is stable and there are no note transition points. Conversely, a large pitch difference within the second sliding window indicates that there may be note transition points within the second sliding window. This is due to drastic pitch changes near the note transition points. The purpose of smoothing in the above scheme is to eliminate pitch inaccuracies caused by consonants, attack, end, glissando, etc., rather than to eliminate the pitch difference between the note transition points and the audio signals of surrounding frames. Therefore, for the audio signals of frames suspected of being note transition points, the smoothing granularity is much smaller than that of other frames, and there is no need to over-smooth the audio signals near the note transition points.

[0085] In some embodiments, for example, the smoothing granularity of the center frame of the second sliding window is determined by a pitch difference threshold. If the maximum pitch difference of the audio signal within the second sliding window is greater than or equal to the pitch difference threshold, the center frame is smoothed with a first smoothing granularity; if the maximum pitch difference of the audio signal within the second sliding window is less than the pitch difference threshold, the center frame is smoothed with a second smoothing granularity; wherein the first smoothing granularity is smaller than the second smoothing granularity.

[0086] Therefore, by using the above method, the pitch of the preceding and following frames of audio signals can be fully considered in the continuous multi-frame audio signal of the audio signal to be cut, so as to smooth the audio signal at the center frame position to eliminate the influence of pitch fluctuation; and reduce the smoothing granularity near the note transition point to retain the pitch characteristics near the note transition point, so that the pitch characteristics of the continuous multi-frame audio signal smoothed by the above method can fully reflect the position of the note transition point.

[0087] Figure 4 A flowchart of a method 400 for predicting the distance to a note transition point according to an embodiment of the present invention is shown. Method 400 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 8 The method is performed at the illustrated electronic device 800. It should be understood that method 400 may also include additional steps not shown and / or the steps shown may be omitted, and the scope of the invention is not limited in this respect.

[0088] In step 402, if the computing device 110 determines that the current frame audio signal is at a predetermined frame position within the first sliding window range, it acquires the audio features of each frame audio signal within the first sliding window range.

[0089] For example, if the predetermined frame position is 0, the first sliding window range is [-16, 16], meaning there are 16 frames of audio signal before and after the predetermined frame position. If 1 frame is 10ms, the range of the first sliding window is 330ms. When the current frame audio signal is at the 0 frame position, the audio features of each frame audio signal in the 33 frames of audio signal in the range of [-16, 16] within the first sliding window are obtained.

[0090] In step 404, the computing device 110 predicts the distance from the current frame audio signal to the nearest note transition point via a first neural network model based on the acquired audio features of each frame.

[0091] For example, for a continuous multi-frame audio signal F = {f1, f2, f3, f4, f5, ..., f...} 1000}, f 17 The distance to the nearest note transition point is 5 frames (i.e., the nearest note transition point is at f). 17 (5 frames backward), f 18 The distance to the nearest note transition point is -2 frames (i.e., the nearest note transition point is at f). 18 (3 frames forward), and so on, without listing them all.

[0092] In step 406, the computing device 110 moves the first sliding window one frame backward based on the time series of consecutive multi-frame audio signals to predict the distance of the next frame audio signal to the nearest note transition point.

[0093] In step 408, the computing device 110 sequentially predicts the distance from each frame of the audio signal in the first sliding window to the nearest note transition point when the first sliding window is within a predetermined frame position, until the rear boundary of the first sliding window overlaps with the last frame of the audio signal in the consecutive multi-frame audio signal.

[0094] For example, a series of consecutive audio frames F = {f1, f2, f3, f4, f5, ..., f...} 1000}, the first sliding window range is [-16, 16], and the output sequence D = {d} is the distance between note transition points. 17 d 18 d 18 、……、d 984It should be understood that the first and last few frames of a continuous audio signal may not be at the predetermined frame positions. Therefore, in the above example, the sequence D of note transition point distances does not include the positions of the nearest note transition points in frames 1-16 and 985-1000. Since human voices are typically captured without immediate sound production within hundreds of milliseconds after pressing record, the first and last few frames of a continuous audio signal usually do not contain human voices. Therefore, the sequence of note transition point distances output by the above method does not affect the fitting of the note transition point positions.

[0095] Therefore, the above scheme can obtain a sequence of distances between each frame of audio signal and the nearest note transition point in a series of consecutive audio signals, so as to perform regression fitting on the position of the note transition point based on the sequence of distances between the note transition points.

[0096] Figure 5 A flowchart of another method 500 for predicting the distance of a note transition point according to an embodiment of the present invention is shown. Method 500 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 8 The method is performed at the illustrated electronic device 800. It should be understood that method 500 may also include additional steps not shown and / or the steps shown may be omitted, and the scope of the invention is not limited in this respect.

[0097] In step 502, if the computing device 110 determines that the first sliding window includes multiple first sliding windows, and the distance from the center frame of any one of the multiple first sliding windows to the predetermined frame position is different from the distance from the center frame of the other first sliding windows to the predetermined frame position.

[0098] In some embodiments, the predetermined frame positions of multiple first sliding windows are all 0 frames. A set of first sliding windows consists of two windows with the same frame span, but different distances from the predetermined frame position to the rear boundary of the first sliding window. For example, the range of the first sliding window b (also known as the negative priority sliding window) is [-24, 8], the center frame of the first sliding window b is -8, the distance from the 0 frame position of the first sliding window b to the center frame is -8, and the distance from the 0 frame position of the first sliding window b to the rear boundary is 8; the range of the first sliding window c (also known as the positive priority sliding window) is [-8, 24], the center frame of the first sliding window c is 8, and the distance from the 0 frame position of the first sliding window c to the rear boundary is 24. By performing predictions using positive priority windows and negative priority windows respectively, when there are no note transition points within the range of the first sliding window, the model will not randomly select the front and rear boundary values ​​of the window, but will instead favor points with larger absolute values ​​of distance from the predetermined frame position, such as 24 for the first sliding window c and -24 for the first sliding window b.

[0099] In step 504, the computing device 110, based on each first sliding window, predicts, via a first neural network model, the distance from each frame of audio signal in a series of consecutive frames of audio signal to the nearest note transition point for each first sliding window, specifically for the audio features within each first sliding window range for each frame of audio signal.

[0100] In step 506, the computing device 110 aggregates the distances of each frame of audio signal in a series of consecutive frames of audio signal for each first sliding window to the nearest note transition point.

[0101] Continuing with the above example, for instance, a series of audio signals F = {f1, f2, f3, f4, f5, ..., f...} 1000}, the first sliding window b ranges from [-24, 8], and the output sequence D is the distance between note transition points. b ={d b25 d b26 d b27 、……、d b992};The first sliding window c has a range of [-8, 24], and outputs a sequence D of note transition point distances. c ={d c9 d c10 d c11 、……、d c976 The two are combined to obtain the sequence D of the distances between note transition points. bc ={d c9 d c10 d c11 、……、d c992}

[0102] Regarding the distance to the nearest note transition point, continuing with the example above, let's consider the aggregation point D. b and D c For example, retain the unique values; for duplicate values, take the smaller absolute value, such as |d. b50 |>|d c50 |, with d c50 As f 50 Distance to the nearest note transition point. Taking a smaller absolute value improves fitting efficiency, while a larger absolute value indicates a greater distance from the note transition point. In some embodiments, the average of repeated values ​​can be taken as the distance of the audio data in that frame to the nearest note transition point; taking the average yields a more comprehensive distance prediction result.

[0103] In step 508, the computing device 110 performs regression fitting on the aggregated distance to determine the location of the note transition points used to cut the audio signal to be cut.

[0104] Therefore, by using multiple first sliding windows, the distance from each frame of audio signal in a series of consecutive audio frames to the nearest note transition point can be predicted for each first sliding window, thereby generating multiple sequences of distances from the nearest note transition point for the series of consecutive audio frames. The obtained sequences of distances from the nearest note transition point are then aggregated, making the obtained distances from the nearest note transition point more accurate and easier to fit the note transition point position.

[0105] Figure 6 A flowchart of a method 600 for regressing and fitting a note transition point position curve according to an embodiment of the present invention is shown. Method 600 can be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 8 The method is performed at the illustrated electronic device 800. It should be understood that method 600 may also include additional steps not shown and / or the steps shown may be omitted, and the scope of the invention is not limited in this respect.

[0106] In step 602, the computing device 110 initializes a fitting curve about the note transition points based on the time series of continuous multi-frame audio signals.

[0107] For example, to initialize the fitted curve y, the distance between each frame of audio signal and the note transition point in the fitted curve is initialized to the maximum value, such as 24, with F = {f1, f2, f3, f4, f5, ..., f...} 1000 Taking} as an example, we get y0 = {x1 = 24, x2 = 24, ..., x} 1000 =24}, x i (i = {1, 1000}), x i This represents the distance of the i-th frame audio signal from the nearest note transition point in the fitted curve y.

[0108] In step 604, the computing device 110, based on the time series of multiple consecutive audio signals, adds a note transition point starting from the first audio signal and moves the note transition point backward step by step with one audio signal as the step size, until it determines that the curve fitted by the added note transition point at the current position is closest to the distance from the nearest note transition point to each audio signal in the region.

[0109] Regarding the area, it refers to the multiple frames within the influence range of a note transition point. For example, a note transition point may affect multiple frames within a sliding window.

[0110] In step 606, after the computing device 110 determines the position of a note transition point, it adds a new note transition point after maintaining a predetermined safe distance from the determined note transition point, and continues to perform regression fitting until no more new note transition points can be added.

[0111] For example, the sequence D with the distance to the nearest note transition point mentioned above. bc ={d bc9 d bc10 d bc11 、……、d bc992 For example, by using regression fitting, after adding each note transition point, y0 = {x1, x2, ..., x} 1000 Multiple x's within the influence range of the note transition point in} i With the corresponding multiple d bc The value of the closest similarity is determined by the fit. For example, if moving the note transition point to frame 10 yields a more accurate fit than moving it to frame 11 or frame 9, then the note transition point is placed at frame 10. It's important to note that the influence distance of a note transition point is finite. Therefore, after determining the previous note transition point, a new note transition point is added after a safe distance, for example, greater than or equal to 150ms, to fit the next note transition point.

[0112] For example, please refer to Figure 7 This illustration shows a schematic diagram of fitting the distance of note transition points through regression using various first sliding windows according to an embodiment of the present invention, such as... Figure 7 The diagram illustrates the nearest note transition point distance prediction results obtained by the first neural network through negative priority sliding windows and positive priority sliding windows (such as the first sliding windows b and c in the example above) (as shown in 7a, illustrating the positive priority model result and the negative priority model result), and the regression fitting result diagram (as shown in 7b). It can be seen that by using multiple first sliding windows to perform positive and negative priority model prediction and aggregating the nearest note transition point distance, a result that is closer to the actual note transition point distance can be obtained.

[0113] Therefore, by gradually adding note transition points and moving the positions of note transition points sequentially, a regression fitting can be performed on the fitting curve for note transition points. This allows for a more accurate determination of the positions of multiple note transition points in the audio signal to be cut, thereby improving the efficiency and accuracy of audio cutting.

[0114] To further understand the solutions provided in the embodiments of the present invention, the following example of audio cutting will be used for further explanation.

[0115] Table 1 below shows the matching and audio segmentation results of the score information and recording information for a song excerpt. The numbers on the left represent the sequence of notes. The score audio information includes the score pitch and score duration; the actual audio includes the actual pitch; duration 1 - segmentation algorithm (the duration of each note obtained by segmenting the actual audio using the segmentation algorithm provided in this embodiment); and duration 2 - manual annotation (the duration of each note obtained by manual annotation). All pitch units are standard MIDI, and all duration units are milliseconds.

[0116] Table 1

[0117]

[0118]

[0119] According to Table 1 above, comparing the duration 1 marked by the cutting algorithm and the duration 2 marked manually, the error between the two is within 20ms for most notes, which shows that the audio cutting method provided by the embodiment of the present invention is very accurate.

[0120] Please refer to Figure 9 This illustrates a comparison between algorithmic annotation and manual annotation results for segmenting a musical phrase recording. Figure 9 The note sequence 1-8 in the text. Figure 9 In the graph, the horizontal axis represents time (in milliseconds), the top row of dots represents the results of the segmentation algorithm, and the bottom row represents the results of manual annotation. The manual annotation results are, for example, the average of annotations completed independently by four people (with adequate musical literacy). As can be seen from the graph, the segmentation algorithm can achieve almost the same accuracy as manual annotation. However, compared to manual annotation, the segmentation algorithm significantly improves annotation efficiency.

[0121] Furthermore, the method for segmenting audio signals provided in this embodiment of the invention can be used in conjunction with methods for matching human vocal melody audio with sheet music, methods for annotating human vocal audio segments, etc.; for example, the human vocal melody audio to be matched is segmented to serve as input audio data for the method of matching human vocal melody audio with sheet music; for example, the annotated audio data generated from the annotated human vocal audio segments is used as the neural network training dataset of this solution to serve as data for training the neural network model of the method for segmenting audio signals provided in this embodiment of the invention.

[0122] Figure 8 A schematic step diagram of an example electronic device 800 that can be used to implement embodiments of the contents of this specification is shown. For example, as Figure 1The computing device 110 shown can be implemented by an electronic device 800. As shown, the electronic device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 802 or loaded from storage unit 808 into random access memory (RAM) 803. The random access memory 803 can also store various programs and data required for the operation of the electronic device 800. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0123] Multiple components in electronic device 800 are connected to input / output interface 805, including: input unit 806, such as keyboard, mouse, microphone, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0124] The various processes and procedures described above, such as methods 200 to 600, can be executed by the central processing unit 801. For example, in some embodiments, methods 200 to 600 can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via read-only memory 802 and / or communication unit 809. When the computer program is loaded into random access memory 803 and executed by the central processing unit 801, one or more actions of methods 200 to 600 described above can be performed.

[0125] This invention relates to methods, apparatus, systems, electronic devices, computer-readable storage media, and / or computer program products. The computer program product may include computer-readable program instructions for performing various aspects of the invention.

[0126] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0127] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge computing devices. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media within the respective computing / processing device.

[0128] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0129] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or step diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each step in the flowchart illustrations and / or step diagrams, as well as combinations of steps in the flowchart illustrations and / or step diagrams, can be implemented by computer-readable program instructions.

[0130] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more steps of the flowchart and / or diagram of steps. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more steps of the flowchart and / or diagram of steps.

[0131] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more steps of a flowchart and / or a diagram of steps.

[0132] The flowcharts and step diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each step in the flowchart or step diagram may represent a module, segment, or part of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the step may occur in a different order than those indicated in the drawings. For example, two consecutive step steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each step in the step diagram and / or flowchart, and combinations of steps in the step diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0133] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for cutting an audio signal, comprising: Acquire multiple consecutive frames of audio signal related to the audio signal to be cut; Extract pitch, volume, and speech information from each frame of the acquired multi-frame audio signal. Based on the extracted information, audio features are generated for each frame of the audio signal in the continuous multi-frame audio signal; For the audio features within a first sliding window range of each frame of audio signal, a first neural network model is used to predict the distance of each frame of audio signal to the nearest note transition point in the consecutive multi-frame audio signals; as well as Regression fitting is performed on the predicted distance to determine the location of the note transition point used to cut the audio signal to be cut.

2. The method according to claim 1, characterized in that, The audio characteristics of each frame of audio signal include one or more of the following: Frequency-related features extracted based on pitch information include frequency change features, cumulative frequency change features, frequency reliability features, and / or frequency reliability change features. as well as Energy-related features extracted from volume information include volume features, volume variation features, phase features, amplitude features, and / or amplitude variation features.

3. The method according to claim 2, characterized in that, The audio characteristics of each frame of audio signal also include: Based on the Mel frequency cepstral coefficient variation features extracted from speech information, the Mel frequency cepstral coefficient variation features and / or frequency reliability variation features are used to determine speech features about note transition points, and the speech features are related to at least one of the following: attack, coda, consonant, vowel, textual attribute of the sound, and mode of sound production.

4. The method according to claim 1, characterized in that, Before extracting the pitch information of each frame of the acquired multi-frame audio signal, the following steps are included: The frequency of each frame of audio signal in the multi-frame audio signal is predicted by the second neural network model, and the frequency reliability information of the predicted frequency of each frame of audio signal by the second neural network model is obtained. Based on the extracted frequency of each frame of audio signal, calculate the pitch of each frame of audio signal; and The pitch of each frame of audio signal is smoothed to obtain the pitch of each frame of audio signal after smoothing. Based on the smoothed pitch, the pitch information of each frame of audio signal in the acquired consecutive multi-frame audio signal is extracted.

5. The method according to claim 4, characterized in that, Smoothing the pitch of each calculated frame of audio signal includes: Based on the analytic hierarchy process (AHP), when the current frame audio signal is considered as the center frame within the second sliding window, the influence weight of the pitch of each frame audio signal within the second sliding window relative to the current frame audio signal is calculated, including: Set each frame in the second sliding window as the scheme layer; The frequency confidence information of the predicted frequencies of the audio signals of each frame within the second sliding window, the distance between the audio signals of each frame within the second sliding window and the current frame audio signal, and the pitch difference between the audio signals of each frame within the second sliding window and the current frame audio signal are set as the criterion layer; and The influence weight of the pitch of each frame of audio signal within the second sliding window range relative to the current frame of audio signal is set as the decision layer; and Based on the calculated influence weights and the pitch of each frame of audio signal within the second sliding window range, the pitch of the current frame of audio signal is smoothed.

6. The method according to claim 1, characterized in that, Predicting the distance of each frame of audio signal to the nearest note transition point includes: In response to determining that the current frame audio signal is at a predetermined frame position within the first sliding window range, the audio characteristics of each frame audio signal within the first sliding window range are obtained; Based on the acquired audio features of each frame, the distance from the current frame's audio signal to the nearest note transition point is predicted via the first neural network model. Based on the time series of the continuous multi-frame audio signals, the first sliding window is moved forward one frame to predict the distance of the next frame audio signal to the nearest note transition point; and The distance from each frame of the continuous multi-frame audio signal to the nearest note transition point is predicted sequentially when the first sliding window is within a predetermined frame position, until the rear boundary of the first sliding window overlaps with the last frame of the continuous multi-frame audio signal.

7. The method according to claim 6, characterized in that, Predicting the distance of each frame of audio signal to the nearest note transition point also includes: In response to determining that the first sliding window includes multiple first sliding windows, the distance from the center frame of any one of the multiple first sliding windows to the predetermined frame position is different from the distance from the center frame of the other first sliding windows to the predetermined frame position; Based on each first sliding window, for the audio features within each first sliding window range for each frame of audio signal, a first neural network model is used to predict the distance from each frame of audio signal in the consecutive multi-frame audio signal to the nearest note transition point for each first sliding window. Aggregate the distances obtained for each frame of the audio signal in the consecutive multi-frame audio signal for each first sliding window to the nearest note transition point; and Regression fitting is performed on the aggregated distance to determine the location of the note transition point used to cut the audio signal to be cut.

8. The method according to claim 1, characterized in that, Regression fitting for the predicted distance includes: Based on the time series of the continuous multi-frame audio signal, initialize the fitting curve about the note transition points; and Based on the time series of the continuous multi-frame audio signals, a note transition point is added starting from the first frame of the audio signal. Using one frame of audio signal as a step size, the note transition point is gradually moved backward until the curve fitted by the added note transition point at its current position is determined to be the distance from the nearest note transition point in each frame of audio signal within the region. After determining the location of a note transition point, a new note transition point is added after a predetermined safe distance from the determined note transition point. The regression fitting continues until no more new note transition points can be added.

9. A computing device, comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method of any one of claims 1-8.

Images