Audio real-time conversion and analysis management system and method based on artificial intelligence

Abstract

The present invention relates to the field of audio conversion, and specifically to an artificial intelligence-based real-time audio conversion and analysis management system and method. The system comprises: an audio processing module, an intelligent network module, a text segmentation module, an embedding transition module, and a reverse conversion module. The audio processing module is used to record reference audio and extended audio, the intelligent network module is used to train a neural network model, the text segmentation module is used to segment standard audio, the embedding transition module is used to embed transition audio and output target audio, and the reverse conversion module is used to compare embedding point position sequences. The present invention can avoid the limitations of traditional speech recognition rule engines, automatically adjust the audio sampling rate, balance bandwidth and quality, improve speech recognition accuracy, enhance the coherence of semantic recognition, increase language conversion speed, optimize audio conversion operation delay, reduce the stagnation and splicing sense of non-natural language, and optimize audio generation quality.

Description

Technical Field

[0001] The present invention relates to the field of audio conversion, and in particular to an artificial intelligence-based real-time audio conversion and analysis management system and method. Background Art

[0002] Real-time audio conversion is the process of instantly processing and converting audio signals. It is often used in combination with audio transcription technology to automatically convert audio content into text or convert text into virtual audio. It is often used in video subtitle generation, streaming media sound beautification, and audio interpretation of works.

[0003] Audio conversion typically involves processes such as audio acquisition, audio recognition, text comparison, and timbre conversion. The original audio is then converted into new audio through operations such as stretching and frequency conversion. The conversion accuracy of this technology depends on the volume of the associated corpus. It is difficult to generate audio text that the user has not provided, in addition to the original audio. Furthermore, the output audio has a spliced ​​feel and lacks smoothness. While limiting filtering can reduce frequency fluctuations, it cannot meet language requirements such as pauses, transitions, and intervals.

[0004] In addition, real-time audio conversion requires recording, but the complex recording process places high demands on equipment and environment. Audio with a lot of noise interference, especially audio containing discontinuous text and punctuation, is difficult to reverse-engineer, affecting the timeliness and accuracy of text-to-audio conversion and reducing the user experience. Summary of the Invention

[0005] The purpose of the present invention is to provide an audio real-time conversion and analysis management system and method based on artificial intelligence to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: an artificial intelligence-based real-time audio conversion and analysis management system, comprising: an audio processing module, an intelligent network module, a text segmentation module, an embedding transition module and a reverse conversion module;

[0007] The audio processing module is used to record human voice reference audio, which contains all typical frequency bands. The reference audio and text are aligned using a dynamic time warping algorithm, and the GMM-HMM model is used to fine-tune the time-frequency boundary. After alignment, the audio sampling rate is converted, and the reference audio is interpolated and extended in the frequency domain to obtain extended audio.

[0008] The intelligent network module is used to detect the spectrum boundary of the extended audio through short-time Fourier transform or wavelet transform, segment the audio using the spectrum boundary and the text boundary corresponding to the extended audio as segmentation points, intercept the paragraphs between two adjacent spectrum boundaries or text boundaries, take the shortest one and output it as morpheme audio, and output the paragraph between the text boundary and the spectrum boundary as transition audio, filter the morpheme audio with a Mel filter or a linear predictive coding filter, calculate the LPC residual between the filtered morpheme audio and the standard audio signal, use the residual as training data, train a neural network to learn the vocalization features in the morpheme audio, and obtain a trained neural network model;

[0009] The text segmentation module is used to receive text input by the user, compile the text into standard audio through the TTS system, verify the error of each morpheme audio under the standard audio one by one using the trained neural network model, detect the time domain, frequency domain and semantics of the error based on the hidden Markov model, and segment the standard audio so that each segmented sub-audio has a unique corresponding morpheme audio, and the morpheme audio contains continuous sampling values, and the errors of these continuous sampling values ​​and the sampling values ​​of the sub-audio are all within a preset range;

[0010] The embedding transition module is used to replace each sub-audio with a corresponding morpheme audio segment, arrange and synthesize the sub-audio segments in the segmentation order to obtain generated audio, model the human ear auditory masking curve, use a dynamic segmenter based on reinforcement learning to analyze the critical band positions in the generated audio, use the Bark scale critical band positions as embedding points, determine transition audio in the corpus, embed the transition audio into the generated audio, ensure that the transition audio satisfies the formant change rate between adjacent sub-audio segments is less than a threshold after applying time domain stretching and frequency domain envelope migration, and output the target audio;

[0011] The reverse conversion module is used to use an artificial intelligence network to check the embedded points after obtaining the audio during the audio reverse conversion process, output the relative position of the embedded points in the audio, compare it with the position sequence of the audio embedded points in the database, determine the natural language environment, segment the audio according to the position sequence with the smallest comparison error, perform morpheme audio comparison, and output the comparison results.

[0012] Furthermore, the audio processing module includes: a corpus recording unit and an interpolation expansion unit;

[0013] The corpus recording unit is used to record human voice reference audio using a swept frequency signal composite sampling technology, and verify the signal-to-noise ratio of each frequency band through FFT, discarding substandard audio;

[0014] The interpolation and extension unit is used to perform dynamic time warping of the reference audio based on the MFCC cepstral coefficients and expand the reference audio.

[0015] Furthermore, the intelligent network module includes: a frequency conversion and division unit, a limiting filter unit and a network training unit;

[0016] The frequency conversion segmentation unit is used to separate morpheme audio and transition audio, wherein the morpheme audio represents a stable audio segment with clear pronunciation, and the transition audio represents a transition audio segment between adjacent morphemes;

[0017] The limiting filter unit is used to process the error residual wave of the morpheme audio in the standard audio filter using the U-Net structure;

[0018] The network training unit is used to perform neural network training, and the neural network includes: CNN, WaveNet and Tacotron.

[0019] Furthermore, the text segmentation module includes: a text management unit and an intelligent segmentation unit;

[0020] The text management unit is used to establish a text input platform and generate a text-to-speech request;

[0021] The intelligent segmentation unit is used to segment the standard audio and determine the corresponding relationship between each audio segment and the morpheme audio.

[0022] Furthermore, the embedding transition module includes: an auditory masking unit, a sampling extraction unit and an audio embedding unit;

[0023] The auditory masking unit is used to establish an acoustic model using the auditory masking characteristics of the human ear to reduce the embedding error to below the masking threshold;

[0024] The sampling extraction unit is used to use Constant-Q transformation to perform phase synchronization transformation and smooth the embedded audio;

[0025] The audio embedding unit is used to optimize the continuity of the transition segment audio through the Griffin-Lim algorithm and output the target audio.

[0026] Furthermore, the reverse conversion module includes: a position memory unit and a text feedback unit;

[0027] The position memory unit is used to generate a sequence according to the audio embedding point position and store it in the corpus;

[0028] The text feedback unit is used to identify the converted audio and determine the converted audio segmentation points based on the historical embedding sequence.

[0029] The method for real-time audio conversion and analysis management based on artificial intelligence includes the following steps:

[0030] Step S1. Recording human voice reference audio, aligning the reference audio with the text using a dynamic time warping algorithm, and then performing frequency domain interpolation and expansion on the reference audio to obtain expanded audio.

[0031] Step S2. Detect the spectrum boundaries of the extended audio, segment the audio using the spectrum boundaries and the text boundaries corresponding to the extended audio as segmentation points, segment the audio into morpheme audio and transition audio, calculate the LPC residual between the filtered morpheme audio and the standard audio signal, and use the residual as training data to train the neural network model;

[0032] Step S3. Compile the input text into standard audio. The neural network model verifies the residual of each morpheme audio under the standard audio. The standard audio is segmented based on the verification results, so that each segmented sub-audio segment has a unique corresponding morpheme audio. The morpheme audio contains continuous sampling values, and the error between these sampling values ​​and the sampling values ​​of the sub-audio is within a preset range.

[0033] Step S4. Each sub-audio is replaced with the corresponding morpheme audio segment, and the generated audio is obtained through time-domain synthesis. The generated audio is modeled according to the human auditory masking curve, and the Bark scale critical band position in the generated audio is determined. The transition audio is embedded into the generated audio using the critical band position as the embedding point to obtain the target audio output;

[0034] Step S5. During the audio reverse conversion process, the audio embedding point position is checked and compared with the position sequence of the audio embedding points in the database. The audio is segmented according to the position sequence with the smallest comparison error, and morpheme audio comparison is performed, and the comparison results are output.

[0035] Furthermore, step S1 includes:

[0036] Step S11. Record human voice reference audio using swept frequency signal composite sampling technology, verify the signal-to-noise ratio of each frequency band through FFT, discard substandard audio, and store the obtained reference audio in the corpus;

[0037] Step S12. Perform dynamic time warping of the reference audio based on the MFCC cepstral coefficients, align the reference audio with the text, and use the GMM-HMM model to fine-tune the time-frequency boundary to obtain the aligned audio;

[0038] Step S13. Use linear interpolation and STFT phase reconstruction method to convert the sampling rate of the aligned audio, perform frequency domain interpolation expansion on the reference audio to match the longer text, and obtain the extended audio.

[0039] Furthermore, step S2 includes:

[0040] Step S21. Detecting the spectral boundaries of the extended audio using a short-time Fourier transform or a wavelet transform, segmenting the audio using the spectral boundaries and the text boundaries corresponding to the extended audio as segmentation points. Segments between two adjacent spectral boundaries or text boundaries are intercepted and output as morpheme audio. If the segments between the spectral boundary and the text boundary overlap, the shortest segment is selected and output as the morpheme audio, and the segment between the text boundary and the spectral boundary is output as transition audio.

[0041] Step S22. Filter the morpheme audio with a Mel filter or a linear predictive coding filter, calculate the LPC residual between the filtered morpheme audio and the standard audio signal, use the residual as training data, and train a neural network to learn the vocal features in the morpheme audio. The neural network includes: CNN, WaveNet and Tacotron, and obtain a trained neural network model.

[0042] Furthermore, step S3 includes:

[0043] Step S31. Establish a text input platform. After receiving user input, generate a text-to-speech request. Compile the text into standard audio through the TTS system. Use the trained neural network model to verify the error of each morpheme audio under the standard audio. Based on the hidden Markov model, detect the time domain, frequency domain, and semantic meaning of the error, and fit the part of the morpheme audio where the error between the morpheme audio and the standard audio is less than a threshold.

[0044] Step S32. Intelligently segment the standard audio to obtain sub-audio segments, and determine the correspondence between each sub-audio segment and the morpheme audio segment so that the segmentation result satisfies the following conditions: each sub-audio segment corresponds to a unique morpheme audio segment, and there are N consecutive sampling points in the morpheme audio segment that are sampled at the same time as the sub-audio segment, and the MSE error is lower than a preset threshold, where N is the sampling point length of the sub-audio segment.

[0045] Furthermore, step S4 includes:

[0046] Step S41: Replace each sub-audio with a corresponding morpheme audio segment, where the morpheme audio segment consists of N consecutive sampling points in the morpheme audio with a sampling error less than a preset threshold at the same time as the sub-audio. The replaced audio segments are synthesized in a time-series order to obtain generated audio.

[0047] Step S42: Utilizing the masking characteristics of the human ear, an acoustic model is established. A dynamic segmenter based on reinforcement learning is used to analyze the critical band locations in the generated audio. Using the Bark scale critical band locations as embedding points, transition audio is determined from the corpus and embedded into the generated audio. This transition audio is embedded in the generated audio such that, after applying time domain stretching and frequency domain envelope shifting, the transition audio satisfies the condition that the formant change rate between adjacent sub-audios is less than a threshold.

[0048] Step S43: Use Constant-Q transformation to perform phase synchronization transformation, smooth the embedded audio, optimize the continuity of the transition audio using the Griffin-Lim algorithm, and then output and play the target audio.

[0049] Furthermore, step S5 includes:

[0050] Step S51. During the audio reverse conversion process, after acquiring the audio, the embedded points are verified using an artificial intelligence network, and the relative positions of the embedded points in the audio are output. The relative positions of the embedded points in the audio are compared with the position sequence of the audio embedded points in the database. The audio is segmented according to the position sequence with the smallest comparison error, and morpheme audio comparison is performed.

[0051] Step S53: Output the morpheme audio comparison result, output text according to the comparison result, and set pause punctuation at the position of the audio embedding point.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The present invention can achieve audio sampling rate conversion through multi-level mixing, interpolate and expand the reference audio, use the text corresponding to the extended audio as the dividing point, and separate the extended audio based on frequency conversion to obtain morpheme audio and transition audio, thereby separating human voice from background noise, avoiding the limitations of traditional speech recognition rule engines, automatically adjusting the audio sampling rate, balancing bandwidth and quality, and improving speech recognition accuracy.

[0054] The present invention verifies the error of each morpheme audio under standard audio one by one by training an artificial intelligence network, and intelligently segments the standard audio, intercepts and arranges the morpheme audio according to the sub-audio segmentation length to obtain generated audio. At the same time, the masking characteristics of human hearing are used to determine the embedding position of the transition audio, so that the frequency error of the embedding position is within the masking threshold, thereby improving the coherence of semantic recognition, optimizing the audio generation quality, and reducing the stagnation and splicing sense of non-natural language.

[0055] The present invention records the embedding position sequence of transition audio, uses an artificial intelligence network to check the embedding points during the audio reverse conversion process, compares them with the position sequence in the database, compares the language transition positions, converts them into punctuation and embeds them in the text, thereby improving the accuracy of two-way conversion between audio and text, increasing the language conversion speed, optimizing the audio conversion operation delay, and enhancing the user's natural language conversion experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0057] Figure 1It is a structural diagram of the audio real-time conversion and analysis management system based on artificial intelligence of the present invention;

[0058] Figure 2 It is a schematic diagram of the steps of the audio real-time conversion and analysis management method based on artificial intelligence of the present invention. DETAILED DESCRIPTION

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0060] See also Figure 1 , the present invention provides a technical solution:

[0061] Embodiment: An artificial intelligence-based audio real-time conversion and analysis management system, comprising: an audio processing module, an intelligent network module, a text segmentation module, an embedding transition module, and a reverse conversion module;

[0062] The audio processing module is used to record human voice reference audio, which contains all typical frequency bands. The reference audio and text are aligned using a dynamic time warping algorithm, and the GMM-HMM model is used to fine-tune the time-frequency boundary. After alignment, the audio sampling rate is converted, and the reference audio is interpolated and extended in the frequency domain to obtain extended audio.

[0063] The audio processing module includes: a corpus recording unit and an interpolation expansion unit;

[0064] The corpus recording unit is used to record human voice reference audio using a swept frequency signal composite sampling technology, and verify the signal-to-noise ratio of each frequency band through FFT, discarding substandard audio;

[0065] The interpolation and extension unit is used to perform dynamic time warping of the reference audio based on the MFCC cepstral coefficients and expand the reference audio.

[0066] The intelligent network module is used to detect the spectrum boundary of the extended audio through short-time Fourier transform or wavelet transform, segment the audio using the spectrum boundary and the text boundary corresponding to the extended audio as segmentation points, intercept the paragraphs between two adjacent spectrum boundaries or text boundaries, take the shortest one and output it as morpheme audio, and output the paragraph between the text boundary and the spectrum boundary as transition audio, filter the morpheme audio with a Mel filter or a linear predictive coding filter, calculate the LPC residual between the filtered morpheme audio and the standard audio signal, use the residual as training data, train a neural network to learn the vocalization features in the morpheme audio, and obtain a trained neural network model;

[0067] The intelligent network module includes: a frequency conversion and division unit, a limiting filter unit and a network training unit;

[0068] The frequency conversion segmentation unit is used to separate morpheme audio and transition audio, wherein the morpheme audio represents a stable audio segment with clear pronunciation, and the transition audio represents a transition audio segment between adjacent morphemes;

[0069] The limiting filter unit is used to process the error residual wave of the morpheme audio in the standard audio filter using the U-Net structure;

[0070] The network training unit is used to perform neural network training, and the neural network includes: CNN, WaveNet and Tacotron.

[0071] The text segmentation module is used to receive text input by the user, compile the text into standard audio through the TTS system, verify the error of each morpheme audio under the standard audio one by one using the trained neural network model, detect the time domain, frequency domain and semantics of the error based on the hidden Markov model, and segment the standard audio so that each segmented sub-audio has a unique corresponding morpheme audio, and the morpheme audio contains continuous sampling values, and the errors of these continuous sampling values ​​and the sampling values ​​of the sub-audio are all within a preset range;

[0072] The text segmentation module includes: a text management unit and an intelligent segmentation unit;

[0073] The text management unit is used to establish a text input platform and generate a text-to-speech request;

[0074] The intelligent segmentation unit is used to segment the standard audio and determine the corresponding relationship between each audio segment and the morpheme audio.

[0075] The embedding transition module is used to replace each sub-audio with a corresponding morpheme audio segment, arrange and synthesize the sub-audio segments in the segmentation order to obtain generated audio, model the human ear auditory masking curve, use a dynamic segmenter based on reinforcement learning to analyze the critical band positions in the generated audio, use the Bark scale critical band positions as embedding points, determine transition audio in the corpus, embed the transition audio into the generated audio, ensure that the transition audio satisfies the formant change rate between adjacent sub-audio segments is less than a threshold after applying time domain stretching and frequency domain envelope migration, and output the target audio;

[0076] The embedding transition module includes: an auditory masking unit, a sampling extraction unit and an audio embedding unit;

[0077] The auditory masking unit is used to establish an acoustic model using the auditory masking characteristics of the human ear to reduce the embedding error to below the masking threshold;

[0078] The sampling extraction unit is used to use Constant-Q transformation to perform phase synchronization transformation and smooth the embedded audio;

[0079] The audio embedding unit is used to optimize the continuity of the transition segment audio through the Griffin-Lim algorithm and output the target audio.

[0080] The reverse conversion module is used to use an artificial intelligence network to check the embedded points after obtaining the audio during the audio reverse conversion process, output the relative position of the embedded points in the audio, compare it with the position sequence of the audio embedded points in the database, determine the natural language environment, segment the audio according to the position sequence with the smallest comparison error, perform morpheme audio comparison, and output the comparison results.

[0081] The reverse conversion module includes: a position memory unit and a text feedback unit;

[0082] The position memory unit is used to generate a sequence according to the audio embedding point position and store it in the corpus;

[0083] The text feedback unit is used to identify the converted audio and determine the converted audio segmentation points based on the historical embedding sequence.

[0084] like Figure 2 As shown, the audio real-time conversion and analysis management method based on artificial intelligence includes the following steps:

[0085] Step S1. Recording human voice reference audio, aligning the reference audio with the text using a dynamic time warping algorithm, and then performing frequency domain interpolation and expansion on the reference audio to obtain expanded audio.

[0086] Step S1 includes:

[0087] Step S11. Record human voice reference audio using swept frequency signal composite sampling technology, verify the signal-to-noise ratio of each frequency band through FFT, discard substandard audio, and store the obtained reference audio in the corpus;

[0088] Step S12. Perform dynamic time warping of the reference audio based on the MFCC cepstral coefficients, align the reference audio with the text, and use the GMM-HMM model to fine-tune the time-frequency boundary to obtain the aligned audio;

[0089] Step S13. Use linear interpolation and STFT phase reconstruction method to convert the sampling rate of the aligned audio, perform frequency domain interpolation expansion on the reference audio to match the longer text, and obtain the extended audio.

[0090] Step S2. Detect the spectrum boundaries of the extended audio, segment the audio using the spectrum boundaries and the text boundaries corresponding to the extended audio as segmentation points, segment the audio into morpheme audio and transition audio, calculate the LPC residual between the filtered morpheme audio and the standard audio signal, and use the residual as training data to train the neural network model;

[0091] Step S2 includes:

[0092] Step S21. Detecting the spectral boundaries of the extended audio using a short-time Fourier transform or a wavelet transform, segmenting the audio using the spectral boundaries and the text boundaries corresponding to the extended audio as segmentation points. Segments between two adjacent spectral boundaries or text boundaries are intercepted and output as morpheme audio. If the segments between the spectral boundary and the text boundary overlap, the shortest segment is selected and output as the morpheme audio, and the segment between the text boundary and the spectral boundary is output as transition audio.

[0093] Step S22. Filter the morpheme audio with a Mel filter or a linear predictive coding filter, calculate the LPC residual between the filtered morpheme audio and the standard audio signal, use the residual as training data, and train a neural network to learn the vocal features in the morpheme audio. The neural network includes: CNN, WaveNet and Tacotron, and obtain a trained neural network model.

[0094] Step S3. Compile the input text into standard audio. The neural network model verifies the residual of each morpheme audio under the standard audio. The standard audio is segmented based on the verification results, so that each segmented sub-audio segment has a unique corresponding morpheme audio. The morpheme audio contains continuous sampling values, and the error between these sampling values ​​and the sampling values ​​of the sub-audio is within a preset range.

[0095] Step S3 includes:

[0096] Step S31. Establish a text input platform. After receiving user input, generate a text-to-speech request. Compile the text into standard audio through the TTS system. Use the trained neural network model to verify the error of each morpheme audio under the standard audio. Based on the hidden Markov model, detect the time domain, frequency domain, and semantic meaning of the error, and fit the part of the morpheme audio where the error between the morpheme audio and the standard audio is less than a threshold.

[0097] Step S32. Intelligently segment the standard audio to obtain sub-audio segments, and determine the correspondence between each sub-audio segment and the morpheme audio segment so that the segmentation result satisfies the following conditions: each sub-audio segment corresponds to a unique morpheme audio segment, and there are N consecutive sampling points in the morpheme audio segment that are sampled at the same time as the sub-audio segment, and the MSE error is lower than a preset threshold, where N is the sampling point length of the sub-audio segment.

[0098] Step S4. Each sub-audio is replaced with the corresponding morpheme audio segment, and the generated audio is obtained through time-domain synthesis. The generated audio is modeled according to the human auditory masking curve, and the Bark scale critical band position in the generated audio is determined. The transition audio is embedded into the generated audio using the critical band position as the embedding point to obtain the target audio output;

[0099] Step S4 includes:

[0100] Step S41: Replace each sub-audio with a corresponding morpheme audio segment, where the morpheme audio segment consists of N consecutive sampling points in the morpheme audio with a sampling error less than a preset threshold at the same time as the sub-audio. The replaced audio segments are synthesized in a time-series order to obtain generated audio.

[0101] Step S42: Utilizing the masking characteristics of the human ear, an acoustic model is established. A dynamic segmenter based on reinforcement learning is used to analyze the critical band locations in the generated audio. Using the Bark scale critical band locations as embedding points, transition audio is determined from the corpus and embedded into the generated audio. This transition audio is embedded in the generated audio such that, after applying time domain stretching and frequency domain envelope shifting, the transition audio satisfies the condition that the formant change rate between adjacent sub-audios is less than a threshold.

[0102] Step S43: Use Constant-Q transformation to perform phase synchronization transformation, smooth the embedded audio, optimize the continuity of the transition audio using the Griffin-Lim algorithm, and then output and play the target audio.

[0103] Step S5. During the audio reverse conversion process, the audio embedding point position is checked and compared with the position sequence of the audio embedding points in the database. The audio is segmented according to the position sequence with the smallest comparison error, and morpheme audio comparison is performed, and the comparison results are output.

[0104] Step S5 includes:

[0105] Step S51. During the audio reverse conversion process, after acquiring the audio, the embedded points are verified using an artificial intelligence network. The relative positions of the embedded points in the audio are output and compared with the position sequence of the audio embedded points in the database. The audio is segmented according to the position sequence with the smallest comparison error and morpheme audio comparison is performed.

[0106] Step S53: Output the morpheme audio comparison result, output text according to the comparison result, and set pause punctuation at the position of the audio embedding point.

[0107] Example: After the user enters text, TTS converts the text into standard audio in the corpus. The morpheme audio in the human voice audio is compared one by one with the standard audio. If a sub-audio in the standard audio has the same sampling value as the morpheme audio, the sub-audio is cut from the standard audio and replaced with the corresponding morpheme audio. The repeated parts are used to obtain all replaced morpheme audios. If the standard audio replacement ratio is less than a threshold, it is fed back that the corpus reference audio does not cover all frequency bands. Otherwise, all morpheme audios are spliced, transition audio is added, and smoothing is performed to obtain the target audio.

[0108] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0109] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. The method for real-time audio conversion and analysis management based on artificial intelligence is characterized by: The method comprises the following steps: Step S1. Recording human voice reference audio, aligning the reference audio with the text using a dynamic time warping algorithm, and then performing frequency domain interpolation and expansion on the reference audio to obtain expanded audio. Step S2. Detect the spectrum boundaries of the extended audio, segment the audio using the spectrum boundaries and the text boundaries corresponding to the extended audio as segmentation points, segment the audio into morpheme audio and transition audio, calculate the LPC residual between the filtered morpheme audio and the standard audio signal, and use the residual as training data to train the neural network model; Step S3. Compile the input text into standard audio. The neural network model verifies the residual of each morpheme audio under the standard audio. The standard audio is segmented based on the verification results, so that each segmented sub-audio segment has a unique corresponding morpheme audio. The morpheme audio contains continuous sampling values, and the error between these sampling values ​​and the sampling values ​​of the sub-audio is within a preset range. Step S4. Each sub-audio is replaced with the corresponding morpheme audio segment, and the generated audio is obtained through time-domain synthesis. The generated audio is modeled according to the human auditory masking curve, and the Bark scale critical band position in the generated audio is determined. The transition audio is embedded into the generated audio using the critical band position as the embedding point to obtain the target audio output; Step S5. During the audio reverse conversion process, the audio embedding point position is checked and compared with the position sequence of the audio embedding points in the database. The audio is segmented according to the position sequence with the smallest comparison error, and morpheme audio comparison is performed, and the comparison results are output.

2. The method for real-time audio conversion and analysis management based on artificial intelligence according to claim 1, characterized in that: Step S1 includes: Step S11. Record human voice reference audio using swept frequency signal composite sampling technology, verify the signal-to-noise ratio of each frequency band through FFT, discard substandard audio, and store the obtained reference audio in the corpus; Step S12. Perform dynamic time warping of the reference audio based on the MFCC cepstral coefficients, align the reference audio with the text, and use the GMM-HMM model to fine-tune the time-frequency boundary to obtain the aligned audio; Step S13. Use linear interpolation and STFT phase reconstruction method to convert the sampling rate of the aligned audio, perform frequency domain interpolation expansion on the reference audio to match the longer text, and obtain the extended audio.

3. The method for real-time audio conversion and analysis management based on artificial intelligence according to claim 2, characterized in that: Step S2 includes: Step S21. Detecting the spectral boundaries of the extended audio using a short-time Fourier transform or a wavelet transform, segmenting the audio using the spectral boundaries and the text boundaries corresponding to the extended audio as segmentation points. Segments between two adjacent spectral boundaries or text boundaries are intercepted and output as morpheme audio. If the segments between the spectral boundary and the text boundary overlap, the shortest segment is selected and output as the morpheme audio, and the segment between the text boundary and the spectral boundary is output as transition audio. Step S22. Filter the morpheme audio with a Mel filter or a linear predictive coding filter, calculate the LPC residual between the filtered morpheme audio and the standard audio signal, use the residual as training data, and train a neural network to learn the vocal features in the morpheme audio. The neural network includes: CNN, WaveNet and Tacotron, and obtain a trained neural network model.

4. The method for real-time audio conversion and analysis management based on artificial intelligence according to claim 3, characterized in that: Step S3 includes: Step S31. Establish a text input platform. After receiving user input, generate a text-to-speech request. Compile the text into standard audio through the TTS system. Use the trained neural network model to verify the error of each morpheme audio under the standard audio. Based on the hidden Markov model, detect the time domain, frequency domain, and semantic meaning of the error, and fit the part of the morpheme audio where the error between the morpheme audio and the standard audio is less than a threshold. Step S32. Intelligently segment the standard audio to obtain sub-audio segments, and determine the correspondence between each sub-audio segment and the morpheme audio segment so that the segmentation result satisfies the following conditions: each sub-audio segment corresponds to a unique morpheme audio segment, and there are N consecutive sampling points in the morpheme audio segment that are sampled at the same time as the sub-audio segment, and the MSE error is lower than a preset threshold, where N is the sampling point length of the sub-audio segment.

5. The method for real-time audio conversion and analysis management based on artificial intelligence according to claim 4 is characterized in that: Step S4 includes: Step S41: Replace each sub-audio with a corresponding morpheme audio segment, where the morpheme audio segment consists of N consecutive sampling points in the morpheme audio with a sampling error less than a preset threshold at the same time as the sub-audio. The replaced audio segments are synthesized in a time-series order to obtain generated audio. Step S42: Utilizing the masking characteristics of the human ear, an acoustic model is established. A dynamic segmenter based on reinforcement learning is used to analyze the critical band locations in the generated audio. Using the Bark scale critical band locations as embedding points, transition audio is determined from the corpus and embedded into the generated audio. This transition audio is embedded in the generated audio such that, after applying time domain stretching and frequency domain envelope shifting, the transition audio satisfies the condition that the formant change rate between adjacent sub-audios is less than a threshold. Step S43: Using Constant-Q transform to perform phase synchronization transformation, smoothing the embedded audio, optimizing the continuity of the transition audio using the Griffin-Lim algorithm, and then outputting and playing the target audio. Step S5 includes: Step S51. During the audio reverse conversion process, after acquiring the audio, the embedded points are verified using an artificial intelligence network. The relative positions of the embedded points in the audio are output and compared with the position sequence of the audio embedded points in the database. The audio is segmented according to the position sequence with the smallest comparison error and morpheme audio comparison is performed. Step S53: Output the morpheme audio comparison result, output text according to the comparison result, and set pause punctuation at the position of the audio embedding point.

6. An artificial intelligence-based audio real-time conversion and analysis management system, characterized by: The system includes the following modules: audio processing module, intelligent network module, text segmentation module, embedding transition module and reverse conversion module; The audio processing module is used to record human voice reference audio, which contains all typical frequency bands. The reference audio and text are aligned using a dynamic time warping algorithm, and the GMM-HMM model is used to fine-tune the time-frequency boundary. After alignment, the audio sampling rate is converted, and the reference audio is interpolated and extended in the frequency domain to obtain extended audio. The intelligent network module is used to detect the spectrum boundary of the extended audio through short-time Fourier transform or wavelet transform, segment the audio using the spectrum boundary and the text boundary corresponding to the extended audio as segmentation points, intercept the paragraphs between two adjacent spectrum boundaries or text boundaries, take the shortest one and output it as morpheme audio, and output the paragraph between the text boundary and the spectrum boundary as transition audio, filter the morpheme audio with a Mel filter or a linear predictive coding filter, calculate the LPC residual between the filtered morpheme audio and the standard audio signal, use the residual as training data, train a neural network to learn the vocalization features in the morpheme audio, and obtain a trained neural network model; The text segmentation module is used to receive text input by the user, compile the text into standard audio through the TTS system, verify the error of each morpheme audio under the standard audio one by one using the trained neural network model, detect the time domain, frequency domain and semantics of the error based on the hidden Markov model, and segment the standard audio so that each segmented sub-audio has a unique corresponding morpheme audio, and the morpheme audio contains continuous sampling values, and the errors of these continuous sampling values ​​and the sampling values ​​of the sub-audio are all within a preset range; The embedding transition module is used to replace each sub-audio with a corresponding morpheme audio segment, arrange and synthesize the sub-audio segments in the segmentation order to obtain generated audio, model the human ear auditory masking curve, use a dynamic segmenter based on reinforcement learning to analyze the critical band positions in the generated audio, use the Bark scale critical band positions as embedding points, determine transition audio in the corpus, embed the transition audio into the generated audio, ensure that the transition audio satisfies the formant change rate between adjacent sub-audio segments is less than a threshold after applying time domain stretching and frequency domain envelope migration, and output the target audio; The reverse conversion module is used to use an artificial intelligence network to check the embedded points after obtaining the audio during the audio reverse conversion process, output the relative position of the embedded points in the audio, compare it with the position sequence of the audio embedded points in the database, determine the natural language environment, segment the audio according to the position sequence with the smallest comparison error, perform morpheme audio comparison, and output the comparison results.

7. The artificial intelligence-based audio real-time conversion and analysis management system according to claim 6, characterized in that: The audio processing module includes: a corpus recording unit and an interpolation expansion unit; The corpus recording unit is used to record human voice reference audio using a swept frequency signal composite sampling technology, and verify the signal-to-noise ratio of each frequency band through FFT, discarding substandard audio; The interpolation and extension unit is used to perform dynamic time warping of the reference audio based on the MFCC cepstral coefficients and expand the reference audio.

8. The artificial intelligence-based audio real-time conversion and analysis management system according to claim 7, characterized in that: The intelligent network module includes: a frequency conversion and division unit, a limiting filter unit and a network training unit; The frequency conversion segmentation unit is used to separate morpheme audio and transition audio, wherein the morpheme audio represents a stable audio segment with clear pronunciation, and the transition audio represents a transition audio segment between adjacent morphemes; The limiting filter unit is used to process the error residual wave of the morpheme audio in the standard audio filter using the U-Net structure; The network training unit is used to perform neural network training, and the neural network includes: CNN, WaveNet and Tacotron.

9. The artificial intelligence-based audio real-time conversion and analysis management system according to claim 8, characterized in that: The text segmentation module includes: a text management unit and an intelligent segmentation unit; The text management unit is used to establish a text input platform and generate a text-to-speech request; The intelligent segmentation unit is used to segment the standard audio and determine the corresponding relationship between each audio segment and the morpheme audio; The embedding transition module includes: an auditory masking unit, a sampling extraction unit and an audio embedding unit; The auditory masking unit is used to establish an acoustic model using the auditory masking characteristics of the human ear to reduce the embedding error to below the masking threshold; The sampling extraction unit is used to use Constant-Q transformation to perform phase synchronization transformation and smooth the embedded audio; The audio embedding unit is used to optimize the continuity of the transition segment audio through the Griffin-Lim algorithm and output the target audio.

10. The artificial intelligence-based real-time audio conversion and analysis management system according to claim 9, characterized in that: The reverse conversion module includes: a position memory unit and a text feedback unit; The position memory unit is used to generate a sequence according to the audio embedding point position and store it in the corpus; The text feedback unit is used to identify the converted audio and determine the converted audio segmentation points based on the historical embedding sequence.

Images