Method for adaptively simulating voice-changing real-time audio communication based on K-m algorithm

Abstract

The invention discloses a method for adaptively simulating voice-changing real-time audio communication based on a K-m algorithm, and the method comprises the following steps: S1, receiving the input of audio in real time through a microphone on an entrance machine, and recognizing and reading the audio data of an audio receiving message buffer area in the initial stage of a building intercom audio call establishment process; and S2, taking the buffer length of the audio data as the total length of sampling judgment data, and packaging the data according to a corresponding audio format. The invention provides an efficient and natural real-time voice changing scheme which can be applied to a real-time communication building talkback scene by self-adaptive human voice detection and effective voice energy recognition aiming at different audio coding formats and dynamically applying a suitable voice changing technical strategy to carry out audio adjustment and voice changing according to a clustering classification algorithm result, so that the real-time voice changing scheme can be applied to a real-time communication building talkback scene, and the user experience is improved. And a more natural and abundant voice changing function is provided for the user.

Description

Technical Field

[0001] The invention relates to the field of real-time audio communication transmission, and in particular to a method for real-time audio communication based on Km algorithm adaptive simulation voice change. Background Art

[0002] In recent years, with the rapid development of online social platforms and mobile communication applications, users have an increasing demand for personalization and fun in daily communication. As an important application in the field of sound processing, voice changing technology has been widely used in online voice chat, game voice interaction and other scenarios. However, in the existing real-time communication application scenarios of building intercom systems, the existing voice changing technology has some limitations in terms of real-time performance, flexibility and naturalness of sound effects. For voices of different age groups and genders, the voice changing effect needs to take into account the differences in pitch, speed, beat and other characteristics. When processing the audio characteristics of different groups (different ages and genders, such as the elderly and children), it is often difficult to achieve the desired effect. In the past, it was not possible to accurately adjust the timbre (pitch, speed, beat, etc.) in real time during audio communication, nor was it possible to adaptively change the voice to match the pitch of different people's audio, resulting in unsatisfactory voice changing effects. In addition, existing fixed voice changing technical strategies cannot be adaptively adjusted according to audio input, and it is difficult to meet the needs of real-time communication scenarios for highly natural and personalized voice changing effects. Therefore, how to adaptively detect the timbre of voices of different age groups and genders during the voice changing process, and adaptively adjust the voice changing according to the timbre feature range, has become a technical problem that needs to be solved urgently. To this end, this application proposes a method for real-time audio communication based on a Km algorithm for adaptive simulated voice changing. Summary of the invention

[0003] Based on the technical problems existing in the background technology, the present invention proposes a method for real-time audio communication based on Km algorithm adaptive simulation voice change.

[0004] The present invention proposes a method for real-time audio communication based on Km algorithm adaptive simulation voice change, comprising the following steps:

[0005] S1: In the initial stage of establishing a building intercom audio call, the microphone on the door machine receives the audio input in real time, and identifies and reads the audio data in the audio receiving message buffer;

[0006] S2: according to the length of the audio data buffer as the total length of the sampled judgment data, the data is packaged according to the corresponding audio format, and the audio data with effective energy is judged according to the package length of different audio encoding formats, and more than three points of effective data in the total length of the sampled audio buffer are selected. If the audio sampling data is valid, proceed to the next step, otherwise it is regarded as other non-human noise and is not processed;

[0007] S3: Perform fast Fourier transform on the effective audio sampling number, convert the time domain signal into the frequency domain signal, calculate the amplitude of each frequency component, and take the highest audio value or the audio value with a larger proportion in the spectrum component as the main audio value;

[0008] S4: The fundamental frequency result obtained by fast Fourier transform is used as the detection input basis for the voices of different age groups and genders, and the corresponding cluster center is adaptively selected according to the result of the cluster classification module to obtain the pitch range;

[0009] S5: According to the pitch range, the voice changing technology strategy and the adaptive voice changing interface are used to adjust the corresponding pitch, beat, sound speed and playback rate, and the 3A module is used to eliminate noise and echo, and automatic gain control is performed;

[0010] S6: The voice changer module changes the audio processed in S5;

[0011] S7: The audio processed in S6 is sent to the cloud server through the sending module. The cloud server forwards the processed audio and plays the audio to the user through the building intercom device.

[0012] Preferably, in S2, the determination of valid data is based on the energy value of the audio, and when the audio energy value is greater than a specified value, it is valid audio sampling data.

[0013] Preferably, in S5, when adjusting the pitch, the audio signal is up-sampled or down-sampled, and an anti-aliasing filter is applied to prevent the aliasing effect. By changing the sampling rate, the pitch of the audio will be increased or decreased accordingly, and the speed of the audio will not be affected, thereby achieving the ability to adjust the pitch. When adjusting the sound speed, the WSOLA (Waveform Similarity Overlap-Add) algorithm is used to change the playback speed of the audio without affecting its tone, thereby achieving the ability to adjust the rhythm. When adjusting the playback rate, it mainly relies on the combined use of resampling technology and WSOLA algorithm, as well as the application of anti-aliasing filters, to achieve the effect of adjusting the pitch while changing the playback rate.

[0014] Preferably, in S5, the 3A module includes an echo cancellation module, a noise cancellation module and an automatic gain control module, the echo cancellation module is used to cancel echoes, the noise cancellation module is used to cancel noises, and the automatic gain control module is used to perform automatic gain control on audio.

[0015] Preferably, in S1, when the user presses the call button of the outdoor machine, the outdoor machine sends a call request to the cloud server, the cloud server forwards the call request to the mobile phone APP, the mobile phone APP sends a monitoring request, the cloud server forwards the monitoring request, and receives the status update or notification information of the outdoor machine.

[0016] Preferably, in S4, the main principle of the cluster classification module is to obtain the average value of one class in the cluster classification, select a suitable cluster interval according to the main frequency of the audio, and re-enter the cluster classification algorithm in the background for each audio sampling data, dynamically adjust the cluster classification dynamic center result in real time, and continuously reversely train and adjust the cluster interval, so as to be closer to the actual value of the sampling frequency actually obtained, match the corresponding timbre adjustment strategy, so as to achieve a better subjective voice changing auditory effect;

[0017] In addition, when adjusting the clustering interval, since in the building intercom application scenario, the users are a relatively fixed group of people, the effect of this learning method will be continuously optimized over a period of time.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The present invention provides an efficient and natural real-time voice changing solution by adaptively detecting human voices for different audio coding formats, identifying effective sound energy, and dynamically applying suitable voice changing technology strategies to adjust and change audio according to the results of clustering and classification algorithms. This solution can be applied to building intercom scenarios for real-time communication, providing users with more natural and rich voice changing functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for real-time audio communication based on Km algorithm adaptive simulation voice change proposed by the present invention;

[0021] Figure 2 This is a block diagram of the connection between the door phone, cloud server and mobile phone APP in the method for real-time audio communication based on Km algorithm adaptive simulation voice change proposed by the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further explained below in conjunction with specific embodiments.

[0023] Example

[0024] Reference Figure 1-2 This embodiment proposes a method for real-time audio communication based on Km algorithm adaptive simulation voice change, including the following steps:

[0025] S1: In the initial stage of establishing a building intercom audio call, the microphone on the door machine receives the audio input in real time, and identifies and reads the audio data in the audio receiving message buffer;

[0026] When the user presses the call button of the outdoor station, the outdoor station sends a call request to the cloud server, and the cloud server forwards the call request to the mobile phone APP. The mobile phone APP sends a monitoring request, and the cloud server forwards the monitoring request and receives the status update or notification information of the outdoor station;

[0027] S2: according to the length of the audio data buffer as the total length of the sampled judgment data, the data is packaged according to the corresponding audio format, and the audio data with effective energy is judged according to the package length of different audio encoding formats, and more than three points of effective data in the total length of the sampled audio buffer are selected. If the audio sampling data is valid, proceed to the next step, otherwise it is regarded as other non-human noise and is not processed;

[0028] The judgment of valid data is based on the energy value of the audio. When the audio energy value is greater than the specified value, it is valid audio sampling data;

[0029] S3: Perform fast Fourier transform on the effective audio sampling number, convert the time domain signal into the frequency domain signal, calculate the amplitude of each frequency component, and take the highest audio value or the audio value with a larger proportion in the spectrum component as the main audio value;

[0030] S4: The fundamental frequency result obtained by fast Fourier transform is used as the detection input basis of the voices of different age groups and genders, and the corresponding cluster interval is adaptively selected according to the result of the cluster classification module to obtain the pitch range;

[0031] The main principle of the cluster classification module is to obtain the average value of a class in the cluster classification, select the appropriate cluster interval according to the main frequency of the audio, and re-enter the cluster classification algorithm in the background for each audio sampling data, dynamically adjust the cluster classification dynamic center result in real time, and continuously reversely train and adjust the cluster interval, so as to be closer to the actual value of the sampling frequency actually obtained, match the corresponding timbre adjustment strategy, so that it can achieve a better subjective voice changing hearing effect;

[0032] In addition, when adjusting the clustering interval, since users in the building intercom application scenario are relatively fixed and of the same type, the effect of this learning method will be continuously optimized over a period of time;

[0033] S5: According to the pitch range, the voice changing technology strategy and the adaptive voice changing interface are used to adjust the corresponding pitch, beat, sound speed and playback rate, and the 3A module is used to eliminate noise and echo, and automatic gain control is performed;

[0034] When adjusting the pitch, the audio signal is upsampled or downsampled, and an anti-aliasing filter is applied to prevent the aliasing effect. By changing the sampling rate, the pitch of the audio will be increased or decreased accordingly, and the speed of the audio will not be affected, thereby realizing the ability to adjust the pitch. When adjusting the speed of sound, the WSOLA (Waveform Similarity Overlap-Add) algorithm is used to change the playback speed of the audio without affecting its tone, thereby realizing the ability to adjust the rhythm. When adjusting the playback rate, it mainly relies on the combination of resampling technology and WSOLA algorithm, as well as the application of anti-aliasing filters, to achieve the effect of adjusting the pitch while changing the playback rate;

[0035] The 3A module includes an echo cancellation module, a noise cancellation module and an automatic gain control module. The echo cancellation module is used to eliminate echoes, the noise cancellation module is used to eliminate noise, and the automatic gain control module is used to perform automatic gain control on audio.

[0036] S6: The voice changer module changes the audio processed in S5;

[0037] S7: The audio processed in S6 is sent to the cloud server through the sending module. The cloud server forwards the processed audio and plays the audio to the user through the building intercom device.

[0038] This embodiment provides an efficient and natural real-time voice changing solution by adaptively detecting human voices for different audio coding formats, identifying effective sound energy, and dynamically applying suitable voice changing technology strategies to adjust and change the audio according to the results of the clustering classification algorithm. This solution can be applied to building intercom scenarios for real-time communications, providing users with more natural and rich voice changing functions.

[0039] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for real-time audio communication based on Km algorithm adaptive simulation voice change, characterized in that: The following steps are involved: S1: In the initial stage of establishing a building intercom audio call, the microphone on the door machine receives the audio input in real time, and identifies and reads the audio data in the audio receiving message buffer; S2: according to the length of the audio data buffer as the total length of the sampled judgment data, the data is packaged according to the corresponding audio format, and the audio data with effective energy is judged according to the package length of different audio encoding formats, and more than three points of effective data in the total length of the sampled audio buffer are selected. If the audio sampling data is valid, proceed to the next step, otherwise it is regarded as other non-human noise and is not processed; S3: Perform fast Fourier transform on the effective audio sampling number, convert the time domain signal into the frequency domain signal, calculate the amplitude of each frequency component, and take the highest audio value or the audio value with a larger proportion in the spectrum component as the main audio value; S4: The fundamental frequency result obtained by fast Fourier transform is used as the detection input basis for the voices of different age groups and genders, and the corresponding cluster center is adaptively selected according to the result of the cluster classification module to obtain the pitch range; S5: According to the pitch range, the voice changing technology strategy and the adaptive voice changing interface are used to adjust the corresponding pitch, beat, sound speed and playback rate, and the 3A module is used to eliminate noise and echo, and automatic gain control is performed; S6: The voice changer module changes the audio processed in S5; S7: The audio processed in S6 is sent to the cloud server through the sending module. The cloud server forwards the processed audio and plays the audio to the user through the building intercom device.

2. The method for real-time audio communication based on Km algorithm adaptive simulation voice change according to claim 1, characterized in that: In S2, the determination of valid data is based on the energy value of the audio. When the audio energy value is greater than a specified value, it is valid audio sampling data.

3. The method for real-time audio communication based on Km algorithm adaptive simulation voice change according to claim 1, characterized in that: In S5, when adjusting the pitch, the audio signal is upsampled or downsampled, and an anti-aliasing filter is applied to prevent the aliasing effect. By changing the sampling rate, the pitch of the audio will be increased or decreased accordingly, and the speed of the audio will not be affected, thereby achieving the ability to adjust the pitch. When adjusting the speed of sound, the WSOLA algorithm is used to change the playback speed of the audio without affecting its tone, thereby achieving the ability to adjust the rhythm. When adjusting the playback rate, it mainly relies on the combined use of the resampling technology and the WSOLA algorithm, as well as the application of the anti-aliasing filter, to achieve the effect of adjusting the pitch while changing the playback rate.

4. The method for real-time audio communication based on Km algorithm adaptive simulation voice change according to claim 1, characterized in that: In the S5, the 3A module includes an echo cancellation module, a noise cancellation module and an automatic gain control module. The echo cancellation module is used to cancel echoes, the noise cancellation module is used to cancel noises, and the automatic gain control module is used to perform automatic gain control on audio.

5. The method for real-time audio communication based on Km algorithm adaptive simulation voice change according to claim 1, characterized in that: In S1, when the user presses the outdoor phone call button, the outdoor phone sends a call request to the cloud server, the cloud server forwards the call request to the mobile phone APP, the mobile phone APP sends a monitoring request, the cloud server forwards the monitoring request, and receives the status update or notification information of the outdoor phone.

6. The method for real-time audio communication based on Km algorithm adaptive simulation voice change according to claim 1, characterized in that: In the S4, the main principle of the cluster classification module is to obtain the average value of a class in the cluster classification, select a suitable clustering interval according to the main frequency of the audio, and re-enter the clustering classification algorithm in the background for each audio sampling data, dynamically adjust the cluster classification dynamic center result in real time, and continuously reversely train and adjust the clustering interval and cluster center, so as to be closer to the actual value of the sampling frequency actually obtained, match the corresponding timbre adjustment strategy, and achieve a better subjective voice changing auditory effect.

Images