Audio processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an audio processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: after initiating audio interaction through a target object identifier, starting audio signal acquisition; calculating a voiceprint vector corresponding to the audio signal collected in real time, and comparing the voiceprint vector obtained by calculation with a target voiceprint vector corresponding to the target object identifier to obtain a comparison result; under the condition that the speaking sound is determined to be from the target object according to the comparison result, sending an audio signal to an opposite end of audio interaction; and under the condition of determining that the speaking sound is not from the target object according to the comparison result, carrying out sound changing processing on the audio signal to obtain a sound changing audio signal, and sending the sound changing audio signal to an opposite end of audio interaction. By adopting the method, the sound information of other personnel can be protected in the audio interaction process, and the personal information and privacy security of the user can be protected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an audio processing method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] With the development of computer technology and Internet technology, audio-based interaction has become an application scenario that can be seen everywhere in daily life, such as online meetings, interactive live broadcasts, online classrooms, etc. Real-time audio and video interaction can greatly facilitate information transmission and improve communication efficiency.

[0003] However, in some audio interaction scenarios, the far-end user can not only hear the speaking voice of the user initiating the audio interaction, but also needs to hear the speaking voice of other personnel, or inadvertently hears the speaking voice of other personnel, which may lead to exposure of the voices of other personnel except the user initiating the audio interaction, resulting in information leakage and personal privacy security cannot be guaranteed. SUMMARY

[0004] Therefore, it is necessary to provide an audio processing method, device, computer equipment, computer readable storage medium and computer program product to protect the voice information of other personnel and protect the personal information of the user during the audio interaction process.

[0005] In a first aspect, the present application provides an audio processing method, comprising:

[0006] starting audio signal collection after initiating audio interaction through a target object identifier;

[0007] calculating a voiceprint vector corresponding to the real-time collected audio signal, comparing the calculated voiceprint vector with a target voiceprint vector corresponding to the target object identifier, and obtaining a comparison result;

[0008] in a case where it is determined according to the comparison result that the speaking voice originates from the target object, sending the audio signal to the opposite end of the audio interaction;

[0009] in a case where it is determined according to the comparison result that the speaking voice does not originate from the target object, performing voice transformation processing on the audio signal to obtain a transformed audio signal, and sending the transformed audio signal to the opposite end of the audio interaction.

[0010] In a second aspect, the present application further provides an audio processing device, comprising:

[0011] an audio interaction initiation module configured to start audio signal collection after initiating audio interaction through a target object identifier;

[0012] The comparison module is configured to calculate a voiceprint vector corresponding to the real-time collected audio signal, compare the calculated voiceprint vector with a target voiceprint vector corresponding to the target object identifier, and obtain a comparison result.

[0013] The sending module is configured to, in a case where it is determined according to the comparison result that the speaking voice is from the target object, send the audio signal to a peer end of the audio interaction.

[0014] The sending module is configured to, in a case where it is determined according to the comparison result that the speaking voice is not from the target object, perform voice conversion processing on the audio signal to obtain a converted audio signal, and send the converted audio signal to the peer end of the audio interaction.

[0015] In an embodiment, the comparison module is further configured to perform activity detection on the real-time collected audio signal to obtain a detection result, and in a case where the detection result indicates that the audio signal contains speaking voice, perform the steps of calculating a voiceprint vector corresponding to the real-time collected audio signal, and comparing the calculated voiceprint vector with a target voiceprint vector corresponding to the target object identifier; and in a case where the detection result indicates that the original audio signal does not contain speaking voice, continue to collect the audio signal and perform activity detection on the real-time collected audio signal.

[0016] In an embodiment, the comparison module is further configured to divide the real-time collected audio signal into a plurality of audio frames, convert each audio frame into a frequency domain signal, calculate speaking voice energy corresponding to speaking spectrum in the audio frame according to the frequency domain signal, calculate total energy of the audio frame according to the frequency domain signal, obtain a detection result that the audio signal contains speaking voice when a proportion of the speaking voice energy to the total energy is greater than or equal to a set threshold, and obtain a detection result that the audio signal does not contain speaking voice when the proportion of the speaking voice energy to the total energy is less than the set threshold.

[0017] In an embodiment, the comparison module is further configured to obtain a general background model trained based on speech samples from different speakers, adjust model parameters of the general background model by using the real-time collected audio signal through an adaptive algorithm to obtain a Gaussian mixture model corresponding to the real-time collected audio signal, the Gaussian mixture model being obtained by linearly combining a plurality of Gaussian distribution functions according to model parameters corresponding to each Gaussian distribution function, obtain a voiceprint vector corresponding to the real-time collected audio signal according to the model parameters corresponding to each Gaussian distribution function, obtain a target voiceprint vector corresponding to the target object identifier, and compare the voiceprint vector corresponding to the real-time collected audio signal with the target voiceprint vector corresponding to the target object identifier to obtain a comparison result.

[0018] In an embodiment, the comparison module is further configured to adjust model parameters of the general background model using the sound source signal of the target object by an adaptive algorithm to obtain a Gaussian mixture model corresponding to the sound source signal of the target object, the Gaussian mixture model being obtained by linearly combining a plurality of Gaussian distribution functions according to model parameters corresponding to each Gaussian distribution function; and obtain a target voiceprint vector corresponding to the target object identifier according to the model parameters corresponding to each Gaussian distribution function.

[0019] In an embodiment, the comparison module is further configured to prompt voiceprint extraction after the target object identifier logs in the audio interaction client, collect the sound source signal of the target object in response to a voiceprint extraction confirmation operation, and save the target voiceprint vector corresponding to the target object identifier in the audio interaction client after obtaining the target voiceprint vector corresponding to the target object identifier using the sound source signal of the target object.

[0020] In an embodiment, the device further comprises:

[0021] The voice changing module is configured to sequentially perform pitch processing and speed processing on the audio signal to obtain a changed audio signal.

[0022] In an embodiment, the voice changing module is further configured to convert the audio signal into a frequency domain signal, sample the frequency domain signal according to frequency to obtain a sampling frequency and a corresponding sampling amplitude, and convert the frequency domain signal with the sampling frequency and the corresponding sampling amplitude into a time domain signal, and take the obtained time domain signal as the audio signal after pitch processing.

[0023] In an embodiment, the voice changing module is further configured to perform pitch detection on the audio signal before converting the audio signal into a frequency domain signal to determine a pitch period of the audio signal, determine a pitch period range to which the pitch period of the audio signal belongs, and obtain a sampling rate corresponding to the pitch period range to which the pitch period of the audio signal belongs according to a pre-set correspondence between pitch period ranges and sampling rates, sample the frequency domain signal according to frequency using the sampling rate to obtain a sampling frequency and a corresponding sampling amplitude.

[0024] In an embodiment, the voice changing module is further configured to divide the audio signal after pitch processing into different audio frames, each audio frame having a preset time length, stretch each audio frame in the time dimension, and then synthesize the stretched audio frames in chronological order to obtain a changed audio signal.

[0025] In one embodiment, the voice-changing module is further configured to input the audio signal into a voice-changing speech model for feature extraction to obtain the speech recognition features of the audio signal; input the sound source signal of a preset object into the voice-changing speech model for feature extraction to obtain the speech recognition features of the sound source signal; and through the voice-changing speech model, map the audio signal to the voice of the preset object based on the speech recognition features of the sound source signal and the speech recognition features of the audio signal to obtain a voice-changing audio signal.

[0026] In one embodiment, the sending module is further configured to, if it is determined from the comparison result that the speaking voice does not originate from the target object, perform voice-changing processing on the audio signal to obtain a voice-changing audio signal; add a sound source identifier to the voice-changing audio signal to indicate that the speaking voice does not originate from the target object; and send the voice-changing audio signal with the added sound source identifier to the other end of the audio interaction to indicate that the current speaking voice does not originate from the target object.

[0027] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0028] After initiating audio interaction through the target object identifier, audio signal acquisition is started;

[0029] Calculate the voiceprint vector corresponding to the real-time acquired audio signal, compare the calculated voiceprint vector with the target voiceprint vector corresponding to the target object identifier, and obtain the comparison result.

[0030] If the comparison results indicate that the voice originates from the target object, the audio signal is sent to the other end of the audio interaction.

[0031] If the comparison results determine that the voice does not originate from the target object, the audio signal is processed to obtain a voice-changing audio signal, which is then sent to the other end of the audio interaction.

[0032] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0033] After initiating audio interaction through the target object identifier, audio signal acquisition is started;

[0034] Calculate the voiceprint vector corresponding to the real-time acquired audio signal, compare the calculated voiceprint vector with the target voiceprint vector corresponding to the target object identifier, and obtain the comparison result.

[0035] in a case where it is determined according to the comparison result that the speaking voice is from the target object, the audio signal is sent to a peer end of the audio interaction;

[0036] in a case where it is determined according to the comparison result that the speaking voice is not from the target object, voice changing processing is performed on the audio signal to obtain a changed audio signal, and the changed audio signal is sent to the peer end of the audio interaction.

[0037] In a fifth aspect, the present application also provides a computer program product, comprising a computer program which, when executed by a processor, implements the following steps:

[0038] After initiating the audio interaction through the target object identifier, audio signal collection is started;

[0039] A voiceprint vector corresponding to the real-time collected audio signal is calculated, and the calculated voiceprint vector is compared with a target voiceprint vector corresponding to the target object identifier to obtain a comparison result;

[0040] in a case where it is determined according to the comparison result that the speaking voice is from the target object, the audio signal is sent to a peer end of the audio interaction;

[0041] in a case where it is determined according to the comparison result that the speaking voice is not from the target object, voice changing processing is performed on the audio signal to obtain a changed audio signal, and the changed audio signal is sent to the peer end of the audio interaction.

[0042] The above audio processing method, device, computer device, computer readable storage medium and computer program product, after initiating the audio interaction through the target object identifier, audio signal collection is started, after starting the audio signal collection, a voiceprint vector corresponding to the real-time collected audio signal is calculated, and the calculated voiceprint vector is compared with a target voiceprint vector corresponding to the target object identifier to obtain a comparison result; in a case where it is determined according to the comparison result that the speaking voice is from the target object, the audio signal is sent to a peer end of the audio interaction; in a case where it is determined according to the comparison result that the speaking voice is not from the target object, voice changing processing is performed on the audio signal to obtain a changed audio signal, and the changed audio signal is sent to the peer end of the audio interaction. In this way, in a scenario where multiple people share the audio interaction initiated through the target object identifier, the voice information of members other than the target object can be prevented from being directly exposed to the peer end of the audio interaction in the audio interaction process, and the audio interaction initiated through the target object identifier can also be unaffected, so that the voice information of other people can be protected in the audio interaction process, and the personal information of the user can be protected. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained on the basis of these drawings without creative effort.

[0044] Figure 1 The application environment diagram of the audio processing method in an embodiment;

[0045] Figure 2 The structural block diagram of the audio processing system provided by an exemplary embodiment of the present application;

[0046] Figure 3 The technical architecture diagram of the audio interaction in the related art;

[0047] Figure 4 The technical architecture diagram of the audio interaction provided by an exemplary embodiment of the present application;

[0048] Figure 5 The flow diagram of the audio processing method in an embodiment;

[0049] Figure 6 The flow diagram of the voice activity detection in an embodiment;

[0050] Figure 7 The step diagram of the sound protection in an embodiment;

[0051] Figure 8 The detailed step diagram of the audio processing in a detailed embodiment;

[0052] Figure 9 The structural block diagram of the audio processing device in an embodiment;

[0053] Figure 10 The internal structure diagram of the computer device in an embodiment;

[0054] Figure 11 The internal structure diagram of the computer device in an embodiment. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the present application clearer, the following will further describe the present application with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0056] The audio processing method provided by the embodiments of the present application can be applied to, for example Figure 1The application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data required by the server 104 to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. In one embodiment, the terminal 102 can initiate audio interaction through the target object identification, start audio signal collection; calculate the voiceprint vector corresponding to the real-time collected audio signal, compare the calculated voiceprint vector with the target voiceprint vector corresponding to the target object identification, and obtain the comparison result; in the case of determining that the speaking voice comes from the target object according to the comparison result, the audio signal is sent to the opposite end of the audio interaction; in the case of determining that the speaking voice does not come from the target object according to the comparison result, the audio signal is processed to obtain a variable voice audio signal, and the variable voice audio signal is sent to the opposite end of the audio interaction through the server 104.

[0057] The server 104 can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal 102 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, a smart television, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0058] Figure 2 The structure block diagram of the audio processing system 100 provided by an exemplary embodiment of the present application is shown. The audio processing system 100 includes a first terminal 102, a server 104 and a second terminal 106.

[0059] The first terminal 102 is connected to the server cluster 102 through a wireless network or a wired network. The first terminal 102 is installed and runs a client (or an application) supporting audio and video functions, which can be any one of a conference client supporting an online conference function, a user content generation client supporting a live broadcast function, and a learning client supporting an online class function. The first terminal 102 is a terminal used by a target object, and the client in the first terminal 102 is logged in with an identifier of the target object, representing that the target object uses the functions of the client through the first terminal 102. The first terminal 102 is connected to the server 104 through a wireless network or a wired network. Optionally, the clients installed on the first terminal 102 and the second terminal 106 are the same, or the clients installed on the two terminals are the same type of clients of different operating system platforms. Different operating systems include an Android operating system, an IOS operating system, a Windows operating system, and an operating system dedicated to a game console. The first terminal 102 can be referred to as one of a plurality of terminals, and the second terminal 106 can be referred to as one of a plurality of terminals. The terminals mentioned in this embodiment can be the first terminal 102 or the second terminal 106 for example. The second terminal 104 is a peer terminal of the first terminal 102 in audio interaction.

[0060] In one application scenario, an online conference client is run on a terminal, a conference host logs in the online conference client through an account of the conference host, after the online conference starts, the terminal starts audio interaction of the online conference, in addition to the conference host, some other personnel share the account to participate in the multi-person conference, so that a user at a peer terminal can hear the voices of the conference host and the other participants. The terminal uses the audio processing method provided in this embodiment, which can protect the voice information of the other participants in the discussion process of the online conference without affecting the voice of the conference host, protect the personal information of the user, and make the other personnel participating in the conference at the peer terminal unable to identify the specific identities of the users around the conference host through voice.

[0061] In one application scenario, a live broadcast client is run on a terminal, a host logs in the live broadcast client through an account of the host, after the live broadcast starts, in addition to the host, some other personnel (for example, some backstage assistant personnel and staff) share the account to participate in the live broadcast, so that a user watching the live broadcast can hear the voices of the host and the other personnel. The terminal uses the audio processing method provided in this embodiment, which can protect the voice information of the other personnel in the live broadcast process without affecting the voice of the host, protect the personal information of the user, and make the audience unable to identify the specific identities of the users around the host through voice.

[0062] In one application scenario, an online class client is run on a terminal, a teacher logs in the online class client through his own account, after starting online teaching, the terminal starts audio interaction of the online class, in addition to the main teacher, some other personnel (for example, students in the same classroom as the main teacher) share the same account to participate in the online class, so that the audio interaction counterpart user can hear the voices of the main teacher and the students (such as the classroom question and answer interaction process between the students and the main teacher). The terminal uses the audio processing method provided in the embodiments of the present application, which can protect the voice information of the students in the question and answer interaction process of the online class without affecting the speaking voice of the main teacher, so that the counterpart personnel participating in the online class cannot identify the specific identity of the user around the main teacher through voice, especially in the case of students being minors, the personal privacy of minors can be protected.

[0063] It should be noted that the audio processing method provided in the embodiments of the present application can also be executed by a server, for example, after the terminal initiates audio interaction through the target object identifier, the audio signal collection is started, and the real-time collected audio signal is directly sent to the server. The server receives the audio signal uploaded by the terminal, calculates the voiceprint vector corresponding to the audio signal, compares the calculated voiceprint vector with the target voiceprint vector corresponding to the target object identifier, obtains the comparison result, and in the case of determining that the speaking voice is derived from the target object according to the comparison result, the server sends the audio signal to the counterpart of the audio interaction. In the case of determining that the speaking voice is not derived from the target object according to the comparison result, the server performs voice changing processing on the audio signal to obtain a changed voice signal, and sends the changed voice signal to the counterpart of the audio interaction.

[0064] As shown in Figure 3 , it is a technical architecture schematic diagram of audio interaction in the related art, referring to Figure 3 , generally, after the near-end terminal initiates audio interaction and starts audio signal collection, the real-time collected audio signal is sequentially subjected to preprocessing and encoding to obtain an encoded code stream, and then is uploaded to the server, and is downloaded to the far-end terminal by the server. The far-end terminal sequentially undergoes decoding and sound mixing processing, and then plays the audio signal. In this way, the audio interaction is realized. As can be seen from the above process, when the near-end user side includes the target object and a plurality of other objects, the voice information of all the objects will be collected and uploaded to the server. At this time, the far-end user can hear the voices of all the objects, and can also identify the corresponding near-end user through tone. In individual scenarios, this will lead to the leakage of the identity of the near-end user, resulting in information security risks.

[0065] As shown in Figure 4 , it is a technical architecture schematic diagram of audio interaction in one embodiment of the present application, referring to Figure 4After the near-end terminal initiates the audio interaction, starts the audio signal collection, and collects the real-time audio signals, the real-time audio signals are sequentially subjected to preprocessing, sound protection processing, and encoding to obtain an encoded stream, which is then uploaded to the server and downloaded to the far-end terminal. The far-end terminal sequentially undergoes decoding and sound mixing processing and then plays the audio signals. In this way, by adding the sound protection processing, the sound information of other persons can be protected during the audio interaction, and the personal information of the user is protected.

[0066] In an exemplary embodiment, as shown in Figure 5 , an audio processing method is provided, which is applied to a computer device (terminal 102 or server 104) in Figure 1 for example, and includes the following steps 502 to 508. Among them:

[0067] Step 502: After initiating the audio interaction through the target object identifier, start the audio signal collection.

[0068] The target object identifier is used to uniquely identify the target object, and the target object identifier can be a mobile phone number, an email address, or other third-party application account used by the target object, etc. After the computer device initiates the audio interaction through the target object identifier, the audio signal collection is started. For example, in the application scenario of a multi-person online conference, after the computer device initiates the online conference, the audio signal collection is started to transmit the sound signals emitted by the conference personnel participating in the multi-person online conference to the far-end of the audio interaction.

[0069] Step 504: Calculate the voiceprint vector corresponding to the real-time collected audio signal, compare the calculated voiceprint vector with the target voiceprint vector corresponding to the target object identifier, and obtain a comparison result.

[0070] The voiceprint is a biological feature carrying language information, which has stability, measurability, and uniqueness. The voiceprint vector is a vectorized expression of the voiceprint, which is used to uniquely represent the voice features of the same speaker, thereby verifying the identity of the speaker. The target voiceprint vector corresponding to the target object identifier is used to uniquely represent the voiceprint information of the target object identifier. The computer device can obtain the target voiceprint vector corresponding to the target object identifier in advance and store it locally, and obtain the target voiceprint vector corresponding to the target object identifier from the local when the voiceprint comparison is needed. Of course, the computer device can also obtain the target voiceprint vector corresponding to the target object identifier from other devices, and the embodiments of the present application do not limit this.

[0071] After the computer device collects the audio signal, the computer device can calculate the voiceprint vector corresponding to the real-time collected audio signal. Alternatively, after the computer device collects the audio signal, the computer device can first pre-process the audio signal, and then calculate the corresponding voiceprint vector. The pre-processing is, for example, 3A processing, that is, the combination of three audio algorithms, namely, acoustic echo cancellation (AEC), background noise suppression (ANS), and automatic gain control (AGC).

[0072] After the voiceprint vector corresponding to the real-time collected audio signal is calculated, the computer device can obtain a target voiceprint vector corresponding to the target object identifier, compare the two, and obtain a comparison result. In the case where the similarity between the voiceprint vector corresponding to the real-time collected audio signal and the target voiceprint vector corresponding to the target object identifier is less than a set threshold, the comparison result is that the speaking voice of the real-time collected audio signal does not come from the target object, and in the case where the similarity between the voiceprint vector corresponding to the real-time collected audio signal and the target voiceprint vector corresponding to the target object identifier is greater than or equal to the set threshold, the comparison result is that the speaking voice of the real-time collected audio signal comes from the target object. The computer device can calculate the Euclidean distance between the voiceprint vector corresponding to the real-time collected audio signal and the target voiceprint vector corresponding to the target object identifier as the similarity.

[0073] Step 506, in the case where it is determined according to the comparison result that the speaking voice comes from the target object, the audio signal is sent to the opposite end of the audio interaction.

[0074] In the case where it is determined according to the comparison result that the speaking voice comes from the target object, it is indicated that the current speaker is the target object, and the voice of the target object does not need to be protected, so the computer device can directly send the audio signal to the opposite end of the audio interaction.

[0075] Step 508, in the case where it is determined according to the comparison result that the speaking voice does not come from the target object, the audio signal is processed to obtain a variable voice audio signal, and the variable voice audio signal is sent to the opposite end of the audio interaction.

[0076] The variable voice processing on the audio signal can be pitch processing, can also be speed processing, or can be both pitch processing and speed processing, to ensure that the voice is more natural and harmonious while being processed. In the case where it is determined according to the comparison result that the speaking voice does not come from the target object, it is indicated that the current speaker is an object other than the target object, and the voice of the other object needs to be protected, so the computer device needs to process the audio signal to obtain a variable voice audio signal, and then send the variable voice audio signal to the opposite end of the audio interaction.

[0077] The audio processing method initiates audio signal collection after initiating the audio interaction through the target object identifier, calculates a voiceprint vector corresponding to the real-time collected audio signal after initiating the audio signal collection, compares the calculated voiceprint vector with a target voiceprint vector corresponding to the target object identifier to obtain a comparison result, sends the audio signal to the opposite end of the audio interaction in a case where it is determined according to the comparison result that the speaking voice is from the target object, and performs voice changing processing on the audio signal to obtain a changed voice signal and sends the changed voice signal to the opposite end of the audio interaction in a case where it is determined according to the comparison result that the speaking voice is not from the target object. In this way, in a scenario where multiple people share the audio interaction initiated through the target object identifier, the voice information of the other members except the target object can be prevented from being directly exposed to the opposite end of the audio interaction in the audio interaction process, and the audio interaction initiated through the target object identifier can also be unaffected, so that the voice information of the other people and the personal information of the user can be protected in the audio interaction process.

[0078] In one embodiment, before calculating the voiceprint vector corresponding to the real-time collected audio signal, the method further includes the step of performing activity detection on the real-time collected audio signal to obtain a detection result.

[0079] That is, the audio processing method includes: initiating audio signal collection after initiating the audio interaction through the target object identifier; performing activity detection on the real-time collected audio signal to obtain a detection result, calculating a voiceprint vector corresponding to the real-time collected audio signal in a case where the detection result indicates that the audio signal contains speaking voice, comparing the calculated voiceprint vector with a target voiceprint vector corresponding to the target object identifier to obtain a comparison result, sending the audio signal to the opposite end of the audio interaction in a case where it is determined according to the comparison result that the speaking voice is from the target object, and performing voice changing processing on the audio signal to obtain a changed voice signal and sending the changed voice signal to the opposite end of the audio interaction in a case where it is determined according to the comparison result that the speaking voice is not from the target object. In a case where the detection result indicates that the original audio signal does not contain speaking voice, the current real-time collected audio signal can be directly transmitted to the opposite end of the audio interaction, and the steps of performing audio signal collection, performing activity detection on the real-time collected audio signal to obtain a detection result are continuously performed.

[0080] The voice activity detection can be used to detect whether the current real-time collected audio signal contains speaker voice, that is, whether it contains human voice. If it contains human voice, the subsequent steps for implementing voice protection are performed, and if it does not contain human voice, the subsequent steps for implementing voice protection are not required to be performed, and the current real-time collected audio signal is directly transmitted to the opposite end of the audio interaction, and the audio signal collection and the activity detection on the real-time collected audio signal are continuously performed.

[0081] In this embodiment, by performing speech activity detection on the real-time collected audio signal, subsequent processing steps for implementing sound protection on audio that does not contain human voice can be avoided, resource waste can be avoided, and audio transmission efficiency can be improved.

[0082] In one embodiment, the real-time collected audio signal is subjected to activity detection to obtain a detection result, including: dividing the real-time collected audio signal into a plurality of audio frames; converting each audio frame into a frequency domain signal; calculating the speech sound energy corresponding to the speech spectrum in the audio frame according to the frequency domain signal; calculating the total energy of the audio frame according to the frequency domain signal; when the proportion of the speech sound energy to the total energy is greater than or equal to a set threshold, obtaining a detection result that the audio signal contains speech sound; and when the proportion of the speech sound energy to the total energy is less than the set threshold, obtaining a detection result that the audio signal does not contain speech sound.

[0083] Specifically, the input of speech activity detection is an audio frame, which is generally a data frame of 20 ms or 10 ms. By analyzing the audio frame, such as converting the audio frame into a frequency domain signal, calculating the proportion of the energy of the part below 2 kHz (the speech spectrum corresponding to the speech sound) to the total energy of the audio frame, and when it is greater than or equal to a certain set threshold, it is considered that the audio frame contains speech sound, otherwise it is considered that the audio frame does not contain speech sound. Alternatively, when the audio frame does not contain human voice for more than a certain time threshold, it is determined that the human voice disappears. As shown in FIG. 1, it is a flowchart of speech activity detection in one embodiment. Referring to FIG. 1, the input is a piece of audio frame, and the energy of the audio frame is calculated to obtain a detection result of whether it contains human voice. Figure 6 Figure 6 As shown in FIG. 1, it is a flowchart of speech activity detection in one embodiment. Referring to FIG. 1, the input is a piece of audio frame, and the energy of the audio frame is calculated to obtain a detection result of whether it contains human voice.

[0084] In this embodiment, by dividing the audio signal into audio frames and converting it into a frequency domain signal, whether the current real-time collected audio signal contains speaker sound can be effectively, quickly and accurately detected.

[0085] ​In one embodiment, the voiceprint vector corresponding to the real-time collected audio signal is calculated, the calculated voiceprint vector is compared with a target voiceprint vector corresponding to a target object identifier, and a comparison result is obtained, including: obtaining a universal background model trained based on speech samples from different speakers, adjusting model parameters of the universal background model by using the real-time collected audio signal through an adaptive algorithm, obtaining a Gaussian mixture model corresponding to the real-time collected audio signal, the Gaussian mixture model being obtained by linearly combining a plurality of Gaussian distribution functions according to model parameters corresponding to each Gaussian distribution function; obtaining the voiceprint vector corresponding to the real-time collected audio signal according to the model parameters corresponding to each Gaussian distribution function; obtaining the target voiceprint vector corresponding to the target object identifier, and comparing the voiceprint vector corresponding to the real-time collected audio signal with the target voiceprint vector corresponding to the target object identifier to obtain the comparison result.

[0086] Due to the differences in oral cavities and the like of different people, the same sound will also have differences in sound, that is, different timbres, which is reflected in the frequency domain as different frequency distributions corresponding to the same sound of different people, and the frequency distribution can be regarded as a continuous probability distribution function. The audio of the target object is modeled by using the universal background model to obtain a Gaussian mixture model, which is a continuous probability distribution function described by a plurality of Gaussian functions, and the parameters of the probability distribution functions corresponding to different objects are also different. Based on the universal background model, the Gaussian mixture model corresponding to the target object is obtained, and the model parameters (a plurality of parameters used for combining the plurality of Gaussian functions) of the Gaussian mixture model can be represented as a voiceprint vector.

[0087] The universal background model (UBM, Universal background model) is a large speech feature distribution model used to represent speaker-independent feature distribution, and the training data thereof comes from background data collected from speech samples of different speakers. In this embodiment, a universal background model is trained by first collecting a large number of speech samples irrelevant to the target object, and then adjusting the model parameters of the universal background model through an adaptive algorithm using a small amount of speech of the target object to obtain a Gaussian mixture model (GMM, Gaussian Mixture Model), that is, a speaker model of the target object, the plurality of Gaussian distribution functions being linearly combined according to the model parameters corresponding to each Gaussian distribution function, the parameters corresponding to different objects being different, and the parameters of all the probability distribution functions being represented as a vector to obtain a voiceprint vector of the object. After obtaining the voiceprint vector corresponding to the real-time collected audio signal, the target voiceprint vector corresponding to the target object identifier is obtained, and the voiceprint vector corresponding to the real-time collected audio signal is compared with the target voiceprint vector corresponding to the target object identifier to obtain a comparison result, and the comparison result can be used to accurately determine whether the real-time collected audio signal is from the target object.

[0088] In an embodiment, the target voiceprint vector corresponding to the target object identifier is obtained by using the sound source signal of the target object, and the method comprises the following steps.

[0089] The model parameters of the general background model are adjusted by using the sound source signal of the target object through an adaptive algorithm to obtain a Gaussian mixture model corresponding to the sound source signal of the target object, and the Gaussian mixture model is obtained by linearly combining a plurality of Gaussian distribution functions according to the model parameters corresponding to each Gaussian distribution function; and the target voiceprint vector corresponding to the target object identifier is obtained according to the model parameters corresponding to each Gaussian distribution function.

[0090] Similarly, the voiceprint vector of the target object is also obtained from the Gaussian mixture model of the target object, and the Gaussian mixture model is obtained based on the general background model and the audio signal originating from the target object. The computer device can determine the target voiceprint vector corresponding to the target object identifier according to the model parameters of the obtained Gaussian mixture model.

[0091] In an embodiment, the method further comprises the following steps: after logging in to the audio interaction client with the target object identifier, prompting to perform voiceprint extraction; in response to a voiceprint extraction confirmation operation, collecting the sound source signal of the target object; and after obtaining the target voiceprint vector corresponding to the target object identifier by using the sound source signal of the target object, saving the target voiceprint vector corresponding to the target object identifier in the audio interaction client.

[0092] In this embodiment, after logging in to the audio interaction client with the target object identifier, the voiceprint extraction is prompted. For example, after logging in to the audio interaction client with the target object identifier, the audio interaction client can immediately prompt the target object to perform voiceprint extraction to protect the sound of other objects except the target object. For another example, after logging in to the audio interaction client with the target object identifier, when the target object creates a new online conference through the audio interaction client, the target object can be prompted to perform voiceprint extraction. Alternatively, before or after the audio interaction is started, the audio interaction client can support starting or stopping the protection of the sound of other objects except the target object. The computer device can collect the sound source signal of the target object in response to a voiceprint extraction confirmation operation triggered by the target object, and obtain the target voiceprint vector corresponding to the target object identifier by using the sound source signal of the target object in the above manner. After obtaining the target voiceprint vector corresponding to the target object identifier, the audio interaction client saves the target voiceprint vector corresponding to the target object identifier, so as to facilitate subsequent voiceprint comparison and improve the comparison efficiency.

[0093] In an embodiment, the audio signal is subjected to voice transformation processing to obtain a transformed audio signal, and the method comprises the following steps.

[0094] Optionally, the pitch conversion processing on the audio signal comprises: converting the audio signal into a frequency domain signal; sampling the frequency domain signal by frequency to obtain a sampling frequency and a corresponding sampling amplitude; and converting the frequency domain signal with the sampling frequency and the corresponding sampling amplitude into a time domain signal, and taking the obtained time domain signal as the audio signal after the pitch conversion processing.

[0095] Since different people sing the same sound, the difference is mainly reflected in the frequency distribution, especially the difference in the fundamental frequency, so resampling the frequency of the audio signal can realize the pitch conversion of the sound. By performing linear interpolation on the audio signal, the frequency of the speech signal can be scaled to realize the pitch conversion of the sound. The sampling frequency here is the frequency that is sampled. According to the obtained sampling frequency and the corresponding sampling amplitude, the computer device can generate a time domain signal as the audio signal after the pitch conversion processing, which can ensure that the pitch of the sound changes greatly and protect the voice information of the user. The sampling of the frequency domain signal by frequency can be upsampling or downsampling.

[0096] Optionally, before converting the audio signal into a frequency domain signal, the method further comprises: performing pitch detection on the audio signal to determine a pitch period of the audio signal; determining a pitch period range to which the pitch period of the audio signal belongs, and obtaining a sampling rate corresponding to the pitch period range to which the pitch period of the audio signal belongs according to a pre-set correspondence between the pitch period range and the sampling rate; and sampling the frequency domain signal by frequency to obtain a sampling frequency and a corresponding sampling amplitude, comprising: sampling the frequency domain signal by frequency using the sampling rate to obtain the sampling frequency and the corresponding sampling amplitude.

[0097] In speech signal processing, the pitch period is an important parameter, which describes the periodicity of the basic frequency of vocal cord vibration, i.e., the fundamental frequency. For example, assuming that the upsampling factor is P and the downsampling factor is Q, the upsampling process is to interpolate P-1 sampling points in the original frequency domain signal, which makes the pitch period P times of the original, the spectrum compression 1 / P times of the original, the time length P times of the original, the pitch 1 / P times of the original, and the speech speed 1 / P times of the original. The downsampling process is to extract every Q-1 points, which makes the pitch period 1 / Q times of the original, the spectrum expansion Q times of the original, the time length 1 / Q times of the original, the pitch Q times of the original, and the speech speed Q times of the original. In order to avoid the unnaturalness of the speech caused by too large differences in the pitch and the speech speed after the pitch conversion, in this embodiment, the computer device can determine the pitch period range to which the pitch period of the audio signal belongs, and determine the corresponding sampling rate according to the pre-set correspondence between the pitch period range and the sampling rate, and sample the frequency domain signal by frequency using the determined sampling rate to obtain the sampling frequency and the corresponding sampling amplitude.

[0098] Optionally, the step of performing the variable speed processing on the pitch-altered audio signal comprises: dividing the pitch-altered audio signal into different audio frames, each audio frame having a preset time length; stretching each audio frame in the time dimension, and then synthesizing the stretched audio frames in the time sequence to obtain the variable audio signal.

[0099] Since the time length of the pitch-altered audio signal has also changed, the time length needs to be recovered. Specifically, the computer device can compress or expand the pitch-altered audio signal in the time domain. Specifically, if the time length of the pitch-altered audio signal is compressed, the computer device can divide the audio signal into different frames, stretch each frame, and then superimpose the frames to obtain the final variable audio signal. If the time length of the pitch-altered audio signal is expanded, the computer device can divide the audio signal into different frames, compress each frame, and then superimpose the frames to obtain the final variable audio signal.

[0100] In one embodiment, the method for performing the variable voice processing on the audio signal to obtain the variable audio signal comprises: inputting the audio signal into a variable voice speech model to extract features to obtain speech recognition features of the audio signal; inputting a sound source signal of a preset object into the variable voice speech model to extract features to obtain speech recognition features of the sound source signal; and mapping the audio signal to the sound of the preset object by the variable voice speech model according to the speech recognition features of the sound source signal and the speech recognition features of the audio signal to obtain the variable audio signal.

[0101] In this embodiment, the computer device can perform voice conversion processing on the audio signal through the trained voice conversion model. The voice conversion model is an artificial intelligence model that has the ability to convert a segment of speech to a specified timbre after being trained on samples. The computer device can input the obtained audio signal into the voice conversion model for feature extraction to obtain speech recognition features of the audio signal, which can capture the linguistic information of the audio signal. The sound source signal of the preset object is input into the voice conversion model for feature extraction to obtain speech recognition features of the sound source signal, which captures the timbre information of the sound source signal. The computer device maps the audio signal to the sound of the preset object according to the speech recognition features of the sound source signal and the speech recognition features of the audio signal through the voice conversion model, and obtains a converted audio signal. The sound source signal of the preset object can be a synthesized sound signal of a virtual object, or the current target object, or any other default preset sound signal. That is, through the voice conversion model, the computer device can convert the sound emitted by other personnel except the target object to the sound of the specified object, thereby realizing sound protection for these other personnel. Of course, the specified object can be the target object. This model-based voice conversion method has good effects in timbre similarity and naturalness, and can ensure natural and harmonious converted audio signals.

[0102] In one embodiment, in the case where it is determined according to the comparison result that the speaking voice does not come from the target object, the audio signal is subjected to voice conversion processing to obtain a converted audio signal, and the converted audio signal is sent to the opposite end of the audio interaction, including: in the case where it is determined according to the comparison result that the speaking voice does not come from the target object, the audio signal is subjected to voice conversion processing to obtain a converted audio signal; adding a sound source identifier indicating that the speaking voice does not come from the target object in the converted audio signal, and sending the converted audio signal with the added sound source identifier to the opposite end of the audio interaction to prompt that the current speaking voice does not come from the target object.

[0103] For example, in one application scenario, the opposite end of the audio interaction can prompt the opposite end user that the current speaking voice does not come from the target object when extracting the sound source identifier indicating that the speaking voice does not come from the target object in the received converted audio signal, such as prompting “XX is not currently speaking”, where “XX” represents the target object that does not need to be protected. In this way, although the opposite end user cannot identify through the voice which other object except the target object is currently speaking at the current end, he or she can know that the current target object is not speaking and will not feel unusual, which can improve the audio interaction experience of the opposite end user.

[0104] For example, in one application scenario, the opposite end of the audio interaction can prompt the opposite end user that the current speaking voice does not come from the target object when extracting the sound source identifier indicating that the speaking voice does not come from the target object in the received converted audio signal, such as prompting “XX is not currently speaking”, where “XX” represents the target object that does not need to be protected. In this way, although the opposite end user cannot identify through the voice which other object except the target object is currently speaking at the current end, he or she can know that the current target object is not speaking and will not feel unusual, which can improve the audio interaction experience of the opposite end user. Figure 7The diagram shown illustrates a module for sound protection in one embodiment, comprising four modules: voiceprint registration, voice activity detection, voiceprint comparison, and sound processing. Voiceprint registration records the voiceprint vector of the target object; voice activity detection detects whether the audio signal collected in real-time during audio interaction contains human voice; voiceprint comparison extracts the voiceprint vector when the audio signal contains human voice and compares it with the target voiceprint vector corresponding to the target object identifier; and sound processing modifies the audio signal when it contains the voice of an object other than the target object.

[0105] For example, in a multi-person conference, the host's voiceprint vector is first recorded through a voiceprint registration module. After the host records their audio segment in the voiceprint registration module, the module generates a corresponding voiceprint vector. When the collected audio signal passes through a voice activity detection system and is found to contain no voice signal, it is considered not to involve privacy and is not processed for voiceprint recognition, being directly sent to the encoding module. When a voice signal is present, voiceprint recognition is performed. The calculated parameter vector is compared with the host's voiceprint vector stored in the voiceprint registration module. If the two are close, it is considered the host's voice; otherwise, it is considered another user's voice. The voiceprint recognition module then includes this information in the voice data frame and sends it to the sound processing module. The sound processing module processes the sound according to the identifier in the voice frame. If the voice frame does not contain a valid human voice, no processing is performed, and the voice is sent to the encoding module. When a voice frame is present, it is checked whether it is the broadcaster's voice. If it is the broadcaster's voice, no processing is performed, and the voice is sent to the encoding module. If it is not the broadcaster's voice, then voice-changing processing is required.

[0106] like Figure 8 The diagram shown illustrates the detailed steps of sound protection in a specific embodiment. (Refer to...) Figure 8 The system collects audio signals and performs voice activity detection on the real-time audio signals. If the detection result indicates the presence of human voice, voiceprint recognition is performed; otherwise, if the detection result indicates the absence of human voice, no voice protection processing is required. If the voiceprint recognition result indicates the presence of other people's voices, voice changing processing is performed; otherwise, no voice changing processing is required. This method protects the identity information of other users around the target without requiring manual user intervention, without affecting normal audio interaction, and is highly accurate with a low risk of misjudgment.

[0107] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0108] Based on the same inventive concept, the embodiments of the present application also provide an audio processing device for implementing the above-mentioned audio processing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more audio processing device embodiments provided below can refer to the limitations of the audio processing method in the above text, which will not be repeated here.

[0109] In one exemplary embodiment, as shown in Figure 9 An audio processing device 900 is provided, comprising an audio interaction initiation module 902, a comparison module 904 and a sending module 906, wherein:

[0110] The audio interaction initiation module 902 is configured to start audio signal collection after initiating the audio interaction through the target object identifier;

[0111] The comparison module 904 is configured to calculate a voiceprint vector corresponding to the real-time collected audio signal, compare the calculated voiceprint vector with a target voiceprint vector corresponding to the target object identifier, and obtain a comparison result;

[0112] The sending module 906 is configured to send the audio signal to the opposite end of the audio interaction in a case where it is determined according to the comparison result that the speaking voice originates from the target object;

[0113] The sending module 906 is configured to perform voice transformation processing on the audio signal to obtain a transformed audio signal and send the transformed audio signal to the opposite end of the audio interaction in a case where it is determined according to the comparison result that the speaking voice does not originate from the target object.

[0114] In an embodiment, the comparison module 904 is further configured to perform activity detection on the real-time collected audio signal to obtain a detection result; in a case where the detection result indicates that the audio signal contains speech sound, perform the steps of calculating a voiceprint vector corresponding to the real-time collected audio signal, and comparing the calculated voiceprint vector with a target voiceprint vector corresponding to the target object identifier; in a case where the detection result indicates that the original audio signal does not contain speech sound, continue to collect the audio signal, and perform activity detection on the real-time collected audio signal.

[0115] In an embodiment, the comparison module 904 is further configured to divide the real-time collected audio signal into a plurality of audio frames; convert each audio frame into a frequency domain signal; calculate a speech sound energy corresponding to speech spectrum in the audio frame according to the frequency domain signal; calculate a total energy of the audio frame according to the frequency domain signal; obtain a detection result that the audio signal contains speech sound when a proportion of the speech sound energy to the total energy is greater than or equal to a set threshold; and obtain a detection result that the audio signal does not contain speech sound when the proportion of the speech sound energy to the total energy is less than the set threshold.

[0116] In an embodiment, the comparison module 904 is further configured to obtain a general background model trained based on speech samples from different speakers, adjust model parameters of the general background model using the real-time collected audio signal through an adaptive algorithm to obtain a Gaussian mixture model corresponding to the real-time collected audio signal, the Gaussian mixture model being obtained by linearly combining a plurality of Gaussian distribution functions according to model parameters corresponding to each Gaussian distribution function; obtain a voiceprint vector corresponding to the real-time collected audio signal according to the model parameters corresponding to each Gaussian distribution function; obtain a target voiceprint vector corresponding to the target object identifier, and compare the voiceprint vector corresponding to the real-time collected audio signal with the target voiceprint vector corresponding to the target object identifier to obtain a comparison result.

[0117] In an embodiment, the comparison module 904 is further configured to adjust model parameters of the general background model using the target object's sound source signal through an adaptive algorithm to obtain a Gaussian mixture model corresponding to the target object's sound source signal, the Gaussian mixture model being obtained by linearly combining a plurality of Gaussian distribution functions according to model parameters corresponding to each Gaussian distribution function; and obtain a target voiceprint vector corresponding to the target object identifier according to the model parameters corresponding to each Gaussian distribution function.

[0118] In an embodiment, the comparison module 904 is further configured to, after logging in to the audio interaction client with the target object identifier, prompt for voiceprint extraction; in response to a voiceprint extraction confirmation operation, collect a sound source signal of the target object; and after obtaining a target voiceprint vector corresponding to the target object identifier using the sound source signal of the target object, save the target voiceprint vector corresponding to the target object identifier in the audio interaction client.

[0119] In an embodiment, the audio processing apparatus 900 further comprises:

[0120] a voice changing module configured to sequentially perform pitch changing processing and speed changing processing on the audio signal to obtain a changed audio signal.

[0121] In an embodiment, the voice changing module is further configured to convert the audio signal into a frequency domain signal, sample the frequency domain signal according to frequency to obtain a sampling frequency and a corresponding sampling amplitude, and convert the frequency domain signal with the sampling frequency and the corresponding sampling amplitude into a time domain signal, and take the obtained time domain signal as the audio signal after pitch changing processing.

[0122] In an embodiment, the voice changing module is further configured to, before converting the audio signal into a frequency domain signal, perform pitch detection on the audio signal to determine a pitch period of the audio signal, determine a pitch period range to which the pitch period of the audio signal belongs, and according to a pre-set correspondence between pitch period ranges and sampling rates, obtain a sampling rate corresponding to the pitch period range to which the pitch period of the audio signal belongs; and sample the frequency domain signal according to frequency using the sampling rate to obtain a sampling frequency and a corresponding sampling amplitude.

[0123] In an embodiment, the voice changing module is further configured to divide the audio signal after pitch changing processing into different audio frames, each audio frame having a preset time length, stretch each audio frame in the time dimension, and then synthesize the stretched audio frames in chronological order to obtain a changed audio signal.

[0124] In an embodiment, the voice changing module is further configured to input the audio signal into a voice changing speech model to extract features of the audio signal, input a sound source signal of a pre-set object into the voice changing speech model to extract features of the sound source signal, and map the audio signal to the sound of the pre-set object according to the features of the sound source signal and the features of the audio signal through the voice changing speech model to obtain a changed audio signal.

[0125] In an embodiment, the sending module 906 is further configured to, in a case where it is determined according to the comparison result that the speaking voice does not come from the target object, perform voice changing processing on the audio signal to obtain a changed voice audio signal, add a sound source identifier indicating that the speaking voice does not come from the target object in the changed voice audio signal, and send the changed voice audio signal with the sound source identifier to the opposite end of the audio interaction to prompt that the current speaking voice does not come from the target object.

[0126] The audio processing apparatus 900 described above, after initiating the audio interaction by the target object identifier, starts audio signal collection, after starting the audio signal collection, calculates a voiceprint vector corresponding to the real-time collected audio signal, compares the calculated voiceprint vector with a target voiceprint vector corresponding to the target object identifier to obtain a comparison result, in a case where it is determined according to the comparison result that the speaking voice comes from the target object, sends the audio signal to the opposite end of the audio interaction, and in a case where it is determined according to the comparison result that the speaking voice does not come from the target object, performs voice changing processing on the audio signal to obtain a changed voice audio signal, and sends the changed voice audio signal to the opposite end of the audio interaction. In this way, in a scenario where multiple people share the audio interaction initiated by the target object identifier, the voice information of the other members except the target object can be avoided to be directly exposed to the opposite end of the audio interaction in the audio interaction process, and the audio interaction initiated by the target object identifier can also not be affected, the voice information of the other people can be protected in the audio interaction process, and the personal information of the user can be protected.

[0127] Each module in the audio processing apparatus 900 described above can be realized by software, hardware, and a combination thereof in whole or in part. Each module described above can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form to be called and executed by a processor to perform the operations corresponding to each module.

[0128] In an exemplary embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in FIG. 8. Figure 10As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an audio processing method.

[0129] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an audio processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0130] Those skilled in the art will understand that Figure 10 , Figure 11The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0131] In an exemplary embodiment, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the audio processing method provided by the embodiments of the present application when executing the computer program, for example:

[0132] After initiating the audio interaction through the target object identification, the audio signal collection is started;

[0133] The voiceprint vector corresponding to the real-time collected audio signal is calculated, and the calculated voiceprint vector is compared with the target voiceprint vector corresponding to the target object identification to obtain a comparison result;

[0134] In a case where it is determined according to the comparison result that the speaking voice originates from the target object, the audio signal is sent to the opposite end of the audio interaction;

[0135] In a case where it is determined according to the comparison result that the speaking voice does not originate from the target object, the audio signal is processed to obtain a changed audio signal, and the changed audio signal is sent to the opposite end of the audio interaction.

[0136] In an embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps of the audio processing method provided by the embodiments of the present application.

[0137] In an embodiment, a computer program product is provided, which includes a computer program, and the computer program is executed by a processor to implement the steps of the audio processing method provided by the embodiments of the present application.

[0138] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0139] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0140] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0141] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. An audio processing method, characterized by, The method comprises: After initiating the audio interaction through the target object identification, starting audio signal collection; Calculating the voiceprint vector corresponding to the real-time collected audio signal, comparing the calculated voiceprint vector with the target voiceprint vector corresponding to the target object identification to obtain a comparison result; In the case where it is determined according to the comparison result that the speaking voice originates from the target object, the audio signal is sent to the opposite end of the audio interaction; In the case where it is determined according to the comparison result that the speaking voice does not originate from the target object, the audio signal is subjected to voice changing processing to obtain a changed audio signal, and the changed audio signal is sent to the opposite end of the audio interaction.

2. The method of claim 1, wherein, Before the calculating the voiceprint vector corresponding to the real-time collected audio signal, the method further comprises: Performing activity detection on the real-time collected audio signal to obtain a detection result; In the case where the detection result indicates that the audio signal contains speaking voice, the calculating the voiceprint vector corresponding to the real-time collected audio signal and the comparing the calculated voiceprint vector with the target voiceprint vector corresponding to the target object identification are performed; In the case where the detection result indicates that the original audio signal does not contain speaking voice, the audio signal collection is continued, and the real-time collected audio signal is subjected to activity detection.

3. The method of claim 2, wherein, The performing activity detection on the real-time collected audio signal to obtain a detection result comprises: Dividing the real-time collected audio signal into multiple audio frames; Converting each audio frame into a frequency domain signal; Calculating the speaking voice energy corresponding to the speaking spectrum in the audio frame according to the frequency domain signal; Calculating the total energy of the audio frame according to the frequency domain signal; When the proportion of the speaking voice energy to the total energy is greater than or equal to a set threshold, a detection result that the audio signal contains speaking voice is obtained; When the proportion of the speaking voice energy to the total energy is less than the set threshold, a detection result that the audio signal does not contain speaking voice is obtained.

4. The method of claim 1, wherein, The calculating the voiceprint vector corresponding to the real-time collected audio signal, the comparing the calculated voiceprint vector with the target voiceprint vector corresponding to the target object identification to obtain a comparison result comprises: Obtaining a general background model trained based on voice samples from different speakers, adjusting model parameters of the general background model through an adaptive algorithm using the real-time collected audio signal to obtain a Gaussian mixture model corresponding to the real-time collected audio signal, the Gaussian mixture model being obtained by linearly combining multiple Gaussian distribution functions according to model parameters corresponding to each Gaussian distribution function; Obtaining a voiceprint vector corresponding to the real-time collected audio signal according to the model parameters corresponding to each Gaussian distribution function; Obtaining a target voiceprint vector corresponding to the target object identification, and comparing the voiceprint vector corresponding to the real-time collected audio signal with the target voiceprint vector corresponding to the target object identification to obtain a comparison result.

5. The method of claim 4, wherein, The obtaining a target voiceprint vector corresponding to the target object identification comprises: The model parameters of the general background model are adjusted by using the sound source signal of the target object through an adaptive algorithm to obtain a Gaussian mixture model corresponding to the sound source signal of the target object, the Gaussian mixture model being obtained by linearly combining a plurality of Gaussian distribution functions according to the model parameters corresponding to each Gaussian distribution function; According to the model parameters corresponding to each Gaussian distribution function, a target voiceprint vector corresponding to the target object identifier is obtained.

6. The method of claim 5, wherein, The method further comprises: After logging in to the audio interaction client with the target object identifier, prompting to perform voiceprint extraction; In response to a voiceprint extraction confirmation operation, collecting the sound source signal of the target object; After obtaining the target voiceprint vector corresponding to the target object identifier using the sound source signal of the target object, saving the target voiceprint vector corresponding to the target object identifier in the audio interaction client.

7. The method of claim 1, wherein, The method further comprises: The audio signal is sequentially subjected to pitch conversion processing and speed conversion processing to obtain a variable sound audio signal.

8. The method of claim 7, wherein, The audio signal is subjected to pitch conversion processing, comprising: Converting the audio signal into a frequency domain signal; Sampling the frequency domain signal according to frequency to obtain a sampling frequency and a corresponding sampling amplitude; Converting the frequency domain signal with the sampling frequency and the corresponding sampling amplitude of the sampling frequency into a time domain signal, and taking the obtained time domain signal as the audio signal after pitch conversion processing.

9. The method of claim 8, wherein, Before converting the audio signal into a frequency domain signal, the method further comprises: Performing pitch detection on the audio signal to determine the pitch period of the audio signal; Determining the pitch period range to which the pitch period of the audio signal belongs, and obtaining the sampling rate corresponding to the pitch period range to which the pitch period of the audio signal belongs according to the pre-set correspondence between the pitch period range and the sampling rate; The frequency domain signal is sampled according to frequency to obtain a sampling frequency and a corresponding sampling amplitude, comprising: the frequency domain signal is sampled according to frequency using the sampling rate to obtain a sampling frequency and a corresponding sampling amplitude.

10. The method of claim 8, wherein, The step of performing speed conversion processing on the audio signal after pitch conversion processing comprises: Dividing the audio signal after pitch conversion processing into different audio frames, and the time length of each audio frame is a pre-set time length; After stretching each audio frame in the time dimension, the stretched audio frames are synthesized according to the time sequence to obtain a variable sound audio signal.

11. The method of claim 1, wherein, The method further comprises: Inputting the audio signal into a variable sound voice model to extract features to obtain speech recognition features of the audio signal; Inputting the sound source signal of the pre-set object into the variable sound voice model to extract features to obtain speech recognition features of the sound source signal; Through the variable sound voice model, the audio signal is mapped to the sound of the pre-set object according to the speech recognition features of the sound source signal and the speech recognition features of the audio signal to obtain a variable sound audio signal.

12. The method according to any one of claims 1 to 11, characterized in that, In a case where it is determined according to the comparison result that the speaking voice does not come from the target object, the audio signal is processed to change the voice to obtain a changed voice audio signal, and the changed voice audio signal is sent to a peer end of the audio interaction, including: In a case where it is determined according to the comparison result that the speaking voice does not come from the target object, the audio signal is processed to change the voice to obtain a changed voice audio signal; The changed voice audio signal is added with a sound source identifier indicating that the speaking voice does not come from the target object, and the changed voice audio signal added with the sound source identifier is sent to the peer end of the audio interaction to prompt that the current speaking voice does not come from the target object.

13. An audio processing apparatus, characterized by: The apparatus includes: An audio interaction initiation module configured to initiate an audio interaction through a target object identifier and start audio signal collection; A comparison module configured to calculate a voiceprint vector corresponding to the audio signal collected in real time, compare the calculated voiceprint vector with a target voiceprint vector corresponding to the target object identifier, and obtain a comparison result; A sending module configured to, in a case where it is determined according to the comparison result that the speaking voice comes from the target object, send the audio signal to a peer end of the audio interaction; The sending module is configured to, in a case where it is determined according to the comparison result that the speaking voice does not come from the target object, process the audio signal to change the voice to obtain a changed voice audio signal, and send the changed voice audio signal to the peer end of the audio interaction.

14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 12.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 12.

16. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 12.

Images