Method and apparatus for training privacy-preserving intent understanding model

Abstract

This specification discloses a method for training a privacy-preserving intent understanding model, wherein the intent understanding model includes a speech encoder and a first decoder. Each iteration of training in this method includes: for any speech sample in the current batch of speech samples, processing the sample using the speech encoder and a decoder from among multiple decoders corresponding to the task type to which the sample's label belongs, to obtain a speech prediction result; the multiple decoders include a first decoder and several second decoders for performing several privacy tasks; performing a first training on the speech encoder and multiple decoders with the goal of reducing the gap between each speech prediction result and its corresponding speech sample label; and then performing a second training on the intent understanding model after the first training, with the goal of increasing the gap between the speech prediction results corresponding to various privacy tasks and the speech sample labels, and reducing the gap between the speech prediction results output by the first decoder and the speech sample labels.

Description

Technical Field

[0001] This specification relates to the field of machine learning technology, and in particular to a privacy-preserving method and apparatus for training an intent understanding model, a privacy-preserving method and apparatus for training an intent understanding model, a privacy-preserving method and apparatus for training a business prediction model, and a privacy-preserving method and apparatus for business prediction. Background Technology

[0002] Spoken Language Understanding (SLU) is a technology that analyzes the user's intent expressed through a speaker's speech. It is widely used in various scenarios, such as in-vehicle voice control and smart homes. SLU is typically implemented by using machine learning techniques to build an SLU model.

[0003] However, in the current process of building and using SLU models, there is a risk that intermediate data generated by the model may be stolen, leading to the leakage of user privacy. Typically, to protect user privacy, data availability must be sacrificed to a significant extent.

[0004] Therefore, a solution is needed that can enhance the protection of user privacy while ensuring or even improving the accuracy of SLU prediction results. Summary of the Invention

[0005] This specification describes a method and apparatus for training a privacy-preserving intent understanding model, which can better meet the needs of practical applications.

[0006] According to a first aspect, a method for training a privacy-preserving intent understanding model is provided, the intent understanding model including a speech encoder and a first decoder; the method involves multiple rounds of iterative training, wherein any round of iterative training includes:

[0007] For any speech sample in the current batch of speech samples, the speech encoder and multiple decoders are used to process the sample using the decoder corresponding to the task type to which the sample's label belongs, to obtain a speech prediction result; wherein the multiple decoders include the first decoder and multiple second decoders for performing several privacy tasks. The speech encoder and multiple decoders are first trained with the goal of reducing the gap between each speech prediction result and its corresponding speech sample label. The intent understanding model trained in the first training is then second trained with the goal of increasing the gap between the speech prediction results corresponding to various privacy tasks and the speech sample labels, and reducing the gap between the speech prediction results output by the first decoder and the speech sample labels.

[0008] In one embodiment, the plurality of privacy tasks includes a speech recognition task and / or a speaker recognition task.

[0009] In one embodiment, the plurality of privacy tasks includes a speech recognition task; wherein, before performing the multi-round iterative training, the method further includes: pre-training the speech encoder and the second decoder for performing the speech recognition task using a training sample set corresponding to the speech recognition task.

[0010] In one embodiment, the plurality of privacy tasks includes a speech recognition task; wherein, for any speech sample in the current batch of speech samples, the sample is processed using the speech encoder and a decoder from among the multiple decoders that corresponds to the label task type of the sample, to obtain a speech prediction result, including:

[0011] For any given speech sample, its corresponding original audio feature vector is segmented and shuffled along the time dimension to obtain a disordered audio feature vector; the disordered audio feature vector is processed by the speech encoder to obtain a hidden layer representation; the original audio feature vector is segmented and shuffled along the time dimension to obtain a disordered audio feature vector; the disordered audio feature vector is processed by the speech encoder to obtain a hidden layer representation; the hidden layer representation is input into the decoder corresponding to the label task type of the sample to obtain the speech prediction result.

[0012] In one embodiment, with the goal of increasing the gap between the speech prediction results and speech sample labels corresponding to various privacy tasks, and decreasing the gap between the speech prediction results and speech sample labels output by the first decoder, a second training is performed on the intent understanding model trained in the first training stage, including:

[0013] Determine the adversarial training loss; the adversarial training loss is positively correlated with the first loss and negatively correlated with the several second losses corresponding to the several privacy tasks; the first loss is determined based on the speech prediction result and speech sample label output by the first decoder, and each second loss is determined based on the speech prediction result and speech sample label corresponding to its privacy task.

[0014] In one embodiment, during the second training of the intent understanding model that has been trained in the first training, the model parameters of the plurality of second decoders remain unchanged.

[0015] According to the second aspect, a privacy-preserving intent understanding method is provided, comprising: acquiring a speech sample to be processed; inputting the speech sample into an intent understanding model trained using the method provided in the first aspect, and obtaining a corresponding intent prediction result.

[0016] According to a third aspect, a method for training a privacy-preserving business prediction model is provided, the business prediction model including an object encoder for business objects and a first decoder for performing a main task; the method involves multiple rounds of iterative training, wherein any round of iterative training includes:

[0017] For any object sample in the current batch of object samples, the sample is processed using the object encoder and a decoder from among the multiple decoders that corresponds to the task type to which the sample's label belongs, to obtain an object prediction result; wherein the multiple decoders include the first decoder and several second decoders for performing several privacy tasks. The object encoder and multiple decoders are first trained with the goal of reducing the gap between each object prediction result and its corresponding object sample label. The business prediction model trained in the first training is then second trained with the goal of increasing the gap between the object prediction results corresponding to various privacy tasks and the object sample labels, and reducing the gap between the object prediction results output by the first decoder and the object sample labels.

[0018] In one embodiment, the business object is voice, image, or text.

[0019] In one embodiment, during the second training of the business prediction model that has been trained in the first training, the model parameters of the plurality of second decoders remain unchanged.

[0020] According to the fourth aspect, a privacy-preserving business prediction method is provided. The method includes: acquiring a sample of objects to be processed; inputting the sample of objects into a business prediction model trained using the method of the third aspect, to obtain the corresponding business prediction result.

[0021] According to the fifth aspect, a training apparatus for a privacy-preserving intent understanding model is provided, the intent understanding model including a speech encoder and a first decoder; the apparatus implements iterative training in any round of multi-round iterative training through the following units:

[0022] A speech prediction unit is configured to process any speech sample in the current batch of speech samples using a speech encoder and a decoder from multiple decoders that corresponds to the task type of the sample's label, to obtain a speech prediction result; wherein the multiple decoders include a first decoder and multiple second decoders for performing several privacy tasks. A first training unit is configured to perform a first training on the speech encoder and multiple decoders with the goal of reducing the gap between each speech prediction result and its corresponding speech sample label. A second training unit is configured to perform a second training on the intent understanding model that has undergone the first training, with the goal of increasing the gap between the speech prediction results corresponding to various privacy tasks and the speech sample labels, and reducing the gap between the speech prediction results output by the first decoder and the speech sample labels.

[0023] According to the sixth aspect, a training apparatus for a privacy-preserving business prediction model is provided, the business prediction model including an object encoder for business objects and a first decoder for performing a main task; the apparatus implements iterative training in any round of multi-round iterative training through the following units:

[0024] An object prediction unit is configured to process any object sample in the current batch of object samples using an object encoder and a decoder from multiple decoders that corresponds to the task type of the sample's label, to obtain an object prediction result; wherein the multiple decoders include a first decoder and multiple second decoders for performing several privacy tasks. A first training unit is configured to perform a first training on the object encoder and multiple decoders with the goal of reducing the gap between each object prediction result and its corresponding object sample label. A second training unit is configured to perform a second training on the business prediction model trained in the first training, with the goal of increasing the gap between the object prediction results corresponding to various privacy tasks and the object sample labels, and reducing the gap between the object prediction results output by the first decoder and the object sample labels.

[0025] According to a seventh aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method provided in any one of the first to fourth aspects.

[0026] According to an eighth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method provided by any one of the first to fourth aspects.

[0027] The training method and apparatus for the privacy-preserving intent understanding model disclosed in the embodiments of this specification, in each round of multi-round iterative training, first trains the speech encoder and multiple decoders corresponding to various tasks to enhance the accuracy of the encoder and decoder in performing the intent understanding SLU task and several privacy tasks. Then, adversarial training is performed on the intent understanding model (including the speech encoder and the first decoder) and several second decoders corresponding to the several privacy tasks. This effectively improves the prediction accuracy of the SLU model, and also effectively improves the accuracy and strength of privacy protection. Furthermore, the accuracy of the SLU task can be guaranteed through pre-training techniques, and / or the original audio feature vectors of the speech samples can be shuffled and recombined to further protect privacy. In summary, it is possible to prevent the completion of several privacy tasks, such as the ASR and VPR tasks, without affecting the SLU task, thereby improving the performance of the intent understanding model while protecting privacy. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic diagram illustrating the training architecture of the intent understanding model disclosed in the embodiments of this specification is shown.

[0030] Figure 2 This diagram illustrates the training method flow of the privacy-preserving intent-understanding model disclosed in the embodiments of this specification;

[0031] Figure 3 This diagram illustrates the data flow of the partitioning and shuffling of the timing coding vectors disclosed in the embodiments of this specification;

[0032] Figure 4 This diagram illustrates a flowchart of the training method for a privacy-preserving business prediction model disclosed in an embodiment of this specification.

[0033] Figure 5 This diagram illustrates the structure of a training device for the privacy-preserving intent-understanding model disclosed in the embodiments of this specification.

[0034] Figure 6 This diagram illustrates the structure of a training device for a privacy-preserving business prediction model disclosed in an embodiment of this specification. Detailed Implementation

[0035] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0036] As mentioned earlier, the intermediate representations, or hidden data, generated by the SLU model from processing raw speech data are easily intercepted by attackers. These attackers can then use techniques to infer speaker identity, emotions, and other biometric information, leading to user privacy leaks. To prevent the model's intermediate representations from leaking user privacy, it is usually necessary to sacrifice some of the model's prediction accuracy.

[0037] Based on the above observations and analysis, this specification discloses a training method for an intent understanding model. It introduces the idea of ​​adversarial learning to train the intent understanding model, so that even if the intermediate representation of the trained intent understanding model is stolen, it is difficult for attackers to infer the user's privacy information.

[0038] Figure 1 This diagram illustrates the training architecture of the intent understanding model disclosed in the embodiments of this specification. Figure 1 As shown, the training architecture includes a speech encoder and multiple decoders for performing various tasks. Specifically, the multiple decoders include a first decoder for performing an intent understanding task and several second decoders for performing several privacy tasks. Figure 1 (The text shows n≥1). It should be understood that the speech encoder and the first decoder constitute the intent understanding model, and the number of certain references in the text is greater than or equal to 1; in addition, the terms "first" and "second" in the first decoder and the second decoder, as well as similar terms in the text, are used to distinguish similar things and do not have any other limiting function such as ordering.

[0039] based on Figure 1 The training architecture shown can be implemented using method A. In each round of iterative training, a batch of speech samples sampled from the speech sample set is used to train the speech encoder and multiple decoders. Further, each round of iterative training includes two phases. In the first phase, the speech encoder and multiple decoders undergo conventional training, during which the speech encoder can simultaneously learn information from multiple tasks. In the second phase, the speech encoder and multiple decoders undergo adversarial training, allowing the speech encoder to contain only information related to the intent understanding task while removing some privacy-related information. Thus, through multiple rounds of iterative training, the prediction accuracy and privacy protection of the intent understanding task model can be effectively improved.

[0040] It is understood that in the above implementation method A, the first and second stages are performed alternately in each round of iterative training. In practice, another implementation method B can also be adopted, in which the first stage is executed multiple times, and then the second stage is executed multiple times. However, since the first and second stages are performed completely separately in this implementation method B, its implementation effect is far inferior to that of the above implementation method A.

[0041] The following is combined Figure 2The steps of the above-described embodiment A will be described in detail below, along with other embodiments.

[0042] Figure 2 This diagram illustrates a flowchart of a training method for a privacy-preserving intent understanding model disclosed in embodiments of this specification. The intent understanding model includes... Figure 1 The text shows a speech encoder and a first decoder. The training method can be implemented by any device, server, or cluster of devices with computing and processing capabilities. For example... Figure 2 As shown, any one of the multiple iterations in this training method includes the following steps:

[0043] Step S210: For any speech sample in the current batch of speech samples, process the sample using the speech encoder and the decoder among multiple decoders that corresponds to the task type to which the label of the sample belongs, to obtain a speech prediction result; wherein the multiple decoders include a first decoder and several second decoders for performing several privacy tasks; Step S220: Perform a first training on the speech encoder and multiple decoders with the goal of reducing the gap between each speech prediction result and the corresponding speech sample label; Step S230: Perform a second training on the intent understanding model that has been trained in the first training with the goal of increasing the gap between the speech prediction results corresponding to various privacy tasks and the speech sample labels, and reducing the gap between the speech prediction results output by the first decoder and the speech sample labels.

[0044] The steps above are explained in detail below:

[0045] First, in step S210, for any speech sample in the current batch of speech samples, the speech encoder and the decoder among multiple decoders that corresponds to the task type to which the label of the sample belongs are used to process the sample to obtain the speech prediction result.

[0046] To make it easier to understand, we will first introduce the speech samples, their sample labels, and how to obtain the speech samples in the current batch.

[0047] Voice samples include a piece of audio or a piece of voice signal, which can be collected in scenarios where users interact with each other via voice, such as smart homes or in-vehicle voice systems.

[0048] The speech sample has at least one sample label corresponding to at least one of the above-mentioned tasks (or task types). It should be understood that "at least one" in this text refers to one or more tasks.

[0049] In one embodiment, in an in-vehicle voice application scenario, the multiple alternative intent labels for the intent understanding task may include: turning Bluetooth on / off, turning navigation on / off, playing / turning music on / off, etc. In another embodiment, in a smart home application scenario, the multiple alternative intent labels may include: turning the TV on / off, opening / closing the curtains, turning the air conditioner on / off, turning the humidifier on / off, etc.

[0050] In one embodiment, the aforementioned privacy tasks include an Automatic Speech Recognition (ASR) task, and correspondingly, the speech recognition label under this task can be the text content corresponding to the speech sample. In another embodiment, the aforementioned privacy tasks include a Speaker Recognition (SR) task, or Voiceprint Recognition (VPR) task, and correspondingly, the speaker identity label under this task can be the speaker identity identifier corresponding to the speech sample. In yet another embodiment, the aforementioned privacy tasks may further include an emotion classification task, and correspondingly, the emotion labels under this task can include happiness, surprise, fright, frustration, agreement, support, disagreement, neutrality, etc.

[0051] The current batch of speech samples is obtained by sampling from a speech sample set. In one embodiment, the speech sample set includes multiple sample subsets corresponding to the various tasks mentioned above, and the samples in each sample subset have sample labels under the corresponding task type. Based on this, multiple sample subsets can be sampled separately to obtain multiple sub-batch samples, which are then collectively classified as the current batch of speech samples. In another embodiment, the speech sample set can be sampled directly to obtain a batch of speech samples, which is then used as the current batch of speech samples.

[0052] The above section introduces speech samples and their labels, as well as the acquisition of the current batch of speech samples. Next, it describes the implementation algorithms and model parameters of the speech encoder and multiple decoders. For brevity, the encoder and multiple decoders will be collectively referred to as the codec. It is important to understand that both the encoder and decoder are learners in machine learning. For any learner in the codec, its implementation algorithm can be: Deep Neural Networks (DNN), Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) networks, Gate Recurrent Units (GRU), or Transformers, etc. Furthermore, the implementation algorithms of any two learners in the codec can be the same or different. In one example, the speech encoder is implemented as an encoder in a Transformer, and the various decoders are implemented as decoders in a Transformer. In another example, the speech encoder is implemented as the encoder in the LAS (listen, attend and spell) model (referred to as listen in the relevant paper), the second decoder used to perform the speech recognition ASR task is implemented as the decoder in the LAS model (referred to as attend and spell in the relevant paper), and the first decoder and other second decoders are implemented as DNN networks.

[0053] Regarding the model parameters of the codec, in the case that this is the first iteration of training, in one implementation, the model parameters in the codec can be randomly initialized as the starting parameters for this iteration of training. In another implementation, considering that adversarial training can lead to a performance degradation in the intent understanding task, the inventors propose pre-training the model part of the codec corresponding to a certain privacy task, and using the pre-trained model parameters as the starting parameters for the corresponding model part in this iteration of training. For other model parts that did not participate in pre-training, their parameters can still be randomly initialized. In one embodiment, several privacy tasks include a speech recognition task, which typically has a large amount of training data. In this case, this large amount of training data can be used to pre-train the speech encoder and the second decoder for the speech recognition task. Thus, by pre-training the model part of the codec corresponding to the privacy task, not only can the adversarial branch for that privacy task be made stronger, but the accuracy of the intent understanding task can also be guaranteed.

[0054] If this round of training iteration is not the first round, the model parameters after the previous round of training iteration can be obtained as the starting parameters of the encoder and decoder in this round of training iteration.

[0055] The above mainly introduces speech samples and codecs. The following describes the process of using the codec to process any one speech sample in the current batch to obtain the speech prediction result. For simplicity, this arbitrary speech sample will be referred to as the target speech sample, and the sample label of the target speech sample will be called the target sample label.

[0056] In this step, the target hidden layer representation can be determined first based on the target speech sample and the speech encoder; then, based on this target hidden layer representation and the target decoder corresponding to the task type to which the target sample label belongs, the target speech prediction result for the target speech sample can be determined.

[0057] In one embodiment, the target speech sample can be directly input into the speech encoder. In another embodiment, features can be extracted from the speech signal in the target speech sample first, and then the extracted feature data can be input into the speech encoder. For example, spectral features such as short-time zero-crossing rate, short-time average amplitude, formant frequency, and Mel-frequency cepstral coefficients can be extracted. This can effectively reduce the processing difficulty of the speech encoder and increase the accuracy of speech prediction results.

[0058] After obtaining the target hidden layer representation, in one embodiment, the target hidden layer representation can be directly input into the target decoder.

[0059] In another embodiment, where several privacy tasks include speech recognition (ASR) tasks, considering that ASR tasks have strong sequence dependencies, while intention recognition (SLU) tasks mainly depend on certain words rather than temporal sequence, it is proposed to further reduce the risk of leakage of ASR privacy information by disrupting sequence dependencies.

[0060] Specifically, see Figure 3 First, feature extraction is performed on the target speech sample (such as extracting the fbank feature of the speech signal) to obtain the original audio feature vector; then, the original audio feature vector is segmented (or divided, partitioned, or cut) according to the time dimension; next, the multiple sub-feature vectors (or sub-vectors) obtained after segmentation are shuffled and recombined to obtain the disordered audio feature vector.

[0061] Furthermore, the aforementioned disordered audio feature vectors are processed using a speech encoder to obtain a target hidden layer representation. This target hidden layer representation is then input into the target decoder to obtain the target speech prediction result. For example, the segmentation can be equal division or random division, etc.

[0062] It is important to understand that the target sample label can be one or more, and correspondingly, the target decoder can be one or more of the aforementioned decoders. For example, assuming there are two or more target decoders, the target hidden layer representation can be input into each target decoder to obtain the corresponding target prediction result.

[0063] From the above, we can obtain the speech prediction results for each speech sample in the current batch of speech samples.

[0064] Then, step S220 is performed to perform the first training on the speech encoder and multiple decoders with the goal of reducing the gap between each speech prediction result and the corresponding speech sample label.

[0065] It's important to understand that training loss measures the difference between the predicted result and the sample label. Specifically, we can first determine the first training loss based on each speech prediction result and the corresponding speech sample label. This first training loss includes sub-losses corresponding to various tasks in the aforementioned tasks.

[0066] For example, the formula for calculating the first training loss is as follows:

[0067] L1 = L slu +L p1 …+L pi +…+L pn (1)

[0068] In equation (1) above, L slu L represents the first sub-loss corresponding to the intent recognition task. pi Let represent the i-th second sub-loss corresponding to the i-th privacy task.

[0069] In this step, the loss and the gap are positively correlated; that is, the smaller the loss, the smaller the gap. Therefore, the first training of the codec can be performed with the goal of minimizing the first training loss. It should be understood that the parameters in the codec can be updated using backpropagation based on the first training loss. An introduction to backpropagation can be found in existing techniques and will not be elaborated upon here. In this way, the first training of the codec can be achieved.

[0070] The above describes one implementation method for steps S210 and S220, in which steps S210 and S220 are executed sequentially. In practice, other implementation methods can also be used. For example, if the current batch of speech samples includes multiple sub-batch samples corresponding to various tasks, then multiple sub-batch samples can be used to perform sub-training. In each sub-training, the corresponding sub-batch samples are used to train the model part of the codec corresponding to the task type of that sub-batch sample. For example, several privacy tasks include speech recognition (ASR) and voiceprint recognition (VPR). Correspondingly, the current speech samples include three sub-batch samples corresponding to intent understanding (SLU), ASR, and VPR tasks. In this case, the speech encoder and first decoder can be trained first using a batch of sub-samples corresponding to the SLU task, then the speech encoder and corresponding second decoder can be trained using a batch of sub-samples corresponding to the ASR task, and finally the speech encoder and corresponding second decoder can be trained using a batch of sub-samples corresponding to the VPR task. This also achieves the first training of the codec.

[0071] The above allows for the initial training of the speech encoder and multiple decoders.

[0072] Then, in step S230, with the goal of increasing the gap between the speech prediction results and speech sample labels corresponding to various privacy tasks, and decreasing the gap between the speech prediction results and speech sample labels output by the first decoder, the intent understanding model that has been trained in the first step is trained in the second step. It should be understood that in this step, the model parameters of several second decoders are fixed and are not updated.

[0073] Specifically, a second training loss can be determined first. This second training loss is positively correlated with the first sub-loss corresponding to the intention understanding SLU task and negatively correlated with several second sub-losses corresponding to several privacy tasks.

[0074] In one embodiment, the first sub-loss and several second sub-losses determined in step S220 can be directly obtained and reused in this step to determine the second training loss. It should be understood that if the original audio feature vectors are divided and shuffled during the speech sample processing in step S210, then in the specific implementation of this step, including this embodiment, the adversarial training in this step can also disrupt the sequence dependencies of the speech recognition ASR task, thereby increasing the difficulty of inferring the correct speech recognition result based on the hidden layer data output by the speech encoder.

[0075] In another embodiment, another batch of training samples, different from the current batch of training samples, can be obtained and input into the codec to determine the first sub-loss and several second sub-losses using the obtained speech prediction results and the corresponding speech sample labels.

[0076] For example, the formula for calculating the second training loss is as follows:

[0077] L2=k*L slu -b1*L p1 …-b i *L pi …-b n *L pn (2)

[0078] In formula (2), L slu L represents the first sub-loss corresponding to the intent recognition task. pi This represents the i-th second sub-loss corresponding to the i-th privacy task; the multiplication coefficients before each sub-loss, such as k or b. i Its weight can be a hyperparameter set by the user; it can be understood that...

[0079] Furthermore, the intent understanding model is trained a second time with the goal of minimizing the second training loss. At this point, equation (2) can also be expressed in the following form:

[0080] minL2=k*minL slu +b1*maxL p1 …+b i *maxL pi +…+b n *maxL pn (3)

[0081] It is understandable that by minimizing the second training loss, it is possible to reduce the gap between the speech prediction results and speech sample labels for intent understanding tasks, and to increase the gap between the speech prediction results and speech sample labels for various privacy tasks. This enables adversarial training.

[0082] By performing multiple rounds of iterative updates until the model parameters of the codec converge, a trained speech encoder and multiple decoders can be obtained. The speech encoder and the first decoder constitute a trained intent understanding model, which can be put into practical use. For example, inputting the speech sample to be processed into the intent understanding model can yield the corresponding intent prediction result. Several second decoders among the multiple decoders can be discarded directly or retained for retraining the codec.

[0083] In summary, the training method for the privacy-preserving intent understanding model disclosed in the embodiments of this specification involves training the speech encoder and multiple decoders corresponding to various tasks in each round of multi-round iterative training. This enhances the accuracy of the encoder and decoder in performing the intent understanding SLU task and several privacy tasks. Then, adversarial training is performed on the intent understanding model (including the speech encoder and the first decoder) and several second decoders corresponding to the privacy tasks. This effectively improves the prediction accuracy of the SLU model and the accuracy and strength of privacy protection. Furthermore, pre-training techniques can be used to ensure the accuracy of the SLU task, and / or the original audio feature vectors can be shuffled and recombined to further protect privacy. In short, it is possible to prevent the completion of several privacy tasks, such as ASR and VPR tasks, without affecting the SLU task, thus improving the performance of the intent understanding model while protecting privacy.

[0084] The above-mentioned training method for protecting privacy mainly introduces the case where the main task is the intention understanding task. In fact, the main task can also be other types of tasks. Accordingly, several privacy tasks, or several attack tasks, can be adaptively set. Figure 4 This diagram illustrates a flowchart of a privacy-preserving business prediction model training method disclosed in an embodiment of this specification. The business prediction model includes an object encoder for business objects and a first decoder for performing the main task. The execution entity of this training method can be any device, server, or device cluster with computing and processing capabilities.

[0085] like Figure 4 As shown, any one of the multiple iterations in this training method includes the following steps:

[0086] Step S410: For any object sample in the current batch of object samples, process the sample using the object encoder and the decoder among multiple decoders that corresponds to the task type to which the label of the sample belongs, to obtain the object prediction result; wherein the multiple decoders include a first decoder and a number of second decoders for performing several privacy tasks; Step S420: Perform a first training on the object encoder and multiple decoders with the goal of reducing the gap between each object prediction result and the corresponding object sample label; Step S430: Perform a second training on the business prediction model that has been trained in the first training with the goal of increasing the gap between the object prediction results corresponding to various privacy tasks and the object sample labels, and reducing the gap between the object prediction results output by the first decoder and the object sample labels.

[0087] Regarding the above steps, in one embodiment, the aforementioned business object is voice. Accordingly, the main task and several privacy tasks can be different voice processing tasks, specifically selectively set from tasks such as Speech Intent Recognition (SLU), Intelligent Speech Recognition (ASR), Voiceprint Recognition (VPR), Voice Emotion Recognition, and Speaker Identity Verification as needed. For example, the main task can be the VPR task, and the several privacy tasks can include the SLU task, the ASR task, and the speaker identity verification task.

[0088] In another embodiment, the aforementioned business object is an image. Accordingly, the main task and several privacy tasks can be different image processing tasks, specifically selectively set from tasks such as face recognition, health status assessment, and emotion recognition as needed. For example, the main task can be health status assessment, and the several privacy tasks can include face recognition and emotion recognition tasks.

[0089] In another embodiment, the aforementioned business object is text. Accordingly, the main task and privacy tasks can be different text processing tasks, specifically selectively set from tasks such as text sentiment classification, text intent classification, and text content compliance determination as needed. For example, the main task can be text intent classification, and several privacy tasks may include text sentiment classification and text content compliance determination.

[0090] It should be noted that, for Figure 4 The steps shown are described in the figure, and you can also refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0091] By performing multiple rounds of iterative updates until the model parameters of the object encoder and multiple decoders converge, a trained object encoder and multiple decoders can be obtained. The object encoder and the first decoder constitute a trained business prediction model, which can be used in actual applications for the main task. For example, inputting the object sample to be processed into the business prediction model will yield the corresponding business prediction result. Several second decoders among the multiple decoders can be directly discarded or retained for retraining the encoder and decoder.

[0092] In summary, the training method for the privacy-preserving business prediction model disclosed in the embodiments of this specification involves training the object encoder and multiple decoders corresponding to various tasks in each round of multi-round iterative training. This enhances the accuracy of the encoder and decoder in performing the main task and several attack tasks. Then, adversarial training is performed on the business prediction model and several second decoders. This effectively improves the prediction accuracy of the business prediction model, as well as the accuracy and strength of privacy protection. Furthermore, pre-training techniques can be used to ensure the accuracy of the main task, and / or the original audio feature vectors can be shuffled and recombined to further protect privacy. In short, it is possible to prevent several privacy tasks from being completed without affecting the main task, thereby improving the performance of the business prediction model while protecting privacy.

[0093] Corresponding to the training methods described above, this specification also discloses a training device in its embodiments. Figure 5 This diagram illustrates the structure of a training device for a privacy-preserving intent understanding model disclosed in embodiments of this specification. The intent understanding model includes a speech encoder and a first decoder. Figure 5 As shown, the device 500 implements iterative training for any round in multi-round iterative training through the following units:

[0094] The speech prediction unit 510 is configured to process any speech sample in the current batch of speech samples using the speech encoder and a decoder from a plurality of decoders that corresponds to the task type to which the sample's label belongs, to obtain a speech prediction result; wherein the plurality of decoders includes a first decoder and a plurality of second decoders for performing a plurality of privacy tasks. The first training unit 520 is configured to perform a first training on the speech encoder and the plurality of decoders with the goal of reducing the gap between each speech prediction result and its corresponding speech sample label. The second training unit 530 is configured to perform a second training on the intent understanding model that has undergone the first training, with the goal of increasing the gap between the speech prediction results corresponding to various privacy tasks and the speech sample labels, and reducing the gap between the speech prediction results output by the first decoder and the speech sample labels.

[0095] In one embodiment, the plurality of privacy tasks includes a speech recognition task and / or a speaker recognition task.

[0096] In one embodiment, the plurality of privacy tasks include a speech recognition task; the apparatus 500 further includes a pre-training unit 540 configured to pre-train the speech encoder and a second decoder for performing the speech recognition task using a training sample set corresponding to the speech recognition task.

[0097] In one embodiment, the speech encoder is a temporal encoder, and the plurality of privacy tasks include a speech recognition task; the speech prediction unit 510 is specifically configured to: for any speech sample, segment and shuffle the corresponding original audio feature vector according to the time dimension to obtain a disordered audio feature vector; process the disordered audio feature vector using the speech encoder to obtain a hidden layer representation; input the hidden layer representation into the decoder corresponding to the label task type of the sample to obtain the speech prediction result.

[0098] In one embodiment, the second training unit 530 is specifically configured to: determine an adversarial training loss; the adversarial training loss is positively correlated with a first loss and negatively correlated with a plurality of second losses corresponding to the plurality of privacy tasks; the first loss is determined based on the speech prediction result and speech sample label output by the first decoder, and each second loss is determined based on the speech prediction result and speech sample label corresponding to its privacy task.

[0099] Figure 6 This diagram illustrates the structure of a training apparatus for a privacy-preserving business prediction model disclosed in embodiments of this specification. The business prediction model includes an object encoder for business objects and a first decoder for performing the main task. Figure 6 As shown, the device 600 implements iterative training for any round in multi-round iterative training through the following units:

[0100] The object prediction unit 610 is configured to process any object sample in the current batch of object samples using the object encoder and a decoder from among multiple decoders that corresponds to the task type of the sample's label, to obtain an object prediction result; wherein the multiple decoders include a first decoder and multiple second decoders for performing several privacy tasks. The first training unit 620 is configured to perform a first training on the object encoder and multiple decoders with the goal of reducing the gap between each object prediction result and its corresponding object sample label. The second training unit 630 is configured to perform a second training on the business prediction model that has undergone the first training, with the goal of increasing the gap between the object prediction results corresponding to various privacy tasks and the object sample labels, and reducing the gap between the object prediction results output by the first decoder and the object sample labels.

[0101] In one embodiment, the aforementioned business object is voice, image, or text.

[0102] In one embodiment, the second training unit 630 is specifically configured to: determine adversarial training loss; the adversarial training loss is positively correlated with the first loss and negatively correlated with a plurality of second losses corresponding to the plurality of privacy tasks; the first loss is determined based on the object prediction result and object sample label output by the first decoder, and each second loss is determined based on the object prediction result and object sample label corresponding to its privacy task.

[0103] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 2 or Figure 4 The method described.

[0104] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements a combination... Figure 2 or Figure 4 The methods described herein. Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0105] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for training a privacy-preserving intent understanding model, the intent understanding model comprising a speech encoder and a first decoder; The method involves multiple rounds of iterative training, where any one round of iterative training includes: For any speech sample in the current batch of speech samples, the speech encoder and a decoder from a plurality of decoders that corresponds to the task type to which the label of the sample belongs are used to process the sample to obtain a speech prediction result; wherein the plurality of decoders include the first decoder and a plurality of second decoders for performing a plurality of privacy tasks; With the goal of reducing the gap between each speech prediction result and the corresponding speech sample label, the speech encoder and multiple decoders are first trained. With the goal of increasing the gap between the speech prediction results and speech sample labels for various privacy tasks, and reducing the gap between the speech prediction results and speech sample labels output by the first decoder, the intent understanding model that has been trained in the first training is trained in the second training.

2. The method according to claim 1, wherein, The privacy tasks include speech recognition tasks and / or speaker recognition tasks.

3. The method according to claim 1, wherein, The privacy-preserving tasks include speech recognition tasks; wherein, prior to the multi-round iterative training, the method further includes: The speech encoder and the second decoder used to perform the speech recognition task are pre-trained using a training sample set corresponding to the speech recognition task.

4. The method according to claim 1, wherein, The privacy-preserving tasks include speech recognition tasks; wherein, for any speech sample in the current batch of speech samples, the speech encoder and a decoder from among the multiple decoders corresponding to the labeled task type of that sample are used to process the sample to obtain a speech prediction result, including: For any given speech sample, its corresponding original audio feature vector is segmented and shuffled according to the time dimension to obtain a disordered audio feature vector. The speech encoder is used to process the disordered audio feature vector to obtain the hidden layer representation; The hidden layer representation is input into the decoder corresponding to the label task type of the sample to obtain the speech prediction result.

5. The method according to claim 1, wherein, With the goal of increasing the gap between the speech prediction results and speech sample labels for various privacy tasks, and decreasing the gap between the speech prediction results and speech sample labels output by the first decoder, a second training is performed on the intent understanding model trained in the first training stage, including: Determine the adversarial training loss; the adversarial training loss is positively correlated with the first loss and negatively correlated with the several second losses corresponding to the several privacy tasks; the first loss is determined based on the speech prediction result and speech sample label output by the first decoder, and each second loss is determined based on the speech prediction result and speech sample label corresponding to its privacy task.

6. The method according to claim 1, wherein, During the second training of the intent understanding model that has been trained in the first training, the model parameters of the several second decoders remain unchanged.

7. A method for understanding the intent to protect privacy, comprising: Obtain the speech sample to be processed; The speech sample is input into the intent understanding model trained using the method described in claim 1 to obtain the corresponding intent prediction result.

8. A method for training a privacy-preserving business prediction model, the business prediction model comprising an object encoder for business objects and a first decoder for performing a main task; The method involves multiple rounds of iterative training, where any one round of iterative training includes: For any object sample in the current batch of object samples, the sample is processed using the object encoder and a decoder from multiple decoders that corresponds to the task type to which the label of the sample belongs, to obtain the object prediction result; wherein the multiple decoders include the first decoder and a number of second decoders for performing several privacy tasks; With the goal of reducing the gap between the prediction results of each object and the corresponding object sample labels, the object encoder and multiple decoders are first trained. With the goal of increasing the gap between the object prediction results and the object sample labels for various privacy tasks, and reducing the gap between the object prediction results and the object sample labels output by the first decoder, a second training is performed on the business prediction model that has been trained in the first training.

9. The method according to claim 8, wherein, The business object is voice, image, or text.

10. The method according to claim 8, wherein, During the second training of the business prediction model that has been trained in the first training, the model parameters of the plurality of second decoders remain unchanged.

11. A privacy-preserving business forecasting method, comprising: Obtain a sample of the object to be processed; The object sample is input into the business prediction model trained using the method described in claim 8 to obtain the corresponding business prediction result.

12. A training apparatus for a privacy-preserving intent understanding model, the intent understanding model comprising a speech encoder and a first decoder; the apparatus implementing iterative training in any round of multi-round iterative training through the following units: The speech prediction unit is configured to process any speech sample in the current batch of speech samples using the speech encoder and a decoder from a plurality of decoders that corresponds to the task type to which the label of the sample belongs, to obtain a speech prediction result; wherein the plurality of decoders includes a first decoder and a plurality of second decoders for performing a plurality of privacy tasks. The first training unit is configured to perform first training on the speech encoder and multiple decoders with the goal of reducing the gap between each speech prediction result and the corresponding speech sample label. The second training unit is configured to perform a second training on the intent understanding model that has been trained in the first training, with the goal of increasing the gap between the speech prediction results and speech sample labels corresponding to various privacy tasks, and reducing the gap between the speech prediction results and speech sample labels output by the first decoder.

13. A training apparatus for a privacy-preserving business prediction model, the business prediction model comprising an object encoder for business objects and a first decoder for performing a main task; the apparatus implements iterative training in any round of multi-round iterative training through the following units: The object prediction unit is configured to process any object sample in the current batch of object samples using the object encoder and a decoder from a plurality of decoders that corresponds to the task type to which the label of the sample belongs, to obtain an object prediction result; wherein the plurality of decoders includes a first decoder and a plurality of second decoders for performing a plurality of privacy tasks. The first training unit is configured to perform first training on the object encoder and multiple decoders with the goal of reducing the gap between the prediction results of each object and the corresponding object sample labels. The second training unit is configured to perform a second training on the business prediction model that has been trained in the first training, with the goal of increasing the gap between the object prediction results and the object sample labels corresponding to various privacy tasks, and reducing the gap between the object prediction results and the object sample labels output by the first decoder.

14. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed in a computer, it causes the computer to perform the method according to any one of claims 1-11.

15. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-11.

Images