Privacy protection audio generation method and device, electronic equipment and storage medium

Abstract

The invention provides a privacy protection audio generation method and device, electronic equipment and a storage medium, and belongs to the technical field of audio processing, and the method comprises the steps: determining an encryption identifier according to the information amount of a target audio; inputting the encryption identifier and the target audio into a target encryption model to obtain an encrypted audio of the target audio; wherein different encryption identifiers correspond to different encryption loss functions in the target encryption model. According to the privacy protection audio generation method and device, the electronic equipment and the storage medium provided by the invention, the reliability and safety of privacy protection audio generation can be improved.

Description

Technical Field

[0001] The present disclosure belongs to the field of audio processing technology, and more specifically, to a privacy-preserving audio generation method and device, an electronic device, and a storage medium. Background Art

[0002] With the popularity of devices such as voice assistants and smart speakers, the risk of audio data leakage is gradually increasing. The need to protect the privacy of personal audio data is urgent. Existing audio processing technologies have certain shortcomings in privacy protection. For example, simple encryption methods may affect audio quality and are easy to crack; some blurring methods excessively lose the comprehensibility of audio. Therefore, a reliable and secure privacy-preserving audio generation method is needed. Summary of the invention

[0003] The purpose of the present disclosure is to provide a privacy-preserving audio generation method and device, an electronic device, and a storage medium to improve the reliability and security of privacy-preserving audio generation.

[0004] According to a first aspect of an embodiment of the present disclosure, a privacy-preserving audio generation method is provided, comprising: Determining an encryption identifier according to the information volume of the target audio; The encryption identifier and the target audio are input into a target encryption model to obtain encrypted audio of the target audio; wherein different encryption identifiers correspond to different encryption loss functions in the target encryption model.

[0005] According to a second aspect of the embodiments of the present disclosure, there is provided a privacy protection audio generation device, including: An encryption identifier determination module, used to determine the encryption identifier according to the information amount of the target audio; The encrypted audio generation module is used to input the encryption identifier and the target audio into a target encryption model to obtain the encrypted audio of the target audio; wherein different encryption identifiers correspond to different encryption loss functions in the target encryption model.

[0006] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above-mentioned privacy protection audio generation method when executing the computer program.

[0007] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned privacy-preserving audio generation method are implemented.

[0008] The privacy-preserving audio generation method and device, electronic device, and storage medium provided by the embodiments of the present disclosure have the following beneficial effects: The disclosed embodiment can accurately determine the encryption identifier by carefully analyzing and evaluating the information volume of the target audio, thereby selecting a suitable encryption method for the target audio, which not only improves the pertinence of encryption, but also effectively ensures the security and privacy of the target audio.

[0009] At the same time, different encryption identifiers correspond to different encryption loss functions, so that the target encryption model can be optimized according to different encryption requirements and security levels, thereby maintaining the audio quality as much as possible while ensuring security.

[0010] Therefore, the embodiments of the present disclosure can improve the reliability and security of privacy-preserving audio generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A flowchart of a privacy protection audio generation method provided by an embodiment of the present disclosure; Figure 2 A structural block diagram of a privacy protection audio generation device provided by an embodiment of the present disclosure; Figure 3 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0013] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0014] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0015] Please refer to Figure 1 , Figure 1 A flowchart of a privacy-preserving audio generation method provided in an embodiment of the present disclosure is provided, and the method includes: S101: Determine an encryption identifier according to the information volume of the target audio.

[0016] In the present embodiment, the target audio is the audio data of the target user, and the target audio may include user audio, background audio and environmental audio. The information amount of the target audio may include multiple information amounts, and the multiple information amounts may be content richness, importance, duration and complexity. The information amount of the target audio is related to the complexity of the target audio content and the importance conveyed by the target audio.

[0017] Collecting audio data of a target user as target audio; Extract features of the target audio to obtain multiple feature information; Perform weighted calculation on multiple feature information to determine the information amount of the target audio; Among them, multiple feature information may include audio duration, sound elements and information density.

[0018] The steps of determining the encryption identifier according to the information volume of the target audio may be as follows: First, analyze the characteristics of the target audio.

[0019] First, perform spectrum analysis on the target audio to understand the frequency distribution of the target audio. Different frequency ranges can correspond to different types of sounds, such as low frequencies can be background noise or low sounds, and high frequencies can be sharp sounds or certain specific sound effects. By analyzing the frequency characteristics, the complexity of the target audio can be preliminarily determined.

[0020] Finally, speech recognition and semantic analysis are performed. If the target audio contains clear speech content and the importance or sensitivity of the information it conveys can be determined through semantic analysis, the information content assessment value of the target audio will also be increased.

[0021] Second, set evaluation criteria.

[0022] According to different application scenarios and requirements, the evaluation criteria for the information volume of the target audio are set. For example, the information volume of the target audio can be divided into three levels: low, medium, and high, and each level corresponds to a different encryption identifier.

[0023] For target audio with low-level information content, you can choose not to encrypt or use a low-strength encryption algorithm. The encryption flag can be set to "0" or other symbols representing a low level of confidentiality, indicating that high confidentiality is not required.

[0024] For target audio with medium information volume, a medium-strength encryption algorithm can be used, and the encryption flag can be set to "1" or other symbols representing a medium level of confidentiality.

[0025] For target audio with a high level of information, a high-strength encryption algorithm can be used, and the encryption flag can be set to "2" or other symbols representing a high degree of confidentiality to ensure the security of the audio.

[0026] Third, dynamic adjustment.

[0027] As time goes by and the target audio content changes, the amount of information in the target audio will change. Therefore, it is necessary to establish a dynamic adjustment mechanism to re-evaluate the amount of information in the target audio periodically or in real time and adjust the encryption flag accordingly.

[0028] For example, if new important content is added to the target audio during transmission, or its usage scenario becomes more sensitive, it is necessary to increase the encryption flag and strengthen the encryption measures.

[0029] The setting of the encryption identifier can guide the selection of the encryption algorithm and facilitate the management and identification of a large number of target audios. Different encryption identifiers correspond to different encryption algorithms and encryption strengths. After the encryption identifier is determined, the present embodiment can automatically select a suitable encryption method to protect the target audio.

[0030] For example, for the target audio with encryption mark "0", no encryption may be performed or only a simple encoding method may be used; for the target audio with encryption mark "1", a symmetric encryption algorithm may be used; for the target audio with encryption mark "2", an asymmetric encryption algorithm may be used.

[0031] The encryption identifier can be used as a label to facilitate the management and identification of a large number of target audios. When storing, transmitting and processing the target audio, the confidentiality level of the target audio can be quickly determined based on the encryption identifier, and corresponding security measures can be taken.

[0032] In summary, this embodiment can select a suitable encryption method for the target audio by analyzing and evaluating the information amount of the target audio to ensure the security and privacy of the target audio.

[0033] S102: Input the encryption identifier and the target audio into the target encryption model to obtain the encrypted audio of the target audio; wherein different encryption identifiers correspond to different encryption loss functions in the target encryption model.

[0034] In this embodiment, the target encryption module can be trained by the historical target audio and the encryption identifier corresponding to the historical target audio to obtain a trained target encryption model. The encryption identifier and the target audio are input into the trained target encryption model to obtain the encrypted audio corresponding to the target audio.

[0035] The target encryption model is a method for encrypting target audio, which can ensure the security and privacy of the target audio and achieve privacy protection for the generation of encrypted audio.

[0036] The loss function is a function used in machine learning and deep learning to measure the difference between the model's prediction and the actual result. In the target encryption model, the loss function is used to evaluate the difference between the encrypted target audio and the original target audio, as well as the performance of the encrypted target audio in terms of security and audio quality.

[0037] Different encryption identifiers represent different encryption requirements and security levels. Therefore, different encryption loss functions are needed to optimize the performance of the target encryption model. For example, for the target audio with an encryption identifier of "0", the encryption loss function focuses on selecting a loss function with an audio quality adjustment precision less than the first precision; for the target audio with an encryption identifier of "1", the encryption loss function focuses on selecting a loss function with an audio quality adjustment precision greater than or equal to the first precision and less than the second precision; for the target audio with an encryption identifier of "2", the encryption loss function focuses on selecting a loss function with an audio quality adjustment precision greater than or equal to the second precision. Among them, the first precision and the second precision can be pre-set.

[0038] Exemplarily, the target encryption module can select Generative Adversarial Networks (GAN) as the model architecture. GAN consists of a generator and a discriminator. The generator is used to generate target audio samples that are as realistic as possible to deceive the discriminator; the discriminator is used to distinguish between real audio and encrypted audio generated by the generator. Through adversarial training, the generator continuously improves the audio quality of the generated encrypted audio, making it closer and closer to the real audio. In the task of generating audio that is recognizable but not easy to detect, the generator can learn the feature distribution of the target audio and generate encrypted audio with similar features but with specific recognizable features.

[0039] The Variational Autoencoder (VAE) consists of an encoder and a decoder. The encoder compresses the input audio into a low-dimensional latent space representation, and the decoder reconstructs the audio from the latent space. VAE can learn the potential distribution of audio data and thus generate new audio samples.

[0040] In the encrypted audio task, VAE can encode and decode the target audio to generate audio that is similar to the target audio but has certain changes, and can embed recognizable information in the latent space.

[0041] A covert trigger is a specific signal or pattern embedded in the target audio that can be triggered under certain conditions, making the target audio recognizable. A covert trigger should be imperceptible to ensure the concealment of the target audio. For example, a specific frequency region can be selected in the spectrum of the target audio, or a small change can be inserted in the time series of the target audio as a covert trigger.

[0042] By combining covert triggers with GAN and VAE, it is possible to embed identifying information in the process of generating audio. For example, covert triggers can be added to the input of the generator or decoder to make the generated audio recognizable under certain conditions. At the same time, in order to ensure that the audio is not easy to discover, the covert triggers need to be carefully designed and processed to minimize their impact on the auditory effect of the audio.

[0043] Specifically, the steps of the encrypted audio generation process can be as follows: First, data preparation.

[0044] Collect a large amount of audio data as training data, which may include various types of music, speech, environmental sounds, etc.

[0045] Preprocess the audio data, such as normalization and filtering, to improve the training effect of the model.

[0046] Second, model training.

[0047] GAN and VAE models are trained separately. For GAN, the generator and discriminator are trained alternately so that the generator can generate realistic audio samples; for VAE, the potential distribution of audio data is learned by minimizing the reconstruction error and regularization term.

[0048] During the training process, hidden triggers can be introduced, such as adding specific signals or patterns to the input of the generator or decoder, and by adjusting the parameters of the model, the generated audio can be made as close to the real audio as possible while containing the hidden triggers.

[0049] Third, encrypted audio generation.

[0050] Using the trained GAN and VAE models (i.e., the target encryption model) and the covert trigger mechanism, audio with identifiable but difficult to detect is generated. This embodiment can generate different encrypted audios by adjusting the input random noise or latent variables and controlling the parameters of the covert trigger.

[0051] Fourth, verification and optimization.

[0052] The generated encrypted audio is verified to check its recognizability and difficulty in detection. In this embodiment, the quality of the audio can be evaluated by methods such as artificial hearing test and spectrum analysis.

[0053] Based on the verification results, the target encryption model and covert trigger are optimized, and the parameters and design are adjusted to improve the audio performance.

[0054] Exemplarily, first, the audio data of the target user is collected to generate the target audio; secondly, the evaluation criteria are set according to the amount of information of the target audio to determine the encryption identifier; then, different encryption loss functions correspond to different encryption identifiers, and then the target encryption model is determined according to the encryption identifier, historical target audio and encryption loss function; finally, the encryption identifier and the target audio are input into the target encryption model to obtain the encrypted audio of the target audio.

[0055] It can be concluded from the above that this embodiment can accurately determine the encryption identifier by carefully analyzing and evaluating the information volume of the target audio, thereby selecting a suitable encryption method for the target audio. This not only improves the pertinence of encryption, but also effectively ensures the security and privacy of the target audio.

[0056] At the same time, different encryption identifiers correspond to different encryption loss functions, so that the target encryption model can be optimized according to different encryption requirements and security levels, thereby maintaining the audio quality as much as possible while ensuring security.

[0057] Therefore, this embodiment can improve the reliability and security of privacy-preserving audio generation.

[0058] In one embodiment of the present disclosure, determining an encryption identifier according to the information amount of the target audio includes: Separating the target audio to obtain first audio information and second audio information; the first audio information is user audio information in the target audio, and the second audio information is environmental audio information in the target audio; The encryption identifier is determined according to the first information amount of the first audio information and the second information amount of the second audio information.

[0059] Extract features of the target audio to obtain multiple feature information; Separating the target audio according to the plurality of feature information to obtain first audio information and second audio information; Separate the target audio based on multiple feature information, including: The plurality of feature information includes first feature information; In response to the feature value of the first feature information being greater than or equal to the first feature threshold, taking the target audio corresponding to the first feature information as the first audio information; In response to the feature value of the first feature information being less than the first feature threshold, taking the target audio corresponding to the first feature information as the second audio information; The first characteristic threshold may be component information, and may be dynamically adjusted according to actual conditions.

[0060] Extracting features from the first audio information to obtain a plurality of first feature information; Performing weighted calculation on the plurality of first feature information to determine the first information amount; Extracting features from the second audio information to obtain a plurality of second feature information; Performing weighted calculation on the plurality of second feature information to determine the second information amount; Among them, the first amount of information is greater than the second amount of information.

[0061] In this embodiment, the target audio is separated into user audio information and environmental audio information in order to more accurately evaluate the amount of information in the audio, thereby determining a suitable encryption identifier. Different types of audio information may have different importance and security requirements. User audio information usually contains personal voice content, which may involve personal privacy, commercial secrets or sensitive information, and therefore requires higher security protection. Environmental audio information may be relatively less important, but it may also provide additional contextual information in some cases.

[0062] Audio separation can be achieved through a variety of technologies, such as blind source separation and deep learning methods. Blind source separation technology uses the statistical characteristics of the signal to separate the mixed signal into individual source signals without knowing the prior knowledge of the source signal. Deep learning algorithms can learn the characteristics of audio by training neural networks, thereby achieving audio separation.

[0063] For example, a deep neural network architecture, such as a convolutional neural network or a recurrent neural network, can be used to process the target audio, and the spectral features, time domain features, etc. of the audio can be learned to separate the user audio information from the ambient audio information.

[0064] For the first audio information (i.e., the user audio information), the following factors may be considered to evaluate its information content: First, the complexity of the language content: this can include the richness of vocabulary, complexity of grammatical structure, depth of expression, etc. If the user audio contains professional terms, complex sentence structures, or in-depth discussions, the amount of information is higher than that of everyday language.

[0065] Second, importance and sensitivity: If the user audio involves personal privacy, business secrets or sensitive topics, the amount of information will be considered high in certain situations, such as personal bank account information, business negotiation content, etc.

[0066] Third, voice features: such as voice clarity, speaking speed, intonation, etc. Clear, fast and rich-intonation voice can convey more information.

[0067] For the second audio information (ambient audio information), the following factors can be considered to evaluate its information content: First, the diversity of sound: the more different sound elements contained in the ambient audio, the greater the amount of information. For example, the amount of ambient audio information containing multiple musical instruments, natural sounds (such as birdsong, water flow), crowd noise, etc. will be relatively increased.

[0068] Second, the relevance to the user audio: If the ambient audio is closely related to the user audio and can provide additional contextual information, then the amount of information will also increase. For example, in a meeting recording, coughing sounds, book flipping sounds, etc. in the ambient audio can hint at the atmosphere of the meeting and the status of the participants.

[0069] Third, the intensity and change of sound: strong sounds or frequently changing sounds can attract more attention and are therefore considered to have a certain amount of information, such as sudden explosions, continuous machine roars, etc.

[0070] When determining the encryption identifier, it is necessary to comprehensively consider the first information amount of the user audio information (i.e., the first audio information) and the second information amount of the environmental audio information (i.e., the second audio information). Different weights can be assigned to different types of audio information according to specific application scenarios and security requirements.

[0071] Different encryption identification standards can be set according to the amount of information comprehensively evaluated. For example, the amount of information can be divided into three levels: low, medium, and high, and each level corresponds to a different encryption identification.

[0072] For target audio with low-level information content, you can choose not to encrypt or use a low-strength encryption algorithm. The encryption flag can be set to "0" or other symbols representing a low level of confidentiality, indicating that high confidentiality is not required.

[0073] For target audio with medium information volume, a medium-strength encryption algorithm can be used, and the encryption flag can be set to "1" or other symbols representing a medium level of confidentiality.

[0074] For target audio with a high level of information, a high-strength encryption algorithm can be used, and the encryption flag can be set to "2" or other symbols representing a high degree of confidentiality to ensure the security of the audio.

[0075] It can be concluded from the above that this embodiment separates the target audio, evaluates the amount of information of the user audio information and the ambient audio information, and comprehensively considers the above factors to determine the encryption identifier. Therefore, this embodiment can select a suitable encryption method for the target audio according to the specific characteristics and security requirements of the target audio to ensure the security and privacy of the target audio.

[0076] In one embodiment of the present disclosure, determining an encryption identifier according to a first amount of information of the first audio information and a second amount of information of the second audio information includes: Performing weighted calculation on the first amount of information and the second amount of information to determine the amount of information of the target audio; In response to the information amount of the target audio being less than the first information amount, using the first encryption identifier as the encryption identifier; In response to the information amount of the target audio being greater than or equal to the first information amount and less than the second information amount, using the second encryption identifier as the encryption identifier; In response to the information amount of the target audio being greater than or equal to the second information amount, using the third encryption identifier as the encryption identifier; The encryption accuracy of the first encryption identifier is less than the encryption accuracy of the second encryption identifier; The encryption accuracy of the second encryption identifier is less than the encryption accuracy of the third encryption identifier.

[0077] In this embodiment, the weighted calculation of the first information amount of the first audio information (user audio information) and the second information amount of the second audio information (environmental audio information) is to comprehensively consider the importance of different types of audio information in the entire target audio. Since the importance of user audio information and environmental audio information to the entire audio may be different, by assigning different weights, the actual information amount of the target audio can be more accurately reflected.

[0078] For example, if in a specific scenario, user audio information is considered more important, then a higher weight can be assigned to the first amount of information; conversely, if environmental audio information also has great value, then the weight distribution can be appropriately adjusted so that both can be reasonably reflected when calculating the target audio information amount.

[0079] Exemplarily, this embodiment can be calculated in a linear weighted manner, that is, the information volume of the target audio = weight 1×first information volume+weight 2×second information volume. The sum of weight 1 and weight 2 is usually 1, which can be adjusted according to specific circumstances.

[0080] For example, suppose weight 1 is , weight 2 is , the first amount of information is , the second amount of information is , then the information amount of the target audio = .

[0081] When the target audio information volume is less than the first information volume, the first encryption identifier is used as the encryption identifier, which means that the target audio information volume is relatively low and does not require excessive encryption accuracy. The encryption accuracy of the first encryption identifier is less than that of the second encryption identifier and the third encryption identifier, which can minimize the resource consumption and performance impact on audio processing while ensuring a certain degree of security.

[0082] For example, some daily, less important audio records, such as ordinary voice memos or environmental sound samples, fall within the above range (i.e., the amount of information of the target audio is less than the first amount of information) and are encrypted using the first encryption identifier.

[0083] When the target audio information volume is greater than or equal to the first information volume and less than the second information volume, the second encryption flag is selected, indicating that the target audio information volume is at a medium level and requires a certain degree of encryption protection. The encryption accuracy of the second encryption flag is higher than that of the first encryption flag, but lower than that of the third encryption flag, and a medium-strength encryption algorithm can be used.

[0084] For example, some recordings of business meetings, important personal conversations, etc. fall within the above range (that is, the amount of information of the target audio is greater than or equal to the first amount of information and less than the second amount of information), and need to be encrypted using the second encryption identifier to ensure a certain degree of security and privacy.

[0085] When the target audio has a larger amount of information than or equal to the second amount of information, the third encryption identifier is used for encryption, which means that the target audio has a very large amount of information, involves extremely sensitive information, and requires the highest level of encryption accuracy. The third encryption identifier usually uses a high-intensity encryption algorithm to ensure the absolute security of the audio.

[0086] For example, audio content involving state secrets, major business negotiations, or highly personal privacy requires encryption protection using a third encryption identifier.

[0087] Encryption identifiers with different encryption precision provide different levels of security. The higher the encryption precision, the more difficult it is to crack, and the higher the security of the audio.

[0088] From the above, it can be concluded that this embodiment determines the information amount of the target audio by performing weighted calculation on the information amount of the first audio information and the second audio information, and selects the corresponding encryption identifier according to different information amount ranges, thereby achieving personalized encryption protection for the audio and finding the best balance between security and encrypted audio quality.

[0089] In one embodiment of the present disclosure, the privacy protection audio generation method further includes: Determine the target encryption model based on the encryption identification and historical target audio; The encryption identifier includes a first encryption identifier, a second encryption identifier and a third encryption identifier.

[0090] In this embodiment, different encryption identifiers represent different security level requirements. This embodiment can ensure that the generated encrypted audio can meet specific security requirements by determining the target encryption model based on the encryption identifier. For example, for highly sensitive audio content, a model with high encryption accuracy is required; while for daily audio content, a simpler encryption model can be used to save resources.

[0091] For example, the first encryption identifier corresponds to a low level of security, which can be suitable for some daily, less sensitive audio, and a relatively simple encryption algorithm and parameter setting model can be selected. The second encryption identifier corresponds to a medium level of security, which requires a more complex encryption algorithm combination. The third encryption identifier corresponds to a high level of security, which requires the use of a strict security policy.

[0092] The historical target audio contains various audio data that have been processed in the past. By analyzing the characteristics, security requirements, and encryption effects of the above audio, we can learn encryption models suitable for different types of audio. In this way, when encrypting new target audio, we can draw on historical experience and choose a more appropriate encryption model.

[0093] For example, if certain types of audio in historical target audio perform well using a specific encryption model, then that model may be preferentially selected for encryption for new similar audio.

[0094] The historical target audio can be used to evaluate the performance of different encryption models in actual applications, thereby optimizing the decision-making process of model selection. Different encryption models can be compared and screened based on indicators such as the encryption success rate of historical audio, security assessment, and audio quality impact.

[0095] For example, if a certain encryption model often fails to decrypt or suffers from severe audio quality degradation in historical applications, you can avoid using that model when selecting a target encryption model.

[0096] Specifically, the process of determining the target encryption model can be as follows: First, collect and analyze historical target audio.

[0097] Collect a large amount of historical target audio data, including different types of audio content, different security requirements and corresponding encryption identifiers. Perform detailed analysis on these audios and extract their characteristic information, such as audio length, frequency distribution, information volume, etc.

[0098] At the same time, the encryption model used for each audio and the effect evaluation after encryption are recorded, including indicators in terms of security, audio quality, etc.

[0099] Second, establish a model library.

[0100] According to different encryption identifiers and historical audio analysis results, a model library containing multiple encryption models is established. Each model has its own specific algorithm, parameter settings and scope of application.

[0101] When there is a new target audio that needs to be encrypted, the most appropriate encryption model is selected from the model library as the target encryption model based on its corresponding encryption identifier and its similarity with the historical target audio.

[0102] A variety of decision-making methods can be used, such as rule-based decision-making, machine learning algorithms, etc. For example, a decision tree algorithm is used to automatically select the best encryption model based on the characteristics of the audio and the encryption identifier.

[0103] When using the target encryption model to encrypt new audio, the encryption effect is constantly monitored and user feedback is collected. If problems are found or there is room for improvement, the target encryption model can be adjusted and optimized.

[0104] For example, if the quality of encrypted audio is found to be seriously degraded, the model parameters can be adjusted to improve the audio quality while ensuring security. Or, based on new security threats, the model algorithm can be updated in a timely manner to enhance its security.

[0105] This embodiment uses chi-square test and t-test to compare the distribution difference between the target encryption model output and the historical target audio, so as to check whether there is any unauthorized data leakage.

[0106] At the same time, fuzzy matching technology can also be used to decompose the audio signal, identify potential trigger patterns through fuzzy matching algorithms, and detect whether there are signs of external intervention or data abuse.

[0107] Important generation and verification processes are performed on the target user's device to ensure that individual data does not leave the target user's local area, reducing the risk of leakage; the parties can also collaborate in training models remotely, and data privacy protection is achieved through encryption technology to ensure that only legitimate users can access the original audio.

[0108] This embodiment also monitors and tunes the actual performance of the target encryption model by collecting feedback from target users, and timely adjusts privacy protection measures to deal with new sensitive issues. In addition, the maintained privacy protection model is regularly updated to prevent the generation technology from solidifying certain trigger features due to batch learning of new data, and to maintain the long-term stability of the encryption effect.

[0109] From the above, it can be concluded that determining the target encryption model based on the encryption identifier and historical target audio can select the most appropriate encryption model according to different security requirements and historical experience, improve encryption efficiency and security, and achieve continuous improvement and optimization.

[0110] In one embodiment of the present disclosure, the first encryption identifier corresponds to a first loss function; The second encrypted identifier corresponds to the second loss function; The third encrypted identifier corresponds to the third loss function; The audio quality adjustment accuracy of the first loss function is less than the audio quality adjustment accuracy of the second loss function; The audio quality adjustment accuracy of the second loss function is less than the audio quality adjustment accuracy of the third loss function.

[0111] In this embodiment, the loss function is used to measure the performance and effect of the encryption model during the encryption process. Different loss functions will have different effects on the quality of the encrypted audio, and will also affect the strength and security of the encryption. A specific encryption identifier corresponds to a specific loss function to ensure that the encryption process can meet the corresponding security requirements and audio quality requirements.

[0112] The first loss function corresponds to the first encryption identifier, and its audio quality adjustment accuracy is the smallest, which means that when the first encryption identifier is used for encryption, the impact on the audio quality is relatively small. It is suitable for scenarios that require high audio quality but relatively low security requirements. For example, some ordinary music audio or daily voice recordings do not require too high encryption strength, but need to maintain good audio quality.

[0113] The second loss function has a greater audio quality adjustment accuracy than the first loss function, corresponding to the second encryption flag, and is suitable for situations where both security and audio quality are required. For example, audio records of some business negotiations require a certain degree of encryption protection to prevent information leakage, but the audio quality cannot be degraded too much to affect subsequent listening and analysis.

[0114] The third loss function corresponds to the third encryption identifier, which has the highest audio quality adjustment accuracy. This means that when the third encryption identifier is used for encryption, the audio quality will be greatly affected, but it can also provide the highest level of security protection. It is suitable for scenarios with extremely high security requirements, such as audio involving state secrets, major commercial secrets, or highly sensitive personal information.

[0115] In this embodiment, different encryption identifiers correspond to loss functions of different audio quality adjustment precisions, so that security and audio quality can be balanced according to actual needs during the encryption process.

[0116] When selecting encryption identifiers and loss functions, factors such as the importance, sensitivity, usage scenarios, and requirements for audio quality of the target audio need to be considered comprehensively.

[0117] From the above, it can be concluded that different encryption identifiers correspond to different loss functions of audio quality adjustment accuracy, and a reasonable trade-off and choice can be made between security and audio quality according to specific needs.

[0118] In one embodiment of the present disclosure, the privacy protection audio generation method further includes: Determine a first target encryption model according to the first loss function, and generate a first encrypted audio; Determine a second target encryption model according to a second loss function, and generate a second encrypted audio; A third target encryption model is determined according to the third loss function to generate third encrypted audio.

[0119] In this embodiment, the first loss function, the second loss function and the third loss function correspond to different audio quality adjustment precisions and encryption requirements respectively. Different loss functions reflect different requirements for encrypted audio, including security, audio quality, and encryption strength. According to a specific loss function, the corresponding encryption algorithm, parameter settings, and model structure can be selected and adjusted to achieve the best encryption effect. For each loss function, a specific target encryption model can be determined to meet different encryption requirements.

[0120] After determining the first target encryption model according to the first loss function, the target audio can be input into the model for encryption processing to generate the first encrypted audio.

[0121] Since the audio quality adjustment accuracy of the first loss function is low, the first target encryption model has relatively little impact on the audio quality during the encryption process. Therefore, the first encrypted audio can provide a certain degree of encryption protection while maintaining good audio quality.

[0122] For example, for some daily voice recordings or music audio, the first target encryption model can be used to encrypt them to prevent unauthorized access, but at the same time it does not affect the normal playback and enjoyment of the audio.

[0123] Similarly, a second target encryption model is determined according to the second loss function, and then the target audio is input into the model to generate a second encrypted audio. The audio quality adjustment accuracy of the second loss function is between the first and third loss functions, so the second target encryption model will balance the audio quality and encryption strength during the encryption process.

[0124] For example, for audio records of some business documents or important meeting recordings, the second target encryption model can be used for encryption, which not only ensures a certain quality of the audio but also provides higher security.

[0125] After determining the third target encryption model according to the third loss function, the target audio is input into the model to generate the third encrypted audio. The third loss function has the highest audio quality adjustment accuracy, which means that the third target encryption model not only considers encryption strength but also needs to balance audio quality.

[0126] For example, audio involving state secrets, major commercial secrets, or highly sensitive personal information must be encrypted using a third-party target encryption model to ensure absolute security while also considering audio quality.

[0127] From the above, we can conclude that determining different target encryption models according to different loss functions and generating different encrypted audios can meet the encryption requirements in different scenarios, provide flexible encryption options, and achieve a balance between security and audio quality.

[0128] In one embodiment of the present disclosure, the privacy protection audio generation method further includes: determining a target audio signal based on the target audio; determining a first encrypted audio signal based on the first encrypted audio; determining a first loss function based on the target audio signal and the first encrypted audio signal; determining a second encrypted audio signal based on the second encrypted audio; determining a second loss function based on the target audio signal and the second encrypted audio signal; determining a third encrypted audio signal based on the third encrypted audio; A third loss function is determined based on the target audio signal and the third encrypted audio signal.

[0129] In this embodiment, the target audio is the original unencrypted audio, which is converted into a target audio signal for subsequent analysis and processing. The target audio signal can be a digital representation of audio, such as an audio data sequence obtained by sampling and quantization.

[0130] The first encrypted audio, the second encrypted audio and the third encrypted audio also need to be converted into corresponding encrypted audio signals, which are encrypted audio data sequences.

[0131] A first loss function is determined based on the target audio signal and the first encrypted audio signal. The first loss function is used to measure the audio quality difference between the first encrypted audio and the target audio.

[0132] Assume that the target audio signal is , the first encrypted audio signal is .

[0133] A first loss function between the target audio signal and the first encrypted audio signal may be calculated, and its expression is as follows:

[0134] in, represents the number of sample points, Indicates The time of the sample point, represents the target audio signal, represents a first encrypted audio signal, represents the weight function; Since the human ear has different sensitivities to different frequency ranges, the weight function can be adjusted according to the human ear's sensitivity curve to frequency. The human ear's sensitivity to frequency is highest in the range of 1000Hz-4000Hz, so a bell-shaped curve can be used to represent the frequency weight. represents the frequency corresponding to the sample point, (in the frequency range with higher sensitivity), represents the parameter that controls the frequency weight distribution, then the expression of the weight function is:

[0135] At the same time, the bell curve can be referred to, and higher weights can be given to frequency areas that are sensitive to the human ear, and lower weights can be given to insensitive areas.

[0136] A second loss function is determined based on the target audio signal and the second encrypted audio signal. The second loss function is used to measure the difference between the second encrypted audio and the target audio. Since the encryption level of the second encrypted audio is higher than that of the first encrypted audio, the second loss function will pay more attention to the strength and security of encryption. Different indicators or weights from the first loss function can be used to determine the second loss function to adapt to higher encryption requirements.

[0137] Assume that the target audio signal is , the second encrypted audio signal is .

[0138] A second loss function between the target audio signal and the second encrypted audio signal may be calculated, and its expression is as follows:

[0139] in, and represent the mean values ​​of the target audio signal and the second encrypted audio signal respectively, and denote the standard deviation of the target audio signal and the second encrypted audio signal, respectively. represents the covariance of the target audio signal and the second encrypted audio signal, and It is a constant set to avoid the denominator being zero, and the value can be determined based on experience.

[0140] This embodiment still adopts As a weight loss function (i.e., a frequency weight function), the mean, standard deviation, and covariance of the target audio signal and the second encrypted audio signal can be weighted and calculated as follows: Perform discrete Fourier transform on each frame of the target audio signal and the second encrypted audio signal to obtain a frequency domain representation and ; Calculate the weighted mean:

[0141]

[0142] in, Indicates the number of frequency domain points; Calculate the weighted standard deviation:

[0143]

[0144] Compute the weighted covariance:

[0145] in, yes The conjugation of.

[0146] In practice, for an audio signal in the time domain, the average amplitude of the signal can be calculated as an approximation of brightness. The variance of the audio signal is calculated as a measure of contrast, and the covariance of the audio signal is calculated as a measure of structure. This embodiment optimizes the model by minimizing the second loss function so that the target audio signal and the second encrypted audio signal are as similar as possible in brightness, contrast, and structure.

[0147] A third loss function is determined based on the target audio signal and the third encrypted audio signal. The third loss function is used to measure the maximum difference between the third encrypted audio and the target audio. Since the third encrypted audio has the highest encryption level, the third loss function may more strictly require minimizing the difference between the encrypted audio and the original audio while ensuring the highest security.

[0148] The third loss function may be determined using an indicator or weight different from that of the second loss function to accommodate higher encryption requirements.

[0149] Assume that the target audio signal is , the third encrypted audio signal is .

[0150] The expression of the third loss function is:

[0151] Since target audio with high information volume requires not only high-level encryption to improve security, but also high-quality audio, the time weight function is introduced. , that is, giving higher weights to the key parts of the audio.

[0152] make Indicates The time corresponding to each sample point is Indicates the key time node, represents the parameter controlling the time weight distribution, then the time weight function The expression is as follows:

[0153] Among them, the key time nodes The setting is made according to the actual situation. In this embodiment, the model is optimized by minimizing the third loss function, so that the target audio signal and the third encrypted audio signal are as similar as possible in brightness, contrast and structure.

[0154] By determining different loss functions, the corresponding encryption model can be optimized according to the difference between the encrypted audio and the target audio. For example, if the value of the first loss function is large, it means that the first target encryption model has a greater impact on the audio quality during the encryption process, and the model parameters or algorithms need to be adjusted to reduce the value of the loss function. Different loss functions can help find a balance between security and audio quality. Adapt to different application scenarios: It can be concluded from the above that determining different loss functions based on the target audio signal and the encrypted audio signal is an important step in the privacy-preserving audio generation method. This embodiment can help optimize the encryption model, balance security and audio quality, and adapt to different application scenarios.

[0155] Corresponding to the privacy protection audio generation method of the above embodiment, Figure 2 This is a structural block diagram of a privacy protection audio generation device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 2The privacy protection audio generating device 20 includes: an encryption identification determining module 21 and an encrypted audio generating module 22.

[0156] The encryption identifier determination module 21 is used to determine the encryption identifier according to the information amount of the target audio; The encrypted audio generation module 22 is used to input the encryption identifier and the target audio into the target encryption model to obtain the encrypted audio of the target audio; wherein different encryption identifiers correspond to different encryption loss functions in the target encryption model.

[0157] In one embodiment of the present disclosure, the encryption identification determination module 21 is further configured to: Separating the target audio to obtain first audio information and second audio information; the first audio information is user audio information in the target audio, and the second audio information is environmental audio information in the target audio; The encryption identifier is determined according to the first information amount of the first audio information and the second information amount of the second audio information.

[0158] In one embodiment of the present disclosure, the encryption identification determination module 21 is further configured to: In one embodiment of the present disclosure, a weighted calculation is performed on the first information amount and the second information amount to determine the information amount of the target audio; In response to the information amount of the target audio being less than the first information amount, using the first encryption identifier as the encryption identifier; In response to the information amount of the target audio being greater than or equal to the first information amount and less than the second information amount, using the second encryption identifier as the encryption identifier; In response to the information amount of the target audio being greater than or equal to the second information amount, using the third encryption identifier as the encryption identifier; The encryption accuracy of the first encryption identifier is less than the encryption accuracy of the second encryption identifier; The encryption accuracy of the second encryption identifier is less than the encryption accuracy of the third encryption identifier.

[0159] In one embodiment of the present disclosure, the privacy protection audio generation device 20 further includes: a model building module; A model building module for determining a target encryption model based on an encryption identifier and historical target audio; The encryption identifier includes a first encryption identifier, a second encryption identifier and a third encryption identifier.

[0160] In one embodiment of the present disclosure, the first encryption identifier corresponds to a first loss function; The second encrypted identifier corresponds to the second loss function; The third encrypted identifier corresponds to the third loss function; The audio quality adjustment accuracy of the first loss function is less than the audio quality adjustment accuracy of the second loss function; The audio quality adjustment accuracy of the second loss function is less than the audio quality adjustment accuracy of the third loss function.

[0161] In one embodiment of the present disclosure, the model building module is specifically used to: Determine a first target encryption model according to the first loss function, and generate a first encrypted audio; Determine a second target encryption model according to a second loss function, and generate a second encrypted audio; A third target encryption model is determined according to the third loss function to generate third encrypted audio.

[0162] In one embodiment of the present disclosure, the model building module is specifically used to: determining a target audio signal based on the target audio; determining a first encrypted audio signal based on the first encrypted audio; determining a first loss function based on the target audio signal and the first encrypted audio signal; determining a second encrypted audio signal based on the second encrypted audio; determining a second loss function based on the target audio signal and the second encrypted audio signal; determining a third encrypted audio signal based on the third encrypted audio; A third loss function is determined based on the target audio signal and the third encrypted audio signal.

[0163] See also Figure 3 , Figure 3 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 3 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303 and one or more memories 304. The processors 301, input devices 302, output devices 303 and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of each module / unit in the above-mentioned device embodiments, such as Figure 2 The functions of modules 21 to 22 are shown.

[0164] It should be understood that in the embodiment of the present disclosure, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0165] The input device 302 may include a touch panel, a fingerprint collection sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.

[0166] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0167] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of the privacy-preserving audio generation method provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device described in the embodiments of the present disclosure, which will not be repeated here.

[0168] In another embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor, all or part of the processes in the above-mentioned embodiment method are implemented, and the computer program can also be completed by instructing the relevant hardware through the computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0169] The computer-readable storage medium may be an internal storage unit of the electronic device of any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0170] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0171] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0172] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or it can be an electrical, mechanical or other form of connection.

[0173] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.

[0174] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0175] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A privacy-preserving audio generation method, characterized in that: include: Determining an encryption identifier according to the information volume of the target audio; The encryption identifier and the target audio are input into a target encryption model to obtain encrypted audio of the target audio; wherein different encryption identifiers correspond to different encryption loss functions in the target encryption model.

2. The privacy-preserving audio generation method according to claim 1, wherein: The step of determining the encryption identifier according to the information amount of the target audio includes: Separating the target audio to obtain first audio information and second audio information; the first audio information is user audio information in the target audio, and the second audio information is environmental audio information in the target audio; An encryption identifier is determined according to a first information amount of the first audio information and a second information amount of the second audio information.

3. The privacy-preserving audio generation method according to claim 2, characterized in that: The step of determining the encryption identifier according to the first information amount of the first audio information and the second information amount of the second audio information includes: Performing weighted calculation on the first amount of information and the second amount of information to determine the amount of information of the target audio; In response to the information amount of the target audio being less than the first information amount, using the first encryption identifier as the encryption identifier; In response to the information amount of the target audio being greater than or equal to the first information amount and less than the second information amount, using the second encryption identifier as the encryption identifier; In response to the information amount of the target audio being greater than or equal to the second information amount, using the third encryption identifier as the encryption identifier; The encryption accuracy of the first encryption identifier is less than the encryption accuracy of the second encryption identifier; The encryption accuracy of the second encryption identifier is less than the encryption accuracy of the third encryption identifier.

4. The privacy-preserving audio generation method according to claim 3, wherein: Also includes: Determining the target encryption model based on the encryption identifier and the historical target audio; The encryption identifier includes the first encryption identifier, the second encryption identifier and the third encryption identifier.

5. The privacy-preserving audio generation method according to claim 4, characterized in that: The first encryption identifier corresponds to a first loss function; The second encrypted identifier corresponds to a second loss function; The third encrypted identifier corresponds to a third loss function; The audio quality adjustment accuracy of the first loss function is less than the audio quality adjustment accuracy of the second loss function; The audio quality adjustment accuracy of the second loss function is less than the audio quality adjustment accuracy of the third loss function.

6. The privacy-preserving audio generation method according to claim 5, characterized in that: Also includes: Determine a first target encryption model according to the first loss function, and generate a first encrypted audio; Determine a second target encryption model according to the second loss function, and generate a second encrypted audio; A third target encryption model is determined according to the third loss function to generate third encrypted audio.

7. The privacy-preserving audio generation method according to claim 6, wherein: Also includes: determining a target audio signal based on the target audio; determining a first encrypted audio signal based on the first encrypted audio; determining the first loss function based on the target audio signal and the first encrypted audio signal; determining the second encrypted audio signal based on the second encrypted audio; determining the second loss function based on the target audio signal and the second encrypted audio signal; determining the third encrypted audio signal based on the third encrypted audio; The third loss function is determined based on the target audio signal and the third encrypted audio signal.

8. A privacy protection audio generation device, characterized in that: include: An encryption identifier determination module, used to determine the encryption identifier according to the information amount of the target audio; The encrypted audio generation module is used to input the encryption identifier and the target audio into a target encryption model to obtain the encrypted audio of the target audio; wherein different encryption identifiers correspond to different encryption loss functions in the target encryption model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images