Sample processing method and related device and storage medium

Abstract

The embodiments of the present application provide a sample processing method and a related device. Among them, a sample processing method may include: obtaining a first feature space sample; performing a feature space transformation on the first feature space sample to obtain a second feature space sample, the second feature space is different from the first feature space; performing a disturbance injection of a stimulus component on the second feature space sample to obtain a second feature space sample after disturbance injection, wherein the disturbance injection does not change the masking state of the second feature space sample; performing a feature space inverse transformation on the second feature space sample after disturbance injection to obtain an inversely transformed first feature space sample; and sending or storing the inversely transformed first feature space sample. The technical solution of the embodiments of the present application is conducive to effectively resisting attacks related to adversarial samples.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to some artificial intelligence sample processing methods, related devices and storage media. Background Art

[0002] In recent years, artificial intelligence (AI) technology and applications have developed rapidly, covering more and more industries. At the same time, the security risks of AI technology have gradually been exposed, and related security issues have far-reaching impacts. For example, security attackers take advantage of AI model vulnerabilities and carefully construct input samples to deceive AI models into making wrong decisions, or implant backdoors in AI models to implement advanced attacks, or interactively query and build similar models through the service interface of the AI ​​model, or query the service interface of the AI ​​model in scenarios where users provide training data to obtain privacy information in user data, etc.

[0003] Therefore, how to improve the anti-attack capability of AI systems and thus improve the security of the application of AI technology is a technical topic that urgently needs to be studied in this field. Summary of the invention

[0004] The embodiments of the present application provide some artificial intelligence sample processing methods, related devices and storage media, etc.

[0005] A first aspect of an embodiment of the present application provides a sample processing method, which may include: acquiring a first feature space sample (the first feature space sample may be acquired by locally generating it or receiving it from other devices); performing a feature space transformation on the first feature space sample to obtain a second feature space sample, the second feature space being different from the first feature space; performing a disturbance injection of a stimulus component on the second feature space sample to obtain a second feature space sample after disturbance injection, the disturbance injection not changing the masking state of the second feature space sample; performing a feature space inverse transformation on the second feature space sample after disturbance injection to obtain an inversely transformed first feature space sample; and sending or storing the inversely transformed first feature space sample.

[0006] The feature space may also be referred to as a feature domain. It can be understood that the first feature space sample represents a sample in the first feature space, and the second feature space sample represents a sample in the second feature space.

[0007] The first feature space may be, for example, one of the following feature spaces: frequency domain space, time domain space, brightness space, pattern space or texture space. The second feature space may be, for example, one of the following feature spaces: frequency domain space, time domain space, brightness space, pattern space or texture space.

[0008] Among them, the samples mentioned in the embodiments of the present application may be, for example, audio samples, video samples, picture samples, or other types of samples.

[0009] Assuming that the first feature space is the time domain space and the first feature space is the frequency domain space, then the first feature space samples are transformed into the second feature space samples, specifically, the samples in the time domain space are transformed into the Fourier transform (time-frequency transform) to obtain the samples in the frequency domain space. Assuming that the first feature space is the frequency domain space and the first feature space is the time domain space, then the first feature space samples are transformed into the second feature space samples, specifically, the samples in the frequency domain space are transformed into the inverse Fourier transform (frequency-time transform) to obtain the samples in the time domain space. The transformation of other samples between feature spaces is analogous.

[0010] Among them, not changing the masking state of the second feature space sample specifically refers to not changing the masking state of each channel of the second feature space sample. For example, the masked channel is still a masked channel before and after the disturbance injection, that is, the masking state of the channel remains masked before and after the disturbance injection. The stimulus component in the masked channel is imperceptible to the user before and after the disturbance injection.

[0011] Among them, the sample processing method exemplified above can be applied to an AI client, an AI server, a third-party sample database, or a transit network device between an AI client and an AI server, etc. Among them, when the sample processing method is applied to an AI client, the AI ​​client can send the inverse transformed first feature space sample to the AI ​​server after obtaining the inverse transformed first feature space sample. When the sample processing method is applied to an AI server, the AI ​​server can store the inverse transformed first feature space sample (for example, it can be stored in a training set or other location) after obtaining the inverse transformed first feature space sample, so that the inverse transformed first feature space sample can be input into the AI ​​model when needed later. When the sample processing method is applied to a third-party sample database, the third-party sample database can store the inverse transformed first feature space sample, so that the stored inverse transformed first feature space sample can be provided to the AI ​​server, etc. when needed.

[0012] It can be seen that the above sample processing method is a sample processing mechanism based on the masking effect. Since the perturbation injection does not change the masking state of the sample, it can destroy the fragile structure of the adversarial sample in a hidden state that the user is not aware of, which is conducive to achieving a good attack prevention effect without affecting the user's perceived experience. In practice, it has been found that this sample processing mechanism based on the masking effect can effectively prevent dodge attacks, theft attacks, and bait attacks based on adversarial samples. For AI input samples such as audio / picture / video samples, the sample processing mechanism based on the masking effect is conducive to not introducing perceptual damage to the sample, which is conducive to improving the multi-scenario adaptability of the solution.

[0013] In some possible implementations, injecting a disturbance of a stimulus component into the second feature space sample to obtain a second feature space sample after the disturbance injection may include:

[0014] Determine N masking channels of the second feature space sample; perform disturbance injection of stimulus components on M masking channels among the N masking channels, wherein N is greater than or equal to M, and the masking states of the M masking channels are not changed by the disturbance injection.

[0015] The M masking channels are, for example, randomly or non-randomly selected from the N masking channels. In the case of randomly selecting the masking channels for disturbance injection, it is helpful to increase the difficulty for attackers to analyze the disturbance injection rules, thereby helping to improve the attack defense capability.

[0016] For example, determining the N masking channels of the second feature space sample includes: calculating the masking thresholds of X channels of the second feature space sample (X is, for example, the total number of channels of the second feature space sample, and the masking thresholds of different channels may be the same or different); selecting N masking channels from the X channels based on the masking thresholds and stimulation components of the X channels (N is, for example, the total number of masking channels of the second feature space sample), the stimulation component of masking channel i is less than or equal to the masking threshold of masking channel i, and the masking channel i is any one of the N masking channels.

[0017] It can be seen that by calculating the masking threshold of each channel of the sample, it is easy to make flexible choices on the adjustment amplitude and adjustment range under the constraint of the masking threshold, which is conducive to increasing the difficulty for attackers to speculate on the disturbance injection law.

[0018] In some possible implementations, the stimulation component perturbation injected into the masking channel i among the M masking channels may be obtained based on a random algorithm, or the stimulation component perturbation injected into the masking channel i among the M masking channels may be obtained based on a preset sequence (the preset sequence is, for example, a digital watermark).

[0019] The stimulus component disturbance injected into the masking channel i among the M masking channels is a positive stimulus component disturbance or a negative stimulus component disturbance. It can be understood that injecting a positive stimulus component disturbance increases the stimulus component, and injecting a negative stimulus component disturbance decreases the stimulus component.

[0020] It can be seen that when the disturbance is obtained based on a random algorithm, it is helpful to further increase the difficulty for attackers to analyze the disturbance injection rules, which is conducive to improving the attack defense capability. In addition, when the stimulus component disturbance is obtained using a digital watermark, the use of a digital watermark can detect whether the sample is credible and whether it has been tampered with, which is conducive to active prevention.

[0021] A second aspect of an embodiment of the present application provides a sample processing method, which may include: acquiring a first feature space sample; injecting a stimulus component perturbation into the first feature space sample to obtain a first feature space sample after perturbation injection, wherein the perturbation injection does not change the masking state of the first feature space sample; and sending or storing the first feature space sample after perturbation injection.

[0022] The first feature space may be, for example, one of the following feature spaces: frequency domain space, time domain space, brightness space, pattern space or texture space.

[0023] Among them, not changing the masking state of the first feature space sample specifically refers to not changing the masking state of each channel of the first feature space sample. For example, the masked channel is still a masked channel before and after the disturbance injection, that is, the masking state of the channel remains masked before and after the disturbance injection. The stimulus component in the masked channel is not perceptible to the user before and after the disturbance injection.

[0024] Among them, the sample processing method of the embodiment of the present application can be applied to an AI client, an AI server, a third-party sample database, or a transit network device between an AI client and an AI server, etc. When the sample processing method is applied to an AI client, the AI ​​client can send the first feature space sample injected with disturbance to the AI ​​server after obtaining the first feature space sample injected with disturbance. When the sample processing method is applied to an AI server, the AI ​​server can store the first feature space sample injected with disturbance (for example, it can be stored in a training set or other location) after obtaining the first feature space sample injected with disturbance, so that the first feature space sample injected with disturbance can be input into the AI ​​model when needed later. When the sample processing method is applied to a third-party sample database, the third-party sample database can store the first feature space sample injected with disturbance, so that the stored first feature space sample injected with disturbance can be provided to the AI ​​server, etc. when needed.

[0025] It can be seen that the above sample processing method is a sample processing mechanism based on the masking effect. Since the perturbation injection does not change the masking state of the sample, it can destroy the fragile structure of the adversarial sample in a hidden state that the user is not aware of, which is conducive to achieving a good attack prevention effect without affecting the user's perceived experience. In practice, it has been found that this sample processing mechanism based on the masking effect can effectively prevent dodge attacks, theft attacks, and bait attacks based on adversarial samples. For AI input samples such as audio / picture / video samples, the sample processing mechanism based on the masking effect is conducive to not introducing perceptual damage to the sample, which is conducive to improving the multi-scenario universality of the solution.

[0026] For example, performing a disturbance injection of a stimulus component on a first feature space sample to obtain a first feature space sample with disturbance injection may include: determining N masking channels of the first feature space sample; performing a disturbance injection of the stimulus component on M of the N masking channels, wherein N is greater than or equal to M, and the masking states of the M masking channels are not changed by the disturbance injection.

[0027] The M masking channels are, for example, randomly or non-randomly selected from the N masking channels. In the case of randomly selecting the masking channels for disturbance injection, it is helpful to increase the difficulty for attackers to analyze the disturbance injection rules, thereby helping to improve the attack defense capability.

[0028] For example, determining the N masking channels of the first feature space sample includes: calculating the masking thresholds of X channels of the first feature space sample (X is, for example, the total number of channels of the first feature space sample, and the masking thresholds of different channels may be the same or different); selecting N masking channels from the X channels based on the masking thresholds and stimulation components of the X channels (N is, for example, the total number of masking channels of the first feature space sample), the stimulation component of masking channel i is less than or equal to the masking threshold of masking channel i, and the masking channel i is any one of the N masking channels.

[0029] In some possible implementations, the stimulation component perturbation injected into the masking channel i among the M masking channels may be obtained based on a random algorithm, or the stimulation component perturbation injected into the masking channel i among the M masking channels may be obtained based on a preset sequence (the preset sequence is, for example, a digital watermark).

[0030] The stimulus component disturbance injected into the masking channel i among the M masking channels is a positive stimulus component disturbance or a negative stimulus component disturbance. It can be understood that injecting a positive stimulus component disturbance increases the stimulus component, and injecting a negative stimulus component disturbance decreases the stimulus component.

[0031] It can be seen that when the disturbance is obtained based on a random algorithm, it is helpful to further increase the difficulty for attackers to analyze the disturbance injection rules, which is conducive to improving the attack defense capability. In addition, when the stimulus component disturbance is obtained using a digital watermark, the use of a digital watermark can detect whether the sample is credible and whether it has been tampered with, which is conducive to active prevention.

[0032] A third aspect of an embodiment of the present application provides a sample processing device, comprising:

[0033] The acquisition unit is used to acquire a first feature space sample.

[0034] The feature space transformation unit is used to perform feature space transformation on the first feature space samples to obtain second feature space samples, where the second feature space is different from the first feature space.

[0035] The disturbance injection unit is used to perform disturbance injection of a stimulus component on the second feature space sample to obtain the second feature space sample after the disturbance injection, wherein the disturbance injection does not change the masking state of the second feature space sample.

[0036] The feature space inverse transformation unit is used to perform feature space inverse transformation on the second feature space samples injected with disturbance to obtain inverse transformed first feature space samples.

[0037] An output unit is used to send or store the inverse transformed first feature space samples.

[0038] In some possible implementations, the disturbance injection unit is specifically used to determine N masking channels of the second feature space samples; perform disturbance injection of stimulation components on M masking channels among the N masking channels, where N is greater than or equal to M, and the masking states of the M masking channels are not changed by the disturbance injection.

[0039] For example, determining the N masking channels of the second feature space sample includes: calculating the masking thresholds of X channels of the second feature space sample (X is, for example, the total number of channels of the second feature space sample, and the masking thresholds of different channels may be the same or different); selecting N masking channels from the X channels based on the masking thresholds and stimulation components of the X channels (N is, for example, the total number of masking channels of the second feature space sample), the stimulation component of masking channel i is less than or equal to the masking threshold of masking channel i, and the masking channel i is any one of the N masking channels.

[0040] A fourth aspect of the present application provides a sample processing device, including:

[0041] The acquisition unit is used to acquire a first feature space sample.

[0042] The disturbance injection unit is used to perform a stimulation component disturbance injection on the first feature space sample to obtain a first feature space sample after the disturbance injection, wherein the disturbance injection does not change the masking state of the first feature space sample.

[0043] An output unit sends or stores the first feature space sample injected with disturbance.

[0044] In some possible implementations, the disturbance injection unit is specifically used to determine N masking channels of the first feature space sample; perform disturbance injection of stimulation components on M masking channels among the N masking channels, where N is greater than or equal to M, and the masking states of the M masking channels are not changed by the disturbance injection.

[0045] For example, determining the N masking channels of the first feature space sample includes: calculating the masking thresholds of X channels of the first feature space sample (X is, for example, the total number of channels of the first feature space sample, and the masking thresholds of different channels may be the same or different); selecting N masking channels from the X channels based on the masking thresholds and stimulation components of the X channels (N is, for example, the total number of masking channels of the first feature space sample), the stimulation component of masking channel i is less than or equal to the masking threshold of masking channel i, and the masking channel i is any one of the N masking channels.

[0046] A fifth aspect of an embodiment of the present application provides a sample processing device, including:

[0047] A processor and a memory coupled to each other; wherein the processor is used to call a computer program stored in the memory to complete some or all steps of any one of the methods provided in the embodiments of the present application. For example, the processor can be used to call a program stored in the memory to complete some or all steps of any one of the methods provided in the first aspect or the second aspect.

[0048] A sixth aspect of the embodiments of the present application provides a computer-readable storage medium, including:

[0049] The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it can implement some or all steps of any method provided in the embodiments of the present application. For example, when the computer program is executed by the processor, it can implement some or all steps of any method provided in the first aspect or the second aspect.

[0050] A sixth aspect of the embodiments of the present application provides a computer program product, wherein when the computer program product is run on a computer device, the computer device is caused to perform some or all steps of any method provided in the embodiments of the present application. For example, when the computer program product is run on a computer device, the computer device is caused to perform some or all steps of any method provided in the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Some drawings related to the embodiments of the present application are described below.

[0052] Figure 1-A and Figure 1-B It is a schematic diagram of a network architecture of an AI system provided in an embodiment of the present application.

[0053] Figure 2-A An example is given to illustrate a possible implementation of the input reconstruction technology.

[0054] Figure 2-B A schematic diagram of a masking effect in an audio scene is shown as an example.

[0055] Figure 3 It is a flow chart of a sample processing method provided in an embodiment of the present application.

[0056] Figure 4 It is a flow chart of a sample processing method provided in an embodiment of the present application.

[0057] Figure 5-A It is a flowchart of an audio sample processing method provided in an embodiment of the present application.

[0058] Figure 5-B It is a curve diagram of the masking state and masking threshold of an audio sample in the frequency domain space provided by an embodiment of the present application.

[0059] Figure 5-C A schematic diagram showing the effect of injecting disturbances into audio samples based on the masking effect is shown.

[0060] Figure 6-A to Figure 6-C It is a schematic diagram of the architecture of several improved AI systems provided as examples in the embodiments of the present application.

[0061] Figure 7 This is a schematic diagram of the architecture of an anti-attack detection in a smart home scenario provided by an embodiment of the present application.

[0062] Figure 8 It is a schematic diagram of a process of injecting a digital watermark into an audio sample based on a masking effect provided in an embodiment of the present application.

[0063] Fig. 9It is a schematic diagram of the signal effect before and after a digital watermark is injected into an audio sample based on a masking effect provided in an embodiment of the present application.

[0064] Fig.10 It is a schematic diagram of a system architecture that can resolve adversarial sample attacks in an end-to-end Internet communication scenario provided by an embodiment of the present application.

[0065] Fig.11 It is a schematic diagram of the system architecture of a model theft defense scenario provided in an embodiment of the present application.

[0066] Fig.12 It is a structural schematic diagram of a sample processing device provided in an embodiment of the present application.

[0067] Fig.13 It is a structural schematic diagram of another sample processing device provided in an embodiment of the present application.

[0068] Fig.14 It is a structural schematic diagram of another sample processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0070] See also Figure 1-A , the following is an example of a network architecture of an AI system. Figure 1-A An AI system shown by way of example may include an AI client (AI Custom) and an AI server (AI Server). The AI ​​Server is a service provider, while the AI ​​Custom is a service requester. The AI ​​Custom may be a normal user or an attacker. The AI ​​client may submit samples to the AI ​​server, and the AI ​​client may also request AI reasoning services and other related AI services from the AI ​​server. The AI ​​server may obtain samples from the AI ​​client, and may use these samples to train the AI ​​model. The AI ​​server may also provide AI reasoning services to the AI ​​client.

[0071] See also Figure 1-B , Figure 1-B The internal functional module structure of AI Server is given as an example. AI Server can include AI model (which can provide reasoning service), training set and training module (which can provide online training service), etc. AIServer can exchange data with AI Custom or other devices through its service interface (API, Application Programming Interface).

[0072] For example, AI Custom can submit samples such as audio / video samples to AI Server and wait for AI Server to return the inference decision results of its AI model for the submitted audio / video samples. Attackers may attack AI clients, AI servers, or transit network devices between AI clients and AI servers or third-party databases used to store samples.

[0073] The inventors of this application have discovered that common AI security-related attack methods include the following categories: evasion attacks, bait attacks, backdoor attacks, and theft attacks.

[0074] Evasion attack: refers to the attacker modifying the input to deceive the AI ​​model into making wrong decisions. This attack mainly occurs in the model reasoning stage; for example, adversarial samples are constructed in the digital world or the physical world. These samples are usually difficult for humans to detect, and the small perturbations in the samples are used to evade the detection of the AI ​​model and mislead it to make wrong reasoning decisions.

[0075] Drug bait attack: In the scenario of retraining AI models, attackers can inject training samples with specific noise pollution into the AI ​​model, causing the trained AI model to fail to work properly. This attack occurs during the model training phase; for example, if an attacker adds a small amount of malicious data to the health database, it will cause significant changes in the dosage recommendations for half of the patients.

[0076] Backdoor attack: Similar to traditional methods, the attacker implants a backdoor in the AI ​​model, making the AI ​​model's judgment of special inputs controlled by the attacker. This attack often occurs during the model generation or transmission stage; since AI models such as neural network models are mainly composed of parameters and network structures and lack interpretability, backdoors in AI models are more difficult to detect.

[0077] Theft attack: Using the service interface provided by the AI ​​model, the attacker infers the model parameters or information in the training data through multiple queries and combined input and output, resulting in the theft of the model itself or private data. The stolen model not only causes intellectual property damage, but also leads to more serious consequences such as evasion attacks.

[0078] AI security technology is in its infancy. Currently, related research is mainly concentrated in academia, mostly focusing on technical exploration of specific business scenarios, and there is still a lack of mature or unified solutions. The inventor of this application has summarized and found that AI security-related defense measures can include the following categories: network distillation technology, adversarial training technology, adversarial sample detection technology, input reconstruction technology, and differential privacy technology.

[0079] Network distillation technology: Multiple models are connected in series during the training phase. This knowledge transfer and sharing can enhance the robustness of the model and partially offset the impact of small perturbations on the model.

[0080] Adversarial training technology: Add generated adversarial samples to the training set, enhance the robustness of the model through retraining, and prevent specific types of adversarial sample attacks in advance.

[0081] Adversarial sample detection technology: Add a detection module in the inference stage to determine whether the model input is an adversarial sample. The judgment rules can be sample distribution, sample historical input characteristics, feature differences between input samples and normal samples, etc.

[0082] Input reconstruction technology: For example, in the inference stage, the input samples are transformed in some way to offset the adversarial sample attack. This transformation must not affect the normal function of the model. Common transformation methods include noise addition and denoising.

[0083] Differential privacy technology: Use differential privacy methods to add noise to data or training steps during the training phase to protect user privacy information potentially contained in the published data, such as using crowdsourcing privacy protection crowdsourcing technology with random responses and homomorphic encryption technology.

[0084] AI technology uses the structure and depth of complex networks to gain powerful learning capabilities, but it also introduces the unexplainability of the model and the one-sidedness of the learned knowledge. Based on these drawbacks, attackers carefully construct samples to discover model vulnerabilities and trick the model into making wrong decisions; in addition, the unexplainability of the model may also make it difficult to detect attacks such as adversarial sample attacks and backdoor attacks.

[0085] Let's explore some possible attack defense solutions.

[0086] See also Figure 2-A , Figure 2-A An example is given of a possible implementation of the input reconstruction technology, in which a Gaussian blur algorithm, a median blur algorithm, a color depth bit quantization algorithm, etc. are executed during the filtering of the input samples, thereby making certain changes to the input samples in order to offset the adversarial sample attack. However, this technology of input sample reconstruction through filtering may affect the user's subjective perception due to excessive changes to the samples.

[0087] The inventors of this application have found that constructing adversarial samples is the biggest attack method currently facing AI security. Adversarial samples are AI model inputs designed by the attacker with the purpose of causing the AI ​​model to output incorrectly. So far, adversarial samples have been most widely studied in the field of images. In the field of images, adversarial samples can be constructed by slight modifications to the image, which leads to misclassification, and adversarial samples are very practical in the real world. Adversarial samples in the field of images are difficult for humans to distinguish, but adversarial samples in the field of speech recognition are often perceptible. However, in recent years, adversarial samples generated by new generation methods in the field of speech have also become difficult to detect. Some researchers have used improved methods to generate imperceptible adversarial samples that are comparable to image adversarial samples. Therefore, audio, images, and videos are facing increasingly severe adversarial sample attack risks, which are often difficult to be detected and can be generated silently in real environments (such as adding disturbances of specific sounds by interfering with a sound source, or modifying certain pixels during image and video acquisition), so there is an urgent need to study new defense methods.

[0088] Research has found that adversarial samples are generally small perturbations added to normal samples, so that the difference is not easily noticed by humans, but can fool the AI ​​model. The generation of adversarial samples is to change the input samples to make the existing model output wrong decisions under the premise of small subjective differences. Generally, the gradient method is used to solve the gradient of the loss function in the sample space. From a certain perspective, adversarial samples themselves are fragile.

[0089] Based on this discovery, the inventor of this application proposed that the fragile structure of the adversarial sample can be destroyed by actively injecting a disturbance component into the sample, while minimizing the impact on the subjective perception and model accuracy of the audio and video sample. The masking effect in visual and auditory perception provides a theoretical basis for this injection method, which helps to ensure that the difference in audio and video perception before and after the injection is not detected. The masking effect refers to the inability of the subjective subject to fully receive all the stimulus information due to the presence of multiple similar stimuli.

[0090] For example Figure 2-BIn the example shown, the minimum sound intensity at which a single pure tone of 0.47kHz can be heard is 8dB; if a pure tone of 0.24kHz frequency and 60dB sound intensity is added at the same time, the previous 0.47kHz pure tone cannot be heard; if these two pure tones are to be heard at the same time, the sound intensity of 0.47kHz usually needs to be increased to more than 29dB. This phenomenon of threshold increase comes from the masking of weak sounds by strong sounds of different frequencies. Sound signals not only have frequency domain masking, but also time domain masking. For image stimuli, there are similar brightness masking and pattern masking. Using the masking effect, as long as a small perturbation is added to the frequency domain / time domain / pattern channel of the weak stimulus, and it is still below the masking threshold after adding the weak stimulus component (that is, after adding the weak stimulus component, the masking state of the channel is still not changed), the added perturbation will not be noticed by people. In addition, since the perturbation amount of the adversarial sample is carefully designed, for example, a perturbation of a specific amplitude needs to be injected at a specific position and frequency. On the contrary, if a sufficient amount of perturbation that conforms to the masking effect is injected into a wider range of perceptual channels and time / spatial locations, it is expected that an effect that can destroy the fragile structure of the adversarial sample can be achieved.

[0091] The following specifically discusses some defense solutions for adversarial samples based on the masking effect.

[0092] See also Figure 3 , Figure 3 : is a flow chart of a sample processing method provided in an embodiment of the present application. A sample processing method may include:

[0093] 301. Obtain a first feature space sample.

[0094] The first feature space sample may be obtained by locally generating it or receiving it from other devices (such as an AI client, etc.).

[0095] 302. Perform feature space transformation on the first feature space sample to obtain a second feature space sample.

[0096] The feature space may also be referred to as a feature domain. It can be understood that the first feature space sample represents a sample in the first feature space, and the second feature space sample represents a sample in the second feature space.

[0097] The second feature space is different from the first feature space. The first feature space may be, for example, one of the following feature spaces: frequency domain space, time domain space, brightness space, pattern space or texture space. The second feature space may be, for example, one of the following feature spaces: frequency domain space, time domain space, brightness space, pattern space or texture space.

[0098] Among them, the samples mentioned in the embodiments of the present application may be, for example, audio samples, video samples, picture samples, or other types of samples.

[0099] Assuming that the first feature space is the time domain space and the first feature space is the frequency domain space, then the first feature space samples are transformed into the second feature space samples, specifically, the samples in the time domain space are transformed into the Fourier transform (time-frequency transform) to obtain the samples in the frequency domain space. Assuming that the first feature space is the frequency domain space and the first feature space is the time domain space, then the first feature space samples are transformed into the second feature space samples, specifically, the samples in the frequency domain space are transformed into the inverse Fourier transform (frequency-time transform) to obtain the samples in the time domain space. The transformation of other samples between feature spaces is analogous.

[0100] 303. Perform disturbance injection of a stimulus component (stimulus) on the second feature space sample to obtain a second feature space sample after disturbance injection.

[0101] The disturbance injection does not change the masking state of the second feature space samples.

[0102] Among them, a sample in a certain feature space can be divided into multiple channels, also called perception channels, and the user's perception of different channels may be different.

[0103] Among them, the masking state of the second feature space sample specifically refers to the masking state of each channel of the second feature space sample. The masking state of a channel can be "masked" or "unmasked". A channel whose masking state is masked can be called a masking channel, wherein a masking channel is also called a masked channel or a suppressed channel or an imperceptible channel. A channel whose masking state is unmasked can be called an unmasked channel or an unsuppressed channel or a perceptible channel.

[0104] Among them, not changing the masking state of the second feature space sample specifically refers to not changing the masking state of each channel of the second feature space sample. For example, the masked channel is still a masked channel before and after the disturbance injection, that is, the masking state of the channel remains masked before and after the disturbance injection. The stimulus component in the masked channel is imperceptible to the user before and after the disturbance injection.

[0105] 304. Perform an inverse feature space transformation on the second feature space samples injected with disturbance to obtain inversely transformed first feature space samples.

[0106] 305. Send or store the inverse transformed first feature space samples.

[0107] Among them, the sample processing method of the embodiment of the present application can be applied to an AI client, an AI server, a third-party sample database, or a transit network device between an AI client and an AI server, etc. When the sample processing method is applied to an AI client, the AI ​​client can send the inverse transformed first feature space sample to the AI ​​server after obtaining the inverse transformed first feature space sample. When the sample processing method is applied to an AI server, the AI ​​server can store the inverse transformed first feature space sample (for example, it can be stored in a training set or other location) after obtaining the inverse transformed first feature space sample, so that the inverse transformed first feature space sample can be input into the AI ​​model when needed later. When the sample processing method is applied to a third-party sample database, the third-party sample database can store the inverse transformed first feature space sample, so that the stored inverse transformed first feature space sample can be provided to the AI ​​server, etc. when needed.

[0108] For example, performing a disturbance injection of a stimulus component on the second feature space sample to obtain a second feature space sample with a disturbance injection may include: determining N masking channels of the second feature space sample; performing a disturbance injection of a stimulus component on M of the N masking channels, wherein N is greater than or equal to M, and the masking states of the M masking channels are not changed by the disturbance injection.

[0109] Here, N may be equal to 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 20, 32, 40, 64, 128, 256, 300 or other values, for example.

[0110] Here, M may be equal to 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 19, 32, 40, 64, 128, 253, 300 or other values, for example.

[0111] The M masking channels are, for example, randomly or non-randomly selected from the N masking channels.

[0112] For example, determining the N masking channels of the second feature space sample includes: calculating the masking thresholds of X channels of the second feature space sample (X is, for example, the total number of channels of the second feature space sample, and the masking thresholds of different channels may be the same or different); selecting N masking channels from the X channels based on the masking thresholds and stimulation components of the X channels (N is, for example, the total number of masking channels of the second feature space sample), the stimulation component of masking channel i is less than or equal to the masking threshold of masking channel i, and the masking channel i is any one of the N masking channels.

[0113] In some possible implementations, the stimulation component perturbation injected into the masking channel i among the M masking channels may be obtained based on a random algorithm, or the stimulation component perturbation injected into the masking channel i among the M masking channels may be obtained based on a preset sequence (the preset sequence is, for example, a digital watermark).

[0114] The stimulus component disturbance injected into the masking channel i among the M masking channels is a positive stimulus component disturbance or a negative stimulus component disturbance. It can be understood that injecting a positive stimulus component disturbance increases the stimulus component, and injecting a negative stimulus component disturbance decreases the stimulus component.

[0115] It can be seen that the above sample processing method is a sample processing mechanism based on the masking effect. Since the perturbation injection does not change the masking state of the sample, it can destroy the fragile structure of the adversarial sample in a hidden state that the user is not aware of, which is conducive to achieving a good attack prevention effect without affecting the user's perceived experience. In practice, it has been found that this sample processing mechanism based on the masking effect can effectively prevent dodge attacks, theft attacks, and bait attacks based on adversarial samples. For AI input samples such as audio / picture / video samples, the sample processing mechanism based on the masking effect is conducive to not introducing perceptual damage to the sample, which is conducive to improving the multi-scenario adaptability of the solution.

[0116] In addition, by calculating the masking threshold of each channel of the sample, it is easy to make flexible choices of the adjustment amplitude and adjustment range under the constraint of the masking threshold, which is conducive to increasing the difficulty for attackers to speculate on the disturbance injection law; further, when using digital watermarks to obtain stimulus component disturbances, the use of digital watermarks can detect whether the sample is credible and whether it has been tampered with, which is conducive to achieving active prevention.

[0117] See also Figure 4 , Figure 4 It is a flowchart of another sample processing method provided in an embodiment of the present application. Figure 4 Example solutions and Figure 3 The main difference between the example schemes is that Figure 4 The example solution does not require transforming the feature space of the sample. Another sample processing method may include:

[0118] 401. Obtain a first feature space sample.

[0119] The first feature space sample may be obtained by locally generating it or receiving it from other devices (such as an AI client, etc.).

[0120] The first feature space may be, for example, one of the following feature spaces: frequency domain space, time domain space, brightness space, pattern space or texture space.

[0121] 402. Perform stimulus component disturbance injection on the first feature space sample to obtain a first feature space sample after disturbance injection. The disturbance injection does not change the masking state of the first feature space sample.

[0122] 403. Send or store the disturbance-injected first feature space sample.

[0123] Among them, a sample in a certain feature space can be divided into multiple channels, also called perception channels, and the user's perception of different channels may be different.

[0124] The masking state of the first feature space sample specifically refers to the masking state of each channel of the first feature space sample. The masking state of a channel can be "masked" or "unmasked". A channel whose masking state is masked can be called a masking channel, and a masking channel is also called a masked channel or an imperceptible channel. A channel whose masking state is unmasked can be called an unmasked channel or a perceptible channel.

[0125] Among them, not changing the masking state of the first feature space sample specifically refers to not changing the masking state of each channel of the first feature space sample. For example, the masked channel is still a masked channel before and after the disturbance injection, that is, the masking state of the channel remains masked before and after the disturbance injection. The stimulus component in the masked channel is not perceptible to the user before and after the disturbance injection.

[0126] Among them, the sample processing method of the embodiment of the present application can be applied to an AI client, an AI server, a third-party sample database, or a transit network device between an AI client and an AI server, etc. When the sample processing method is applied to an AI client, the AI ​​client can send the first feature space sample injected with disturbance to the AI ​​server after obtaining the first feature space sample injected with disturbance. When the sample processing method is applied to an AI server, the AI ​​server can store the first feature space sample injected with disturbance (for example, it can be stored in a training set or other location) after obtaining the first feature space sample injected with disturbance, so that the first feature space sample injected with disturbance can be input into the AI ​​model when needed later. When the sample processing method is applied to a third-party sample database, the third-party sample database can store the first feature space sample injected with disturbance, so that the stored first feature space sample injected with disturbance can be provided to the AI ​​server, etc. when needed.

[0127] For example, performing a disturbance injection of a stimulus component on a first feature space sample to obtain a first feature space sample with disturbance injection may include: determining N masking channels of the first feature space sample; performing a disturbance injection of the stimulus component on M of the N masking channels, wherein N is greater than or equal to M, and the masking states of the M masking channels are not changed by the disturbance injection.

[0128] Here, N may be equal to 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 20, 32, 40, 64, 128, 256, 300 or other values, for example.

[0129] Here, M may be equal to 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 19, 32, 40, 64, 128, 253, 300 or other values, for example.

[0130] The M masking channels are, for example, randomly or non-randomly selected from the N masking channels.

[0131] For example, determining the N masking channels of the first feature space sample includes: calculating the masking thresholds of X channels of the first feature space sample (X is, for example, the total number of channels of the first feature space sample, and the masking thresholds of different channels may be the same or different); selecting N masking channels from the X channels based on the masking thresholds and stimulation components of the X channels (N is, for example, the total number of masking channels of the first feature space sample), the stimulation component of masking channel i is less than or equal to the masking threshold of masking channel i, and the masking channel i is any one of the N masking channels.

[0132] In some possible implementations, the stimulation component perturbation injected into the masking channel i among the M masking channels may be obtained based on a random algorithm, or the stimulation component perturbation injected into the masking channel i among the M masking channels may be obtained based on a preset sequence (the preset sequence is, for example, a digital watermark).

[0133] The stimulus component disturbance injected into the masking channel i among the M masking channels is a positive stimulus component disturbance or a negative stimulus component disturbance. It can be understood that injecting a positive stimulus component disturbance increases the stimulus component, and injecting a negative stimulus component disturbance decreases the stimulus component.

[0134] It can be seen that the above sample processing method is a sample processing mechanism based on the masking effect. Since the perturbation injection does not change the masking state of the sample, it can destroy the fragile structure of the adversarial sample in a hidden state that the user is not aware of, which is conducive to achieving a good attack prevention effect without affecting the user's perceived experience. In practice, it has been found that this sample processing mechanism based on the masking effect can effectively prevent dodge attacks, theft attacks, and bait attacks based on adversarial samples. For AI input samples such as audio / picture / video samples, the sample processing mechanism based on the masking effect is conducive to not introducing perceptual damage to the sample, which is conducive to improving the multi-scenario universality of the solution.

[0135] In addition, by calculating the masking threshold of each channel of the sample, it is easy to make flexible choices of the adjustment amplitude and adjustment range under the constraint of the masking threshold, which is conducive to increasing the difficulty for attackers to speculate on the disturbance injection law; further, when using digital watermarks to obtain stimulus component disturbances, the use of digital watermarks can detect whether the sample is credible and whether it has been tampered with, which is conducive to achieving active prevention.

[0136] See also Figure 5-A , Figure 5-A In the example, the audio sample is taken as an example, and the disturbance of the stimulus component is injected after the audio sample is transformed from the time domain space to the frequency domain space.

[0137] In the specific implementation, the original audio sample is first transformed from the time domain space to the frequency domain space using Fourier transform. Then the masking state and masking threshold of each channel (here refers to the frequency band) are calculated in the frequency domain space. For example, see Figure 5-B , Figure 5-B The fluctuation curve in represents the spectrum of the audio sample (frequency vs. sound pressure amplitude), and the stepped line represents the masking threshold corresponding to each frequency band. The frequency position of the spectrum curve above the threshold line corresponds to the unmasked frequency band (perceptible channel), which can be perceived by hearing; on the contrary, the frequency position below the threshold line corresponds to the masked frequency band (masked channel), which cannot be perceived by hearing because it is suppressed by other frequency bands.

[0138] One or more suppressed frequency bands (partial masking channels) are randomly selected, and disturbances are injected into the sound pressure amplitudes of these frequency bands in a certain manner, so that the sound pressure amplitude after the injection of disturbances is still lower than the masking threshold. The manner of injecting disturbances can, for example, follow a certain probability distribution, such as Gaussian distribution, uniform distribution, etc.; the injected disturbances can be disturbances with specific physical meanings, such as digital watermarks, etc.

[0139] Through the inverse Fourier transform, the frequency domain spatial sample injected with disturbance is transformed into the time domain spatial sample. The time domain spatial sample obtained by this transformation is a sample that can resist adversarial attacks. Figure 5-C , Figure 5-CAn example of the effect of perturbation injection on audio samples based on the masking effect is shown. After the perturbation injection, the audio samples do not change their masking state, so the user cannot perceive these perturbation injections, which does not affect the user's audio perception experience.

[0140] in, Figure 5-A What is shown is the frequency domain masking of audio samples. Of course, masking of other feature spaces of audio samples can also be performed. For image samples, frequency domain masking / texture masking of image samples can be performed. For video samples, brightness masking / texture masking of video samples can be performed. For specific masking methods, please refer to the frequency domain masking mechanism of audio samples, which will not be described here one by one.

[0141] In some specific products, a software functional device may be used to implement the sample processing method exemplified above. This software functional device may be called, for example, a mask based positive perturbation injection (MPI) module. The MPI module may be used to perform user-imperceptible perturbation injection on the input audio / video / image samples, and then output the audio / video / image samples after the perturbation injection.

[0142] See also Figure 6-A to Figure 6-C , Figure 6-A to Figure 6-C Several improved AI system architectures are given as examples. The MPI module can be flexibly integrated in multiple optional locations within the AI ​​system.

[0143] For example Figure 6-A As shown in the example, the MPI module can be integrated into AI Custom, that is, AI Custom implements the sample processing method in the above example. Before submitting the input sample to AI Server, the MPI module in AI Custom can first perform positive perturbation injection on the sample.

[0144] Another example Figure 6-B As shown in the example, the MPI module can be integrated into the AI ​​Server, that is, the AI ​​Server implements the sample processing method in the above example. After the AI ​​Server receives the input sample through the API, the MPI module can first inject a positive disturbance into the sample, and then submit the sample with the disturbance injection to the subsequent module for further processing (such as submitting it to the AI ​​model, training set, online training module, etc.).

[0145] Another example Figure 6-CAs shown in the example, the MPI module can be integrated into the third-party sample database, that is, the third-party sample database can be used to implement the sample processing method in the above example. After AI Custom submits the input sample, the input sample can be redirected to the third-party sample database for forward perturbation injection, and then the sample with perturbation injection can be submitted to AIServer.

[0146] The following is an example of a counter-attack detection scenario.

[0147] In the smart home scenario, an AI server in the network provides AI services to an AI client, and the AI ​​server can return inference results to guide the AI ​​client to perform operations. An attacker may tamper with the samples submitted by the AI ​​client and add content that the user cannot perceive, fooling the AI ​​server into making incorrect inference results.

[0148] See also Figure 7 For this scenario, the sample processing solution described above is applied in AI Client (i.e., the MPI module is integrated into AI Custom). Before submitting the audio or video sample, AI Client injects a positive disturbance containing a digital watermark into the audio or video sample. Taking audio samples as an example, the situation of injecting a digital watermark into an audio sample based on the masking effect can be as follows: Figure 8 The comparison of the audio time domain sound pressure before and after the digital watermark is injected into the audio sample can be shown as follows: Fig. 9 shown. Fig. 9 The signal graph shown in (a) represents the original audio sample, (b) represents the audio sample after the digital watermark is injected, and (c) represents the digital watermark. Fig. 9 It can be seen that the audio samples before and after the digital watermark is injected do not change much, and users cannot perceive the difference.

[0149] After the AI ​​server obtains the input sample, it can compare the digital watermark in the input sample to verify the integrity of the sample. If the sample has not been tampered with and the identity is credible, the sample is submitted to the AI ​​model for decision making, or submitted to the training set for online training; otherwise, for example, an alarm may be generated and the service may be rejected.

[0150] The following is another example of a scenario that can resolve adversarial sample attacks.

[0151] In an end-to-end Internet communication scenario, the AI ​​Client and the AI ​​Server are two communicating parties, and a certain intermediate communication server (such as an instant messaging server) provides AI services and communication relay for the AI ​​Client. During the communication process, attackers may tamper with the communication object, construct adversarial samples, etc., in order to fool the AI ​​Server into making incorrect inferences (such as making the expected voice / video enhancement function invalid). Typical scenarios include AI security issues for audio and video samples that real-time communication (RTC) systems may face.

[0152] For this application scenario, see Fig.10 , the sample processing solution in the above example can be applied in AI Server (i.e., the MPI module is integrated into the AI ​​Server). After receiving the input audio and video samples, the MPI module injects positive disturbances into the samples, destroys the malicious disturbances in the potential adversarial samples, and passes the samples injected with disturbances to the AIModel or training set for reasoning or online training. Since this disturbance injection operation has no effect on the user's audio and video perception, the MPI module will not cause damage to audio and video communication and AI reasoning.

[0153] The following is an example of a model stealing defense scenario.

[0154] by Fig.11 For example, an AI Server that provides AI reasoning query functions to the outside world. An attacker can use the reasoning results returned by this AI Server through multiple queries to train the local AI model. After multiple trainings, the local AI model can replace the reasoning function of the AI ​​model in the online AI Server, thereby achieving the purpose of stealing the AI ​​model.

[0155] For this application scenario, see Fig.11 The sample processing solution mentioned above can be applied in the AI ​​Server (i.e., the MPI module is integrated into the AI ​​Server). The MPI module can inject positive disturbances into the input samples before the AI ​​model in the AI ​​Server obtains the input samples. The disturbance injection satisfies the requirements of imperceptibility and randomness of the injection pattern, so that the attacker cannot obtain the actual input of the online AI model, making it difficult for the attacker's local training to infer the online AI model, which is conducive to reducing the risk of model theft.

[0156] It can be seen that in the several application scenarios given as examples, implementing perturbation injection based on the masking effect on samples can effectively resist adversarial attacks without affecting the user's perception of the samples.

[0157] Some device embodiments are also provided below.

[0158] See also Fig.12 , the present application embodiment provides a sample processing device 1200, which may include:

[0159] The acquisition unit 1210 is used to acquire a first feature space sample.

[0160] The feature space transformation unit 1220 is configured to perform feature space transformation on the first feature space samples to obtain second feature space samples, where the second feature space is different from the first feature space.

[0161] The disturbance injection unit 1230 is used to perform disturbance injection of a stimulus component on the second feature space sample to obtain a second feature space sample after disturbance injection, wherein the disturbance injection does not change the masking state of the second feature space sample.

[0162] The feature space inverse transformation unit 1240 is used to perform feature space inverse transformation on the second feature space samples injected with disturbance to obtain inverse transformed first feature space samples.

[0163] The output unit 1250 is configured to send or store the inverse transformed first feature space samples.

[0164] In some possible implementations, the disturbance injection unit 1230 may be specifically used to determine N masking channels of the second feature space samples; perform disturbance injection of stimulation components on M masking channels among the N masking channels, where N is greater than or equal to M, and the masking states of the M masking channels are not changed by the disturbance injection.

[0165] For example, determining the N masking channels of the second feature space sample includes: calculating the masking thresholds of X channels of the second feature space sample (X is, for example, the total number of channels of the second feature space sample, and the masking thresholds of different channels may be the same or different); selecting N masking channels from the X channels based on the masking thresholds and stimulation components of the X channels (N is, for example, the total number of masking channels of the second feature space sample), the stimulation component of masking channel i is less than or equal to the masking threshold of masking channel i, and the masking channel i is any one of the N masking channels.

[0166] It is understood that the functions of the various functional modules of the sample processing device 1200 may be based on the attached Figure 3 The specific implementation is based on the relevant description of the corresponding embodiment, which will not be repeated here.

[0167] See also Fig.13 , the present application embodiment provides a sample processing device 1300, including:

[0168] The acquisition unit 1310 is configured to acquire a first feature space sample.

[0169] The disturbance injection unit 1320 is used to perform a stimulus component disturbance injection on the first feature space sample to obtain a first feature space sample after the disturbance injection, wherein the disturbance injection does not change the masking state of the first feature space sample.

[0170] The output unit 1330 sends or stores the first feature space sample after the disturbance injection.

[0171] In some possible implementations, the disturbance injection unit 1320 is specifically used to determine N masking channels of the first feature space samples; perform disturbance injection of stimulation components on M masking channels among the N masking channels, where N is greater than or equal to M, and the masking states of the M masking channels are not changed by the disturbance injection.

[0172] For example, determining the N masking channels of the first feature space sample includes: calculating the masking thresholds of X channels of the first feature space sample (X is, for example, the total number of channels of the first feature space sample, and the masking thresholds of different channels may be the same or different); selecting N masking channels from the X channels based on the masking thresholds and stimulation components of the X channels (N is, for example, the total number of masking channels of the first feature space sample), the stimulation component of masking channel i is less than or equal to the masking threshold of masking channel i, and the masking channel i is any one of the N masking channels.

[0173] It is understood that the functions of the various functional modules of the sample processing device 1300 may be based on the attached Figure 3 The specific implementation is based on the relevant description of the corresponding embodiment, which will not be repeated here.

[0174] See also Fig.14 The embodiment of the present application also provides a sample processing device 1400, which can implement some or all of the functions of the AI ​​client, AI server or third-party sample database provided in the above embodiments. The sample processing device 1400 specifically includes: a processor 1410 and a memory 1420 coupled to each other.

[0175] Among them, the processor 1410 can be used to call the computer program stored in the memory 1420 to complete part or all of the steps of any method that can be executed by the AI ​​client, AI server or third-party sample database or MPI module in the embodiment of the present application.

[0176] Among them, the processor 1410 is also called a central processing unit (CPU). In a specific application, the components of the image prediction device are coupled together, for example, through a bus system. Among them, the bus system may include a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, various buses are marked as bus system 1430 in the figure. The method disclosed in the above embodiment of the present application can be applied to the processor 1410, or implemented by the processor 1410. The processor 1410 may be an integrated circuit chip with signal processing capabilities. In some implementation processes, some or all of the steps of the above method can be completed by hardware integrated logic circuits or software instructions in the processor 1410. The processor 1410 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a ready-made programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The processor 1410 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor 1410 may be a microprocessor or the processor may also be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory or a register. The storage medium is located in the memory 1420, for example, the processor 1410 can read the information in the memory 1420, and complete some or all of the steps of the above method in combination with its hardware.

[0177] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program is executed by hardware (such as a processor, etc.) to perform part or all of the steps of any method performed by any device in an embodiment of the present application.

[0178] The embodiment of the present application also provides a computer program product including instructions, and when the computer program product is run on a computer device, the computer device is caused to execute part or all of the steps of any one of the above methods.

[0179] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a service end, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, service end, or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a service end, a data center, etc. that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., an optical disk), or a semiconductor medium (e.g., a solid-state hard disk), etc. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0180] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0181] In the several embodiments provided in the present application, it should be understood that the disclosed device can also be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the indirect coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0182] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0184] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium may include, for example: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

Claims

1. A sample processing method, characterized in that: include: Obtaining a first feature space sample; Performing feature space transformation on the first feature space samples to obtain second feature space samples, where the second feature space is different from the first feature space; Performing disturbance injection of a stimulus component on the second feature space sample to obtain a second feature space sample after disturbance injection, wherein the disturbance injection does not change a masking state of the second feature space sample; Performing an inverse feature space transformation on the second feature space samples injected with the disturbance to obtain an inversely transformed first feature space sample; sending or storing the inverse transformed first feature space samples, Wherein, the first feature space sample is an audio sample, a video sample or a picture sample; The step of injecting a disturbance of a stimulus component into the second feature space sample to obtain a second feature space sample after the disturbance injection comprises: Determine N masking channels of the second feature space sample; perform disturbance injection of stimulus components on M masking channels among the N masking channels, wherein N is greater than or equal to M, and the masking states of the M masking channels are not changed by the disturbance injection; in, The stimulation component perturbation injected into the masking channel i among the M masking channels is obtained based on a random algorithm; The perturbation injection of the stimulus component is used to destroy the structure of the adversarial sample.

2. The method according to claim 1, characterized in that The determining of the N masking channels of the second feature space samples comprises: Calculating masking thresholds of X channels of the second feature space sample; selecting N masking channels from the X channels based on the masking thresholds and stimulus components of the X channels, wherein the stimulus component of masking channel i is less than or equal to the masking threshold of masking channel i, and the masking channel i is any one of the N masking channels.

3. The method according to claim 1 or 2, characterized in that: The M masking channels are randomly selected from the N masking channels.

4. The method according to any one of claims 1 to 2, characterized in that: The stimulation component perturbation injected into the masking channel i among the M masking channels is a positive stimulation component perturbation or a negative stimulation component perturbation.

5. The method according to claim 3, characterized in that: The stimulation component perturbation injected into the masking channel i among the M masking channels is a positive stimulation component perturbation or a negative stimulation component perturbation.

6. The method according to any one of claims 1 to 2, characterized in that: The second feature space is one of the following feature spaces: frequency domain space, time domain space, brightness space, pattern space or texture space.

7. The method according to claim 3, characterized in that The second feature space is one of the following feature spaces: frequency domain space, time domain space, brightness space, pattern space or texture space.

8. A sample processing method, characterized in that: include: Obtaining a first feature space sample; Performing stimulation component disturbance injection on the first feature space sample to obtain a first feature space sample after disturbance injection, wherein the disturbance injection does not change the masking state of the first feature space sample; Sending or storing the disturbance-injected first feature space sample; Wherein, the first feature space sample is an audio sample, a video sample or a picture sample; The step of injecting a stimulus component disturbance into the first feature space sample to obtain a first feature space sample after the disturbance injection comprises: Determine N masking channels of the first feature space sample; perform disturbance injection of stimulus components into M masking channels among the N masking channels, wherein the masking states of the M masking channels are not changed by the disturbance injection, wherein N and M are positive integers, and N is greater than or equal to M; in, The stimulation component perturbation injected into the masking channel i among the M masking channels is obtained based on a random algorithm; The perturbation injection of the stimulus component is used to destroy the structure of the adversarial sample.

9. A sample processing device, characterized in that: include: A processor and memory coupled to each other; The processor is used to call the computer program stored in the memory to execute the method described in any one of claims 1 to 7 or 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to implement the method of any one of claims 1 to 7 or 8 when executed by hardware.

Images