A sample processing method based on deep learning and related equipment

Abstract

The embodiment of the present application provides a sample processing method based on deep learning and related equipment for filtering backdoor samples in deep learning models. The method of the embodiment of the present application includes: obtaining key feature information of the sample to be processed; wherein the key feature information is used to describe the feature information of the key position point in the sample to be processed; determining the target feature information located at the identification position point in the key feature information of the key position point; wherein the target feature information is the feature information in the key feature information used to identify the sample to be processed; inputting the target feature information to the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted; wherein the target position point is the position point corresponding to the identification position point.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of information security technology, and in particular to a sample processing method based on deep learning and related equipment. Background Art

[0002] With the widespread application of deep learning in the fields of medicine and autonomous driving, the security issues of deep learning models have also attracted people's attention. During the training phase of deep learning, attackers add triggers to some training samples and change their classification labels to target category labels, so that the trained deep neural network is implanted with a "backdoor".

[0003] During the testing phase, if the samples to be predicted are given the same trigger, the deep neural network model will output the target category instead of the true classification label of the sample. This is a backdoor attack. When there are not enough computing resources to train the deep neural network model, customers often hand over the model training task to a third party. If a malicious third party acts as an attacker and implants a backdoor into the model, when the model is used to predict samples with triggers, the model will give the target category specified by the attacker as a prediction. The system based on the deep neural network model will make wrong judgments, seriously affecting the security of the system application.

[0004] The triggers contained in the backdoor samples have a great influence. If they are found and pasted onto other clean images, these clean samples with pasted triggers will also be misclassified as target classes by the model. Summary of the invention

[0005] The embodiments of the present application provide a deep learning-based sample processing method and related equipment for filtering backdoor samples in a deep learning model.

[0006] A first aspect of an embodiment of the present application provides a sample processing method based on deep learning, comprising:

[0007] Acquire key feature information of the sample to be processed; wherein the key feature information is used to describe feature information at key positions in the sample to be processed;

[0008] Determine the target feature information located at the identification position point in the key feature information of the key position point; wherein the target feature information is the feature information in the key feature information used to identify the sample to be processed;

[0009] The target feature information is input into the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted; wherein the target position point is the position point corresponding to the identification position point.

[0010] Optionally, obtaining key feature information of the sample to be processed includes:

[0011] Obtaining position feature information of all position points in the sample to be processed;

[0012] Determine the representation parameter information of the position feature information of all the position points;

[0013] The target parameter information in the characterization parameter information is determined, and the position point corresponding to the target parameter information is determined as the key position point, and the feature information corresponding to the key position point is determined as the key feature information.

[0014] Optionally, in the key feature information for determining the key position point, the target feature information located at the identification position point includes:

[0015] Acquire all the position points to be identified in the key position points, and the feature information to be identified corresponding to the position points to be identified; wherein the key feature information includes all the feature information to be identified;

[0016] Selecting a first position point and a second position point from all the position points to be identified, and determining difference information of first feature information and second feature information respectively corresponding to the first position point and the second position point; wherein the first position point and the second position point are any two position points from all the position points to be identified;

[0017] According to the difference information, the feature information to be identified of the third position point is mutated to obtain the third feature information corresponding to the third position point; wherein the third position point is any one of all the position points to be identified;

[0018] Cross-value the third feature information and the feature information to be identified corresponding to the third position point to determine the feature information to be selected for the third position point;

[0019] When the feature information to be selected meets the preset fitness information, the third position point is determined as the identification position point, and the feature information to be identified corresponding to the third position point is determined as the target feature information.

[0020] Optionally, before determining the third position point as the identification position point and determining the to-be-identified feature information corresponding to the third position point as the target feature information, the method further includes:

[0021] Determining the color characteristic information corresponding to the third position point in the sample to be processed;

[0022] Determining fitness information corresponding to the third position point according to the chromaticity feature information;

[0023] Inputting the feature information to be selected of the third position point into a clean sample, identifying the classification category label of the clean sample, so as to determine the prediction probability information corresponding to the third position point; wherein the classification category label of the clean sample is a first category label, and the prediction probability information is used to describe the probability information of identifying the classification category label as a second category label;

[0024] The preset fitness information is determined according to the fitness information and the predicted probability information.

[0025] Optionally, the method further comprises:

[0026] When the feature information to be identified at the third position point is mutated, determining a target number of iterations for the third position point;

[0027] When the target iteration number satisfies a preset number threshold, determining the third position point that satisfies the preset fitness information as the identification position point;

[0028] Or, when the target number of iterations meets a preset number threshold and the feature information to be selected does not meet preset fitness information, it is determined that the classification category label of the sample to be predicted is the first category label, and the sample to be predicted is a clean sample.

[0029] Optionally, inputting the target feature information to the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted, includes:

[0030] Acquire multiple groups of the reference samples, and set the position points in the reference samples corresponding to the identified position points as the target position points; wherein the reference samples are clean samples;

[0031] Paste the image feature information corresponding to the target feature information to the target position point to obtain multiple groups of samples to be predicted;

[0032] Determine whether the classification category labels of all the samples to be predicted are the same as the classification category labels of the reference samples;

[0033] If the classification category label of the sample to be predicted is the same as that of the reference sample, determining that the sample to be processed is the clean sample;

[0034] If the classification category labels of the sample to be predicted and the reference sample are different, it is determined that the sample to be predicted is a backdoor sample.

[0035] Optionally, after determining the classification category label of the sample to be predicted, the method further includes:

[0036] If the classification category label is the first category label, it is determined that the sample to be processed is a clean sample.

[0037] If the classification category label is the second category label, it is determined that the target feature information is a trigger and the sample to be processed is a backdoor sample.

[0038] A second aspect of an embodiment of the present application provides a sample processing system based on deep learning, including:

[0039] An acquisition unit, used to acquire key feature information of the sample to be processed; wherein the key feature information is used to describe feature information at key positions in the sample to be processed;

[0040] A determination unit, used to determine target feature information located at an identification position point in the key feature information of the key position point; wherein the target feature information is feature information in the key feature information used to identify the sample to be processed;

[0041] An input unit is used to input the target feature information to the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted; wherein the target position point is the position point corresponding to the identification position point.

[0042] The second aspect of the embodiments of the present application provides a sample processing method based on deep learning for executing the first aspect.

[0043] A third aspect of an embodiment of the present application provides a sample processing device based on deep learning, including:

[0044] CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply;

[0045] The memory is a short-term storage memory or a persistent storage memory;

[0046] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the deep learning-based sample processing method described in the first aspect.

[0047] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium includes instructions, which, when executed on a computer, enable the computer to execute the deep learning-based sample processing method described in the first aspect.

[0048] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: through a sample processing method based on deep learning disclosed in the embodiments of the present application, by obtaining the key feature information of the sample to be processed; wherein the key feature information is used to describe the feature information of the key position point in the sample to be processed; then determine the target feature information located at the identification position point in the key feature information of the key position point; wherein the target feature information is the feature information used to identify the sample to be processed in the key feature information; finally, input the target feature information to the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted; wherein the target position point is the position point corresponding to the identification position point. Thus, the key features of the sample to be processed are combined with the reference sample to determine the classification label of the sample to be predicted, and the classification label of the sample to be processed is correspondingly determined, thereby maintaining the classification accuracy while significantly reducing the success rate of the attack. Correspondingly, the defense method based on sample preprocessing is simple to implement and highly practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0050] Figure 1 A schematic diagram of an existing process of forming a backdoor network model;

[0051] Figure 2 This is a schematic diagram of the prediction of a backdoor network model in an existing backdoor attack;

[0052] Figure 3 A schematic diagram of a flow chart of a sample processing method based on deep learning disclosed in an embodiment of the present application;

[0053] Figure 4 A flowchart of another sample processing method based on deep learning disclosed in an embodiment of the present application;

[0054] Figure 5 A technical flow chart of a sample processing method based on deep learning disclosed in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of the structure of a sample processing system based on deep learning disclosed in an embodiment of the present application;

[0056] Figure 7 This is a schematic diagram of the structure of a sample processing device based on deep learning disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0058] It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0059] See also Figure 1 and Figure 2 , Figure 1 The figure is a flow chart of an existing backdoor network model. Figure 2 This is a schematic diagram of the prediction of the backdoor network model in an existing backdoor attack.

[0060] With the widespread application of deep learning in the fields of medicine and autonomous driving, the security issues of deep learning models have also attracted people's attention. In the training stage of deep learning, the attacker adds triggers to some training samples and changes their classification labels to target category labels, so that the trained deep neural network is equipped with a "backdoor". In the testing stage, if the same trigger is added to the sample to be predicted, the deep neural network model will output the target category instead of the real classification label of the sample. This is a backdoor attack. When there are not enough computing resources to train the deep neural network model, customers often hand over the task of model training to a third party. If there is a malicious third-party organization that implants a backdoor into the model as an attacker, then when the model is used to predict samples with triggers, the model will give the target category specified by the attacker as a prediction. The system based on the deep neural network model will make wrong judgments, which seriously affects the security of the system application. The trigger contained in the backdoor sample has a great influence. If it is found and pasted on other clean images, these clean samples with the trigger pasted will also be misclassified as the target class by the model. The technical solution of this application is mainly used to filter samples with triggers in the testing stage, so that the deep neural network will not output wrong predictions due to backdoor attacks. For example, the true category of a cat is cat. If the attacker wants to attack this image and wants the system to recognize it as a dog, then dog is the target category. In other words, the target category is the category that the backdoor attacker intends to misjudge by the system.

[0061] Specifically, by Figure 1 It can be seen that the current backdoor network model is generally obtained by training clean samples and backdoor samples to obtain the corresponding trained backdoor model. Figure 2 It can be seen that in an actual backdoor attack, different samples are identified and predicted through the backdoor network model, so as to identify the category of the clean sample or the backdoor sample as the true category or the target category.

[0062] In the prior art solution, as a defense method based on sample preprocessing, the STRIP function applies a linear superposition strategy to an input image and a set of clean images, and then determines whether the input is clean or with a trigger by the diversity of the predictions of the superimposed images. Specifically, the predictions of the input samples formed by the superposition of clean samples and clean images tend to be more diverse, while the input formed by the superposition of samples with triggers and clean images is more likely to be judged as the target class due to the influence of the trigger. However, when the fragile trigger is destroyed during the superposition process, the clean and poisonous inputs cannot be completely distinguished. Unlike this method of linearly superimposing pixels, the technical solution of the present application replaces the pixels on the clean image with the pixels of the trigger to ensure the integrity of the trigger.

[0063] For example, in another prior art solution, the Februus technique can be used. Specifically, the gradient weight-based class activation map GradCAM is used, and the influential area is deleted from the input according to the class activation map. Subsequently, it regenerates the area with a generative adversarial network. By indiscriminately deleting and redrawing each input, regardless of whether the input sample is clean or poisonous. Therefore, it is difficult to maintain accuracy on clean samples because the generative adversarial network GAN cannot always accurately restore the removed area. In addition, the threshold of the class activation map directly determines the size of the area to be removed, and the defender usually does not know the size of the trigger area, resulting in a lack of universality of a certain threshold.

[0064] NEO randomly searches for regions on the sample and masks them with the dominant color to see if the region causes a change in prediction, and if so, it could be a trigger or a feature of a clean sample. To confirm if it is a trigger, the region is extracted and placed on clean images to check if the amount of change in prediction in those images exceeds a threshold. However, it is also possible that a feature region of a clean image could exceed the threshold, since the threshold is calculated from the amount of change in prediction caused by a randomly extracted region in the clean image.

[0065] In summary, a good defense method needs to take two points into consideration: (1) not affecting the classification accuracy of the model for clean samples; (2) being able to reduce the attack success rate of backdoor samples. There are many types of existing defense methods against backdoor attacks. Some defense methods are based on the modification of model parameters or structures, such as Neural Cleanse and fine-tuning. These methods are often more complex and require large computing resources, which are conditions that many customers do not have. This type of defense is not convenient for practical application. Another part of the defense method is based on sample filtering, that is, to identify and judge the samples input into the model, find backdoor samples with triggers and prevent them from entering the model. This type of defense method is more convenient for defenders with limited resources.

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] To facilitate the solution of the above mentioned technical problems, please refer to Figure 3 , Figure 3 This is a flowchart of a sample processing method based on deep learning disclosed in an embodiment of the present application, including steps 301 to 303.

[0068] 301. Obtain key feature information of the sample to be processed.

[0069] In order to identify the sample, it is first necessary to obtain the key feature information of the sample to be processed. Among them, the key feature information is mainly used to describe the feature information of the key position points in the sample to be processed. It should be noted that the key position point can be a specific pixel position point or a specific area, which is not limited here. The corresponding sample to be processed can be expressed in the form of a picture image or a video image, etc., which is not limited here. For the convenience of understanding and description, the subsequent understanding of the sample to be processed can be understood by picture images.

[0070] In one specific embodiment, in the model prediction stage, the sample needs to be input into the model for running. Specifically, GradCAM is used to roughly find the key positions that affect image classification. GradCAM is used to mark the importance of each position on the sample to be processed (ranging from 0 to 1), so that the most important position is used as the key position of the sample to be processed, thereby obtaining the key feature information of the key position.

[0071] Based on the above embodiment, in another specific embodiment, the key position is the place with the most obvious features found by GradCAM (in this embodiment, it can be the brightest place). Therefore, in the GradCAM image, the brightest place is the place in the entire image that has the greatest impact on classification. Correspondingly, the key position is a specific range, wherein the range must be as small as possible, but must include relevant important positions. Therefore, in one of the embodiments, the key position is a minimum circumscribed matrix.

[0072] Furthermore, it is important to understand that for clean samples, the key positions that affect image classification include the main features of the sample. For backdoor samples, the key positions that affect image classification include the triggers of the sample. For ease of understanding, this will be described in detail later.

[0073] 302. Determine target feature information located at the identification position point in the key feature information of the key position point.

[0074] After determining the key feature information of the key position point, it is necessary to further find the target feature information at the identification position point in the key feature information. The target feature information is the feature information in the key feature information that can be used to identify the sample to be processed. The identification position point is a specific position point in the key position point, which can be a certain area or a specific position point. There is no specific limitation here.

[0075] In one specific embodiment, an evolutionary algorithm is used to search in the key area in step 301 to find the area with the greatest influence. For a clean sample, the most influential part is the main feature of the sample; for a backdoor sample, the most influential part is the trigger of the sample. Thus, the evolutionary algorithm is used to find the area with the greatest influence, that is, the identification position point, and determine the target feature information of the identification position point.

[0076] 303. Input the target feature information to the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted.

[0077] After the target feature information is determined, the target feature information can be input into the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted. Among them, the target position point is the position point corresponding to the identification position point. It is not difficult to understand that the target position point can also be a partial area or a specific position point, which will not be described in detail here. In one embodiment, the reference sample is a sample that has been determined to be a clean sample. It should be noted that the reference sample is a sample in a verification set, wherein the images in the verification set are guaranteed to be clean.

[0078] In one specific embodiment, in order to determine whether the influential part found by the evolutionary algorithm is a trigger or a clean input feature, it is placed on a set of images (the image is the reference sample mentioned above), and its position is the same as the original position on the input sample (the sample to be processed mentioned above). Thus, the sample to be predicted is formed. Then, the algorithm predicts the sample to be predicted, determines the classification category label of the sample to be predicted, and thus determines whether the sample to be processed is a clean sample or a backdoor sample.

[0079] A sample processing method based on deep learning disclosed in this embodiment obtains key feature information of the sample to be processed; wherein the key feature information is used to describe the feature information of the key position point in the sample to be processed; then the target feature information located at the identification position point in the key feature information of the key position point is determined; wherein the target feature information is the feature information used to identify the sample to be processed in the key feature information; finally, the target feature information is input into the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted; wherein the target position point is the position point corresponding to the identification position point. Thus, the key features of the sample to be processed are combined with the reference sample to determine the classification label of the sample to be predicted, and the classification label of the sample to be processed is correspondingly determined, thereby maintaining the classification accuracy while significantly reducing the success rate of the attack. Correspondingly, the defense method based on sample preprocessing is simple to implement and highly practical.

[0080] Furthermore, in order to facilitate the above Figure 3 For a detailed description of the sample processing methods described in , see Figure 4 , Figure 4 This is a flowchart of another deep learning-based sample processing method disclosed in an embodiment of the present application, including steps 401 to 413.

[0081] 401. Obtain position feature information of all position points in the sample to be processed, and determine representation parameter information of the position feature information of all position points.

[0082] In order to identify the sample and determine whether the sample has a trigger, so as to filter the backdoor sample, it is necessary to identify and scan the sample to be processed. Specifically, the position feature information of all position points in the sample to be processed is obtained, and the characterization parameter information of the position feature information of all position points is determined. Among them, the position point can be a specific pixel point or a corresponding partial area, which is not limited here.

[0083] In one specific embodiment, the four-dimensional feature information of all position points in the sample to be processed is obtained, including the horizontal and vertical coordinates of the corresponding pixel points, the length and width of the area, etc. (i.e., the position feature information mentioned above), which is not limited here. Correspondingly, the image color (RGB) of the corresponding position point, i.e., the characterization parameter information mentioned above, can also be obtained. For ease of understanding, the position feature information or characterization parameter information mentioned above can be understood as the feature information of the corresponding position point, and will not be described in detail later.

[0084] Furthermore, in another specific embodiment, GradCAM can be used to roughly obtain the feature information of all the position points in the sample to be processed, and thereby obtain the characterization parameter information corresponding to the position points. Correspondingly, GradCAM can also mark the relevant importance (ranging from 0 to 1) for each position point on the sample to be processed, so as to facilitate the subsequent determination of key feature information.

[0085] 402. Determine target parameter information in the characterization parameter information, determine a position point corresponding to the target parameter information as a key position point, and determine feature information corresponding to the key position point as key feature information.

[0086] After determining the characterization parameter information, the target parameter information in the characterization parameter information can be determined, and then the position point corresponding to the target parameter information can be determined as the key position point, thereby determining the feature information at the key position point as the key feature information. It is not difficult to understand that in this embodiment, the target parameter information can be understood as the feature information of the area with the greatest influence in the characterization parameter information corresponding to each position point. Since step 401 marks the importance of each position point, the target parameter information can be understood as the pre-set importance that needs to be met. In this embodiment, it is positioned as 0.7.

[0087] In one specific embodiment, step 402 in this embodiment is similar to the above Figure 3 The step 301 is similar to that in the above, and will not be described in detail here. However, it should be noted that please refer to Figure 5 , Figure 5 This is a technical flow chart of a sample processing method based on deep learning disclosed in an embodiment of the present application. Figure 5 As can be seen from the left 1, GradCAM is used to roughly find the key positions that affect image classification. The left 2 is the class activation map formed by GradCAM, where Figure 5 The place where the left 2 in the figure has obvious differences from the surrounding area is the place with the greatest impact on classification. Correspondingly, the feature information of the place where there is obvious difference is the target parameter information described above, and the details are not repeated here.

[0088] 403. Obtain all the position points to be identified in the key position points and the feature information to be identified corresponding to the position points to be identified.

[0089] After the key position points are determined, all the position points to be identified in the key position points and the feature information to be identified corresponding to the position points to be identified can be obtained. It is not difficult to understand that the key feature information includes all the feature information to be identified.

[0090] In one specific embodiment, steps 401 and 402 reduce the possible area of ​​the trigger as much as possible, so step 403 needs to find the part with the best influence on the final classification in the area found above. Specifically, the feature information of each area location point found in the above steps is first determined.

[0091] Furthermore, in another specific embodiment, the differential evolution algorithm is used to select the best individual in each generation and continuously iterate to find the global optimal solution. The operations that need to be performed on the individuals in each generation include initialization, mutation, crossover and selection. Specifically, in one of the feasible embodiments, the population P is initialized first, and the population includes 20 individuals. Each individual represents a part of the searched area. The individual is represented by four dimensions, which refer to the position points to be identified. The corresponding band is the horizontal and vertical coordinates of the identified position points, and the length and width of the area. It is not difficult to understand that the number of populations is only one of the specific implementation methods. Other numbers of populations can also be set, such as 50, but it should be noted that 20 is an effective and fast solution formulated during the experiment.

[0092] Based on the above embodiment, it is not difficult to understand that step 403 is an initialization operation of the differential evolution algorithm.

[0093] 404. Select a first position point and a second position point from all the position points to be identified, and determine difference information of first feature information and second feature information corresponding to the first position point and the second position point respectively.

[0094] Based on step 403, a first position point and a second position point are selected from all the position points to be identified, and first feature information corresponding to the first position point and second feature information corresponding to the second position point are determined respectively, and then difference information between the first position point and the second position point is determined according to the first feature information and the second feature information. It is not difficult to understand that the first position point and the second position point are any two position points among all the position points to be identified.

[0095] In one specific embodiment, the first position point and the second position point are two individuals randomly selected from the population, and their difference can be used to represent the difference between any two individuals in the current population.

[0096] 405. Mutate the feature information to be identified at the third position point according to the difference information to obtain the third feature information corresponding to the third position point, and cross-sample the third feature information corresponding to the third position point and the feature information to be identified to determine the feature information to be selected at the third position point.

[0097] Based on the above steps, the feature information to be identified at the third position point can be mutated according to the difference information, thereby obtaining the third feature information corresponding to the third position point. Then, the third feature information of the third position point and the feature information to be identified are cross-valued to determine the feature information to be selected at the third position point. It is not difficult to understand that the third position point is any one of all the position points to be identified.

[0098] In one specific embodiment, each individual u i (t) The generated variant individual v i (t) is:

[0099] v i (t) = u i (t)+α(u i1(t)-u i2 (t)).

[0100] Among them, t represents the number of iterations of the current mutation. i1 (t) and u i2 (t) are two individuals randomly selected from the population, i.e., the first position point and the second position point in the above. Their difference is used to represent the difference between any two individuals in the current population. α is used to control the degree of differential evolution. In this embodiment, α is 0.5. It is not difficult to understand that the above value is only one of the achievable embodiments, and the value of α is not specifically limited here. It should also be noted that u i (t) is the feature information to be identified at the third position point described above, v i (t) is the third feature information of the third position point.

[0101] After the third feature information and identification feature information of the third position point are determined, the above feature information needs to be cross-processed, so that after the cross-processing, the offspring individual From the parent individual u i (t) and the individual v derived from it i (t) to choose from. Specifically,

[0102]

[0103] Here, j represents each dimension, and θ belongs to the uniform distribution U(0,1). The crossover hyperparameter ρ here is 0.5. It is not difficult to understand that the dimensions described above are the horizontal and vertical coordinates of the corresponding position points and the length and width of the area. Different dimensions can be selected differently. For example, the first dimension can be the parent individual u i The first dimension of (t), the second dimension can be the offspring individual The second dimension of the third position point is determined, that is, the final four-dimensional dimension of the third position point.

[0104] Based on the above embodiment, it is not difficult to understand that step 404 to step 405 are mutation operations and crossover operations of the differential evolution algorithm.

[0105] 406 . Determine chromaticity feature information corresponding to the third position point in the sample to be processed, and determine fitness information corresponding to the third position point according to the chromaticity feature information.

[0106] In order to ensure that the searched area has both the backdoor effect and is as small as possible, and does not include too many parts belonging to the clean image, it is necessary to determine the color feature information in the third position point, so as to determine the fitness information corresponding to the third position point.

[0107] In one specific embodiment, the average color of the third position point in the sample to be processed is input into the corresponding model. If it causes the predicted label of the model to change, 1 is returned; otherwise, 0 is returned. It is not difficult to understand that before the sample to be processed is detected, it is not certain whether it is a clean sample or a backdoor sample. Therefore, the average color of the sample to be processed is obtained to replace the color feature information of the corresponding third position point. In other words, it is determined by u, and the corresponding color feature information is input into the model to directly determine whether the output label has changed. For easy understanding, please refer to Figure 5 However, since it is not certain whether it is a clean sample before detection, it may be the first in the first row or the first in the second row.

[0108] Furthermore, from the above description, it can be seen that the fitness information of the third position point is 0 or 1.

[0109] 407. Input the feature information to be selected of the third position point into the clean sample, identify the classification category label of the clean sample, and determine the prediction probability information corresponding to the third position point.

[0110] In order to determine the probability that the third position point may change the label, it is necessary to determine the probability that the prediction result changes when the area corresponding to the third position point is pasted onto other clean images. Specifically, the feature information to be selected of the third position point is input into the clean sample, and the classification category label of the clean sample is identified to determine the prediction probability information corresponding to the third position point. It should be noted in advance that the classification category label of the clean sample is the first category label, and the prediction probability information is used to describe the probability information of the identification classification category label being the second category label.

[0111] In one specific embodiment, the evaluation is performed by pasting the feature information to be selected at the third position point onto a clean image set. In other words, it is equal to the ratio of the predicted label of the model to the label of the sample after the region corresponding to the third position point is pasted into the clean image set. For example, if there are a total of h clean images in the set, and q samples among them change the prediction after being pasted into the current region u, then the corresponding predicted probability information is q / h. It is not difficult to understand that, as can be seen in Figure 5 , u is a rectangular position of an individual. For example, u is a square block with the upper left corner coordinates (1,1) and a length and width of 3. A piece cut off from this position of the man's image can be pasted to the same position of the woman's head portrait, such as (1,1) of the woman's head portrait.

[0112] For ease of understanding, the following example is given. i Refers to a certain position of the sample to be processed (which can be understood as the third position point). The purpose of the evolutionary algorithm is to use ui Iterative update, finally u i Aim at the trigger position or the clean sample feature position. Correspondingly, set a clean sample verification set, take h = 10, and put the u i Dig out the piece at the location and paste it on the u of the 10 confirmed clean images i The q clean images are then input into the model for reclassification. The classification of the q clean images is changed after the images are pasted. i Position (u i Contains coordinates and length and width, so it can be found accurately) and paste this position on 10 clean images. These 10 clean images can be correctly classified by the model at the beginning, for example, there are 5 pigs, 3 dogs, and 2 chickens. Now paste what I just dug on the cat, and suddenly 9 pictures are classified as cats. So now q = 9.

[0113] 408. Determine preset fitness information according to the fitness information and the predicted probability information.

[0114] Based on the above steps 406 and 407, a fitness function can be designed to determine the preset fitness information according to the fitness information and the prediction probability information.

[0115] In one specific embodiment, the fitness function is:

[0116]

[0117] Wherein, f(u) is the preset fitness information. flag(u) is a binary value, i.e., the fitness information mentioned above. flips(u) is the predicted probability information mentioned above. s(u) is the area of ​​the search area, i.e., the area of ​​the third position point, and S is the total area of ​​the input image, i.e., the area of ​​the sample to be processed.

[0118] Therefore, the preset fitness information can be determined through the fitness information and the predicted probability information.

[0119] 409. When the feature information to be selected satisfies the preset fitness information, the third position point is determined as the identification position point, and the feature information to be identified corresponding to the third position point is determined as the target feature information.

[0120] When the feature information to be selected satisfies the preset fitness information, the third position point can be determined as the identification position point, and the feature information to be identified corresponding to the third position point can be determined as the target feature information.

[0121] Specifically, in one of the specific embodiments, in the selection stage, by evaluating the fitness values ​​of the individuals in the current generation population and the corresponding offspring individuals, the new generation population P is selected.+ Individuals with larger fitness values ​​f(u) are selected and retained. It is not difficult to understand that the specific method is to determine whether the feature information to be selected meets the preset fitness information. If it does, the third position point is determined as the identification position point, and the feature information to be identified corresponding to the third position point is determined as the target feature information. The relevant selection function is:

[0122]

[0123] in, is the selected offspring individual. Thus, the most influential part can be determined. Based on the above embodiment, it is not difficult to understand that steps 406 to 409 are selection operations of the differential evolution algorithm. For details, please refer to Figure 3 In the third and fourth columns, the differential evolution algorithm is used to further narrow down the regions with large influence.

[0124] Further, in another specific embodiment, when the feature information to be identified of the third position point is mutated, the target iteration number of the third position point is determined. When the target iteration number meets the preset number threshold, the third position point that meets the preset fitness information is determined to be the identification position point. Alternatively, when the target iteration number meets the preset number threshold and the feature information to be selected does not meet the preset fitness information, the classification category label of the sample to be predicted is determined to be the first category label, and the sample to be predicted is a clean sample.

[0125] Specifically, when the fitness function value does not improve significantly, or when the number of generations reaches 100, the evolution terminates. The region corresponding to the optimal fitness at this time is the region with the greatest influence we found. If no region that meets the conditions is found until 100 generations, the current sample is directly determined to be a clean sample. There is no need to execute subsequent steps 410-413.

[0126] 410. Acquire multiple groups of reference samples, and set the position points in the reference samples corresponding to the identified position points as target position points, so as to paste the image feature information corresponding to the target feature information to the target position points, and obtain multiple groups of samples to be predicted.

[0127] In order to determine the category labels of the influential parts found by the differential evolution algorithm, it is necessary to compare and verify with reference samples. Specifically, multiple groups of reference samples are obtained, and the position points corresponding to the identification position points in the reference samples are set as target position points, and then the image feature information corresponding to the target feature information is pasted to the target position points to obtain multiple groups of samples to be predicted.

[0128] In one specific embodiment, the target feature information of the identification position point is placed on a group of images (the group of images can be 100 or 50, which is not limited here), and the corresponding position is the same as the original position on the sample to be processed, so as to obtain multiple groups of samples to be predicted. Figure 3 In the fourth and fifth columns, the image feature information corresponding to the target feature information is pasted to the target position point.

[0129] 411. Determine whether the classification category labels of all samples to be predicted are the same as the classification category labels of the reference samples. If they are the same, execute step 412; if they are not the same, execute step 413.

[0130] Thus, the samples to be predicted can be predicted, so as to determine whether the classification category labels of all samples to be predicted are the same as the classification category labels of the reference samples. If they are the same, step 412 is executed; if they are not the same, step 413 is executed. It is not difficult to understand that the classification category label of the reference sample is the first category label, that is, the reference sample is a clean sample.

[0131] Specifically, the sample to be predicted is put into the backdoor network model to statistically determine whether the classification category label of the sample to be predicted in the model is the same as the classification category label of the reference sample. If most of them are the same, step 412 is executed; if most of them are not the same, step 413 is executed.

[0132] 412. Determine that the sample to be processed is a clean sample.

[0133] When the classification category labels of most of the samples to be predicted are the same as the classification category labels of the reference samples, and the classification category labels of the samples to be predicted are the first category labels, the samples to be processed are determined to be clean samples.

[0134] 413. Determine that the sample to be predicted is a backdoor sample.

[0135] When the classification category labels of most of the samples to be predicted are different from the classification category labels of the reference samples, and the classification category labels of the samples to be predicted are the second category labels, it is determined that the samples to be processed are backdoor samples.

[0136] A sample processing method based on deep learning disclosed in this embodiment combines evolutionary algorithms with backdoor defense. At the same time, searching for triggers in backdoor samples of a backdoor network model is an effective and efficient method. Furthermore, the technical solution of the present application can significantly reduce the success rate of attacks while maintaining classification accuracy. The defense method based on sample preprocessing is simple to implement and has strong practicality. Furthermore, in the technical solution of the present application, the only attribute used is the backdoor effect of the backdoor trigger, and it has high robustness to the attributes such as the content and size of the trigger, and has strong universality.

[0137] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] If the plan involves sensitive information (such as user information, corporate information), it should be stated that the collection, use and processing of sensitive information must comply with the laws, regulations and standards of relevant countries and regions, and must be carried out with the permission or consent of the relevant entities (such as users or companies).

[0139] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of a sample processing system based on deep learning disclosed in an embodiment of the present application.

[0140] The acquisition unit 601 is used to acquire key feature information of the sample to be processed; wherein the key feature information is used to describe feature information of key positions in the sample to be processed;

[0141] A determination unit 602 is used to determine target feature information located at an identification position point in key feature information of a key position point; wherein the target feature information is feature information in the key feature information used to identify a sample to be processed;

[0142] The input unit 603 is used to input the target feature information to the target position point in the reference sample to obtain the sample to be predicted so as to determine the classification category label of the sample to be predicted; wherein the target position point is the position point corresponding to the identification position point.

[0143] Exemplarily, the system includes:

[0144] The acquisition unit 601 is specifically used to acquire the position feature information of all position points in the sample to be processed;

[0145] A determination unit 602, specifically configured to determine the characterization parameter information of the location feature information of all location points;

[0146] The determination unit 602 is further configured to determine target parameter information in the characterization parameter information, determine a position point corresponding to the target parameter information as a key position point, and determine feature information corresponding to the key position point as key feature information.

[0147] Exemplarily, the system includes:

[0148] The acquisition unit 601 is specifically used to acquire all the position points to be identified in the key position points, and the feature information to be identified corresponding to the position points to be identified; wherein the key feature information includes all the feature information to be identified;

[0149] The determination unit 602 is specifically configured to select a first position point and a second position point from all the position points to be identified, and determine difference information of first feature information and second feature information corresponding to the first position point and the second position point, respectively; wherein the first position point and the second position point are any two position points from all the position points to be identified;

[0150] The acquisition unit 601 is further used to mutate the feature information to be identified of the third position point according to the difference information, and acquire third feature information corresponding to the third position point; wherein the third position point is any one of all the position points to be identified;

[0151] The determination unit 602 is further configured to cross-value the third feature information and the feature information to be identified corresponding to the third position point to determine the feature information to be selected at the third position point;

[0152] The determination unit 602 is further configured to determine the third position point as the identification position point when the feature information to be selected satisfies the preset fitness information, and determine the feature information to be identified corresponding to the third position point as the target feature information.

[0153] Exemplarily, the system further includes:

[0154] The determination unit 602 is further configured to determine the color characteristic information corresponding to the third position point in the sample to be processed;

[0155] The determining unit 602 is further configured to determine fitness information corresponding to the third position point according to the chromaticity feature information;

[0156] The determination unit 602 is further used to input the feature information to be selected of the third position point into the clean sample, identify the classification category label of the clean sample, and determine the prediction probability information corresponding to the third position point; wherein the classification category label of the clean sample is the first category label, and the prediction probability information is used to describe the probability information of identifying the classification category label as the second category label;

[0157] The determination unit 602 is further configured to determine preset fitness information according to the fitness information and the prediction probability information.

[0158] Exemplarily, the system further includes:

[0159] The determination unit 602 is further configured to determine a target number of iterations of the third position point when mutating the feature information to be identified at the third position point;

[0160] The determination unit 602 is further configured to determine, when the target iteration number satisfies a preset number threshold, a third position point that satisfies preset fitness information as an identification position point;

[0161] Alternatively, the determination unit 602 is further configured to determine that the classification category label of the sample to be predicted is the first category label and the sample to be predicted is a clean sample when the target iteration number satisfies a preset number threshold and the feature information to be selected does not satisfy preset fitness information.

[0162] Exemplarily, the system further includes: an execution unit 605 and a judgment unit 604;

[0163] The acquisition unit 601 is specifically used to acquire multiple groups of reference samples, and set the position points in the reference samples corresponding to the identified position points as target position points; wherein the reference samples are clean samples;

[0164] An execution unit 605 is used to paste the image feature information corresponding to the target feature information to the target position point to obtain multiple groups of samples to be predicted;

[0165] A judging unit 604 is used to judge whether the classification category labels of all samples to be predicted are the same as the classification category labels of the reference samples;

[0166] A determination unit 602 is specifically configured to determine that the sample to be processed is a clean sample when the classification category labels of the sample to be predicted and the reference sample are the same;

[0167] The determination unit 602 is further configured to determine that the sample to be predicted is a backdoor sample when the classification category labels of the sample to be predicted and the reference sample are different.

[0168] Exemplarily, the system further includes:

[0169] The determination unit 602 is further configured to determine that the sample to be processed is a clean sample when the classification category label is the first category label.

[0170] The determination unit 602 is further configured to determine, when the classification category label is the second category label, that the target feature information is a trigger and that the sample to be processed is a backdoor sample.

[0171] See below Figure 7 , a schematic diagram of a sample processing device based on deep learning disclosed in an embodiment of the present application includes:

[0172] CPU 701, memory 705, input / output interface 704, wired or wireless network interface 703 and power supply 702;

[0173] The memory 705 is a temporary storage memory or a permanent storage memory;

[0174] The CPU 701 is configured to communicate with the memory 705 and execute the instructions in the memory 705 to perform the aforementioned Figure 3 or Figure 4 A sample processing method based on deep learning in any of the embodiments shown.

[0175] The embodiment of the present application also provides a chip system, characterized in that the chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the aforementioned Figure 3 or Figure 4 A sample processing method based on deep learning in any of the embodiments shown.

[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A sample processing method based on deep learning, It is characterized in that The method comprises: Acquire key feature information of the sample to be processed; wherein the key feature information is used to describe feature information at key positions in the sample to be processed; Determine the target feature information located at the identification position point in the key feature information of the key position point; wherein the target feature information is the feature information in the key feature information used to identify the sample to be processed; Input the target feature information to the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted; wherein the target position point is the position point corresponding to the identification position point; The target feature information located at the identification position point in the key feature information for determining the key position point includes: Acquire all the position points to be identified in the key position points, and the feature information to be identified corresponding to the position points to be identified; wherein the key feature information includes all the feature information to be identified; Selecting a first position point and a second position point from all the position points to be identified, and determining difference information of first feature information and second feature information respectively corresponding to the first position point and the second position point; wherein the first position point and the second position point are any two position points from all the position points to be identified; According to the difference information, the feature information to be identified of the third position point is mutated to obtain the third feature information corresponding to the third position point; wherein the third position point is any one of all the position points to be identified; Cross-value the third feature information and the feature information to be identified corresponding to the third position point to determine the feature information to be selected for the third position point; When the feature information to be selected satisfies the preset fitness information, determining the third position point as the identification position point, and determining the feature information to be identified corresponding to the third position point as the target feature information; Before determining the third position point as the identification position point and determining the to-be-identified feature information corresponding to the third position point as the target feature information, the method further includes: Determining the color characteristic information corresponding to the third position point in the sample to be processed; Determining fitness information corresponding to the third position point according to the chromaticity feature information; Inputting the feature information to be selected of the third position point into a clean sample, identifying the classification category label of the clean sample, so as to determine the prediction probability information corresponding to the third position point; wherein the classification category label of the clean sample is a first category label, and the prediction probability information is used to describe the probability information of identifying the classification category label as a second category label; The preset fitness information is determined according to the fitness information and the predicted probability information.

2. The sample processing method based on deep learning according to claim 1, It is characterized in that The step of obtaining key feature information of the sample to be processed includes: Obtaining position feature information of all position points in the sample to be processed; Determine the representation parameter information of the position feature information of all the position points; The target parameter information in the characterization parameter information is determined, and the position point corresponding to the target parameter information is determined as the key position point, and the feature information corresponding to the key position point is determined as the key feature information.

3. The sample processing method based on deep learning according to claim 1, It is characterized in that The method further comprises: When the feature information to be identified at the third position point is mutated, determining a target number of iterations for the third position point; When the target iteration number satisfies a preset number threshold, determining the third position point that satisfies the preset fitness information as the identification position point; Or, when the target number of iterations meets a preset number threshold and the feature information to be selected does not meet preset fitness information, it is determined that the classification category label of the sample to be predicted is the first category label, and the sample to be predicted is a clean sample.

4. The sample processing method based on deep learning according to claim 1, It is characterized in that The step of inputting the target feature information to the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted, includes: Acquire multiple groups of the reference samples, and set the position points in the reference samples corresponding to the identified position points as the target position points; wherein the reference samples are clean samples; Paste the image feature information corresponding to the target feature information to the target position point to obtain multiple groups of samples to be predicted; Determine whether the classification category labels of all the samples to be predicted are the same as the classification category labels of the reference samples; If the classification category label of the sample to be predicted is the same as that of the reference sample, determining that the sample to be processed is the clean sample; If the classification category labels of the sample to be predicted and the reference sample are different, it is determined that the sample to be predicted is a backdoor sample.

5. The sample processing method based on deep learning according to claim 1, It is characterized in that After determining the classification category label of the sample to be predicted, the method further includes: If the classification category label is the first category label, determining that the sample to be processed is a clean sample; If the classification category label is the second category label, it is determined that the target feature information is a trigger and the sample to be processed is a backdoor sample.

6. A sample processing system based on deep learning, It is characterized in that The system comprises: An acquisition unit, used to acquire key feature information of the sample to be processed; wherein the key feature information is used to describe feature information at key positions in the sample to be processed; A determination unit, used to determine target feature information located at an identification position point in the key feature information of the key position point; wherein the target feature information is feature information in the key feature information used to identify the sample to be processed; An input unit, used to input the target feature information to the target position point in the reference sample to obtain the sample to be predicted, so as to determine the classification category label of the sample to be predicted; wherein the target position point is the position point corresponding to the identification position point; The system includes: An acquisition unit, specifically used to acquire all the position points to be identified in the key position points, and the feature information to be identified corresponding to the position points to be identified; wherein the key feature information includes all the feature information to be identified; A determination unit, specifically used to select a first position point and a second position point from all the position points to be identified, and determine difference information of first feature information and second feature information corresponding to the first position point and the second position point respectively; wherein the first position point and the second position point are any two position points from all the position points to be identified; The acquisition unit is further used to mutate the feature information to be identified of the third position point according to the difference information, and acquire third feature information corresponding to the third position point; wherein the third position point is any one of all the position points to be identified; The determination unit is further used to cross-value the third feature information and the feature information to be identified corresponding to the third position point to determine the feature information to be selected at the third position point; The determination unit is further configured to determine, when the feature information to be selected satisfies the preset fitness information, the third position point as the identification position point, and determine the feature information to be identified corresponding to the third position point as the target feature information; The system also includes: The determination unit is further used to determine the color characteristic information corresponding to the third position point in the sample to be processed; The determination unit is further used to determine the fitness information corresponding to the third position point according to the chromaticity feature information; The determination unit is further used to input the feature information to be selected of the third position point into the clean sample, identify the classification category label of the clean sample, so as to determine the prediction probability information corresponding to the third position point; wherein the classification category label of the clean sample is the first category label, and the prediction probability information is used to describe the probability information of the identification classification category label being the second category label; The determination unit is further used to determine preset fitness information according to the fitness information and the predicted probability information.

7. A sample processing device based on deep learning, It is characterized in that The device comprises: CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply; The memory is a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the deep learning-based sample processing method described in any one of claims 1 to 5.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium includes instructions, which, when executed on a computer, enable the computer to execute the deep learning-based sample processing method according to any one of claims 1 to 5.

Images