Black-box adversarial sample generation method, device, equipment and medium

Abstract

The application discloses a black box adversarial sample generation method and device, equipment and medium, and relates to the technical field of deep neural networks. The method comprises the following steps: acquiring an original sample for a skeletal action, and initializing a random sample different from the category of the original sample; based on the original sample and the random sample, a first adversarial sample is determined through binary search, and the gradient direction symbol of the first adversarial sample is estimated based on historical information corresponding to a successful black box attack, so as to determine a second adversarial sample iterated along the gradient direction; based on the second adversarial sample and the random sample, a third adversarial sample close to a decision boundary is determined through binary search; and the third adversarial sample is projected on a first-order manifold by using a preset adversarial sample generation formula, so as to obtain a final adversarial sample on a natural posture manifold. Through the technical scheme of the application, an adversarial sample that is not easy to perceive can be generated under the premise of limited query budget.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep neural networks, and particularly relates to a black-box adversarial sample generation method, device, equipment and medium. BACKGROUND

[0002] Deep neural networks have shown excellent performance in the field of skeleton-based action recognition, and have become a standard method for processing skeleton sequences, and are widely used in human-computer interaction, video monitoring, video search, automatic driving, virtual reality and motion analysis, and have a profound impact on people's production and life. For example, early warning and timely treatment of the falling action of the elderly, identifying whether the driver is in a state of fatigue during driving, and providing a fatigue warning, identifying whether the worker's operation process is in line with the standard or whether there is a danger in the factory workshop. However, recent studies have shown that deep neural networks are vulnerable to adversarial attacks. For example, in the skeleton data sequence, the enemy can affect the recognition accuracy of the machine by perturbing the joint position. According to the amount of model information obtained by querying, the adversarial attack methods for the skeleton-based action recognition model are mainly divided into two categories: white-box attack and black-box attack. One is white-box attack, and the enemy can obtain all the internal information of the action recognition model by querying to construct adversarial samples, but this type of attack cannot be applied to real application scenarios. The other is black-box attack, and the enemy does not need the complete knowledge of the model, but only needs to query the input and output of the model to construct adversarial samples. According to whether the prediction result of the model is a class label, the black-box attack is divided into score-based attack and decision-based attack. The former can obtain the class confidence of the model output by querying, and estimate the gradient by numerically quantifying the confidence to generate adversarial samples, but compared with the decision-based attack method, it is less practical. The latter only relies on the last prediction, estimates the gradient direction by accessing the top-1 label of the target model to generate adversarial samples, and is more challenging and practical. In order to be close to the real application scenario, the current focus is mainly on the decision-based black-box adversarial attack method. However, this type of method usually faces two problems: high query level and poor imperceptibility.

[0003] In summary, how to generate an adversarial sample that is not easy to be perceived under the premise of limited query budget is a problem to be solved at present. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a black-box adversarial sample generation method, device, equipment and medium, which can generate an adversarial sample that is not easy to be perceived under the premise of limited query budget. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a black-box adversarial sample generation method, comprising:

[0006] An original sample is obtained according to a skeleton action, and a random sample different from a category of the original sample is initialized;

[0007] Based on the original sample and the random sample, a first adversarial sample is determined through binary search, and a gradient direction symbol of the first adversarial sample is estimated based on historical information corresponding to a successful black-box attack, so as to determine a second adversarial sample iterated along the gradient direction;

[0008] Based on the second adversarial sample and the random sample, a third adversarial sample close to a decision boundary is determined through binary search;

[0009] The third adversarial sample is projected onto a first-order manifold by using a preset adversarial sample generation formula, so as to obtain a final adversarial sample on a natural pose manifold.

[0010] Optionally, the gradient direction symbol of the first adversarial sample is estimated based on historical information corresponding to a successful black-box attack, so as to determine a second adversarial sample iterated along the gradient direction, including:

[0011] A difference function is defined in advance by using the first adversarial sample based on historical information corresponding to a successful black-box attack, and a sign function of the difference function is determined;

[0012] A gradient direction of the second adversarial sample is determined through a Monte Carlo method, then a gradient direction symbol is estimated based on the gradient direction, and a step length in the gradient direction is determined through a first parameter adjustment formula, so as to determine a second adversarial sample iterated along the gradient direction according to the gradient direction symbol and the step length.

[0013] Optionally, the difference function is Wherein, x' is an adversarial sample of an original motion; F is a trained motion recognizer with k action categories F: x→R k , F x =F y (x)=argmax j∈k f(x);f(x)∈R k is a confidence score of a skeleton action of each category, F y′ (x′), F y (x′), represents a predicted category label of a sample;

[0014] The sign function of the difference function is φ(x′),

[0015] Optionally, the gradient direction is δ is a preset positive parameter, and R is an action category; from a self-defined Gaussian distribution N(μ,σ2 ) is derived in the following, ξ∈(0, 1) is the degree of inclination of the information obtained by the query;

[0016] The gradient direction sign is

[0017] The first parameter adjustment formula is x i , x i+1 respectively represent the i, i+1 step of the adversarial sample, the parameter a is the size of the step, the initial value of the parameter a of the i iteration is θ is the first preset threshold.

[0018] Optionally, based on the second adversarial sample and the random sample, a third adversarial sample close to the decision boundary is determined by binary search, comprising:

[0019] Based on the second adversarial sample and the random sample, the target parameter in the second parameter adjustment formula is determined by binary search, so as to determine the third adversarial sample close to the decision boundary based on the target parameter by using the second parameter adjustment formula.

[0020] Optionally, the second parameter adjustment formula is x i+1 , x i+2 respectively represent the i+1, i+2 step of the adversarial sample; x0-x i+1 represents the direction of the adversarial sample x i+1 to the original sample x0;

[0021] The target parameter Wherein, ε is a second preset threshold, if the label obtained by the query of the adversarial sample is inconsistent with the original sample and greater than the second preset threshold ε, the value of the target parameter β is half of the original, otherwise the value of β is unchanged.

[0022] Optionally, the third adversarial sample is projected on the first order manifold by using the preset adversarial sample generation formula, to obtain the final adversarial sample on the natural pose manifold, comprising:

[0023] The third adversarial sample is projected on the first order manifold by using to obtain the final adversarial sample on the natural pose manifold; wherein D is the Euclidean distance, x' and x" are the adversarial samples before and after the manifold projection respectively, and are the first derivatives of x' and x", ω is the weight; B i and B irespectively the length of the i th bone in the original sample and the skeleton action in the adversarial sample; and the θ i is the joint angle, and is in the range of .

[0024] In a second aspect, the present application discloses a black-box adversarial sample generation device, comprising:

[0025] a sample acquisition module, configured to acquire an original sample for a skeleton action, and initialize a random sample different from the category of the original sample;

[0026] a gradient direction sign estimation module, configured to determine a first adversarial sample based on the original sample and the random sample through binary search, and perform gradient direction sign estimation on the first adversarial sample based on historical information corresponding to a successful black-box attack to determine a second adversarial sample iterated along the gradient direction;

[0027] a boundary point approximation module, configured to determine a third adversarial sample close to a decision boundary based on the second adversarial sample and the random sample through binary search;

[0028] a first-order manifold projection module, configured to perform first-order manifold projection on the third adversarial sample by using a preset adversarial sample generation formula to obtain a final adversarial sample on a natural pose manifold.

[0029] In a third aspect, the present application discloses an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, the computer program is loaded and executed by the processor to realize the black-box adversarial sample generation method as described above.

[0030] In a fourth aspect, the present application discloses a computer readable storage medium, used to store a computer program; wherein the computer program is executed by a processor to realize the black-box adversarial sample generation method as described above.

[0031] The application provides a black-box adversarial sample generation method, obtains an original sample of a skeletal action, and initializes a random sample different from the category of the original sample; based on the original sample and the random sample, a first adversarial sample is determined through binary search, and the first adversarial sample is subjected to gradient direction symbol estimation based on historical information corresponding to a successful black-box attack, so as to determine a second adversarial sample iterated along the gradient direction; based on the second adversarial sample and the random sample, a third adversarial sample close to a decision boundary is determined through binary search; and the third adversarial sample is subjected to first-order manifold projection by using a preset adversarial sample generation formula, so as to obtain a final adversarial sample on a natural posture manifold. It can be seen that the technical scheme of the application generates a skeletal adversarial sample based on gradient direction symbol estimation, and does not depend on accurate gradient, so that the generated adversarial sample has lower query amount; meanwhile, the historical information obtained through query is used for gradient symbol direction estimation, so that the skeletal adversarial sample closest to the original sample is obtained along the boundary faster, and the enemy is guided to quickly determine an effective attack direction; the skeletal action sample is projected onto the manifold, the overall skeletal movement is coordinated, and the imperceptibility of the skeletal adversarial sample is enhanced. In this way, an adversarial sample that is not easy to perceive is generated under the premise of limited query budget.

[0032] In addition, the application provides a black-box adversarial sample generation device, equipment and storage medium, which correspond to the above-mentioned black-box adversarial sample generation method, and have the same effect. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description only constitute the embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0034] Figure 1 A black-box adversarial sample generation method flow chart disclosed by the application;

[0035] Figure 2 A gradient symbol-based fast guided decision attack diagram for skeletal action recognition on a single frame disclosed by the application;

[0036] Figure 3 A black-box adversarial sample generation device structure diagram disclosed by the application;

[0037] Figure 4 An electronic equipment structure diagram disclosed by the application. DETAILED DESCRIPTION

[0038] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0039] Currently, for decision-based black-box adversarial attacks, there are usually two problems of high query level and poor imperceptibility. To this end, the present application provides a black-box adversarial sample generation scheme, which can generate an adversarial sample that is not easy to be perceived under the premise of limited query budget.

[0040] The embodiments of the present application disclose a black-box adversarial sample generation method, as shown in Figure 1 The method comprises the following steps:

[0041] Step S11: Obtain an original sample for a skeletal action, and initialize a random sample different from the category of the original sample.

[0042] In the embodiments of the present application, given an original sample x of a skeletal action and a category label y. The original sample of an action sequence is x={x 1 ,...,x n}, wherein n represents the number of skeletal motion frames. x t ={q1,...,q m} represents the mth joint of the tth frame. A random sample x * different from the category of the original sample is initialized for the original sample.

[0043] Step S12: Based on the original sample and the random sample, a first adversarial sample is determined through binary search, and the gradient direction sign of the first adversarial sample is estimated based on historical information corresponding to a successful black-box attack, to determine a second adversarial sample iterated along the gradient direction.

[0044] In the field of image classification, researchers have carried out a lot of research on the problems of large query amount and poor imperceptibility of adversarial attacks. For the problem of large query amount, Cheng et al. found that gradient estimation requires a large number of queries to detect the decision boundary, and estimating the direction of the gradient through queries can effectively reduce the query amount. At the same time, Chen et al. showed that the method of estimating the gradient direction only near the decision boundary is effective. Therefore, in the embodiments of the present application, the method of estimating the gradient direction is used to reduce the query amount by using the gradient direction estimation method on the adversarial sample on the decision boundary. After initializing a random sample different from the category (or a specific category) of the original sample, the first adversarial sample between the original sample and the random sample is determined through binary search as the initial adversarial sample.

[0045] Most of the existing adversarial attack methods randomly sample the direction of the next step, which causes the next step of the enemy to have strong uncertainty, resulting in low sampling efficiency. Therefore, random sampling has a great influence on the efficiency of the attack method. According to some previous researches, the curvature of the decision boundary of the deep neural network is small near the data sample, so the direction of the last iteration attack success has a great probability of success when estimating the direction of the current iteration. At the same time, some previous researchers found that historical information can affect the current sampling, so the historical information obtained by querying is beneficial to improve the sampling efficiency. Therefore, in order to improve the efficiency of the next step direction sampling, the embodiment of the present application samples the gradient direction based on the historical information.

[0046] In the embodiment of the present application, the information of the successful sampling attack is obtained and used for the estimation of the gradient symbol to improve the accuracy of the next step attack direction of the enemy. Specifically, the first adversarial sample is used to predefine a difference function based on the historical information corresponding to the successful black box attack, and the sign function of the difference function is determined; the gradient direction of the second adversarial sample is determined by the Monte Carlo method, then the gradient direction symbol is estimated based on the gradient direction, and the step size in the gradient direction is determined by the first parameter adjustment formula, so as to determine the second adversarial sample along the gradient direction according to the gradient direction symbol and the step size.

[0047] Since the previous black box attack method based on skeleton action recognition requires to estimate the gradient itself when estimating the gradient, a large number of queries are required. In the embodiment of the present application, the gradient symbol is estimated by single query symbol oracle based on the feature that the gradient direction symbols on both sides of the decision boundary are inconsistent. First, a difference function S(x') is predefined, φ(x') is the sign function of S(x'): Where x' is the adversarial sample of the original motion; F is the trained action recognizer with k action categories F: x→R k , F x = F y (x) = argmax j∈k f(x); f(x) ∈ R k , is the confidence score of each category of skeleton action, F y′ (x'), F y (x'), indicates the predicted category label of the sample.

[0048] In the boundary attack, the gradient of S(x') is estimated by querying φ(x'). Further, the Monte Carlo method is used to estimate the direction of the next step, the gradient direction symbol is estimated based on the gradient direction, and the step size to be taken in the gradient direction is adjusted, the gradient direction of the boundary point denoted by η, then The gradient direction sign is wherein R is the action category, and δ is a preset positive parameter; derived from a custom Gaussian distribution N(μ, σ 2 ). It should be noted that the random direction is sampled from the custom Gaussian distribution N(μ, σ 2 ), and the variance σ 2 is related to the dispersion degree of the data distribution and cannot well guide the position of the successful attack direction, so in the embodiments of the present application, only the value of the expectation μ is considered, and the value of μ is calculated through the successful historical query information. In the experiment, we found that the unsuccessful historical information is very little and has little influence, so we only use the successful historical information. Specifically, if the query obtains the information of the successful attack, the value of the expectation is biased to the direction of the successful attack. Thus, the expectation μ is updated as: wherein ξ∈(0, 1) represents the degree of inclination to the information obtained by the query.

[0049] In the embodiments of the present application, the step size in the gradient direction is adjusted through a first parameter adjustment formula, and the first parameter adjustment formula is x i , x i+1 respectively represent the i-th and i+1-th steps of the adversarial sample, and the parameter α is the size of the step, and the initial value of the parameter α of the i-th iteration is θ is a first preset threshold; is the estimated gradient direction sign.

[0050] Step S13: based on the second adversarial sample and the random sample, a third adversarial sample close to the decision boundary is determined through binary search.

[0051] In the embodiments of the present application, after the second adversarial sample is determined through the gradient direction sign estimation in the iteration, in order to make the obtained adversarial sample closer to the decision boundary, a binary search is used to approximate the boundary point after the gradient direction sign estimation step, so as to utilize the different characteristics of the sample class labels inside and outside the decision boundary, and the gradient direction sign estimation is performed on the point close to the boundary in the next iteration process. Therefore, the target parameter is continuously adjusted by using the second parameter adjustment formula, and then the third adversarial sample close to the decision boundary is determined based on the target parameter through the second parameter adjustment formula.

[0052] The second parameter adjustment formula is The distance to be moved to approach the target boundary point is found by adjusting the size of the target parameter β. x i+1 , x i+2 respectively represent the i+1-th and i+2-th steps of the adversarial sample; and x0-x i+1 represents the adversarial sample xi+1 to the original sample x0; the target parameter wherein ε is a second preset threshold, if the label obtained by querying the adversarial sample is inconsistent with the original sample and greater than the second preset threshold ε, then the value of the target parameter β is half of the original value, otherwise the value of β remains unchanged. It can be seen that the parameter β changes with whether the queried adversarial sample is consistent with the original label, that is, if the sample is outside the boundary, then the value of β is reduced, otherwise the value of β remains unchanged.

[0053] Step S14: performing first-order manifold projection on the third adversarial sample by using a preset adversarial sample generation formula to obtain a final adversarial sample on a natural pose manifold.

[0054] In view of the problem of imperceptible difference, Diao et al. enhanced the dynamic similarity between the original sample and the adversarial sample by using the method of manifold projection to deceive the visual perception of the human eye. Therefore, in the embodiments of the present application, a first-order manifold projection function is designed to generate a manifold adversarial sample to ensure the imperceptibility of the adversarial sample. Specifically, after performing gradient direction symbol estimation and boundary point approximation, the manifold projection is continued, and the unnatural adversarial sample after perturbation is projected onto the manifold to maintain the imperceptibility of the finally generated adversarial sample.

[0055] Consider a trained action recognizer F: x→R k , the predicted label F x (x) can be obtained by querying y (x) = arg max j∈k f(x), where f(x)∈R k indicates the confidence score of each category of skeletal action. In the embodiments of the present application, the adversarial sample is generated by wherein D is the Euclidean distance, x represents the original motion, x' represents the adversarial sample of the original motion; M represents the manifold, and the obtained adversarial sample is constrained to the manifold to form an adversarial sample more consistent with human kinematics; F x represents the category label of the original sample, and F x′ represents the category label of the adversarial sample.

[0056] It can be understood that the manifold x'∈M in the above formula for generating the adversarial sample represents the bone length and joint angle constraint. The bone length can be calculated by the distance between two joint positions, and the joint rotation is calculated by Euler angle, and only the first-order derivative is used to maintain the imperceptibility of the adversarial sample. Therefore, the above formula is replaced by wherein x' and x" are the adversarial samples before and after manifold projection, and are the first-order derivatives of x' and x", and ω is the weight; Bi and B i "respectively are the length of the i-th bone in the original sample and the adversarial sample, we constrain the bone length to be constant in each frame, we can automatically preserve the bone length by inverse kinematics; the θ i "are the joint angles, in the range of and After inverse kinematics, we match the appropriate joint angles, and then get the joint positions according to forward kinematics.

[0057] In summary, we find a random sample close to the boundary and different from the original sample by binary search, and then in the algorithm iteration, first, we estimate the gradient direction sign by , adjust the step size to be taken in the gradient direction by , and then find the next sample point along the gradient direction by and ; then, we approach the boundary point on the decision boundary by binary search; finally, we find the adversarial sample on the manifold by first-order manifold projection. For example, the pseudo code of the Fast Guided Decision Attack based on Gradient Signs for Skeletal Action Recognition (FGDGS) algorithm is as follows:

[0058]

[0059]

[0060] In the above pseudo code, formula (1): formula (2): formula (3): formula (4):

[0061] This application provides a black-box adversarial example generation method. It obtains original samples of skeletal motion and initializes a random sample of a different category from the original sample. Based on the original sample and the random sample, a first adversarial sample is determined through binary search. Gradient direction sign estimation is performed on the first adversarial sample based on historical information corresponding to successful black-box attacks to determine a second adversarial sample iteratively along the gradient direction. Based on the second adversarial sample and the random sample, a third adversarial sample close to the decision boundary is determined through binary search. Using a preset adversarial example generation formula, a first-order manifold projection is performed on the third adversarial sample to obtain the final adversarial sample on the natural pose manifold. It is evident that the technical solution of this application generates skeletal adversarial examples based on gradient direction sign estimation, without relying on precise gradients, thus generating adversarial examples with lower query volume. Simultaneously, using historical information obtained from queries for gradient sign direction estimation allows for faster acquisition of the skeletal adversarial sample closest to the original sample along the boundary, guiding the adversary to quickly determine an effective attack direction. Projecting the skeletal motion sample onto the manifold coordinates the overall skeletal movement, enhancing the imperceptibility of the skeletal adversarial example. In this way, a difficult-to-detect adversarial example is generated under the premise of limited query budget.

[0062] like Figure 2 The diagram illustrates a specific black-box adversarial attack method, showcasing a gradient-symbol-based fast guided decision attack for skeletal motion recognition on a single frame. Here, B is the decision boundary, M is the natural pose manifold, and x is the original sample. adv It is an adversarial sample obtained from the attack, x * It is a random sample (of a different class than the original sample). and These are sample points during the iteration process.

[0063] First, initialize an initial sample x of a different (or specific) class than the original sample. * Then, the boundary point is approximated by binary search. Black-box adversarial attacks based on skeletal motion recognition involve three iterative steps: gradient direction estimation, approximation of boundary points, and first-order manifold projection. Gradient direction estimation is based on historical information from successful attacks to estimate points. The next direction to move in, and the step size to search in that direction. walk to Approaching the boundary point is achieved by finding the point on the decision boundary through binary search. Finally, by projecting the first-order manifold... Projecting onto the manifold determines the final adversarial examples.

[0064] Accordingly, this application also discloses a black-box adversarial sample generation device, see [link to relevant documentation].Figure 3 The device comprises:

[0065] a sample obtaining module 11, configured to obtain an original sample for a skeletal action, and initialize a random sample different from a category of the original sample;

[0066] a gradient direction symbol estimation module 12, configured to determine a first adversarial sample through dichotomy based on the original sample and the random sample, and perform gradient direction symbol estimation on the first adversarial sample based on historical information corresponding to a successful black-box attack, to determine a second adversarial sample iterated along a gradient direction;

[0067] a boundary point approximation module 13, configured to determine a third adversarial sample close to a decision boundary through dichotomy based on the second adversarial sample and the random sample;

[0068] a first-order manifold projection module 14, configured to perform first-order manifold projection on the third adversarial sample by using a preset adversarial sample generation formula, to obtain a final adversarial sample on a natural pose manifold.

[0069] The more specific working processes of the above modules can refer to the corresponding contents disclosed in the foregoing embodiments, which will not be repeated here.

[0070] As can be seen, the above scheme of the embodiment obtains an original sample for a skeletal action, and initializes a random sample different from a category of the original sample; determines a first adversarial sample through dichotomy based on the original sample and the random sample, and performs gradient direction symbol estimation on the first adversarial sample based on historical information corresponding to a successful black-box attack, to determine a second adversarial sample iterated along a gradient direction; determines a third adversarial sample close to a decision boundary through dichotomy based on the second adversarial sample and the random sample; and performs first-order manifold projection on the third adversarial sample by using a preset adversarial sample generation formula, to obtain a final adversarial sample on a natural pose manifold. It can be seen that the technical scheme of the application generates a skeletal adversarial sample based on gradient direction symbol estimation, and does not depend on accurate gradient, so that the generated adversarial sample has lower query amount; meanwhile, the historical information obtained through query is used for gradient symbol direction estimation, so that the skeletal adversarial sample closest to the original sample is obtained along the boundary more quickly, guiding the enemy to quickly determine an effective attack direction; the skeletal action sample is projected onto a manifold, coordinating the overall skeletal movement and enhancing the imperceptibility of the skeletal adversarial sample. In this way, an adversarial sample that is not easy to be perceived is generated under the premise of limited query budget.

[0071] Further, the embodiment of the application also discloses an electronic device, Figure 4is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content of the diagram should not be considered as any limitation on the scope of use of the present application.

[0072] Figure 4 A structural diagram of an electronic device 20 is provided for the embodiments of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, which is loaded and executed by the processor 21 to implement the related steps in the black-box adversarial sample generation method disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the present embodiment can be a computer.

[0073] In the present embodiment, the power supply 23 is configured to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 is capable of creating a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solutions of the present application, which is not specifically limited herein; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not specifically limited herein.

[0074] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, and data 223, etc., and the data 223 can include various data. The storage mode can be temporary storage or permanent storage.

[0075] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the black-box adversarial sample generation method executed by the electronic device 20 disclosed in any of the preceding embodiments, the computer program 222 can further include computer programs capable of completing other specific work.

[0076] Further, the embodiments of the present application also disclose a computer readable storage medium, which comprises a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a magnetic or optical disk or any other form of storage medium known in the technical field. The computer program is executed by a processor to implement the foregoing black-box adversarial sample generation method. For the specific steps of the method, refer to the corresponding content disclosed in the foregoing embodiments, which will not be described here again.

[0077] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0078] The steps of the black-box adversarial sample generation or algorithm described in combination with the embodiments disclosed in the present text can be implemented directly by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a CD-ROM or any other form of storage medium known in the technical field.

[0079] Finally, it should be further noted that, in the present text, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or sequence between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0080] The above describes in detail the black-box adversarial sample generation method, device, equipment and medium provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above example description is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A black-box adversarial example generation method, characterized in that, include: Obtain the original sample for skeletal motion, and initialize a random sample of a different category from the original sample; Based on the original sample and the random sample, the first adversarial sample is determined by binary search, and the gradient direction sign is estimated on the first adversarial sample based on the historical information corresponding to the successful black-box attack, so as to determine the second adversarial sample iterating along the gradient direction. Based on the second adversarial sample and the random sample, a third adversarial sample close to the decision boundary is determined by binary search; Using a preset adversarial sample generation formula, the third adversarial sample is projected onto a first-order manifold to obtain the final adversarial sample on the natural attitude manifold. The step of performing gradient direction sign estimation on the first adversarial sample based on historical information corresponding to a successful black-box attack, in order to determine the second adversarial sample iterating along the gradient direction, includes: Based on the historical information corresponding to a successful black-box attack, the first adversarial sample predefines a difference function and determines the sign function of the difference function; The gradient direction of the second adversarial sample is determined by the Monte Carlo method, then the sign of the gradient direction is estimated based on the gradient direction, and the step size in the gradient direction is determined by the first parameter adjustment formula, so as to determine the second adversarial sample iterating along the gradient direction according to the gradient direction sign and the step size; The step of using a preset adversarial sample generation formula to project the third adversarial sample onto a first-order manifold to obtain the final adversarial sample on the natural attitude manifold includes: use The third adversarial sample is projected onto a first-order manifold to obtain the final adversarial sample on the natural pose manifold; wherein, It is Euclidean distance. and These are adversarial examples before and after manifold projection. and for and The first derivative, It is weight; and These are the first and second skeletal actions in the original sample and the first skeletal action in the adversarial sample, respectively. The length of the root bone; the stated For joint angle, in and Within the range, This indicates the category label of the adversarial example.

2. The black-box adversarial example generation method according to claim 1, characterized in that, The difference function is ,in, These are adversarial examples of the original motion; For trained with Action Recognizer by Action Category , ; Confidence scores for skeletal movements in each category. , , Indicates the predicted class label of the sample; The sign function of the difference function is , .

3. The black-box adversarial example generation method according to claim 1, characterized in that, The gradient direction is , As a preset positive parameter, Action category; By custom Gaussian distribution From this, we can conclude that , The degree of bias in the information retrieved; The gradient direction symbol is ; The formula for adjusting the first parameter is: ; , They represent the first , Adversarial examples of the step, parameters Let the step size be the number of steps. Parameters of the next iteration The initial value is , , This is the first preset threshold.

4. The black-box adversarial example generation method according to claim 1, characterized in that, The step of determining a third adversarial sample close to the decision boundary through binary search based on the second adversarial sample and the random sample includes: Based on the second adversarial sample and the random sample, the target parameter in the second parameter adjustment formula is determined by binary search, so that a third adversarial sample close to the decision boundary can be determined based on the target parameter using the second parameter adjustment formula.

5. The black-box adversarial example generation method according to claim 4, characterized in that, The formula for adjusting the second parameter is as follows: ; , They represent the first , Adversarial examples of the steps; The adversarial sample To the original sample The direction; The target parameters ,in, The second preset threshold is set if the label obtained from the adversarial sample query is inconsistent with the original sample and is greater than the second preset threshold. Then the target parameter The value is half of the original value, otherwise The value remains unchanged.

6. A black-box adversarial sample generation device, characterized in that, include: The sample acquisition module is used to acquire raw samples for skeletal movements and initialize a random sample of a different category than the raw samples. The gradient direction sign estimation module is used to determine the first adversarial sample by binary search based on the original sample and the random sample, and to perform gradient direction sign estimation on the first adversarial sample based on the historical information corresponding to the successful black-box attack, so as to determine the second adversarial sample iterating along the gradient direction. The boundary point approximation module is used to determine a third adversarial sample that is close to the decision boundary through binary search based on the second adversarial sample and the random sample. A first-order manifold projection module is used to project the third adversarial sample onto a first-order manifold using a preset adversarial sample generation formula, so as to obtain the final adversarial sample on the natural attitude manifold. Specifically, the gradient direction sign estimation module is used to predefine a difference function based on the historical information corresponding to the successful black-box attack using the first adversarial sample, and determine the sign function of the difference function; determine the gradient direction of the second adversarial sample using the Monte Carlo method, then estimate the gradient direction sign based on the gradient direction, and determine the step size in the gradient direction using a first parameter adjustment formula, so as to determine the second adversarial sample iterating along the gradient direction according to the gradient direction sign and the step size; The first-order manifold projection module is specifically used to utilize... The third adversarial sample is projected onto a first-order manifold to obtain the final adversarial sample on the natural pose manifold; wherein, It is Euclidean distance. and These are adversarial examples before and after manifold projection. and for and The first derivative, It is weight; and These are the first and second skeletal actions in the original sample and the first skeletal action in the adversarial sample, respectively. The length of the root bone; the stated For joint angle, in and Within the range, This indicates the category label of the adversarial example.

7. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the black-box adversarial sample generation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein the computer programs, when executed by a processor, implement the black-box adversarial sample generation method as described in any one of claims 1 to 5.

Images