Android malicious application confrontation sample generation method based on Actor-Critic framework

Abstract

The invention discloses an Android malicious application adversarial sample generation method based on an Actor-Critic framework. The method comprises the following steps: firstly, extracting permission, API (Application Program Interface) calling and DEX bytecode features of an Android malicious application by utilizing a tool, and generating an RGB (Red, Green and Blue) feature image through word frequency statistics and normalization processing; then, a reinforcement learning agent comprising a state encoder, a strategy network (Actor), a value evaluation network (Critic) and a disturbance decoder is constructed; the state encoder is fused with a space attention mechanism and can adaptively pay attention to the key area of the image; the intelligent body outputs an action vector and maps the action vector into initial disturbance, and a gradient attack algorithm with a momentum mechanism and self-adaptive disturbance truncation is guided to generate an adversarial sample; according to the method, a reinforcement learning Actor-Critic framework is used, and the momentum is combined, so that the generation time of the generated adversarial sample is shortened, and the method has extremely high structural similarity.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence security and Android malware detection technology, specifically to a method for generating adversarial examples of Android malicious applications based on reinforcement learning agent guidance and combining momentum mechanisms and adaptive perturbations. The following are some key technical areas related to the method for generating adversarial examples of Android malicious applications:

[0002] Feature extraction and preprocessing: Android software has static features such as permissions, API calls, and opcodes. Feature extraction and preprocessing involves extracting these features from the Android software and using technologies such as artificial intelligence to process them, making the model easier to understand.

[0003] Machine Learning and Deep Learning: Deep learning-based visualization-based malware detection is a technique that transforms abstract program analysis into an image recognition problem. Its core lies in preprocessing static features of Android application (APK) data, such as permissions, API call sequences, and DEX bytecode, through word frequency statistics and normalization, mapping and fusing them into the red, green, and blue color channels of an RGB image. Subsequently, mature convolutional neural network models (such as ResNet and DenseNet) are used to train and classify these images to identify malicious structures, achieving high-efficiency and high-precision malware family classification or benign / malicious determination. While this method excels in resisting code obfuscation, its fundamental flaw, like other image classification models, is extremely sensitive to tiny pixel-level perturbations (adversarial examples), easily leading to misjudgments. Background Technology

[0004] With the widespread adoption of the Android operating system, the number of malicious applications has surged, rendering traditional signature-based detection methods inadequate. While deep learning-based detection methods, particularly those that classify APK features into RGB images, offer high accuracy, they have proven vulnerable to adversarial attacks.

[0005] Existing adversarial attack methods, such as Fast Signed Gradient Method (FGSM) and Projective Gradient Descent (PGD), can generate effective adversarial examples, but they have obvious drawbacks: Traditional PGD methods rely on fixed step size and gradient direction, which makes them prone to getting trapped in local optima, and they have slow convergence speed and low generation efficiency; in addition, existing methods often fail to balance the success rate of the attack with the structural similarity of the image, making the generated adversarial examples easy to be recognized by the human eye or filtered by defense mechanisms. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework, which addresses the above-mentioned shortcomings.

[0007] To solve the above technical problems, the present invention adopts the following solution:

[0008] A method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework includes the following steps:

[0009] Step 1: Obtain an existing Android application dataset, extract its permission features, API call features, and DEX file features, and map these three types of features to the R, G, and B channels of an RGB image respectively to generate feature images of uniform specifications and form a training set;

[0010] Step 2: Initialize the adversarial example generation agent. The adversarial example generation agent is based on the Actor-Critic architecture and includes a state encoder for extracting features, a policy network for generating actions, a value evaluation network for evaluating the value of the policy, and a perturbation decoder for generating initial perturbations.

[0011] Step 3: Extract feature image samples from the training set and input them into the state encoder of the reinforcement learning agent to generate adversarial examples. Extract the state vector through a convolutional neural network and spatial attention mechanism.

[0012] Step 4: The policy network outputs a high-dimensional action vector based on the state vector. The perturbation decoder maps the action vector to an initial perturbation matrix with the same size as the feature image. The value evaluation network jointly receives the state vector and the action vector and outputs the expected Q value of the current state-action pair.

[0013] Step 5: Using the initial perturbation matrix as the starting point, the feature image is iteratively attacked using the momentum projection gradient descent and adaptive perturbation truncation algorithm to generate adversarial examples; the momentum projection gradient descent and adaptive perturbation truncation algorithm introduces a momentum term to smooth the gradient direction during the iteration process and dynamically adjusts the perturbation amplitude according to the attack result.

[0014] Step 6: Input the adversarial examples into the target detection model, obtain the classification results, attack success rate, and structural similarity, calculate the comprehensive reward value, and use the comprehensive reward value and expected Q value to update the agent network parameters in Step 2 using the backpropagation algorithm;

[0015] Step 7: Continue to iterate through Steps 3 to 6 until the preset number of training rounds or convergence conditions are reached to obtain a trained adversarial example generation reinforcement learning agent. Then, use the trained adversarial example generation agent to generate adversarial examples.

[0016] Furthermore, step 1 specifically includes: extracting permission features, API call features, and DEX bytecode features from the APK file; generating a grayscale image through word frequency statistics, logarithmic normalization, and normalization operations; mapping the above three types of features to the R, G, and B channels of the RGB image respectively; and fusing them to generate a unified feature image.

[0017] Furthermore, the adversarial example generation agent in step 2 includes a state encoder, a policy network, and a value evaluation network.

[0018] The state encoder uses a multi-layer convolutional neural network and the LeakyReLU activation function to extract features, and a spatial attention module is connected after the convolutional layer to enhance attention to regions sensitive to adversarial perturbations by calculating feature weight maps; finally, a fixed-dimensional state vector is output through a fully connected layer. ;

[0019] The policy network is a multilayer perceptron network used to receive the state vector. Output high-dimensional action vectors This vector represents the potential direction of the initial perturbation;

[0020] The perturbation decoder has two parallel paths, one of which processes the action vector. The mapping is based on the fundamental perturbation features, and another is used to generate a spatial weight map; this is used to generate an initial perturbation matrix with spatial guidance that is consistent with the image size by multiplying the two element-wise. ;

[0021] The value assessment network is a multilayer perceptron network, used to jointly receive the state vector s and the action vector. The expected Q-value of the current state-action pair is output to guide the optimization of the Actor policy.

[0022] Furthermore, step 3 specifically includes the following steps:

[0023] Step 3-1: Extract the feature image samples As input to the state encoder;

[0024] Step 3-2: The state encoder processes the feature image samples sequentially through convolutional layers and a spatial attention module. Then, the pixel-level weights are used to weight the features to obtain the weighted feature map.

[0025] Step 3-3: Map the weighted feature map to a fixed-dimensional state vector through a fully connected layer. , It can reflect the distribution and importance of key features in an image.

[0026] Furthermore, step 4 specifically includes the following steps:

[0027] Step 4-1: The state vector generated in step 3... Input is fed into the policy network to execute the current policy. Output high-dimensional action vectors ;

[0028] Step 4-2: Convert the motion vectors The input is fed into the perturbation decoder, where feature transformation is performed through its internal basic perturbation path and spatial weight path to obtain the basic perturbation features and spatial weight map.

[0029] Step 4-3: Multiply the basic perturbation features with the spatial weight map to finally output the initial perturbation matrix. The matrix This is used to provide a high-quality starting point for subsequent gradient attacks.

[0030] Furthermore, step 5 specifically includes the following steps:

[0031] Step 5-1: Set the iteration parameters, including the momentum factor. Step length The maximum number of iterations T, and the initial perturbation As a perturbation in the first iteration;

[0032] Step 5-2: After multiple iterations until the maximum number of iterations T is reached, output the final adversarial example; the process of each iteration is as follows:

[0033] In the t-th iteration, calculate the gradient of the loss function J of the object detection model. And combined with the momentum from the previous iteration With current momentum Perform an update, then update the perturbation again to obtain the perturbation for local iteration. Apply the current perturbation limits during the update. Perform projection truncation to ensure the perturbation amplitude does not exceed the limit, based on the adversarial examples obtained in the previous iteration. Add to The adversarial examples generated in this iteration are obtained. ;

[0034] Step 5-3: Based on the attack results of the adversarial examples generated in the current iteration, dynamically adjust the perturbation constraints for the next iteration. If the attack is successful, the damage will decrease. If the attack fails, then remain The design remains unchanged, balancing attack success rate and image concealment.

[0035] Furthermore, step 6 specifically includes the following steps:

[0036] Step 6-1: Analyze the adversarial examples output in Step 5. Input the data into the object detection model and obtain the model's classification results. and loss function L;

[0037] Step 6-2: Calculate adversarial examples Compared with the original feature image Structural similarity metrics and attack success rate between them;

[0038] Step 6-3: Combining attack success rate and structural similarity, a preset reward function is applied. Calculate the overall reward value used to guide agent training. The agent network parameters in step 2 are updated using a backpropagation algorithm based on the comprehensive reward value and the expected Q value. The design goal of the reward function is to simultaneously maximize the attack success rate of the model and the image structural similarity of the adversarial examples.

[0039] The beneficial effects of this invention are as follows:

[0040] (1) This invention effectively smooths the gradient direction by introducing a momentum mechanism into gradient descent, avoids attack path oscillations, and significantly accelerates convergence. At the same time, the policy guidance of the reinforcement learning agent reduces the number of iterations required, and the generation time of adversarial examples can be shortened to the millisecond level, realizing efficient and fast attacks;

[0041] (2) This invention utilizes the Actor-Critic reinforcement learning framework to endow the perturbation generation process with intelligent decision-making capabilities. The agent can adaptively adjust the perturbation strategy according to the state of the target model, effectively avoiding the problem that traditional PGD is prone to getting trapped in local optima, and ensuring that adversarial examples can maintain a high level of attack success rate when facing different detection models;

[0042] (3) This invention designs an adaptive perturbation truncation strategy to immediately attenuate the perturbation amplitude when the attack is successful. This forces the model to make misjudgments with minimal perturbation. This ensures that the generated adversarial examples maintain a very high structural similarity to the original feature images, greatly improving the stealth of the adversarial examples;

[0043] (4) The reinforcement learning agent has global policy search capability through the closed loop of "decision-feedback-optimization". It can learn the most aggressive perturbation direction from the high-dimensional feature space, overcome the limitations of attacking purely based on local gradient information, and make the generated adversarial examples more robust.

[0044] (5) This invention employs a strategy of fusing and mapping three types of features—permissions, API calls, and DEX bytecode—to RGB images, ensuring that adversarial examples perturb all key static features contained in the image. This makes the generated adversarial examples effective not only against single target models but also against various CNN detection models (such as ResNet and DenseNet) trained on different feature subsets.

[0045] The present invention will now be described in detail with reference to the accompanying drawings and examples. Attached Figure Description

[0046] Figure 1 The flowchart of the Android malicious application adversarial sample generation method based on the Actor-Critic framework is shown.

[0047] Figure 2 The flowchart for generating RGB feature images by fusing multiple features in an APK is shown.

[0048] Figure 3 The flowchart of the iterative process of the core attack algorithm (M-PGD-AP) is shown;

[0049] Figure 4 A schematic diagram of the reinforcement learning agent (AC agent) network structure is shown; Detailed Implementation

[0051] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are for illustrative purposes only and are not intended to limit the scope of the invention.

[0052] This invention provides a method for generating adversarial examples for Android malicious applications based on the Actor-Critic framework. It aims to address the problems of low generation efficiency and difficulty in balancing attack success rate and stealth in existing adversarial attack methods (such as PGD) when attacking deep learning-based visual Android malware detection models. This invention introduces the Actor-Critic reinforcement learning framework to achieve intelligent decision-making on the perturbation direction and efficiently generates hard-to-detect adversarial examples by improving the momentum mechanism and adaptive perturbation truncation strategy in the gradient descent process.

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions used in this invention will be described clearly and completely below with reference to the accompanying drawings. The examples given are only for explaining this invention and are not intended to limit the scope of this invention.

[0054] This invention provides a method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework, the flowchart of which is shown below. Figure 1 As shown, it includes:

[0055] S1: Feature extraction and feature image generation;

[0056] In step S1, this invention constructs two datasets using two publicly available datasets and one widely used dataset. Dataset 1 is the CICmalDroid2020 dataset created by the Canadian Cyber ​​Security Institute; Dataset 2 includes datasets collected using the AndroZoo tool. Dataset 1, CICmalDroid, contains 17,341 malware samples from different malware families, including Adware, Banking, SMS, and Riskware. Using professional tools such as AndroGuard, APK files are decompiled and statically analyzed to accurately extract three core features: permission features, API call features, and DEX bytecode features. The processed feature matrices are then mapped to the three channels of a standard RGB image.

[0057] S2: Initialize the reinforcement learning agent architecture.

[0058] This step is one of the core innovations of this invention, aiming to construct an Actor-Critic agent for generating initial perturbation guidance. The state encoder structure is defined as follows: a three-layer convolutional network is used for feature extraction, with the number of channels adjustable from 3 to 32 to 64 to 128. The kernel size is 3×3, the stride is 2, and the activation function is LeakyReLU. A spatial attention module is embedded after the convolutional layers to learn the pixel-level weight distribution on the feature map. Finally, a fully connected layer outputs a 256-dimensional state vector. Policy network definition: Construct a multilayer perceptron network to receive a 256-dimensional state vector. Output action vectors with a dimension of 64. This vector represents the direction of the perturbation. The perturbation decoder is defined as follows: two parallel paths are designed to receive the action vector. One path generates the basic perturbation features (using Tanh activation), and the other path generates the spatial weight map (using Sigmoid activation); the two are multiplied element-wise to generate the feature image. Initial perturbation matrix of uniform size Value assessment network definition: Construct a multilayer perceptron network to jointly receive state vectors. and action vectors Output a scalar, i.e., the current state-action pair. The expected Q-value is used to guide the optimization of the Actor strategy.

[0059] S3: Extract the state vector.

[0060] This step is the process of the intelligent agent perceiving the environment: The feature image generated in step 1 is... The image is input into the perturbation decoder defined in step 2. After feature extraction via convolutional layers, a spatial attention mechanism is used for weighted processing, making the feature vector more focused on regions sensitive to adversarial attacks. The encoder then transforms the weighted feature map into a fixed-dimensional 256-dimensional state vector. This serves as the input for the Actor network to make decisions.

[0061] S4: Generate the initial perturbation matrix.

[0062] Initial perturbation matrix The generation of the action vector is the core step in this invention, connecting the reinforcement learning policy with the actual image attack. It is entirely executed by the perturbation decoder, aiming to transform the 64-dimensional action vector output by the policy network into a single action vector. This is transformed into a spatially guided perturbation matrix with the same size as the feature image. The decoder cleverly designs two parallel processing paths to ensure the quality of the perturbation: the first path receives the action vector... Subsequently, a multi-layer fully connected network combined with deconvolution or upsampling operations expands its dimensions from an abstract 64-dimensional array to a 3×W×H size and number of channels consistent with image features, generating basic perturbation features. A Tanh activation function is then used at the output to strictly constrain the perturbation values ​​to the range (-1, 1), ensuring that the initial perturbation amplitude remains within a controllable and reasonable range. The second path generates a single-channel 3×1×W×H spatial weight map through another independent fully connected network and upsampling operations. A Sigmoid activation function is used to limit its value range to (0, 1). This weight map acts as a spatial attention mask, indicating which pixel regions in the image are most sensitive to the classification results of the target model, i.e., requiring more perturbation. Finally, the perturbation decoder generates the final initial perturbation matrix by element-wise multiplying the basic perturbation features with the spatial weight map. This integration method ensures It not only includes the attack direction determined by the Actor policy, but also has pixel-level spatial selectivity, which can accurately concentrate the perturbation on key feature regions, thus providing efficient and high-quality initial guidance for subsequent momentum projection gradient descent and adaptive perturbation truncation attacks.

[0063] S5: Perform momentum projection gradient descent with adaptive perturbation truncation attack.

[0064] This step is the core iterative process of adversarial attacks, with step 4 as an example. Fine-tuning is done from the starting point, and momentum is accumulated from past gradient information to define a more stable and effective attack direction. Adversarial example generation has a fundamental formula... These are generated adversarial examples. It is the original sample. The given perturbation:

[0065]

[0066] Initialization and Guidance: Setting the Momentum Factor The weights used to control the influence of the momentum from the previous step on the current direction are used to smooth the gradient direction, accelerate convergence, and adjust the step size. Its function is to determine the magnitude of the perturbation's propagation in the gradient direction during each iteration step, with a maximum number of iterations T=10. As the initial perturbation, the gradient of the target detection model loss function J is calculated in each iteration t. And combined with historical momentum Update current cumulative momentum :

[0067]

[0068] in It is the L1 norm, which means that the current gradient vector is normalized to ensure that the gradient vector has a unit length. This reflects the momentum mechanism, specifically in the gradient of the loss function. middle, This represents the parameter set of the object detection model. The formula for the adversarial sample generated in round t-1 is: , This represents the actual classification label of the original malware sample. Indicates the current input sample The gradient indicates the direction in which the loss increases. This allows the attack process to utilize historical gradient information, accelerating convergence.

[0069]

[0070] Regarding the disturbance Update, apply current perturbation limits Perform projection truncation Ensure that the disturbance amplitude does not exceed .in, The function is a sign function, which records the accumulated momentum. direction, The function is a projection operation that projects the disturbance. Limited to superior, It is the first The result of the second perturbation. Setting initial perturbation limits. and minimum limit After each iteration, if the attack is successful, the perturbation is multiplied by the momentum factor. Compare with minimum constraints, based on the generated adversarial examples The attack results are dynamically adjusted. After iterating to round T, the final adversarial example is output. .

[0071] S6: Calculate reward value and update strategy.

[0072] This step is the training loop of reinforcement learning, optimizing the agent's strategy through environmental feedback. Input the adversarial examples generated in step 5. The object detection model is used to obtain the model's classification result (ASR) and its comparison with the original image. Structural similarity score (SSIM). Combine ASR and SSIM to calculate the overall reward value. The reward function is designed to simultaneously maximize attack success rate and image stealth (i.e., SSIM). The calculated reward value is then used... Based on Q-value estimation, and utilizing the TD (Temporal Difference) learning principle, the parameters of the Critic network's Q-value are updated using the backpropagation algorithm to improve its accuracy in evaluating the value of state-action pairs. Based on the Q-value evaluated by the Critic network, the Actor network is optimized using the policy gradient method and the backpropagation algorithm. The parameters enable the agent's policy to tend to generate initial perturbations that yield higher long-term rewards (R value). Steps 3 through 6 are iterated repeatedly until the preset number of training rounds or convergence criteria are reached.

[0073] S7: Calculate reward value and update strategy.

[0074] This step is the training loop of reinforcement learning, optimizing the agent's strategy through environmental feedback. Input the adversarial examples generated in step 5. The object detection model is used to obtain the model's classification result (ASR) and its comparison with the original image. Structural similarity score (SSIM). Combine ASR and SSIM to calculate the overall reward value. The reward function is designed to simultaneously maximize attack success rate and image stealth (i.e., SSIM). The calculated reward value is then used... Based on Q-value estimation, and utilizing the TD (Temporal Difference) learning principle, the parameters of the Critic network's Q-value are updated using the backpropagation algorithm to improve its accuracy in evaluating the value of state-action pairs. Based on the Q-value evaluated by the Critic network, the Actor network is optimized using the policy gradient method and the backpropagation algorithm. The parameters enable the agent's policy to tend to generate initial perturbations that yield higher long-term rewards (R value). Steps 3 through 6 are iterated repeatedly until the preset number of training rounds or convergence criteria are reached.

[0075] The method provided by this invention has the following advantages or beneficial technical effects:

[0076] Android malware is on the rise, posing a serious security threat to users' privacy and property. This invention introduces a momentum mechanism into gradient attacks, effectively smoothing the gradient update direction, avoiding oscillations during iteration, and significantly accelerating the convergence speed of the attack. Simultaneously, the reinforcement learning agent (AC Agent) can adaptively generate high-quality initial perturbation guidance, further reducing the number of iterations and ultimately achieving millisecond-level generation speed of adversarial examples, greatly improving generation efficiency. The Actor-Critic reinforcement learning framework is employed, endowing the perturbation generation process with intelligent decision-making capabilities. The agent can adaptively adjust the perturbation direction and magnitude based on the environmental state (feature image) and attack feedback (reward), effectively avoiding the local optima that gradient descent is prone to trap in.

[0077] The above description provides examples of the preferred embodiments of the present invention. Parts not detailed herein are common knowledge to those skilled in the art. The scope of protection of the present invention is determined by the claims. Any equivalent modifications based on the technical teachings of the present invention are also within the scope of protection of the present invention.

Claims

1. A method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework, comprising the following steps: Step 1: Obtain an existing Android application dataset, extract its permission features, API call features, and DEX file features, and map these three types of features to the R, G, and B channels of an RGB image respectively to generate feature images of uniform specifications and form a training set; Step 2: Initialize the adversarial example generation agent. The adversarial example generation agent is based on the Actor-Critic architecture and includes a state encoder for extracting features, a policy network for generating actions, a value evaluation network for evaluating the value of the policy, and a perturbation decoder for generating initial perturbations. Step 3: Extract feature image samples from the training set and input them into the state encoder of the reinforcement learning agent to generate adversarial examples. Extract the state vector through a convolutional neural network and spatial attention mechanism. Step 4: The policy network outputs a high-dimensional action vector based on the state vector. The perturbation decoder maps the action vector to an initial perturbation matrix with the same size as the feature image. The value evaluation network jointly receives the state vector and the action vector and outputs the expected Q value of the current state-action pair. Step 5: Using the initial perturbation matrix as the starting point, the feature image is iteratively attacked using the momentum projection gradient descent and adaptive perturbation truncation algorithm to generate adversarial examples; the momentum projection gradient descent and adaptive perturbation truncation algorithm introduces a momentum term to smooth the gradient direction during the iteration process and dynamically adjusts the perturbation amplitude according to the attack result. Step 6: Input the adversarial examples into the target detection model, obtain the classification results, attack success rate, and structural similarity, calculate the comprehensive reward value, and use the comprehensive reward value and expected Q value to update the agent network parameters in Step 2 using the backpropagation algorithm; Step 7: Continue to iterate through Steps 3 to 6 until the preset number of training rounds or convergence conditions are reached to obtain a trained adversarial example generation reinforcement learning agent. Then, use the trained adversarial example generation agent to generate adversarial examples.

2. The method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework according to claim 1, wherein step 1 specifically includes: Extract permission features, API call features, and DEX bytecode features from APK files; generate grayscale images through word frequency statistics, logarithmic normalization, and normalization operations; map the above three types of features to the R, G, and B channels of RGB images respectively, and fuse them to generate a unified feature image.

3. The method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework according to claim 1, wherein the adversarial sample generating agent in step 2 includes a state encoder, a policy network, and a value evaluation network: The state encoder uses a multi-layer convolutional neural network and the LeakyReLU activation function to extract features, and connects a spatial attention module after the convolutional layers to enhance attention to regions sensitive to adversarial perturbations by calculating feature weight maps; finally, it outputs a fixed-dimensional state vector through a fully connected layer. The policy network is a multilayer perceptron network, used to receive the state vector and output a high-dimensional action vector, which represents the potential direction of the initial perturbation. The perturbation decoder has two parallel paths: one maps the action vector to the basic perturbation feature, and the other generates a spatial weight map. These paths are used to generate an initial perturbation matrix with spatial guidance that is consistent with the image size by multiplying the two elements one by one. The value evaluation network is a multilayer perceptron network, used to jointly receive the state vector s and the action vector, and output the expected Q value of the current state-action pair to guide the optimization of the Actor policy.

4. The method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework according to claim 1, wherein step 3 specifically includes the following steps: Step 3-1: Use the extracted feature image samples as input to the state encoder; Step 3-2: The state encoder processes the feature image samples sequentially through convolutional layers and spatial attention modules, and then uses pixel-level weights to weight them to obtain the weighted feature map. Step 3-3: Map the weighted feature map to a fixed-dimensional state vector through a fully connected layer, which can reflect the distribution and importance of key features in the image.

5. The method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework according to claim 1, wherein step 4 specifically includes the following steps: Step 4-1: Input the state vector generated in step 3 into the policy network, execute the current policy, and output a high-dimensional action vector; Step 4-2: Input the action vector into the perturbation decoder, and perform feature transformation through its internal basic perturbation path and spatial weight path to obtain the basic perturbation feature and spatial weight map; Step 4-3: Multiply the basic perturbation features with the spatial weight graph to finally output the initial perturbation matrix, which is used to provide a high-quality starting point for subsequent gradient attacks.

6. The method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework according to claim 1, wherein step 5 specifically includes the following steps: Step 5-1: Set the iteration parameters, including the momentum factor. Step length The maximum number of iterations T, and the initial perturbation As a perturbation in the first iteration; Step 5-2: After multiple iterations until the maximum number of iterations T is reached, output the final adversarial example; the process of each iteration is as follows: In the t-th iteration, calculate the gradient of the loss function J of the object detection model. And combined with the momentum from the previous iteration With current momentum Perform an update, then update the perturbation again to obtain the perturbation for local iteration. Apply the current perturbation limits during the update. Perform projection truncation to ensure the perturbation amplitude does not exceed the limit, based on the adversarial examples obtained in the previous iteration. Add to The adversarial samples generated in this iteration are obtained. ; Step 5-3: Based on the attack results of the adversarial examples generated in the current iteration, dynamically adjust the perturbation constraints for the next iteration. If the attack is successful, the decay will occur. If the attack fails, then remain The design remains unchanged, balancing attack success rate and image concealment.

7. The method for generating adversarial samples for Android malicious applications based on the Actor-Critic framework according to claim 1, wherein step 6 specifically includes the following steps: Step 6-1: Analyze the adversarial examples output in Step 5. Input the data into the object detection model and obtain the model's classification results. and loss function L; Step 6-2: Calculate adversarial examples Compared with the original feature image Structural similarity metrics and attack success rate between them; Step 6-3: Combining attack success rate and structural similarity, a preset reward function is applied. Calculate the overall reward value used to guide agent training. The agent network parameters in step 2 are updated using a backpropagation algorithm based on the comprehensive reward value and the expected Q value. The design goal of the reward function is to simultaneously maximize the attack success rate of the model and the image structural similarity of the adversarial examples.

Images