An Active Defense AI Auto-Aiming Method Based on Differentiable Rendering

Abstract

The present invention provides an active defense AI self-aiming method based on micro-rendering, comprising: step 1, superimposing disturbances on a two-dimensional texture map to generate an initial adversarial texture; step 2, setting a micro-rendering environment and performing parameter randomization; step 3, performing micro-rendering and background fusion to generate a synthetic image closer to the real game screen; step 4, obtaining the detection results of each target detection model on the image; step 5, calculating the multi-target joint adversarial loss function; step 6, back propagation optimization and perturbation update; step 7, performing iterative optimization to generate the final texture with the ability to deceive AI self-aiming; step 8, replacing the original texture image of the corresponding character in the game client with the generated adversarial texture image, and the game engine will automatically load and use the adversarial texture image to render the character when running. The present invention can effectively interfere with the recognition process of cheating before it occurs by introducing adversarial perturbations at the texture level.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision technology, and in particular relates to an active defense AI self-aiming method based on differentiable rendering. Background Art

[0002] In recent years, first-person shooter (FPS) games have continued to be popular around the world, with a huge player base. However, as games become more popular, cheating problems have become increasingly serious, especially vision-based aiming robots (referred to as "AI self-aiming" or "plug-ins"). This type of plug-in captures the game screen, uses target detection models in computer vision (such as YOLO, Faster R-CNN, etc.) to automatically identify and lock enemy characters, and then simulates mouse operations to aim and shoot, which seriously undermines the fair competition environment of the game and damages the player experience and the interests of game manufacturers.

[0003] Traditional anti-cheating methods, such as memory scanning to detect the behavior of modifying game memory, or post-analysis based on player behavior data (such as VAC, BattleEye, etc.), are difficult to effectively deal with AI aiming. Because AI aiming usually runs outside the game process, does not directly modify game files or memory, and may simulate human operation modes, making detection very difficult and lagging.

[0004] To address this challenge, researchers began to explore active defense strategies. Adversarial sample technology was introduced into this field as a method to interfere with the judgment of deep learning models. By adding tiny perturbations that are difficult for the human eye to detect to the input image, the target detection model can produce incorrect recognition results. A small number of works have attempted to apply adversarial samples to FPS game anti-cheating, such as superimposing two-dimensional perturbations on the screen or modifying the texture of the scene (such as the wall). However, these methods have some inherent defects:

[0005] (1) Visual interference: Adding disturbances directly to the screen or background may significantly affect the visual experience of normal players and the aesthetics of the game screen.

[0006] (2) Lack of robustness: Two-dimensional perturbations are sensitive to dynamically changing factors such as lighting, viewing angle, and distance within the game, and the defensive effect is unstable.

[0007] (3) Limited transferability: Perturbations generated for a specific model may be ineffective for other types of visual self-aiming models, and the type of model used by the attacker is unknown.

[0008] (4) Performance overhead: Some methods that require real-time generation or adjustment of disturbances may impose additional computational burden on the client, affecting the game frame rate.

[0009] Therefore, there is an urgent need for a technical solution that can actively, effectively, robustly defend against visual aimbot cheating with the least impact on the player experience. Summary of the Invention

[0010] Object of the Invention: Aiming at the problems in the prior art that the anti-cheating means have poor visual aimbot effect, and the existing adversarial attack methods have visual interference, poor robustness, insufficient transferability, and affect performance, etc., the present invention aims to provide an active defense AI aimbot method based on differentiable rendering.

[0011] The present invention injects a carefully designed adversarial perturbation directly into the three-dimensional model texture of the game character, and uses a differentiable renderer for optimization, so that the final character image rendered on the screen can effectively deceive or interfere with the target detection model relied on by visual aimbot, realizing an effective defense against visual aimbot cheating.

[0012] The method of the present invention includes the following steps:

[0013] Step 1, Initialize adversarial perturbation and texture fusion: Obtain the original three-dimensional model mesh of the game character and the corresponding two-dimensional texture map T, where , R is the real number space, and represent the length and width of the texture map respectively; initialize an optimizable perturbation , , using random values uniformly distributed in the interval for initialization; superimpose the perturbation on the two-dimensional texture map T to generate an initial adversarial texture : , where function is used to limit the pixel value within the range;

[0014] Step 2, Set the differentiable rendering environment and randomize parameters: Select a differentiable renderer R, and the differentiable renderer R is used to calculate the gradient of the rendered output image with respect to the input parameters (texture); to simulate the variable observation conditions in the game, randomize the rendering parameters, and the rendering parameters include camera parameters and lighting parameters ;

[0015] Step 3, Perform differentiable rendering and background fusion: Apply the initial adversarial texture generated in Step 1 to the original three-dimensional model mesh through UV mapping (the process of mapping two-dimensional texture coordinates to the three-dimensional model surface. UV mapping is the core technology for texture mapping in 3D modeling. By unfolding the 3D model surface onto a 2D plane (U and V axes), the 2D texture can be accurately projected onto the model surface) on the surface, and then use the camera parameters randomly generated in step 2 and lighting parameters , through a differentiable renderer render the model with perturbed texture into a 2D image . During the rendering process, the texture sampling operation calculates the pixel color based on the vertex color affected by the perturbation, thereby transferring the perturbation to the rendered image. Randomly select a background image from the pre-prepared background image dataset , and combine the 2D image with the background image to generate a composite image closer to the real game screen ;

[0016] Step 4, perform image enhancement and multi-model evaluation: Perform image preprocessing operations on the composite image to simulate the processing steps that an attacker may adopt, and obtain the preprocessed image ; Input the image in parallel into a group of pre-selected general object detection models representing the prior art , , denotes the k-th object detection model (mainly using Faster RCNN and YOLO series), and obtain the detection results of each object detection model for the image ;

[0017] Step 5, calculate the multi-object joint adversarial loss function ;

[0018] Step 6, backpropagation optimization and perturbation update;

[0019] Step 7, repeat steps 2 to 6 for iterative optimization until the loss function converges, and the finally obtained optimized perturbation is added to the 2D texture map T to generate the final texture with the ability to deceive AI auto-aim;

[0020] Step 8, deployment and application: Replace the original texture image of the corresponding character in the game client with the generated adversarial texture image. The game engine will automatically load and use the adversarial texture image to render the character during runtime. When the AI auto-aim system observes a character carrying this adversarial texture, its detection performance will be significantly interfered, making it difficult to accurately lock and identify the target, thus achieving the purpose of active defense.

[0021] In step 2, the camera parameters include distance , pitch angle and azimuth 。Illumination parameters include light source intensity. The following camera parameters are randomly generated at each iteration:

[0022] ,

[0023] ,

[0024] ,

[0025] where s is the size of the game character model, and rand(1) represents a uniformly distributed random number in the interval [0, 1), represents an angle uniformly sampled from the interval to simulate the changing perspectives in the game.

[0026] In step 3, a two-dimensional image is fused with the generated mask m to obtain a synthesized image :

[0027] ,

[0028] where the mask m identifies the region of the rendered character in the two-dimensional image : that is, the mask region shows the rendered character, and the rest shows the background; ⊙ represents element-wise multiplication; step 3 aims to let the optimization process consider background interference so that the perturbation remains effective in complex scenarios.

[0029] In step 4, the preprocessing operations include random color jittering (randomly changing brightness, contrast, saturation, hue), cropping (simulating the situation where the target is incomplete in the picture), and scaling (simulating the target size at different distances).

[0030] In step 4, the k-th object detection model detects the image and the detection result is expressed as:

[0031] .

[0032] In step 4, the detection result includes the detected object category, confidence score, and bounding box information.

[0033] Step 5 includes: designing a comprehensive loss function to guide the optimization of the perturbation , and the loss function includes a category loss , a total variation loss , and a perturbation amplitude constraint loss :

[0034] ,

[0035] ,

[0036] ,

[0037] where represents the value of the perturbation at the \(i\)-th row, \(j\)-th column, and \(c\)-th channel, in the \(i\)-th row, \(j\)-th column, and \(c\)-th channel, represents the maximum value of the absolute values of all elements in the perturbation ; is the confidence score of the \(k\)-th model for the \(i\)-th detection of the target class, \(N\) is the number of detections, is the confidence score of the \(k\)-th model for the \(i\)-th detection of the human class, is the activation function;

[0038] The class loss aims to reduce the confidence of the surrogate model in the true target class (such as "person") and mislead it to an irrelevant class so that the target cannot be detected.

[0039] The total variation loss is used to penalize the differences between adjacent pixel values in the perturbation to improve the spatial smoothness of the perturbation and make it look more natural.

[0040] The perturbation magnitude constraint loss limits the maximum absolute value of the perturbation over all pixels and channels, ensuring that each component of the perturbation is not too large, thus more directly controlling the imperceptibility of the perturbation.

[0041] The various losses are weighted and combined as follows:

[0042] ;

[0043] Step 6 includes: Using the automatic differentiation function of the deep learning framework, calculate the gradient of the total loss function with respect to the optimizable perturbation . The calculation of this gradient uses automatic differentiation technology to achieve end-to-end backpropagation, quantifying the sensitivity of the loss function to the final optimization objective (perturbation ). This process specifically includes: The gradient signal starts from the loss , passes through the object detection model , image preprocessing operations, and background fusion in sequence, and is transmitted to the rendered image . Using the differentiable renderer \(R\), the gradient in the image space ( ), backpropagate and map to the texture space on the 3D model surface to obtain the gradient of the adversarial texture ; finally, through the (differentiable) reverse calculation of the texture overlay operation, obtain the final gradient required for guiding the perturbation update ; ; denotes the partial derivative.

[0044] Update the perturbation according to the gradient :

[0045] ,

[0046] where is the learning rate, is a constant, is the loss function with respect to the perturbation partial derivative, denotes the perturbation after the (t + 1)-th iteration update, denotes the perturbation at the t-th iteration.

[0047] Using the above steps, natural adversarial patches in the physical world can be generated. After applying the natural adversarial patches to the attack target, the target detector can be made to make misjudgments.

[0048] The present invention also provides an electronic device, including a processor and a memory. The memory stores program code, and when the program code is executed by the processor, the processor is caused to execute the steps of the method.

[0049] The present invention also provides a storage medium storing a computer program or instruction, and when the computer program or instruction runs on a computer, the steps of the method are executed.

[0050] The present invention has the following beneficial effects: (1) High - efficient active defense ability: This method directly acts on the input (rendered character image) on which the target detection model, the core of the AI auto - aiming tool, depends. By introducing adversarial perturbations at the texture level, it can effectively interfere with the recognition process before cheating behavior occurs. Experiments show that this method can significantly reduce the detection success rate (DSR) of various AI auto - aiming models (such as YOLO series, RTMDet, etc.), and in some models, it can even reach more than 99% (such as for YOLOv5s).

[0051] (2) Excellent visual concealment: By strictly restricting the perturbation amplitude ( norm constraint) and using the total variation loss ( To ensure the smoothness of perturbations, the generated adversarial textures are almost indistinguishable from the original textures to the human eye. Both subjective perception studies and objective image quality evaluation metrics confirm that the adversarial textures generated by this method have minimal impact on the visual experience of players, without affecting the recognition of character appearance and the immersion of the game.

[0052] (3) Strong robustness and generalization ability: By jointly optimizing proxy models with various different architectures, the perturbation patterns learned by the adversarial textures are universal and can effectively defend against other visual auto-aim models that were not involved in the training. Experiments have shown that the adversarial textures trained using YOLOv5x and RTMDet also exhibit good defense effects against models such as Faster R-CNN, YOLOv8n, and YOLOv3n. Secondly, during the generation process, by randomizing the camera perspective, distance, lighting conditions, integrating diverse background images, and simulating image preprocessing, the adversarial textures can maintain a high defense success rate in various complex game environments (different maps, open / closed spaces) and different observation conditions.

[0053] (4) Seamless integration and zero performance overhead: The generation process of this method is completely completed offline. During game runtime, the rendering engine only needs to load the replaced adversarial textures without any additional real-time calculations. Therefore, this method has no negative impact on the frame rate (FPS) of the game and the performance of the client, ensuring a smooth gaming experience, and is particularly suitable for FPS games with extremely high real-time requirements.

[0054] (5) Easy to deploy and expand: For game developers, deploying the adversarial textures generated by this method only requires replacing the texture files in the game resource package, which is simple to operate. This method can be applied to models of different characters and factions in the game and has good scalability.

[0055] In summary, the present invention provides a novel, efficient, stealthy, robust, and performance-lossless active defense solution, effectively addressing the deficiencies of existing technologies in dealing with visual auto-aim cheating and providing strong technical support for maintaining a fair competitive environment in FPS games. Brief Description of the Drawings

[0056] Figure 1 is a schematic diagram of the AI auto-aim workflow.

[0057] Figure 2 is a schematic diagram of the relationship between game character textures and AI auto-aim.

[0058] Figure 3 is the overall flowchart of the present invention.

[0059] Figure 4 is a qualitative comparison diagram of the defense effect of this method.

[0060] Figure 5 It is a curve graph showing the influence of the learning rate on the defense success rate (DSR) of this method.

[0061] Figure 6 It is a curve graph showing the influence of the perturbation amplitude on the defense success rate (DSR) of this method. Specific implementation manners

[0062] The following further specifically describes the present invention in conjunction with the accompanying drawings and specific implementation manners, and the above and / or other advantages of the present invention will become clearer.

[0063] The embodiment of the present invention provides an active defense AI aimbot method based on differentiable rendering and demonstrates its application effect in a real game scenario. Although this method has wide applicability and can be applied to various first-person shooting (FPS) games, the present invention will take a certain first-person shooting game as a specific example for illustration.

[0064] Figure 1 It shows the implementation scenario of the present invention, that is, a typical AI aimbot work process. The core of the AI aimbot is a target detection model. Therefore, the fundamental strategy for defending against the AI aimbot lies in interfering with or deceiving the target detection model. The conventional AI aimbot process is as follows: First, a real-time game screen is obtained through a screen capture software (such as common OBS, etc.); subsequently, the screen is input into the target detection model to identify and locate the enemy characters; finally, the operating system-provided API (such as Windows API) is used to control the mouse, precisely move it to the target position and trigger a shot. The whole process can usually be completed within 10 milliseconds, far exceeding the reaction speed of ordinary players, which is the reason why the AI aimbot poses a serious threat.

[0065] Figure 2 It explains how the in-game texture image is presented in the player's perspective. The visual presentation of game characters mainly depends on the three-dimensional mesh (Mesh) and texture (Texture). The mesh defines the basic geometric shape of the model, while the texture endows the model surface with rich details and colors. Essentially, the visual AI aimbot works by identifying the texture features of the character model. In order to endow the model surface with detailed images, UV unwrapping needs to be performed during the character creation process, that is, mapping the three-dimensional model surface to the two-dimensional UV coordinate space. The art designer then draws details such as skin and clothing on this two-dimensional texture image. As Figure 2As shown in the figure, these two-dimensional texture images are "fitted" to the surface of the three-dimensional model through UV mapping technology, that is, the pixels (texels) on the texture image are corresponding to the vertices of the model. Finally, the three-dimensional model with texture is processed by the rendering pipeline, combined with parameters such as model position, lighting conditions, camera angle, etc., and projected onto the two-dimensional screen to form the image that the player finally sees. This means that the pixel color and details displayed on the screen are directly derived from the texture information on the surface of the model. The visual AI self-aiming captures the screen image and uses the target detection model to analyze this texture information to identify the enemy character.

[0066] Combine the following Figure 3 , the specific steps of the method proposed by the present invention are described in detail:

[0067] First, for the dataset, obtain the game's character 3D models (mesh M) and their corresponding original texture maps T. For example, use a dataset containing 66 models of character 1 (CAT) and a dataset containing 73 models of character 2 (TAT). Then, take high-quality screenshots of different scenes from multiple game maps (such as Dust2, Mirage, Anubis, etc.), crop them to a uniform resolution (such as 640x640 pixels), and build a background dataset containing about 1,800 images ( ).

[0068] Step 1. Original texture for the selected character , initialize a Perturbation graph of the same size , whose value is given by Fill with uniform random numbers in the range. Generate initial adversarial texture , make sure the texture value is in Within the valid range.

[0069] Step 2, randomly generate the following camera parameters at each iteration:

[0070] ,

[0071] ,

[0072] ,

[0073] Where s is the size of the game character model, rand(1) represents a uniformly distributed random number in the interval [0, 1). Indicates from The angle sampled uniformly within the interval to simulate the changing viewing angles in the game. The lighting parameters mainly refer to the lighting intensity.

[0074] Step 3: Change the current adversarial texture Applied to the 3D model M. Using the random parameters generated in Step 2 , render the foreground image containing the adversarial model through a differentiable renderer (implemented using the Pytorch3D framework). At the same time, generate a mask according to (where the pixel value is 1 at non-zero positions and 0 at zero positions). Randomly select a background image from the background dataset and use the mask to fuse with to obtain a mixed image . .

[0075] Step 4, Apply data augmentation operations such as random scaling and color jitter to the fused image to obtain the final image input to the model . Input into the proxy models YOLOv5x and RTMDet to obtain the detection results , .

[0076] Step 5, Calculate the joint adversarial loss , where and are hyperparameters, set to 0.2 and 0.05 according to the experimental results. The class loss , is the number of models, is the number of detection boxes, is the confidence of human detection, is the confidence of non-human detection, used to reduce the model's detection confidence for the "human" category. The total variation loss calculates the total variation of the perturbation to control the smooth change of the perturbation. The perturbation amplitude constraint loss is used to control the maximum value of the perturbation.

[0077] Step 6, Optimization and update. Use the Adam optimizer to update according to . According to the ablation study results, the learning rate η = 0.005 gives better results. After the update, clip : , and keep the perturbation in the black area zero.

[0078] Step 7, Repeat Steps 2 - 6 for a fixed number of iterations (2000 times) until the loss converges. Obtain the final optimized perturbation . Generate the final adversarial texture .

[0079] The following combines Figure 4 , Figure 5 , Figure 6 to demonstrate the effectiveness of the method of the present invention in experimental and real game scenarios:

[0080] Figure 4 Intuitively shows the comparison of the detection effects of AI auto-aim before and after applying this method. Without using this method, AI auto-aim can accurately identify game characters, thus achieving cheating. After generating adversarial textures using this method, AI auto-aim will misidentify the target as other categories (it only attacks targets identified as "humans") or completely fails to detect the target, thus effectively defending against AI auto-aim cheating devices.

[0081] Figure 5 Shows the influence of different learning rates on the final generated adversarial texture effect. The experimental results show that when the learning rate is set around 0.005, the adversarial texture trained has the best defense effect against AI auto-aim.

[0082] Figure 6 Discusses the influence of the perturbation magnitude ( ) on the attack success rate (i.e., the defense effect). The results show that the larger the perturbation magnitude, the stronger the adversarial nature and the better the defense effect, but the more obvious the visual change of the texture. To achieve a balance between the defense effect and the player's visual experience, the present invention selects as the final setting. Under this setting, the visual impact on players by the perturbation is minimized, while still maintaining excellent defense performance against AI auto-aim.

[0083] To further evaluate the performance of the present invention, Table 1 in the appendix compares it with two benchmark methods: random perturbation (adding unoptimized noise to the texture) and AdvMap (adding perturbation to the screen image). Two key metrics, the defense success rate (DSR) and the structural similarity (SSIM), are used for the evaluation. DSR quantifies the degree of failure of AI auto-aim (i.e., the proportion of frames in which AI auto-aim fails to identify the target as "human"), and a higher value represents more effective defense; SSIM measures the visual similarity between the adversarial texture and the original texture, and the closer it is to 1, the smaller the impact on the player's visual experience. The experimental results (testing on multiple AI auto-aim cheating device models for the CAT and TAT datasets) clearly show that the method of the present invention is significantly superior to the two benchmark methods in terms of DSR. At the same time, its SSIM score is extremely close to 1, proving that this method can achieve a powerful defense against AI auto-aim with almost no impact on the original visual effect of the game.

[0084] Table 1

[0085]

[0086] The present invention provides an active defense AI self-aiming method based on differentiable rendering. There are many methods and ways to specifically implement this technical solution. The above is only the preferred implementation mode of the present invention. It should be noted that for those of ordinary skill in the art of this technology, without departing from the principle of the present invention, several improvements and retouches can be made, and these improvements and retouches should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by using the prior art.

Claims

1. An active defense AI self-aiming method based on differentiable rendering, characterized in that Including the following steps: Step 1: Initialize adversarial perturbation and texture fusion: Get the original 3D model mesh of the game character and the corresponding two-dimensional texture map T, where , R is the real number space, and Represent the length and width of the texture map respectively; initialize an optimizable perturbation with the same dimension as the two-dimensional texture map T , , used in Initialize with a random value uniformly distributed in the interval; Superimposed on the two-dimensional texture map T to generate the initial adversarial texture : ,in The function is used to limit the pixel value to within the scope; Step 2, set up a differentiable rendering environment and randomize parameters: Select a differentiable renderer R, which is used to calculate the gradient of the rendered output image with respect to the input parameters; randomize the rendering parameters, where the rendering parameters include camera parameters and lighting parameters ; Step 3, perform differentiable rendering and background fusion: Apply the initial adversarial texture generated in Step 1 to the surface of the original 3D model mesh through UV mapping , then use the randomly generated camera parameters in Step 2 and lighting parameters , and render the model with the perturbed texture into a 2D image through a differentiable renderer . During the rendering process, the texture sampling operation calculates the pixel color based on the vertex color affected by the perturbation, thereby transferring the perturbation to the rendered image. Randomly select a background image from the pre-prepared background image dataset , and fuse the 2D image with the background image to generate a synthetic image closer to the real game scene ; ; Step 4, perform image enhancement and multi-model evaluation: For the synthetic image perform image preprocessing operations to obtain the preprocessed image ; Parallelly input the image into a set of object detection models . Let denote the k-th object detection model, and obtain the detection results of each object detection model for the image ; ​ Step 5, calculate the multi-objective joint confrontation loss function ; Step 6, Backpropagation optimization and perturbation update; Step 7, repeat Steps 2 to 6 for iterative optimization until the loss function converges, and the finally obtained optimized perturbation is added to the two-dimensional texture map T to generate the final texture that can deceive AI auto-aiming; Step 8, Deployment and application: Replace the original texture image of the corresponding character in the game client with the generated adversarial texture image, and the game engine will automatically load and use the adversarial texture image to render the character during runtime.

2. The method according to claim 1, wherein In step 2, the camera parameters include distance, pitch angle, and azimuth angle, and the lighting parameters include light source intensity.

3. The method according to claim 2, characterized in that, In step 3, a two-dimensional image generated mask is used for fusion to obtain a composite image : ， where the mask m identifies the region of the rendered character in the two-dimensional image ; ⊙ represents element-wise multiplication.

4. The method according to claim 3, wherein In Step 4, the preprocessing operations include random color jittering, cropping, and scaling.

5. The method according to claim 4, wherein In step 4, the k-th object detection model detects the image and the detection result is expressed as: 。 6. The method according to claim 5, wherein In Step 4, the detection results include the detected target category, confidence score, and bounding box information.

7. The method according to claim 6, wherein Step 5 includes: designing a comprehensive loss function , which is used to guide the optimization of the perturbation , and the loss function includes a classification loss , a total variation loss and a perturbation amplitude constraint loss : ， ， ， Among them represents the perturbation at the row, the column, and the value at the channel; represents the maximum value of the absolute values of all elements in the perturbation ; is the confidence score of the i-th detection of the k-th model for the target category. N is the number of detections, is the confidence score of the i-th detection of the k-th model for the human category, is the activation function; Weighted combination of losses for each part: ， Among them and are weight parameters.

8. The method according to claim 7, wherein Step 6 includes: calculating the total loss function using the automatic differentiation function of the deep learning framework with respect to the optimizable perturbation gradient . The gradient signal starts from the loss , passes through the object detection model , image preprocessing operations, and background fusion in sequence, and is transmitted to the rendered image . Using the differentiable renderer R, the gradient in the image space is backpropagated and mapped to the texture space on the surface of the 3D model to obtain the gradient with respect to the adversarial texture ; finally, through the reverse calculation of the texture overlay operation, the final gradient required for updating the guidance perturbation is obtained; denotes partial differentiation; Update the perturbation according to the gradient : ， where is the learning rate, is a constant, represents the perturbation after the (t + 1)-th iteration update, represents the perturbation at the t-th iteration.

9. An electronic device, characterized in that, Including a processor and a memory, the memory stores program code, and when the program code is executed by the processor, the processor is caused to execute the steps of the method according to any one of claims 1 to 8.

10. A storage medium, characterized in that, Stored with a computer program or instructions, when the computer program or instructions are run on a computer, the steps of the method according to any one of claims 1 to 8 are executed.

Images