Three-dimensional Gaussian splash-driven multi-view robust physical confrontation camouflage generation method

Abstract

The invention provides a three-dimensional Gaussian splash-driven multi-view robust physical confrontation camouflage generation method, and the method comprises the steps: taking a multi-view picture around a to-be-camouflaged vehicle, taking the multi-view picture as a training set for scene reconstruction, and obtaining a sparse initial point cloud position; initializing a three-dimensional Gaussian ellipsoid by taking each point of the sparse point cloud as a center to form an initial three-dimensional Gaussian splashing scene; iterating for multiple times to obtain a trained three-bit Gaussian splashing scene, and extracting masks of a to-be-disguised vehicle in each multi-view photo; rendering each visual angle to obtain a rendered image, combining the to-be-disguised vehicle in the rendered image and the background in the clean picture, and combining to obtain a to-be-detected picture; sending a to-be-detected picture into a damaged target detector, calculating to obtain a detection loss, back-transmitting the loss and calculating a gradient so as to optimize an adversarial disguise color; and extracting a final deployable confrontation camouflage texture. According to the method, the problems of insufficient confrontation effect and robustness in different visual angles and physical environments can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of countermeasure camouflage generation, and in particular to a multi-perspective robust physical countermeasure camouflage generation method driven by three-dimensional Gaussian splashing. Background Art

[0002] Physical adversarial attack methods reveal the vulnerabilities of deep neural networks and pose a significant threat to safety-critical scenarios such as autonomous driving. Camouflage-based physical attacks are more promising than patch attacks, providing stronger adversarial effects in complex physical environments. However, most previous works rely on mesh priors of the target object and virtual environments constructed by simulators, which are time-consuming to acquire and inevitably differ from the real world. In addition, due to the limitations of the training image background, previous methods often fail to generate multi-viewpoint robust adversarial camouflages and are prone to falling into suboptimal solutions. As a result, these works lack adversarial effectiveness and robustness across different viewpoints and physical environments. Summary of the Invention

[0003] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide a three-dimensional Gaussian splash-driven multi-view robust physical adversarial camouflage generation method to solve the technical problems of insufficient adversarial effect and robustness in different viewpoints and physical environments in the prior art.

[0004] To achieve the above technical objectives, in a first aspect, the technical solution of the present invention provides a three-dimensional Gaussian splash-driven multi-view robust physical adversarial camouflage generation method, comprising the steps of:

[0005] Taking multi-view photos around the vehicle to be camouflaged as a training set for scene reconstruction, performing feature detection and feature matching on the multi-view photos, and recovering the camera pose and sparse scene structure from the multi-view photos to obtain a sparse initial point cloud position;

[0006] Initialize a 3D Gaussian ellipsoid with each point in the sparse point cloud as the center, and all initialized 3D Gaussian ellipsoids constitute the initial 3D Gaussian splash scene;

[0007] Iteratively updating the three-dimensional Gaussian splash scene multiple times to approximate the real scene to obtain a trained three-dimensional Gaussian splash scene, and extracting and saving the mask of the vehicle to be camouflaged in each of the multi-view photos in the training set;

[0008] Traversing the view angles in the training set, rendering each view angle to obtain a rendered image, and combining the vehicle to be disguised in the rendered image with the background in the clean image using the extracted mask to obtain a picture to be detected;

[0009] The image to be detected is fed into a victim detector to calculate a detection loss, which is then back-propagated to calculate the gradient. The zero-order term of the spherical harmonic coefficients of the surface color of each three-dimensional Gaussian in the three-dimensional Gaussian splash scene is updated by gradient descent to optimize the anti-camouflage color of the surface of the vehicle to be camouflaged.

[0010] When the optimization conditions are met, the final deployable adversarial camouflage texture is extracted based on the optimized vehicle three-dimensional Gaussian. By deploying the adversarial camouflage texture, the target vehicle cannot be correctly detected by the target detector.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] This patent proposes a physical attack framework based on 3D Gaussian Splatting (3DGS), named PGA. This framework provides fast and accurate reconstruction using a small number of images, and has photorealistic rendering capabilities. Furthermore, this patent enhances cross-viewpoint robustness and adversarial effectiveness by preventing mutual occlusion and self-occlusion between Gaussians and using a minimax optimization strategy to adjust the imaging background for each viewpoint, helping the algorithm filter out non-robust adversarial features.

[0013] 1. Fast and accurate reconstruction: Using 3DGS technology, only a few images are needed to quickly reconstruct the target and its background, without the need to manually construct a complex environment.

[0014] 2. Photorealistic rendering: Fast and differentiable rendering from a specified camera viewpoint ensures realistic image results during iterative attacks.

[0015] 3. Prevent occlusion: Take measures to solve the problem of inconsistent camouflage imaging at different viewing angles to ensure the effectiveness and consistency of camouflage at various viewing angles.

[0016] 4. Mini-max optimization strategy: By adding perturbations to the image background of each view to maximize the detection loss, and then optimizing the camouflage to minimize the loss, multi-view robust adversarial features are obtained.

[0017] According to some embodiments of the present invention, taking multi-view photos around a vehicle to be disguised includes the following steps:

[0018] Take no less than 50 multi-angle photos around the vehicle to be camouflaged.

[0019] According to some embodiments of the present invention, performing feature detection and feature matching on the multi-view photos, and recovering camera poses and sparse scene structures from the multi-view photos to obtain sparse initial point cloud positions, includes the steps of:

[0020] The multi-view photos are processed using COLMAP software: feature detection and feature matching are performed using the SIFT algorithm, and the SfM technology is used to recover the camera pose and sparse scene structure from the multi-view photos, and a sparse initial point cloud position is obtained based on the camera pose and the sparse scene structure.

[0021] According to some embodiments of the present invention, iteratively updating the three-dimensional Gaussian splash scene multiple times to approximate the real scene to obtain a trained three-dimensional Gaussian splash scene includes the following steps:

[0022] Given the camera intrinsic and extrinsic parameters in the training set, render the scene image under each multi-view photo and compare it with the ground-truth image in the training set, and calculate the average of their pixel differences as the loss function;

[0023] The loss function is back-propagated and the center position, anisotropic covariance matrix, transparency, and surface spherical harmonic coefficients of the three-dimensional Gaussian ellipsoid are optimized through the gradient descent method to update the position, shape, color, and transparency of the three-dimensional Gaussian ellipsoid. The three-dimensional Gaussian splash scene is updated iteratively multiple times to approximate the real scene.

[0024] According to some embodiments of the present invention, for the multi-view photos in the training set, extracting and saving the mask of the vehicle to be disguised in each of the multi-view photos includes the steps of:

[0025] For each of the multi-view photos in the training set, a two-dimensional image segmentation network is used, the original multi-view photos and the corresponding coordinates of several points are input as prompt information, and the mask of the vehicle to be camouflaged in each of the multi-view photos is extracted and saved.

[0026] According to some embodiments of the present invention, after the image to be detected is obtained by combination, the following steps are included:

[0027] A "reverse adversarial attack" is performed on the background to make the camouflage more robust to background changes: before each camouflage optimization iteration, noise is added to the background and iterative optimization is performed using the Iterative Fast Gradient Signed Method (I-FGSM). The optimization process stops when the target detector can correctly detect the target object or the upper limit of the iteration number is reached.

[0028] According to some embodiments of the present invention, extracting a final deployable adversarial camouflage texture based on the optimized vehicle three-dimensional Gaussian includes the following steps:

[0029] The mesh file is extracted based on the optimized vehicle 3D Gaussian splash scene and added to the Blender software for UV unfolding to obtain the final deployable adversarial camouflage texture.

[0030] In a second aspect, the technical solution of the present invention provides a three-dimensional Gaussian splash-driven multi-view robust physical adversarial camouflage generation system, comprising:

[0031] The point cloud construction module takes multi-view photos around the vehicle to be camouflaged as a training set for scene reconstruction, performs feature detection and feature matching on the multi-view photos, and recovers the camera pose and sparse scene structure from the multi-view photos to obtain a sparse initial point cloud position;

[0032] Initialization module, which initializes a 3D Gaussian ellipsoid with each point in the sparse point cloud as the center. All initialized 3D Gaussian ellipsoids constitute the initial 3D Gaussian splash scene;

[0033] a scene iteration module that iteratively updates the three-dimensional Gaussian splash scene multiple times to approximate the real scene, obtaining a trained three-dimensional Gaussian splash scene, and extracts and saves a mask of the vehicle to be camouflaged in each of the multi-view photos in the training set;

[0034] A module for constructing images to be detected is configured to traverse the view angles in the training set, render each view angle, obtain a rendered image, and use the extracted mask to combine the camouflaged vehicle in the rendered image with the background in the clean image to obtain the image to be detected.

[0035] The adversarial camouflage color optimization module feeds the image to be detected into the victim detector, calculates the detection loss, backpropagates the loss and calculates the gradient, and updates the zero-order term of the spherical harmonic function coefficient of the surface color of each three-dimensional Gaussian in the three-dimensional Gaussian splash scene through gradient descent to optimize the adversarial camouflage color of the surface of the vehicle to be camouflaged;

[0036] The texture deployment module, when the optimization conditions are met, extracts the final deployable adversarial camouflage texture based on the optimized vehicle's three-dimensional Gaussian. By deploying the adversarial camouflage texture, the target vehicle cannot be correctly detected by the target detector.

[0037] In a third aspect, the technical solution of the present invention provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the three-dimensional Gaussian splash-driven multi-perspective robust physical adversarial camouflage generation method as described in any one of the first aspects.

[0038] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, wherein the abstract drawing is identical to one of the drawings in the specification:

[0040] Figure 1 A flowchart of a method for generating multi-view robust physical adversarial camouflage driven by three-dimensional Gaussian splattering is provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] It should be noted that although the system diagrams illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the system or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0043] Reference Figure 1 , Figure 1 A flowchart of a method for generating multi-view robust physical adversarial camouflage driven by three-dimensional Gaussian splattering is provided in accordance with an embodiment of the present invention.

[0044] In one embodiment, a three-dimensional Gaussian splash-driven multi-perspective robust physical adversarial camouflage generation method includes the following steps: taking multi-perspective photos around the vehicle to be camouflaged as a training set for scene reconstruction, performing feature detection and feature matching on the multi-perspective photos, and recovering the camera posture and sparse scene structure from the multi-perspective photos to obtain a sparse initial point cloud position; initializing a three-dimensional Gaussian ellipsoid with each point of the sparse point cloud as the center, and all initialized three-dimensional Gaussian ellipsoids constitute an initial three-dimensional Gaussian splash scene; iteratively updating the three-dimensional Gaussian splash scene multiple times to approximate the real scene to obtain a trained three-dimensional Gaussian splash scene, and for the multi-perspective photos in the training set, extracting and saving the vehicle to be camouflaged in each of the multi-perspective photos. ; traverse the view angles in the training set, render each view angle to obtain a rendered image, and use the extracted mask to combine the vehicle to be camouflaged in the rendered image and the background in the clean image to obtain a picture to be detected; send the picture to be detected to the victim target detector, calculate the detection loss, back-propagate the loss and calculate the gradient, and update the zero-order term of the spherical harmonic function coefficient of the surface color of each three-dimensional Gaussian in the three-dimensional Gaussian splash scene by gradient descent to optimize the surface adversarial camouflage color of the vehicle to be camouflaged; when the optimization conditions are met, extract the final deployable adversarial camouflage texture based on the optimized three-dimensional Gaussian of the vehicle. By deploying the adversarial camouflage texture, the target vehicle cannot be correctly detected by the target detector.

[0045] Step S1: Take at least 50 multi-view photos around the vehicle to be disguised. ori , used as a training set for reconstructing scenes;

[0046] Step S2: Use COLMAP software to process the multi-view photos taken in the previous step. Specifically, the SIFT algorithm is used for feature detection and feature matching, and the SfM technique is used to recover the camera pose and sparse scene structure from the image sequence. Based on the above results, the sparse initial point cloud position is obtained.

[0047] Step S3, initializing a three-dimensional Gaussian ellipsoid with each point of the sparse point cloud as the center, and all initialized three-dimensional Gaussian ellipsoids constitute an initial three-dimensional Gaussian splash scene G;

[0048] Step S4: Given the camera intrinsic and extrinsic parameters in the training set, render the scene image at each viewpoint and compare it with the ground-truth image in the training set. Calculate the average of the pixel differences as the loss function. Backpropagate the loss function and optimize the center position, anisotropic covariance matrix, transparency, and surface spherical harmonics coefficients of the 3D Gaussian ellipsoid using gradient descent to update the position, shape, color, and transparency of the 3D Gaussian ellipsoid. Iterate G multiple times to approximate the real scene.

[0049] Step S5: For each photo in the training set, use a two-dimensional image segmentation network, such as SegmentAnything, input the original image and the corresponding coordinates of several points as prompt information, and extract and save the mask M of the vehicle to be disguised in each photo;

[0050] Step S6: After obtaining the trained three-dimensional Gaussian splash scene, traverse the viewing angles in the training set and render each viewing angle first to obtain the rendered image I r = {I1, I2, ...}, by combining the disguised vehicle in the rendered image and the background in the clean image using the extracted mask M, the image to be detected I is obtained. det , the formula is:

[0051] I det =(I r ·M)+(I ori (1-M))

[0052] I det : The image to be detected (the combined image used to input the target detector).

[0053] I r : Rendered image of the 3D Gaussian splash scene (including camouflaged vehicles).

[0054] M: Mask of the vehicle to be camouflaged (binarized matrix, vehicle area is 1, background is 0).

[0055] I ori : Originally captured multi-view photos (unprocessed real scene images).

[0056] : Matrix element-wise multiplication (used to extract vehicle regions or background regions).

[0057] In step S7, considering that camouflage can be considered as a feature, where higher robustness corresponds to greater versatility, this patent performs a "reverse adversarial attack" on the background to make the camouflage more robust to background changes. Specifically, before each camouflage optimization iteration, point noise is added to the background and iterative optimization is performed using the Iterative Fast Gradient Signed Method (I-FGSM). It should be noted that the optimization process stops once the detector can correctly detect the target object or the upper limit of the iteration number is reached;

[0058] Step S8: For the victim detector F to be attacked, the image I to be detected is det The detection result is calculated and consists of multiple detection boxes:

[0059]

[0060] And calculate the detection loss:

[0061]

[0062] in:

[0063] L det : Detection loss (measures the detection effect of the target detector).

[0064] I det : The image to be detected (the image input to the detector).

[0065] ∑ I : Sum the images from all training set perspectives.

[0066] The confidence of the detection box with the largest intersection over union (IoU) with the ground-truth box (gt) at the I-th perspective.

[0067] m * : The index of the optimal detection box (determined by maximizing IoU).

[0068] gt I : The real target box (marked vehicle position) under the I-th perspective.

[0069] The mth detection box output by the detector at the Ith perspective.

[0070] IoU(·,·): Intersection over Union function (measures the degree of overlap between two boxes).

[0071] Step S9: After obtaining the detection loss, backpropagate the loss and calculate the gradient. Then, update the zero-order term of the spherical harmonic function coefficient of the surface color of each three-dimensional Gaussian in the three-dimensional Gaussian splash scene by gradient descent to optimize the anti-camouflage color of the surface of the vehicle to be camouflaged.

[0072]

[0073] G′: optimized three-dimensional Gaussian splash scene (updated Gaussian parameters).

[0074] G: Original 3D Gaussian splash scene (Gaussian parameters to be optimized).

[0075] Minimize G (optimize camouflage to minimize loss).

[0076] Maximize σ (add perturbations to the background to maximize the loss).

[0077] σ: background perturbation parameter (noise added to the background area).

[0078] θ c: Camera parameters (intrinsic and extrinsic).

[0079] I det (θ c ,G): Based on camera parameters θ c And the image to be detected rendered by scene G.

[0080] (1-M): Background region mask (multiplied by σ to add perturbations only to the background).

[0081] ‖σ‖ ∞ ≤∈: Constraint condition, the infinite norm of σ does not exceed ∈ (limiting the perturbation amplitude).

[0082] ∈: perturbation amplitude threshold (hyperparameter, controlling the maximum intensity of background perturbation).

[0083] Step S10, repeat steps S6-S9 to perform iterative updates until the target number of iterations (100 times) is reached;

[0084] In step S11, a mesh file is extracted based on the optimized three-dimensional Gaussian of the vehicle and added to the Blender software for UV unfolding to obtain the final deployable adversarial camouflage texture. By deploying the adversarial camouflage texture, the target vehicle cannot be correctly detected by the target detector.

[0085] The present invention uses 3DGS technology, based on a small number of multi-view photos taken around the vehicle to be camouflaged, and combines the SIFT algorithm and SfM technology through COLMAP software to achieve feature detection, matching, and recovery of camera posture and sparse scene structure, thereby obtaining a sparse point cloud and initializing a three-dimensional Gaussian ellipsoid, and quickly approximating the real scene through iterative optimization. The present invention breaks through the traditional method's reliance on complex manual modeling, greatly shortens the scene reconstruction time, and improves the efficiency of attack deployment. The present invention only requires a few images to complete the reconstruction, reducing the cost and difficulty of data acquisition, and is particularly suitable for scenarios that require rapid response, such as autonomous driving. The reconstruction results of the present invention are close to the real environment, providing a precise basis for subsequent anti-camouflage generation, and avoiding the attenuation of attack effects due to environmental modeling errors.

[0086] The present invention utilizes a differentiable rendering pipeline to render images from a specified perspective based on a trained three-dimensional Gaussian splash scene and camera intrinsic and extrinsic parameters, and optimizes Gaussian parameters (such as color, transparency, etc.) through backpropagation of the loss function to ensure that the rendered image is highly consistent with the real scene.

[0087] The rendered images have realistic visual effects, making the generated adversarial camouflage more difficult to detect in physical environments and improving its stealth. The differentiability of the algorithm supports end-to-end optimization, facilitating the direct linking of the rendering process with adversarial attack objectives (such as detection loss), improving optimization efficiency. The realistic rendering results more accurately simulate imaging effects under different viewing angles, lighting conditions, and other conditions, providing reliable data for multi-viewpoint robustness optimization.

[0088] The present invention solves the problem of inconsistent camouflage imaging under different viewing angles through specific measures (such as optimizing the position, shape, transparency and other parameters of the Gaussian ellipsoid), prevents self-occlusion and mutual occlusion between Gaussians, and ensures the integrity and effectiveness of camouflage under various viewing angles.

[0089] This invention can avoid partial occlusion or distortion of the camouflage due to changes in viewing angle, ensuring the stability of the countermeasure attack in multiple scenarios and angles. It also improves the adaptability of the camouflage in complex environments. For example, in autonomous driving scenarios, the camouflage can effectively interfere with detection regardless of the angle from which the vehicle is photographed. This invention ensures the consistency of the countermeasure features across different viewing angles, preventing detectors from misidentifying normal targets due to occlusion, and enhancing the reliability of the attack.

[0090] The present invention adopts the idea of ​​"reverse adversarial attack", first adding point noise to the background (such as through the I-FGSM algorithm) to maximize the detection loss, and then optimizing the camouflage to minimize the loss. Through this minimax iterative process, non-robust features are filtered to obtain adversarial features that are universal to multiple perspectives. The robustness of the camouflage to background changes is enhanced, so that the attack does not rely on a specific background and is applicable to more complex real environments. Through multi-perspective joint optimization, it is ensured that the adversarial features can effectively interfere with the detector at different perspectives, avoiding the problem of insufficient robustness of traditional methods due to a single perspective. This strategy can actively adapt to the characteristics of the detector, dynamically adjust the adversarial features, and improve the pertinence and effectiveness of the attack, especially when facing different types of target detectors. It has stronger generalization capabilities.

[0091] In one embodiment, a three-dimensional Gaussian splash-driven multi-view robust physical adversarial camouflage generation system includes: a point cloud construction module, which takes multi-view photos around the vehicle to be camouflaged as a training set for scene reconstruction, performs feature detection and feature matching on the multi-view photos, and recovers the camera pose and sparse scene structure from the multi-view photos to obtain a sparse initial point cloud position; an initialization module, which initializes a three-dimensional Gaussian ellipsoid with each point in the sparse point cloud as the center, and all initialized three-dimensional Gaussian ellipsoids constitute an initial three-dimensional Gaussian splash scene; a scene iteration module, which iteratively updates the three-dimensional Gaussian splash scene multiple times to approximate the real scene to obtain a trained three-dimensional Gaussian splash scene, and extracts and saves the mask of the vehicle to be camouflaged in each of the multi-view photos in the training set; The module for constructing the image to be detected traverses the view angles in the training set, renders each view angle to obtain a rendered image, and uses the extracted mask to combine the camouflaged vehicle in the rendered image and the background in the clean image to obtain the image to be detected; the module for optimizing the camouflage color sends the image to be detected into the victim target detector, calculates the detection loss, back-propagates the loss and calculates the gradient, and updates the zero-order term of the spherical harmonic function coefficient of the surface color of each three-dimensional Gaussian in the three-dimensional Gaussian splash scene by gradient descent to optimize the surface camouflage color of the vehicle to be camouflaged; the module for deploying the texture extracts the final deployable camouflage texture based on the optimized three-dimensional Gaussian of the vehicle when the optimization conditions are met. By deploying the camouflage texture, the target vehicle cannot be correctly detected by the target detector.

[0092] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0093] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate and may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0094] In addition, an embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by a processor or controller, for example, by a processor in the above-mentioned terminal embodiment, so that the above-mentioned processor can execute the three-dimensional Gaussian splash-driven multi-perspective robust physical adversarial camouflage generation method in the above-mentioned embodiment.

[0095] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage, or other magnetic storage devices, or can be used to store desired information and any other medium that can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0096] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above implementation. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

[0097] The specific embodiments of the present invention described above do not limit the scope of protection of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A three-dimensional Gaussian splash-driven multi-view robust physical adversarial camouflage generation method, characterized by: Including steps: Taking multi-view photos around the vehicle to be camouflaged as a training set for scene reconstruction, performing feature detection and feature matching on the multi-view photos, and recovering the camera pose and sparse scene structure from the multi-view photos to obtain a sparse initial point cloud position; Initialize a 3D Gaussian ellipsoid with each point in the sparse point cloud as the center, and all initialized 3D Gaussian ellipsoids constitute the initial 3D Gaussian splash scene; Iteratively updating the three-dimensional Gaussian splash scene multiple times to approximate the real scene to obtain a trained three-dimensional Gaussian splash scene, and extracting and saving the mask of the vehicle to be camouflaged in each of the multi-view photos in the training set; Traversing the view angles in the training set, rendering each view angle to obtain a rendered image, and combining the vehicle to be disguised in the rendered image with the background in the clean image using the extracted mask to obtain a picture to be detected; The image to be detected is fed into a victim detector to calculate a detection loss, which is then back-propagated to calculate the gradient. The zero-order term of the spherical harmonic coefficients of the surface color of each three-dimensional Gaussian in the three-dimensional Gaussian splash scene is updated by gradient descent to optimize the anti-camouflage color of the surface of the vehicle to be camouflaged. When the optimization conditions are met, the final deployable adversarial camouflage texture is extracted based on the optimized vehicle three-dimensional Gaussian. By deploying the adversarial camouflage texture, the target vehicle cannot be correctly detected by the target detector.

2. The multi-view robust physical adversarial camouflage generation method driven by three-dimensional Gaussian splattering according to claim 1 is characterized in that: Taking multi-angle photos around the vehicle to be disguised includes the following steps: Take no less than 50 multi-angle photos around the vehicle to be camouflaged.

3. The multi-view robust physical adversarial camouflage generation method driven by three-dimensional Gaussian splattering according to claim 1 is characterized in that: Performing feature detection and feature matching on the multi-view photos, and recovering camera poses and sparse scene structures from the multi-view photos to obtain sparse initial point cloud positions, including the steps of: The multi-view photos are processed using COLMAP software: feature detection and feature matching are performed using the SIFT algorithm, and the SfM technology is used to recover the camera pose and sparse scene structure from the multi-view photos, and a sparse initial point cloud position is obtained based on the camera pose and the sparse scene structure.

4. The multi-view robust physical adversarial camouflage generation method driven by three-dimensional Gaussian splattering according to claim 1 is characterized in that: Iteratively updating the three-dimensional Gaussian splash scene multiple times to approximate the real scene to obtain a trained three-dimensional Gaussian splash scene, including the steps of: Given the camera intrinsic and extrinsic parameters in the training set, render the scene image under each multi-view photo and compare it with the ground-truth image in the training set, and calculate the average of their pixel differences as the loss function; The loss function is back-propagated and the center position, anisotropic covariance matrix, transparency, and surface spherical harmonic coefficients of the three-dimensional Gaussian ellipsoid are optimized through the gradient descent method to update the position, shape, color, and transparency of the three-dimensional Gaussian ellipsoid. The three-dimensional Gaussian splash scene is updated iteratively multiple times to approximate the real scene.

5. The multi-view robust physical adversarial camouflage generation method driven by three-dimensional Gaussian splashing according to claim 1 is characterized in that: For the multi-view photos in the training set, extracting and saving the mask of the vehicle to be disguised in each of the multi-view photos includes the following steps: For each of the multi-view photos in the training set, a two-dimensional image segmentation network is used, the original multi-view photos and the corresponding coordinates of several points are input as prompt information, and the mask of the vehicle to be camouflaged in each of the multi-view photos is extracted and saved.

6. The multi-view robust physical adversarial camouflage generation method driven by three-dimensional Gaussian splattering according to claim 1 is characterized in that: After the image to be detected is obtained by combining, the following steps are included: A "reverse adversarial attack" is performed on the background to make the camouflage more robust to background changes: before each camouflage optimization iteration, noise is added to the background and iteratively optimized using the Iterative Fast Gradient Signed Method (I-FGSM). The optimization process stops when the target detector can correctly detect the target object or the upper limit of the iteration number is reached.

7. The three-dimensional Gaussian splash driven multi-view robust physical adversarial camouflage generation method according to claim 1, characterized in that: The final deployable adversarial camouflage texture is extracted based on the optimized vehicle 3D Gaussian, including the following steps: The mesh file is extracted based on the optimized vehicle 3D Gaussian splash scene and added to the Blender software for UV unfolding to obtain the final deployable adversarial camouflage texture.

8. A three-dimensional Gaussian splash-driven multi-view robust physical adversarial camouflage generation system, characterized by: include: The point cloud construction module takes multi-view photos around the vehicle to be camouflaged as a training set for scene reconstruction, performs feature detection and feature matching on the multi-view photos, and recovers the camera pose and sparse scene structure from the multi-view photos to obtain a sparse initial point cloud position; Initialization module, which initializes a 3D Gaussian ellipsoid with each point in the sparse point cloud as the center. All initialized 3D Gaussian ellipsoids constitute the initial 3D Gaussian splash scene; a scene iteration module that iteratively updates the three-dimensional Gaussian splash scene multiple times to approximate the real scene, obtaining a trained three-dimensional Gaussian splash scene, and extracts and saves a mask of the vehicle to be camouflaged in each of the multi-view photos in the training set; A module for constructing images to be detected is configured to traverse the view angles in the training set, render each view angle, obtain a rendered image, and use the extracted mask to combine the camouflaged vehicle in the rendered image with the background in the clean image to obtain the image to be detected. The adversarial camouflage color optimization module feeds the image to be detected into the victim detector, calculates the detection loss, backpropagates the loss and calculates the gradient, and updates the zero-order term of the spherical harmonic function coefficient of the surface color of each three-dimensional Gaussian in the three-dimensional Gaussian splash scene through gradient descent to optimize the adversarial camouflage color of the surface of the vehicle to be camouflaged; The texture deployment module, when the optimization conditions are met, extracts the final deployable adversarial camouflage texture based on the optimized vehicle's three-dimensional Gaussian. By deploying the adversarial camouflage texture, the target vehicle cannot be correctly detected by the target detector.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the three-dimensional Gaussian splash-driven multi-view robust physical adversarial camouflage generation method according to any one of claims 1 to 7.

Images