Three-dimensional gaussian sputtering-based physically realistic adversarial sample joint optimization generation method

Abstract

The application provides a physically realistic adversarial sample joint optimization generation method based on three-dimensional Gaussian sputtering, wherein the method comprises the following steps: initializing a 3D Gaussian model from multi-view images through a 3D Gaussian sputtering generator; when it is detected that the 3D Gaussian model has geometric distortion or noise, starting a physical filtering module to obtain a target 3D model; mapping the target 3D model to a 2D image space through a differentiable Gaussian rasterization renderer; if there are illumination changes, sensor noise or occlusion interference in the deployment environment of the target 3D model, simulating the environmental interference through a physical enhancement module; based on the output confidence of the target 3D model and the geometric deviation of the original 3D model, calculating an adversarial loss and a shape preservation loss through a double-loss optimization module, and dynamically adjusting the weight coefficients of the two losses according to the optimization stage; according to the attack requirement, selecting the optimization dimension through a selective optimization controller; and outputting a physically realistic and highly attackable adversarial 3D model, which is used for deployment to a real vehicle or a simulation environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and computer vision, and particularly relates to a physical-realistic adversarial sample joint optimization generation method based on three-dimensional Gaussian sputtering. BACKGROUND

[0002] With the rapid development of autonomous driving technology, camera-based target detection models play a crucial role in intelligent transportation systems and are widely used in vehicle recognition, pedestrian detection, and traffic sign recognition in key scenarios.

[0003] In related technologies, through the collaborative work of 2D adversarial mapping, 3D texture optimization, and geometric deformation, various physical world adversarial attack methods are constructed. Specifically, this technical system covers the whole process from adversarial sample generation, environment simulation to attack deployment, including Adversarial Patch, DAS, FCA, MeshAdv, and other representative methods. Among them, 2D methods interfere with model feature extraction through high-contrast textures, and 3D methods attempt to improve multi-view compatibility of attacks through differentiable rendering or mesh optimization. However, in existing physical adversarial sample generation methods, low-dimensional parameter optimization strategies are directly used, and joint modeling of geometry and texture is not achieved, which may result in adversarial samples failing in real-world scenarios due to structural distortion or light mismatch, or significantly reduced attack stability in dynamic perspectives and complex environments (such as rain, fog, and occlusion), thereby affecting their actual deployment effect in autonomous driving systems. In addition, existing technologies usually use fragmented optimization objectives and lack collaborative control mechanisms for physical realism and attack strength, making it difficult to balance visual concealment and model deception, limiting the generalization ability and robustness of adversarial samples in the real world. SUMMARY

[0004] The present application aims to at least partially solve one of the technical problems in the related art.

[0005] To this end, the first object of the present application is to propose a physical-realistic adversarial sample joint optimization generation method based on three-dimensional Gaussian sputtering.

[0006] The second object of the present application is to propose a physical-realistic adversarial sample joint optimization generation device based on three-dimensional Gaussian sputtering.

[0007] The third object of the present application is to propose an electronic device.

[0008] The fourth object of the present application is to propose a computer-readable storage medium.

[0009] The fifth object of the present application is to propose a computer program product.

[0010] To achieve the above purpose, the first aspect of the embodiment of the present application proposes a physical real adversarial sample joint optimization generation method based on three-dimensional Gaussian sputtering, comprising:

[0011] In response to a multi-view image input request, a 3D Gaussian model is initialized from a multi-view image by a pre-trained 3D Gaussian sputtering generator;

[0012] When it is detected that the 3D Gaussian model has geometric distortion or noise, a physical filtering module is started to obtain a target 3D model;

[0013] The target 3D model is mapped to a 2D image space by a differentiable Gaussian rasterization renderer to support gradient back propagation;

[0014] If there are illumination changes, sensor noise or occlusion interference in the deployment environment of the target 3D model, a physical enhancement module is used to simulate environmental interference;

[0015] Based on the output confidence of the target 3D model and the geometric deviation of the original 3D model, an adversarial loss and a shape preservation loss are calculated by a double loss optimization module, and the weight coefficients of the two are dynamically adjusted according to the optimization stage to balance the attack strength and physical reality. According to the attack requirement, the optimization dimension is selected by a selective optimization controller;

[0016] An adversarial 3D model that is physically real and has strong attackability is output and deployed to a real vehicle or a simulation environment.

[0017] Optionally, the 3D Gaussian sputtering generator generates a high-fidelity 3D Gaussian model by modeling the surface geometry and texture features of the target object based on multi-view image input through a deep neural network, and the model contains at least 14 optimization parameters, including position, rotation, scale, color and opacity.

[0018] Optionally, the physical filtering module includes a topology-aware pruning submodule and a structure denoising submodule, the topology-aware pruning submodule dynamically removes outlier Gaussian points based on local density estimation, and the structure denoising submodule adaptively adjusts Gaussian scale and opacity through camera-aware anisotropic filtering to eliminate geometric jitter and preserve structural continuity.

[0019] Optionally, the physical enhancement module includes an imaging degradation submodule, a photometric variation submodule, a shadow projection submodule and an adaptive occlusion submodule, which are used to inject depth-dependent sensor noise, simulate color shift caused by dynamic lighting, generate soft shadow boundaries and simulate real occlusion with random masks.

[0020] Optionally, the optimization dimension includes a geometry selection mode and an appearance selection mode, which are used to optimize position, rotation, scale parameters or color, opacity parameters, respectively.

[0021] To achieve the above object, the second aspect of the present application proposes a device for generating a physically realistic adversarial sample based on three-dimensional Gaussian sputtering joint optimization, comprising:

[0022] An initialization unit is configured to initialize a 3D Gaussian model from a multi-view image by a pre-trained 3D Gaussian sputtering generator in response to a multi-view image input request;

[0023] A physical filtering unit is configured to obtain a target 3D model by starting a physical filtering module when it is detected that the 3D Gaussian model has geometric distortion or noise;

[0024] A rendering unit is configured to map the target 3D model to a 2D image space by a differentiable Gaussian rasterization renderer to support gradient backpropagation;

[0025] A physical enhancement unit is configured to simulate environmental interference by a physical enhancement module if the target 3D model deployment environment has changes in illumination, sensor noise or occlusion interference;

[0026] An adversarial optimization unit is configured to calculate an adversarial loss and a shape preservation loss by a double loss optimization module based on the output confidence of the target 3D model and the geometric deviation of the original 3D model, and dynamically adjust the weight coefficients of the two according to the optimization stage to achieve a balance between attack strength and physical reality; according to the attack requirement, the optimization dimension is selected by a selective optimization controller;

[0027] An output unit is configured to output a physically realistic and attackable adversarial 3D model for deployment to a real vehicle or a simulation environment.

[0028] To achieve the above object, the third aspect of the present application proposes an electronic device, comprising a processor and a memory in communication with the processor;

[0029] The memory stores computer execution instructions;

[0030] The processor executes the computer execution instructions stored in the memory to implement the method of any one of the first aspect.

[0031] To achieve the above object, the fourth aspect of the present application proposes a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the method of any one of the first aspect.

[0032] To achieve the above object, the fifth aspect of the present application proposes a computer program product, which is executed by a processor to implement the method of any one of the first aspect.

[0033] The embodiments of the present application provide technical solutions with at least the following beneficial effects:

[0034] 1. Improved physical authenticity: The present application realizes consistent adjustment of geometry and texture through joint optimization of 14-dimensional parameters, significantly improving the physical authenticity of adversarial samples. In real environment testing, the detection escape rate of adversarial vehicles under dynamic lighting is improved to 92.62%, much higher than the 64.11% of the prior art FCA.

[0035] 2. Enhanced attack stability: The present application simulates various environmental disturbances (such as rain, fog, night, and occlusion) through a physical enhancement module, enabling adversarial samples to maintain attack effectiveness in complex scenarios. The Log Confidence Reduction (LCR) index is improved to 3.5628, verifying that the attack strength has reached the current optimal level (SOTA).

[0036] 3. Improved environmental adaptability: The present application simulates lighting changes, sensor noise, shadow projection, and occlusion interference in real environments, enabling adversarial samples to maintain attack effectiveness under different time periods, weather conditions, and changes in viewing angle, significantly improving their deployment adaptability in the physical world.

[0037] 4. Optimized computing efficiency: The present application adopts a selective dimension optimization strategy, flexibly selecting optimization parameters according to attack requirements, effectively reducing computing overhead. In the NVIDIA RTX 4090 hardware environment, single adversarial sample generation only takes 1 minute, which is significantly improved compared to the prior art.

[0038] 5. Improved attack concealment: The adversarial samples generated by the present application are visually closer to real objects and are not easily detected by humans or high-precision detection systems, thereby improving the concealment and practicality of attacks.

[0039] Additional aspects and advantages of the present application will be partially given in the following description, partially will become apparent from the following description, or will be understood by those skilled in the art through practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0040] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0041] Figure 1 A textual flowchart of a physical real adversarial sample joint optimization generation method based on three-dimensional Gaussian sputtering provided by the embodiments of the present application;

[0042] Figure 2 A structural flowchart of a physical real adversarial sample joint optimization generation method based on three-dimensional Gaussian sputtering provided by the embodiments of the present application. DETAILED DESCRIPTION

[0043] Embodiments of the present application are described below in detail, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0044] Current camera-based target detection models (such as vehicle detection systems in autonomous driving) have significant vulnerability when facing physical adversarial attacks. In the prior art, the 2D adversarial patch method only optimizes local textures, and it is difficult to maintain attack effects under multi-view or occlusion conditions; while the 3D texture optimization method (such as DAS, FCA) improves the transferability of attacks to a certain extent, but its optimization process is usually limited to the RGB channel, ignoring the geometric properties such as position, rotation, and scale, resulting in the failure of the generated adversarial samples in the real environment due to structural distortion or light mismatch. In addition, although the existing 3D geometric attack method (such as MeshAdv) adjusts the shape parameters, it does not optimize the texture jointly, resulting in unnatural vision and being easily identified by human observers or high-precision detection models.

[0045] To solve this problem, the embodiments of the present application provide a three-dimensional Gaussian splatting-based physical real adversarial sample joint optimization generation method, which realizes high attack success rate and high visual concealment of adversarial samples in the physical world by fusing multi-dimensional parameter optimization of geometry and texture, enhancing physical constraints, and designing environmental robustness, thereby improving the robustness evaluation and defense ability of the target detection model in the real environment.

[0046] Figure 1 A flowchart of a three-dimensional Gaussian splatting-based physical real adversarial sample joint optimization generation method provided by the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the method comprises the following steps: Figure 1

[0047] Step 101, in response to a multi-view image input request, initializing a 3D Gaussian model from a multi-view image by a pre-trained 3D Gaussian splatting generator.

[0048] In the embodiments of the present application, step 101 provides a method of initializing a three-dimensional Gaussian model from a multi-view image by a pre-trained three-dimensional Gaussian splatting (3D Gaussian Splatting, abbreviated as 3DGS) generator in response to a multi-view image input request.

[0049] ​Specifically, the 3D Gaussian sputtering generator adopted by the embodiments of the present application is based on a deep neural network structure, which has learned a large number of geometric and texture distribution characteristics of real objects under multiple perspectives in a pre-training stage. In actual operation, when receiving multi-perspective image inputs provided by a user or a system (for example, multiple angle RGB images of a target vehicle obtained in an automatic driving scene), the generator can uniformly model and analyze the input images, thereby outputting a high-fidelity three-dimensional Gaussian representation.

[0050] The three-dimensional Gaussian model generated in the embodiments of the present application contains at least fourteen optimizable parameters, which specifically include: position parameters (x, y, z) in three-dimensional space, rotation parameters (which can be Euler angles or quaternion form), scale parameters (expansion scale in each principal axis direction), color parameters (RGB or HSV space), opacity parameters, and covariance or radial information related to the Gaussian distribution shape. The above parameters can be jointly updated in the subsequent adversarial optimization process to simultaneously control the geometric morphology and surface texture performance of the target object.

[0051] Compared with the prior art, the three-dimensional Gaussian model proposed in the present application has a high-dimensional, multi-degree-of-freedom parameter space, which is significantly superior to the way of only perturbing in the RGB texture space (such as the DAS and FCA methods) or only applying surface modification on a fixed geometric structure. By embedding texture and geometry into the multi-dimensional parameters of the three-dimensional Gaussian point, the present application supports the recovery of a high-dimensional 3D representation with strong physical consistency from low-dimensional image input, laying a solid foundation for subsequent stable physical adversarial samples.

[0052] In summary, step 101 improves the expressiveness of physical adversarial samples by unified modeling of multi-perspective images and initialization of three-dimensional Gaussian structures, and provides a unified and efficient representation framework for subsequent physical constraints, environment enhancement and joint optimization mechanism, which is significantly different from the strategy of splitting texture and geometry in the prior art, and has stronger physical reality and attack adaptation ability.

[0053] Step 102, when detecting that the 3D Gaussian model has geometric distortion or noise, starting a physical filtering module to obtain a target 3D model.

[0054] In the embodiments of the present application, step 102 provides a processing flow for geometric structure optimization of an initial three-dimensional Gaussian model. When detecting that the 3D Gaussian model has obvious geometric distortion or noise characteristics, a physical filtering module will be automatically started to obtain a target 3D model with stronger physical structure continuity and higher visual reality.

[0055] The physical filtering module includes two key sub-modules: a topology-aware pruning sub-module and a structural denoising sub-module. These two sub-modules introduce structural constraints while maintaining the flexibility of Gaussian representation to ensure that the finally generated adversarial samples have high physical rationality and visual naturalness.

[0056] Specifically:

[0057] The topology-aware pruning sub-module is based on a point cloud density estimation mechanism. By calculating the density distribution of each Gaussian point in the local space, it identifies outlier points that are discontinuous with the main structure of the target object or have geometric abnormalities (such as isolated points suspended outside the object surface, Gaussian redundant areas, etc.). In the embodiments of the present application, a dynamic threshold mechanism is used to adaptively prune based on different object categories or scene densities, significantly improving the overall coherence of the model structure. For curved objects such as vehicle side doors and roofs, smooth transition effects can be achieved, avoiding structural abnormalities such as breaks, overlaps, or abruptness.

[0058] The structural denoising sub-module further reconstructs and adjusts the scale of the remaining Gaussian points. This module relies on camera parameters (including viewing angle, focal length, exposure information, etc.) extracted from multiple-view images to perform anisotropic filtering. By analyzing the stability of the same Gaussian point in the imaging process under different viewing angles, it adjusts its scale factor and opacity parameters, effectively suppressing common distortion phenomena such as geometric jitter and jagged edges. For example, in areas with sharp changes in curvature or at the edges of vehicles, the structural denoising sub-module can actively shrink the Gaussian scale and reduce transparency to soften the visual profile, improving consistency and naturalness during multi-view rendering.

[0059] Compared with existing technologies, the physical filtering module in the embodiments of the present application does not rely on rough mesh modification or manual filtering operations. Instead, it introduces adaptive and differentiable density and geometric constraint mechanisms to avoid structural damage caused by directly editing mesh vertices in methods such as MeshAdv, ensuring that the generated results not only conform to the continuity of the physical world but also have high adversarial sample concealment.

[0060] In summary, step 102 introduces topology-aware pruning and structural denoising sub-modules to jointly optimize the geometric consistency and physical reality of the 3D Gaussian model, making the physical adversarial samples generated by the present application deployable and stable across scenes, providing a solid structural foundation for subsequent adversarial optimization.

[0061] Step 103 maps the target 3D model to a 2D image space using a differentiable Gaussian rasterization renderer to support gradient backpropagation.

[0062] In the embodiment of the present application, step 103 provides a differentiable rendering process for converting the optimized three-dimensional Gaussian model into a two-dimensional image representation to realize subsequent gradient backpropagation and parameter joint optimization based on image space.

[0063] Specifically, the present application adopts a differentiable Gaussian rasterization renderer as the image generation core module. This renderer can accurately project the corresponding visual representation in the two-dimensional image space according to the multi-dimensional parameters of each Gaussian point in the 3D Gaussian model, including position, scale, rotation, color, opacity, etc., and supports differentiable modeling of the rendering result at each pixel level. This feature enables the renderer not only to generate realistic images, but also to participate in the gradient propagation process in the entire adversarial optimization process.

[0064] In the embodiment of the present application, the rendering process not only considers the influence of camera intrinsic parameters (such as focal length, image plane size) and extrinsic parameters (such as position, orientation) on the projection result, but also integrates the spatial scale and transparency of each Gaussian point to control the mixing and covering method of colors in the rendered image. For example, in the overlapping area, a flexible transition can be achieved by superimposing Gaussian weights, avoiding the problem of jagged edges or discontinuous jumps in traditional rendering, thereby improving the realism and naturalness of the image.

[0065] More importantly, the differentiable Gaussian renderer supports gradient backpropagation based on loss functions (such as adversarial loss, physical constraint loss, etc.), that is, after calculating the error in the image space, the gradient can be accurately transmitted to each parameter in the original Gaussian model through the chain rule. This mechanism is the key foundation for the present application to achieve efficient and high-quality adversarial sample generation.

[0066] Unlike the non-differentiable rendering method based on mesh or point cloud in the prior art, the embodiment of the present application solves the problem that traditional three-dimensional adversarial attack methods are difficult to unify image space and physical parameter space optimization by introducing a differentiable Gaussian rendering structure, and realizes efficient collaboration between physical real modeling and target detection deception attack.

[0067] In summary, step 103 effectively maps the optimized three-dimensional model to the image space through the differentiable Gaussian rasterization renderer and supports gradient backpropagation throughout the process, providing a controllable and trainable rendering framework for subsequent attack target-oriented optimization and physical environment enhancement of the present application, which is significantly different from the optimization bottleneck caused by the existing non-differentiable rendering pipeline.

[0068] Step 104, if there are changes in lighting, sensor noise or occlusion interference in the deployment environment of the target 3D model, the physical enhancement module simulates environmental interference.

[0069] In the embodiment of the present application, step 104 provides an environment adaptability modeling method based on a physical enhancement mechanism, aiming to cope with complex environmental factors such as illumination changes, sensor noise and occlusion interference that the target three-dimensional model may encounter when deployed in a real scene, thereby improving the robustness and concealment of the generated adversarial samples under cross-scene conditions.

[0070] Specifically, when it is detected that there are non-ideal imaging conditions in the deployment environment (such as uneven lighting, blurred imaging, local occlusion, etc.), the embodiment of the present application will invoke a physical enhancement module to simulate and strengthen the performance of the target 3D model in the image space. The module includes the following four sub-modules, which introduce realistic interference from four angles of imaging quality, illumination change, shadow effect and occlusion effect.

[0071] Specifically:

[0072] Imaging degradation sub-module: This sub-module injects sensor noise with depth-dependent characteristics into different depth regions in the image according to the depth map estimation result. For example, stronger blur or low contrast is introduced in the far region to simulate rain and fog weather imaging distortion, thereby improving the attack transfer ability of the adversarial sample under poor image conditions.

[0073] Illumination change sub-module: This sub-module adjusts the pixel values in the rendered image by simulating the color shift phenomenon brought by different lighting conditions (such as direct sunlight, dusk low light, backlit environment). In specific implementation, physical modeling strategies such as brightness enhancement, color imbalance, exposure drift, etc. can be combined to simulate the interference of dynamic lighting in real scenes on image texture.

[0074] Shadow projection sub-module: This module randomly sets the position of the simulation light source, and calculates the shadow projection area of the target three-dimensional model in the image plane under the current lighting condition through the rendering engine. By using a soft edge masking function to simulate the flexible shadow boundary under natural lighting, the fusion degree and masking consistency of the adversarial sample in the natural environment are enhanced.

[0075] Adaptive occlusion sub-module: This sub-module implements partial occlusion operation on the target image by randomly sampling mask regions (such as partial leaves, pedestrians, traffic signs, etc. to simulate foreground elements), simulates dynamic occlusion interference in real scenes, and improves the attack retention ability of the adversarial sample under the condition of view switching or occlusion.

[0076] Through the above enhancement mechanism, the embodiment of the application can generate high-robustness adversarial samples that remain effective in various extreme environments (such as rainy days, backlight, and complex backgrounds). Unlike existing methods such as TT3D, traditional techniques rely on fixed environmental parameters or ignore sensor modeling during the imaging process, making it difficult to cope with the uncertainty of environmental conditions in real deployment. The application actively introduces diversified scene factors in the training stage through interference injection oriented to physical reality, thereby realizing strong transferability of adversarial performance.

[0077] In summary, step 104 simulates real environment interference by introducing a physical enhancement module, which not only effectively improves the cross-scene adaptability of adversarial samples, but also enhances their deception stability to detection models. It is one of the important links for the application to achieve stable physical attack capability.

[0078] Step 105, based on the output confidence of the target 3D model and the geometric deviation of the original 3D model, calculates the adversarial loss and shape preservation loss through the double-loss optimization module, and dynamically adjusts the weight coefficients of the two according to the optimization stage, to achieve the balance between attack strength and physical reality; according to the attack demand, the optimization dimension is selected by the selective optimization controller.

[0079] In the embodiment of the application, step 105 provides an adversarial sample optimization mechanism based on joint driving of double-loss function and selective dimension control, aiming to dynamically balance between attack strength and physical reality, and flexibly adjust the optimization direction according to different attack scenarios, in order to balance the actual deployment requirements and system computing efficiency.

[0080] Specifically, for the difference information between the output results of the target 3D model and the original model, the application introduces a double-loss optimization module for joint parameter update. This module contains two main loss functions:

[0081] Adversarial loss function: used to measure the deception ability of the current adversarial sample to the target detection model, the optimization goal is to minimize the recognition confidence of the model to the sample, for example, the bounding box score of the target class of detectors such as YOLO, Faster R-CNN, etc. This loss can be directly constructed based on the classification score, confidence or IoU index of the model output.

[0082] Shape preservation loss function: used to limit the geometric disturbance amplitude of the current three-dimensional Gaussian model, to ensure that no serious deformation that can be visually detected is introduced during the implementation of the attack. This loss is constructed by calculating the Euclidean distance or angular deviation between the optimized Gaussian parameters (position, rotation, scale) and the original undisturbed model.

[0083] To coordinate the above two goals, the embodiment of the present application further designs a dynamic weight adjustment mechanism. That is, by introducing a weight coefficient, the relative importance of the adversarial loss and the shape preservation loss is automatically adjusted according to the optimization phase or the current performance indicator of the sample. For example, in the initial stage, the weight is biased towards the adversarial loss to quickly improve the attack effect, and in the later stage, the shape preservation loss is strengthened to control the geometric acceptability of the final model, thereby achieving a dynamic balance between physical realism and attack effect.

[0084] At the same time, in order to avoid the calculation redundancy and optimization interference brought by full parameter optimization, the present application further proposes a selective dimension optimization strategy. This strategy controls the current optimization dimension through a selective optimization controller, which specifically includes two modes:

[0085] Geometry selection mode: only activate the gradient calculation and update of position, rotation, scale and other geometric related parameters, which is used to quickly generate structural disturbance, such as adjusting the overall size or attitude direction of the vehicle, to achieve structural deception.

[0086] Appearance selection mode: only optimize color and opacity and other visual appearance related parameters, which is used to generate texture disturbance with high concealment, such as gradient camouflage pattern or light mimicry texture.

[0087] The selective optimization mechanism of the embodiment of the present application is significantly different from the optimization strategy of the fixed parameter space in the prior art. For example, traditional methods such as FCA only perform texture disturbance optimization, and DTA fixedly performs specific conversion dimensions, lacking adaptability and flexibility. While the present application supports autonomous selection of the most suitable optimization dimension combination for specific attack targets, running platforms or environmental restrictions, thereby reducing unnecessary computational consumption and optimization conflict risks while ensuring attack performance.

[0088] In summary, step 105 realizes fine balance between attack strength and geometric fidelity through joint optimization of double loss function and selective dimension control strategy, and provides flexible dimension regulation capability for system deployment, which is one of the core technologies of the present application to improve the practicality and efficiency of physical adversarial samples.

[0089] Step 106, output the physical and strong adversarial 3D model, for deployment to real vehicles or simulation environment.

[0090] In the embodiment of the present application, step 106 provides an implementation of outputting and deploying the final optimized adversarial three-dimensional model to the target application environment. The model is generated after processing by multiple modules such as three-dimensional Gaussian initialization, physical filtering, differentiable rendering, physical enhancement and double loss joint optimization in the aforementioned steps, and has high physical realism and cross-scene attack ability.

[0091] Specifically, the output adversarial 3D model completely retains the fourteen-dimensional optimization parameters of each Gaussian point in the parameter space, including spatial position, rotation angle, scale coefficient, color information, opacity value, etc. These parameters have been verified in the image space through the differentiable optimization process, ensuring that they can significantly interfere with the recognition judgment of mainstream target detection models (such as YOLO series, Faster R-CNN, DETR, etc.) under two-dimensional imaging, while maintaining high naturalness and consistency in physical structure and visual appearance.

[0092] The adversarial 3D model has good deployment versatility and supports the following two types of scene applications:

[0093] Real vehicle deployment: The model can be attached to the surface of a real vehicle through 3D printing, high-resolution texture film, or projection enhancement system, enabling the vehicle to mislead detection systems in actual road scenarios. Especially in autonomous driving and intelligent transportation scenarios, it can be used to evaluate the safety and robustness of existing perception systems.

[0094] Simulation environment deployment: The model can be directly imported into virtual simulation platforms (such as CARLA, AirSim, Unity, or Blender environments) for large-scale adversarial training and algorithm verification. Due to the high precision and strong physical consistency of the output model parameters, it can effectively support multi-view, multi-weather, and multi-illumination simulation testing requirements, providing a real and effective testing benchmark for the safety evaluation and robustness improvement of target detection models.

[0095] Unlike existing methods that output attack samples only suitable for local mapping or two-dimensional image deception, the adversarial three-dimensional model output by the embodiments of the present application has complete three-dimensional spatial expression ability and real environment interaction attributes. It not only can play a sustained and stable deceptive role in multi-view, multi-platform, and multi-task scenarios, but also can produce consistent perception effects with physical sensing systems (such as multi-camera, laser radar), further improving attack transferability and versatility.

[0096] In summary, step 106 outputs high-quality three-dimensional adversarial models with physical reality and attack effectiveness, ensuring that the adversarial sample generation method proposed in the present application can be directly applied to real-world scenarios and industrial systems, which is a key link for the landing and transformation of the invention.

[0097] To implement the above embodiments, the present application further proposes a physical real adversarial sample joint optimization generation device based on three-dimensional Gaussian sputtering. The device comprises:

[0098] An initialization unit for initializing a 3D Gaussian model from multi-view images through a pre-trained 3D Gaussian sputtering generator in response to a multi-view image input request;

[0099] a physical filtering unit configured to activate a physical filtering module to obtain a target 3D model when it is detected that the 3D Gaussian model has geometric distortion or noise;

[0100] a rendering unit configured to map the target 3D model to a 2D image space by a differentiable Gaussian rasterizer to support gradient backpropagation;

[0101] a physical enhancement unit configured to simulate environmental interference by a physical enhancement module if the target 3D model deployment environment has illumination changes, sensor noise or occlusion interference;

[0102] an adversarial optimization unit configured to calculate an adversarial loss and a shape preservation loss by a dual loss optimization module based on the output confidence of the target 3D model and the geometric deviation of the original 3D model, and dynamically adjust the weight coefficients of the two losses according to the optimization stage to achieve a balance between attack strength and physical authenticity; and select an optimization dimension by a selective optimization controller according to attack requirements;

[0103] an output unit configured to output an adversarial 3D model that is physically authentic and highly attackable for deployment to a real vehicle or a simulation environment.

[0104] As to the apparatus in the above-mentioned embodiments, the specific manners in which the various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0105] To achieve the above-mentioned embodiments, the present application further provides an electronic device, comprising a processor and a memory connected to the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided by the foregoing embodiments.

[0106] To achieve the above-mentioned embodiments, the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the method provided by the foregoing embodiments.

[0107] To achieve the above-mentioned embodiments, the present application further provides a computer program product, comprising a computer program, which is executed by a processor to implement the method provided by the foregoing embodiments.

[0108] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the present application comply with relevant laws and regulations and do not violate public order and good customs.

[0109] It is important to note that user's personal information shall be collected for legitimate and reasonable uses of the service and not shared or sold outside of those legitimate uses. Further, such collection / sharing shall occur after receiving the consent of the users, including but not limited to informing the users to read the user agreement / user notice before using the function, and signing the agreement / authorization including authorization of relevant user information. In addition, any necessary steps shall be taken to protect and secure access to such personal information data, and ensure that other individuals with access to the personal information data follow their privacy policies and procedures.

[0110] The present application contemplates providing an implementation in which the user has the option to opt in or opt out of allowing the collection and / or access to personal information data. That is, the present disclosure contemplates providing users with control over whether and how to allow personal information data to be collected and used or made available to a third party. In some embodiments, examples, or implementations, the present disclosure contemplates providing users with the ability to prevent sharing, and / or deleting, their personal information data with the service.

[0111] In the foregoing detailed description, reference is made to descriptive terms such as "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" etc. for describing various embodiments or examples of the application. The descriptive terms are used for the convenience of the reader in understanding the application but are not intended to limit the application. Furthermore, these descriptive terms do not imply that all embodiments or examples of the application are identical or similar. In addition, the specific features, structures, materials or characteristics described in the above description can be combined in any suitable manner in one or more embodiments or examples of the application. Moreover, the described embodiments or examples are not necessarily mutually exclusive, and can be combined in any suitable manner. Furthermore, the skilled person will appreciate that the features illustrated in the drawings and described in the specification can be combined with any other features or method steps, or combinations thereof, unless specifically stated otherwise.

[0112] In addition, the terms "first", "second", etc. are used herein only to describe various embodiments or examples and do not imply or suggest relative importance or a number of indicated technical features. Thus, the features defined with "first", "second" can include at least one of the features explicitly or implicitly. In the description of the application, the meaning of "a plurality" is at least two, for example two, three, etc., unless specifically limited otherwise.

[0113] Any process or method descriptions or descriptions of the flow diagrams in the present application can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions (or steps) in the process, and / or that the various processes described can be understood as being stored in a non-transitory machine-readable or computer-readable medium or memories, executed by a computer or used by a computer. The various embodiments and / or examples of the application can include variations of these processes described, including but not limited to, the use of software, hardware, and / or other computer program products for performing the various processes described herein. The various processes described can be implemented in software and / or code, which can be stored in any form of non-transitory machine-readable or computer-readable medium or memories, which be accessed by a computer or used by a computer. Any actuators described can be updated, replaced, or otherwise changed providing mechanisms of actuation vary. The various embodiments and / or examples of the application can include variations of these actuators described, including but not limited to, the use of software, hardware, and / or other computer program products for performing the various processes described herein. The various processes described can be implemented in software and / or code, which can be stored in any form of non-transitory machine-readable or computer-readable medium or memories, which be accessed by a computer or used by a computer. The various processes described can be implemented in software and / or code, which can be stored in any form of non-transitory machine-readable or computer-readable medium or memories, which be accessed by a computer or used by a computer.

[0114] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of executable instructions stored in a computer readable medium, which can be executed by an instruction execution system, apparatus or device, such as a computer-based system, processor- based system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination thereof. For the purposes of this specification, a "computer readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can specifically include the following, which are non-exhaustive list: electrical connection (electrical device), portable computer diskette (magnetic device), Random Access Memory (RAM), Read-Only Memory (ROM), Erasable Programmable Read-Only Memory (EPROM or Flash memory), optical fiber device, and portable compact disc read-only memory (CDROM). Additionally, the computer readable medium can even be paper or other suitable medium upon which the program is printed, as the program can be electronically captured, for example, by optically scanning the paper or other suitable medium, then electronically converted into a form that can be further processed by a computer based system into an electronically accessible form in computer memory.

[0115] It should be understood that portions of the present application can be realized by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if realized by hardware, as in another embodiment, any one or a combination of the following technologies known in the art can be used: discrete logic circuit having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0116] Those skilled in the art can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium. When the programs are executed, one or a combination of the steps of the method embodiments is included.

[0117] In addition, each of the function units in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0118] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

[0119] It should be understood that the various forms of flow shown above can be reordered, added or deleted steps. For example, each step described in the present application can be executed in parallel, sequentially or in different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0120] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and replacements can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A three-dimensional Gaussian sputtering-based physical real adversarial sample joint optimization generation method, characterized in that, Includes the following steps: In response to multi-view image input requests, a 3D Gaussian model is initialized from the multi-view images using a pre-trained 3D Gaussian sputtering generator; When geometric distortion or noise is detected in the 3D Gaussian model, the physical filtering module is activated to obtain the target 3D model. The physical filtering module includes a topology-aware pruning submodule and a structure denoising submodule. The topology-aware pruning submodule dynamically removes outlier Gaussian points based on local density estimation. The structure denoising submodule adaptively adjusts the Gaussian scale and opacity through camera-aware anisotropic filtering to eliminate geometric jitter and preserve structural continuity. The target 3D model is mapped to a 2D image space using a differentiable Gaussian rasterizer to support gradient backpropagation. If the target 3D model deployment environment has lighting changes, sensor noise, or occlusion interference, the environmental interference is simulated by the physical enhancement module. The physical enhancement module includes an imaging degradation submodule, a photometric change submodule, a shadow projection submodule, and an adaptive occlusion submodule, which are used to inject depth-related sensor noise, simulate color shifts caused by dynamic lighting, generate soft shadow boundaries, and simulate real occlusion with random masks. Based on the output confidence of the target 3D model and the geometric deviation of the original 3D model, an adversarial loss and a shape-preserving loss are calculated through a dual-loss optimization module. The weight coefficients of both are dynamically adjusted according to the optimization stage to achieve a balance between attack strength and physical realism. The optimization dimension is selected through a selective optimization controller according to the attack requirements. The dual-loss optimization module includes an adversarial loss function and a shape-preserving loss function. The adversarial loss function is used to measure the ability of the current adversarial example to deceive the target detection model. The optimization objective is to minimize the model's recognition confidence of the current adversarial example. The adversarial loss is constructed based on the classification score, confidence, or IoU index of the model output. The shape-preserving loss function is used to limit the geometric perturbation amplitude of the current 3D Gaussian model to ensure that no severe deformation that can be perceived by the naked eye is introduced when the attack is carried out. The shape-preserving loss is constructed by calculating the Euclidean distance or angular deviation between the optimized Gaussian parameters and the original unperturbed model. Output physically realistic and highly aggressive adversarial 3D models for deployment in real vehicles or simulation environments.

2. The method of claim 1, wherein, The 3D Gaussian sputtering generator is based on multi-view image input and uses a deep neural network to model the surface geometry and texture features of the target object to generate a high-fidelity 3D Gaussian model. The model contains at least 14 optimizable parameters, including position, rotation, scale, color, and opacity.

3. The method of claim 2, wherein, The optimization dimensions include geometric selection mode and appearance selection mode, which are used to optimize position, rotation, scale parameters or color, opacity parameters, respectively.

4. A three-dimensional Gaussian sputtering-based physical real adversarial sample joint optimization generation device, characterized in that, include: An initialization unit is used to initialize a 3D Gaussian model from the multi-view image in response to a multi-view image input request, using a pre-trained 3D Gaussian sputtering generator. A physical filtering unit is used to activate the physical filtering module to obtain the target 3D model when geometric distortion or noise is detected in the 3D Gaussian model. The physical filtering module includes a topology-aware pruning submodule and a structure denoising submodule. The topology-aware pruning submodule dynamically removes outlier Gaussian points based on local density estimation. The structure denoising submodule adaptively adjusts the Gaussian scale and opacity through camera-aware anisotropic filtering to eliminate geometric jitter and preserve structural continuity. A rendering unit is used to map the target 3D model to a 2D image space using a differentiable Gaussian rasterization renderer to support gradient backpropagation. The physical enhancement unit is used to simulate environmental interference if there are changes in lighting, sensor noise, or occlusion interference in the deployment environment of the target 3D model. The physical enhancement module includes an imaging degradation submodule, a photometric change submodule, a shadow projection submodule, and an adaptive occlusion submodule, which are used to inject depth-related sensor noise, simulate color shift caused by dynamic lighting, generate soft shadow boundaries, and simulate real occlusion with random masks. The adversarial optimization unit is used to calculate adversarial loss and shape preservation loss based on the output confidence of the target 3D model and the geometric deviation of the original 3D model. The weight coefficients of both are dynamically adjusted according to the optimization stage to achieve a balance between attack strength and physical realism. The optimization dimension is selected by a selective optimization controller according to the attack requirements. The dual-loss optimization module includes an adversarial loss function and a shape preservation loss function. The adversarial loss function measures the ability of the current adversarial example to deceive the target detection model. The optimization objective is to minimize the model's recognition confidence against the current adversarial example. The adversarial loss is constructed based on the model's output classification score, confidence, or IoU metric. The shape preservation loss function limits the geometric perturbation amplitude of the current 3D Gaussian model, ensuring that no visually perceptible severe deformation is introduced during the attack. The shape preservation loss is constructed by calculating the Euclidean distance or angular deviation between the optimized Gaussian parameters and the original unperturbed model. The output unit is used to output physically realistic and highly aggressive adversarial 3D models for deployment in real vehicles or simulation environments.

5. An electronic device, comprising: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-3.

6. A computer readable storage medium characterized by, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-3.

7. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-3.

Images