Adversarial patch generation method and device based on feature fragmentation, equipment and storage medium

Abstract

The application discloses a feature fragmentation-based adversarial patch generation method and device, equipment and storage medium, and the method comprises the following steps: acquiring a reference physical image, an attacked vehicle image and a target detection model to be attacked, constructing a text prompt word, and initializing the text prompt word through reverse mapping and manifold anchoring to generate a disturbed patch latent variable; inputting the disturbed patch latent variable into a diffusion decoder to generate a digital adversarial patch, and attaching the digital adversarial patch to the attacked vehicle image to construct a synthetic adversarial image; inputting the synthetic adversarial image into the target detection model, and jointly optimizing the disturbed patch latent variable under the constraints of feature fragmentation, self-adaptive rollback mechanism, expected transformation and scale low-pass filtering, and projection constraint to generate an optimized patch latent variable; generating a final adversarial patch that can be physically deployed according to the optimized patch latent variable, and mapping the final adversarial patch to a target area. Through the feature fragmentation constraint, the deployment capability of the adversarial patch is improved.

Description

Technical Field

[0001] This invention belongs to the field of physical adversarial image generation technology, and relates to a method, apparatus, device and storage medium for generating adversarial patches based on feature fragmentation. Background Technology

[0002] Object detection models are a core component of the visual perception system for autonomous vehicles, and their detection robustness directly affects vehicle driving safety. Physical adversarial patches, by attaching specific texture patterns to the surface of target objects, can interfere with the model's recognition output and are an important means of evaluating the safety of autonomous driving perception systems.

[0003] Existing adversarial patching methods often employ high-frequency noise or simple texture optimization, resulting in visually jarring patches that are easily detected by the human eye and anomaly detection modules. Furthermore, adversarial perturbations tend to cluster locally in the feature space, leading to gradient vanishing during the optimization process, resulting in poor attack stability and physical robustness, making covert deployment difficult in real-world road scenarios. Current technologies fail to balance the visual naturalness, attack effectiveness, and environmental adaptability of adversarial patches, limiting their engineering application in autonomous driving safety evaluation and model robustness enhancement.

[0004] Therefore, it is necessary to provide improved technical solutions to overcome the above-mentioned technical problems existing in the prior art. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, device and storage medium for generating adversarial patches based on feature fragmentation. By combining a diffusion generation framework with deep feature fragmentation constraints, adaptive gradient backoff and physical domain enhancement, it generates visually natural, attack-stable and robust physical adversarial patches, solving the problems of abrupt textures, easy optimization interruption and weak physical deployment capability of traditional methods.

[0006] To achieve the above objectives: In a first aspect, embodiments of this application provide a method for generating adversarial patches based on feature fragmentation, including: Obtain reference physical images, images of attacked vehicles, and target detection models to be attacked. Construct text prompt words and initialize them through inverse mapping and manifold anchoring to generate disturbed patch latent variables. The harassed patch latent variable is input into the diffusion decoder to generate a digital adversarial patch, and the digital adversarial patch is attached to the attacked vehicle image to construct a synthetic adversarial image; The synthetic adversarial image is input into the target detection model, and under the constraints of feature fragmentation, adaptive backoff mechanism, expectation transformation and scale low-pass filtering, and projection constraint, the disturbed patch latent variables are jointly optimized to generate optimized patch latent variables. Based on the latent variables of the optimized patch, a physically deployable final adversarial patch is generated, and the final adversarial patch is mapped to the target area.

[0007] In one embodiment, the step of constructing and initializing text prompt words to generate disturbed patch latent variables includes: Based on the reference physical image, the image of the attacked vehicle, and the target detection model, the attack target, attack category, and target area are determined. Develop text hints related to the appearance of the patch and determine the generation and optimization parameters; By inversely mapping the reference physical image using a denoising diffusion implicit model, the target latent variables are obtained and a latent manifold conforming to the physical appearance distribution is established. The target latent variable is anchored in a manifold and an initial adversarial perturbation is injected to generate a perturbed patch latent variable.

[0008] In one embodiment, inputting the disturbed patch latent variable into the diffusion decoder to generate a digital adversarial patch includes: The latent variables of the disturbed patch are input into the diffusion decoder to generate a digital adversarial patch that combines natural texture with optimizable attack attributes. The digital adversarial patch is applied to the target region, and the synthetic adversarial image is preprocessed to adapt it to the input requirements of the target detection model.

[0009] In one embodiment, the joint optimization of the disturbed patch latent variables includes: The synthetic adversarial image is input into the target detection model to extract the detection output, target confidence, and intermediate layer depth features; Feature fragmentation constraints are applied to the target region based on the patch mask; Real-time monitoring of target confidence level; when the target confidence level falls below a preset confidence threshold, an adaptive backoff mechanism is triggered. A physical domain expectation transformation is applied to the digital adversarial patch, the physical domain expectation transformation including at least one of perspective, rotation, translation, lighting, color perturbation and scale low-pass filtering; Apply the perturbation projection constraint and backpropagate the gradient to update the perturbation patch latent variables.

[0010] In one embodiment, the feature fragmentation constraint includes: The feature activation space variance is calculated for the patch mask region of the multi-layer feature map, and the variances of each layer are weighted and summed to obtain the feature fragmentation loss.

[0011] In one embodiment, before updating the disturbed patch latent variable, the method further includes: When the target confidence level is not less than the preset confidence threshold, the global loss is obtained by weighted summation of the detection attack loss and the feature fragmentation loss. When the target confidence level is less than the preset confidence threshold, the weighted sum of the backoff loss and the feature fragmentation loss is used as the global loss.

[0012] In one embodiment, the iterative joint optimization of the disturbed patch latent variables further includes: When the joint loss satisfies the preset convergence iteration condition, the optimized patch latent variable is output, and the final adversarial patch image is generated through the diffusion decoder.

[0013] Secondly, embodiments of this application provide an adversarial patch generation apparatus based on feature fragmentation, comprising: The latent variable initialization module is used to acquire the reference physical image, the image of the attacked vehicle, and the target detection model to be attacked, construct text prompt words, and generate the disturbed patch latent variables through inverse mapping and manifold anchoring. The patch generation module is used to input the harassed patch latent variable into the diffusion decoder to generate a digital adversarial patch, and to attach the digital adversarial patch to the attacked vehicle image to construct a synthetic adversarial image; The joint optimization module is used to input the synthetic adversarial image into the target detection model and perform joint optimization on the disturbed patch latent variables under feature fragmentation constraints, adaptive backoff mechanism, expectation transformation and scale low-pass filtering, and projection constraints to generate optimized patch latent variables. The result output module is used to generate a physically deployable final adversarial patch based on the latent variables of the optimized patch, and to map the final adversarial patch to the corresponding region of the target vehicle image.

[0014] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores executable program code, and when the executable program code is executed by the processor, it implements the steps of the feature fragmentation-based adversarial patch generation method described in the first aspect.

[0015] Fourthly, embodiments of this application provide a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the feature fragmentation-based adversarial patch generation method described in the first aspect.

[0016] The adversarial patch generation method, apparatus, device, and storage medium based on feature fragmentation provided in this application ensure that the initial texture of the patch closely resembles the real physical environment through latent manifold anchoring, disperses the perturbation distribution in the feature space through feature fragmentation constraints, maintains the continuity of the optimized gradient through an adaptive backoff mechanism, and improves the robustness of the patch through multi-physics domain transformation. Finally, it generates visually natural, attack-stable, and physically deployable adversarial patches, which can be widely used in engineering scenarios such as safety evaluation of autonomous driving perception systems and model robustness enhancement, and has high practical value and application prospects.

[0017] This application provides a method, apparatus, device, and storage medium for generating adversarial patches based on feature fragmentation. The method includes: acquiring a reference physical image, an image of the attacked vehicle, and a target detection model; constructing text prompts and initializing them through inverse mapping and manifold anchoring to generate a disturbed patch latent variable; inputting the disturbed patch latent variable into a diffusion decoder to generate a digital adversarial patch, and attaching the digital adversarial patch to the image of the attacked vehicle to construct a synthetic adversarial image; inputting the synthetic adversarial image into the target detection model, and jointly optimizing the disturbed patch latent variable under feature fragmentation constraints, an adaptive backoff mechanism, expectation transformation and scale low-pass filtering, and projection constraints to generate an optimized patch latent variable; generating a physically deployable final adversarial patch based on the optimized patch latent variable, and mapping the final adversarial patch to the target region. This application improves the security of adversarial patches through feature fragmentation constraints. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the adversarial patch generation method based on feature fragmentation provided in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the structure generated by adversarial patching based on feature fragmentation, provided in an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0024] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0025] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0026] It should be noted that step designations such as S110 and S120 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S120 first and then S110, etc., but these should all be within the protection scope of this application.

[0027] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0028] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0029] First Embodiment Referring to Figure 1, an embodiment of this application provides a method for generating adversarial patches based on feature fragmentation. This method can be executed by an adversarial patch generation device provided in this application. The device can be implemented in software and / or hardware. In this embodiment, the adversarial patch generation device is taken as the executing entity of the method. The adversarial patch generation method provided in this embodiment includes the following steps: Step S110: Obtain the reference physical image, the image of the attacked vehicle, and the target detection model to be attacked. Construct text prompt words and initialize them through inverse mapping and manifold anchoring to generate the disturbed patch latent variables.

[0030] It is understandable that textual prompts related to the appearance of the target patch are constructed, while the patch application area, initial mask area, patch size parameters, perturbation boundary, diffusion steps, confidence threshold, loss weight, and physical transformation parameters are determined. Manifold anchoring is then performed in conjunction with the textual prompts, and initial adversarial perturbations are injected into the anchored latent representation to form the latent variables of the perturbed patch. Through inverse mapping of the real physical image and anchoring to the latent manifold, the initial latent variables are made to conform to the texture distribution of the road scene, reducing the visual abruptness of the patch.

[0031] In one implementation, constructing and initializing text prompts to generate latent variables for the obstructed patch includes: Based on the reference physical image, the image of the attacked vehicle, and the target detection model, the attack target, attack category, and target region are determined; text prompts related to the patch appearance are constructed, and generation and optimization parameters are determined; the reference physical image is inversely mapped through a denoising diffusion implicit model to obtain target latent variables and establish a latent manifold that conforms to the physical appearance distribution; the target latent variables are manifold anchored and initial adversarial perturbations are injected to generate perturbed patch latent variables.

[0032] Specifically, the reference physical image is inversely mapped using Denoising Diffusion Implicit Models (DDIM) to obtain a latent representation corresponding to the reference physical image, and a latent manifold state conforming to the distribution of the real physical appearance is established based on this representation. In each time step t of the denoising process, the unconditional text embedding is alternately optimized and updated. This allows the latent representation to progressively approximate the target manifold under semantic and physical texture constraints. The calculation formula for its single-step optimization objective is as follows:

[0033] Where C represents the text prompt word embedding. The target latent variable is obtained by inverse mapping from the reference physical image. As an intermediate latent variable, This represents the reconstruction result at the current time step.

[0034] Step S120: Input the latent variables of the harassed patch into the diffusion decoder to generate a digital adversarial patch, and attach the digital adversarial patch to the image of the attacked vehicle to construct a synthetic adversarial image.

[0035] It can be understood that a digital adversarial patch is attached to the target region of the attacked vehicle image to generate a corresponding synthetic adversarial image, which allows the patch and the host image to be spatially superimposed at the target location. The generated synthetic adversarial image is preprocessed to meet the input requirements of the target detection model and to ensure the consistency between the patch region and the host image in terms of spatial location, scale relationship and texture transition, thereby enhancing the naturalness of the synthetic result. The synthetic adversarial image is used as the current iteration sample to input the target detection model, providing the input basis for subsequent attack loss calculation, deep feature extraction and joint optimization.

[0036] In one implementation, the perturbed patch latent variable is input into a diffusion decoder to generate a digital adversarial patch, including: The latent variables of the disturbed patch are input into the diffusion decoder to generate a digital adversarial patch that combines natural texture with optimizable attack attributes. The digital adversarial patch is then attached to the target region, and the synthetic adversarial image is preprocessed to adapt it to the input requirements of the target detection model.

[0037] It is understandable that the latent variables of the disturbed patch are input into the diffusion decoder, which decodes and generates a digital adversarial patch that combines natural texture distribution and optimizable attack attributes. This makes the generated patch visually as close as possible to the natural texture in the real road scene and smoothly blends it with the vehicle image to ensure visual consistency.

[0038] Step S130: Input the synthetic adversarial image into the target detection model, and under the constraints of feature fragmentation, adaptive backoff mechanism, expectation transformation and scale low-pass filtering, and projection constraint, jointly optimize the disturbed patch latent variables to generate optimized patch latent variables.

[0039] It is understandable that by inputting synthetic adversarial images into a target detection model, the target detection results, target confidence, and deep feature maps of the preset intermediate layers are obtained from the model output, and a balance between attack effectiveness, naturalness, and robustness is achieved through multiple constraints.

[0040] In one implementation, joint optimization of the latent variables of the disturbed patch includes: The synthetic adversarial image is input into the target detection model, and the detection output, target confidence, and intermediate layer depth features are extracted. Feature fragmentation constraints are applied to the target region based on the patch mask. The target confidence is monitored in real time, and an adaptive backoff mechanism is triggered when the target confidence is lower than a preset confidence threshold. A physical domain expectation transformation is applied to the digital adversarial patch, which includes at least one of perspective, rotation, translation, illumination, color perturbation, and scale low-pass filtering. Perturbation projection constraints are performed, and the gradient is backpropagated to update the latent variables of the perturbed patch.

[0041] In one embodiment, the feature fragmentation constraint includes: calculating the feature activation space variance for the patch mask region of the multi-layer feature map, and performing a weighted summation of the variances of each layer to obtain the feature fragmentation loss.

[0042] Specifically, feature responses corresponding to the patch regions are extracted from the multi-layer feature map, and feature fragmentation constraints are applied to these regions using a patch mask. This enhances the discreteness and discontinuity of feature activation in the patch regions, weakening the detection model's stable representation of the target structure. Its feature fragmentation... The formula for calculating the loss is as follows:

[0043] Where L is the selected set of feature layers, To balance the weights of each layer, For the feature map of layer l, For the patch mask at the corresponding scale, Var( ) represents the spatial variance of feature activation within the masked region.

[0044] In one embodiment, before updating the disturbed patch latent variable, the method further includes: when the target confidence is not less than a preset confidence threshold, using a weighted sum of detection attack loss and feature fragmentation loss as the global loss; when the target confidence is less than the preset confidence threshold, using a weighted sum of backoff loss and feature fragmentation loss as the global loss.

[0045] Specifically, when the target confidence level is lower than a preset threshold Furthermore, when conventional detection loss fails to provide effective gradients, an adaptive backoff mechanism is triggered. This mechanism relaxes the matching constraints between predicted candidate boxes and the real target, or directly extracts weak response target information from deep mesh features to construct the backoff loss. This ensures a continuous and effective gradient during the optimization process. The formula for calculating the global joint loss is as follows:

[0046] in, For target confidence level, α represents the backoff loss weight, and α represents the feature fragmentation loss weight.

[0047] In one embodiment, the iterative joint optimization of the disturbed patch latent variables further includes: when the joint loss satisfies the preset convergence iteration condition, outputting the optimized patch latent variables, and generating the final adversarial patch image through a diffusion decoder.

[0048] Specifically, to improve the adaptability of the patch in real physical environments, a desired transformation is applied to the digital adversarial patch in each optimization iteration. This transformation includes spatial perspective transformation, rotation transformation, translation transformation, scale transformation, brightness perturbation, contrast perturbation, color perturbation, and scale low-pass filtering to simulate changes in viewpoint, distance, and illumination that may occur during vehicle movement. The optimization objective is calculated using the following formula:

[0049] in, Let P be the predefined physical transformation distribution, P be the adversarial patch, and x be the host image. This indicates that the patch is attached to the host image, where T is the set of physical transformations. For parameters The target detection model, where y is the label of the attack target; Furthermore, after each parameter update, a perturbation projection constraint is applied to ensure that the patch always satisfies the preset perturbation boundary and naturalness requirements, and the optimization gradient is backpropagated to the latent variables of the perturbed patch. The calculation formula is as follows:

[0050] in, Let be the perturbation variable for the i-th iteration. This represents the update amount for the current iteration. For the perturbation boundary, Indicates based on Projection operations on norms.

[0051] Step S140: Generate a physically deployable final adversarial patch based on the latent variables of the optimized patch, and map the final adversarial patch to the target area.

[0052] It is understandable that when the joint loss meets the convergence condition or reaches the maximum number of iterations, the final adversarial patch image is generated by the diffusion decoder based on the optimized latent variables, and then mapped to the corresponding area of ​​the target vehicle image to form a physical adversarial patch that can be directly printed and attached for deployment.

[0053] Second Embodiment Based on the first embodiment of this application, this embodiment provides an adversarial patch generation device based on feature fragmentation. Referring to FIG2, the device includes: The latent variable initialization module 21 is used to acquire the reference physical image, the image of the attacked vehicle, and the target detection model to be attacked, construct text prompt words, and initialize them through inverse mapping and manifold anchoring to generate the disturbed patch latent variables.

[0054] The patch generation module 22 is used to input the latent variables of the harassed patch into the diffusion decoder to generate a digital adversarial patch, and to attach the digital adversarial patch to the attacked vehicle image to construct a synthetic adversarial image.

[0055] The joint optimization module 23 is used to input the synthetic adversarial image into the target detection model and perform joint optimization on the disturbed patch latent variables under feature fragmentation constraints, adaptive backoff mechanism, expectation transformation and scale low-pass filtering, and projection constraints to generate optimized patch latent variables.

[0056] The output module 24 is used to generate a physically deployable final adversarial patch based on the latent variables of the optimized patch, and to map the final adversarial patch to the corresponding area of ​​the target vehicle image.

[0057] The specific functions and execution steps of each of the above modules have been described in detail in the method embodiments, and will not be repeated here.

[0058] Third Embodiment Based on the same inventive concept as the foregoing embodiments, this application provides an electronic device, such as... Figure 3 As shown, the device includes: a processor 301 and a memory 302 storing a computer program; wherein, Figure 3 The processor 301 shown in the diagram does not indicate that there is only one processor 301, but only indicates the positional relationship of processor 301 relative to other devices. In practical applications, there can be one or more processors 301; similarly, Figure 3 The memory 302 shown in the diagram has the same meaning, that is, it is only used to indicate the positional relationship of memory 302 relative to other devices. In practical applications, there can be one or more memories 302. When the processor 301 runs the computer program, it implements the aforementioned feature fragmentation-based adversarial patch generation method.

[0059] The device may also include at least one network interface 303. The various components of the device are coupled together via a bus system 304. It is understood that the bus system 304 is used to implement communication between these components. In addition to a data bus, the bus system 304 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 3 The general designated all buses as Bus System 304.

[0060] The memory 302 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 302 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0061] Fourth embodiment Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program stored in the computer-readable storage medium is executed by a processor, it implements the above-described feature fragmentation-based adversarial patch generation method. For the specific steps implemented when the computer program is executed by the processor, please refer to [link to relevant documentation]. Figure 2 The description of the illustrated embodiments will not be repeated here.

[0062] In this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions are generally described in detail only when they appear for the first time. When they appear again, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, the same or similar terms, concepts, technical solutions and / or application scenario descriptions that are not described in detail later can be referred to their previous relevant detailed descriptions.

[0063] In this application, the descriptions of the various embodiments have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0064] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0065] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. For those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating adversarial patches based on feature fragmentation, characterized in that, The method includes: Obtain reference physical images, images of attacked vehicles, and target detection models to be attacked. Construct text prompt words and initialize them through inverse mapping and manifold anchoring to generate disturbed patch latent variables. The harassed patch latent variable is input into the diffusion decoder to generate a digital adversarial patch, and the digital adversarial patch is attached to the attacked vehicle image to construct a synthetic adversarial image; The synthetic adversarial image is input into the target detection model. Under the constraints of feature fragmentation, adaptive backoff mechanism, expectation transformation including scale low-pass filtering, and projection constraint, the perturbed patch latent variables are jointly optimized to generate optimized patch latent variables. The feature fragmentation constraint includes calculating the feature activation space variance of the patch mask region of the multi-layer feature map and weighted summing of the variances of each layer to obtain the feature fragmentation loss. Based on the latent variables of the optimized patch, a physically deployable final adversarial patch is generated, and the final adversarial patch is mapped to the target area.

2. The method according to claim 1, characterized in that, The process of constructing and initializing text prompts to generate latent variables for the obstructed patch includes: Based on the reference physical image, the image of the attacked vehicle, and the target detection model, the attack target, attack category, and target area are determined. Develop text hints related to the appearance of the patch and determine the generation and optimization parameters; By inversely mapping the reference physical image using a denoising diffusion implicit model, the target latent variables are obtained and a latent manifold conforming to the physical appearance distribution is established. The target latent variable is anchored in a manifold and an initial adversarial perturbation is injected to generate a perturbed patch latent variable.

3. The method according to claim 1, characterized in that, The step of inputting the disturbed patch latent variable into the diffusion decoder to generate a digital adversarial patch includes: The latent variables of the disturbed patch are input into the diffusion decoder to generate a digital adversarial patch with natural textures that can optimize attack attributes. The digital adversarial patch is applied to the target region, and the synthetic adversarial image is preprocessed to adapt it to the input requirements of the target detection model.

4. The method according to claim 1, characterized in that, The joint optimization of the disturbed patch latent variables includes: The synthetic adversarial image is input into the target detection model to extract the detection output, target confidence, and intermediate layer depth features; Feature fragmentation constraints are applied to the target region based on the patch mask; The target confidence level is monitored in real time, and an adaptive backoff mechanism is triggered when the target confidence level is lower than a preset confidence threshold. A physical domain expectation transformation is applied to the digital adversarial patch, the physical domain expectation transformation including at least one of perspective, rotation, translation, lighting, color perturbation and scale low-pass filtering; Apply the perturbation projection constraint and backpropagate the gradient to update the perturbation patch latent variables.

5. The method according to claim 4, characterized in that, Before updating the disturbed patch latent variable, the following is also included: When the target confidence level is not less than the preset confidence threshold, the global loss is obtained by weighted summation of the detection attack loss and the feature fragmentation loss. When the target confidence level is less than the preset confidence threshold, the weighted sum of the backoff loss and the feature fragmentation loss is used as the global loss.

6. The method according to claim 1, characterized in that, The iterative joint optimization of the disturbed patch latent variables also includes: When the joint loss satisfies the preset convergence iteration condition, the optimized patch latent variable is output, and the final adversarial patch image is generated through the diffusion decoder.

7. An adversarial patch generation device based on feature fragmentation, characterized in that, The device includes: The latent variable initialization module is used to acquire the reference physical image, the image of the attacked vehicle, and the target detection model to be attacked, construct text prompt words, and initialize them through inverse mapping and manifold anchoring to generate the disturbed patch latent variables; The patch generation module is used to input the harassed patch latent variable into the diffusion decoder to generate a digital adversarial patch, and to attach the digital adversarial patch to the attacked vehicle image to construct a synthetic adversarial image; The joint optimization module is used to input the synthetic adversarial image into the target detection model and perform joint optimization on the disturbed patch latent variables under feature fragmentation constraints, adaptive backoff mechanism, expectation transformation and scale low-pass filtering, and projection constraints to generate optimized patch latent variables. The result output module is used to generate a physically deployable final adversarial patch based on the latent variables of the optimized patch, and to map the final adversarial patch to the corresponding region of the target vehicle image.

8. An electronic device, characterized in that, The electronic device includes: a memory and a processor, wherein the memory stores executable program code, and when the executable program code is executed by the processor, it implements the steps of the feature fragmentation-based adversarial patch generation method as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the feature-fragmented adversarial patch generation method as described in any one of claims 1 to 6.

Images