A dual erasing method and device for multi-view diffusion model

Abstract

The application discloses a double erasing method and device for a multi-view diffusion model, and the method comprises the following steps: acquiring an input image; processing the input image to obtain an adversarial sample image; and processing the input image and the adversarial sample image to obtain adversarial noise information. The application extracts the latent features of the adversarial sample, and designs a latent feature erasing loss to make it deviate from the distribution of clean images. In each iteration, a time step is randomly sampled, and an attention erasing loss is established to transfer the attention of the region of interest to other regions, thereby destroying the geometric and visual consistency between the generated multi-views. The two loss parts are combined to form a final double erasing loss, and then the disturbance can be updated through a gradient descent algorithm. The method can simultaneously erase the latent features and the attention, and intellectual property protection is realized.

Description

Technical Field

[0001] This invention relates to the fields of 3D set reconstruction and adversarial learning technology, and in particular to a dual erasure method and apparatus for multi-view diffusion models. Background Technology

[0002] 3D geometry reconstruction is a crucial task in computer graphics, providing fundamental support for applications such as video games and virtual reality. Recently, significant progress has been made in this field with the introduction of diffusion models, enabling the accurate generation of multi-view images and the reconstruction of 3D geometry in seconds. While this brings immense convenience, intellectual property issues are increasingly concerning, as malicious users can exploit sample images from the internet, even without permission, to mimic the 3D structure of any product and gain illicit profits. Therefore, it is essential to develop effective intellectual property protection technologies to prevent the theft of 3D assets.

[0003] Multi-view diffusion models (MVDMs) are a widely used generative model for 3D geometry reconstruction. Early work utilized stable diffusion to generate multi-view images from single-view images. However, these methods primarily relied on self-attention, considering only a single view at each iteration, resulting in inconsistent 3D shapes. Subsequently, multi-view attention mechanisms and cross-domain attention mechanisms were introduced into the stable diffusion framework to improve the consistency and quality of generated images. According to our extensive research, adversarial attacks on diffusion models show great potential to prevent unauthorized image generation. All current attack methods focus on single-image generation tasks that only require consideration of the image's internal features, such as DreamBooth, LoRA, and custom diffusions. Therefore, they optimize adversarial examples to disrupt the latent feature distribution of the input image, thereby reducing the quality of the generated image. However, in multi-view image generation tasks, attacks on descendant features are insufficient because they lack consideration for disrupting the geometric and visual consistency of the generated multi-view images. Directly transferring previous attack methods (WAE) to MVDMs leads to some chaotic effects by disrupting the distribution of latent features, but the overall structure remains intact because the consistency between multi-view images is not disturbed. In addition to disrupting the distribution of latent features, researchers believe that attacking the attention mechanism in MVDM can effectively destroy the geometric and visual consistency of the generated multi-view images. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a dual erasure method and apparatus for multi-view diffusion models. This invention fully considers various attention mechanisms in MVDM, including self-attention, multi-view attention, and cross-domain attention, and then proposes the same attention erasure loss to reduce attention to the region of interest. In this way, the attention of MVDM shifts from the region of interest (i.e., the foreground region) to the background region, thereby disrupting the consistency between the generated multi-view images. Furthermore, this invention incorporates an additional function erasure loss, causing descendant functions to deviate from the original distribution to achieve the proposed double erasure attack. Experiments on object datasets such as the SOTAMVDM and GoogleScannedObjects datasets demonstrate that the method of this invention achieves superior performance in terms of attack effectiveness, transferability, and robustness of the defense.

[0005] To address the aforementioned technical problems, a first aspect of this invention discloses a dual erasure method for a multi-view diffusion model, the method comprising: The input image acquisition module is used to acquire the input image; An adversarial sample image generation module is used to process the input image to obtain an adversarial sample image; An adversarial noise information generation module is used to process the input image and the adversarial sample image to obtain adversarial noise information; The dual erasure module is used to process the input image using the anti-noise information to obtain a dual erasure result.

[0006] As an optional implementation, in the first aspect of the present invention, processing the input image to obtain adversarial sample images includes: S21, Obtain initialization noise information; S22, the initial noise information and the input image are processed to obtain adversarial sample images.

[0007] As an optional implementation, in the first aspect of the present invention, processing the input image and the adversarial sample image to obtain adversarial noise information includes: S31, Process the input image and the adversarial sample image to obtain the feature erasure loss; S32, process the input image to obtain attention erasure loss; S33, integrate the feature erasure loss and the attention erasure loss to obtain the total erasure loss; S34, using the gradient descent method, the total erasure loss is processed to obtain anti-noise information.

[0008] As an optional implementation, in the first aspect of the present invention, the processing of the input image and the adversarial example image to obtain the feature erasure loss includes: S311, Process the input image to obtain input image features. ; S312, Process the adversarial sample image to obtain adversarial sample image features. ; S313, for the input image features and the adversarial sample image features The feature erasure loss is obtained by processing the feature.

[0009] As an optional implementation, in the first aspect of the present invention, processing the input image to obtain attention erasure loss includes: S321, Process the input image to obtain the foreground mask of the input image. ; S322, Foreground mask of the input image The process is performed to obtain the attention mask. ; S323, regarding the attention mask Processing is performed to remove the loss of attention; The expression for the attention erasure loss is: in, To erase the loss of attention, for The first in line, number Column elements; For attention graph The first in line, number Column elements, , express The function, where T is the transpose.

[0010] As an optional implementation, in the first aspect of the present invention, the input image is processed to obtain a foreground mask of the input image. ,include; S3211, Encode the adversarial sample image to obtain the adversarial sample image features; S3212, Using the Unet model, the adversarial sample image features are processed to obtain denoised adversarial sample image features; S3213, Process the features of the denoised adversarial sample image to obtain the foreground mask of the input image. .

[0011] As an optional implementation, in the first aspect of the present invention, the attention mask... The process involves attention erasure loss, including: S3231, regarding the attention mask The process yields self-attention masks, multi-view attention masks, and cross-domain attention masks. S3232, Process the self-attention mask to obtain a self-attention map; S3233, Process the multi-view attention mask to obtain a multi-view attention map; S3234, Process the cross-domain attention mask to obtain a cross-domain attention map; S3235, Process the self-attention map, the multi-view attention map, and the cross-domain attention map to obtain the attention erasure loss.

[0012] A second aspect of this invention discloses a dual erasure and apparatus for a multi-view diffusion model, the apparatus comprising: The input image acquisition module is used to acquire the input image; An adversarial sample image generation module is used to process the input image to obtain an adversarial sample image; The adversarial noise information generation module is used to process the input image and the adversarial sample image to obtain adversarial noise information.

[0013] As an optional implementation, in a second aspect of the present invention, processing the input image to obtain adversarial sample images includes: S21, Obtain initialization noise information; S22, the initial noise information and the input image are processed to obtain adversarial sample images.

[0014] As an optional implementation, in a second aspect of the present invention, processing the input image and the adversarial sample image to obtain adversarial noise information includes: S31, Process the input image and the adversarial sample image to obtain the feature erasure loss; S32, process the input image to obtain attention erasure loss; S33, integrate the feature erasure loss and the attention erasure loss to obtain the total erasure loss; S34, using the gradient descent method, the total erasure loss is processed to obtain anti-noise information.

[0015] As an optional implementation, in a second aspect of the present invention, the processing of the input image and the adversarial example image to obtain the feature erasure loss includes: S311, Process the input image to obtain input image features. ; S312, Process the adversarial sample image to obtain adversarial sample image features. ; S313, for the input image features and the adversarial sample image features The feature erasure loss is obtained by processing the feature.

[0016] As an optional implementation, in a second aspect of the present invention, processing the input image to obtain attention erasure loss includes: S321, Process the input image to obtain the foreground mask of the input image. ; S322, Foreground mask of the input image The process is performed to obtain the attention mask. ; S323, regarding the attention mask Processing is performed to remove the loss of attention; The expression for the attention erasure loss is: in, To erase the loss of attention, for The first in line, number Column elements; For attention graph The first in line, number Column elements, , express The function, where T is the transpose.

[0017] As an optional implementation, in the second aspect of the present invention, the input image is processed to obtain a foreground mask of the input image. ,include; S3211, Encode the adversarial sample image to obtain the adversarial sample image features; S3212, Using the Unet model, the adversarial sample image features are processed to obtain denoised adversarial sample image features; S3213, Process the features of the denoised adversarial sample image to obtain the foreground mask of the input image. .

[0018] As an optional implementation, in a second aspect of the present invention, the attention mask... The process involves attention erasure loss, including: S3231, regarding the attention mask The process yields self-attention masks, multi-view attention masks, and cross-domain attention masks. S3232, Process the self-attention mask to obtain a self-attention map; S3233, Process the multi-view attention mask to obtain a multi-view attention map; S3234, Process the cross-domain attention mask to obtain a cross-domain attention map; S3235, Process the self-attention map, the multi-view attention map, and the cross-domain attention map to obtain the attention erasure loss.

[0019] A third aspect of the present invention discloses another dual erasure device for multi-view diffusion models, the device comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute some or all of the steps in the dual erasure method for a multi-view diffusion model disclosed in the first aspect of the present invention.

[0020] The fourth aspect of the present invention discloses a computer-storable medium storing computer instructions, which, when invoked, are used to execute some or all of the steps in the dual erasure method for a multi-view diffusion model disclosed in the first aspect of the present invention.

[0021] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: This invention extracts latent features from adversarial examples and designs a latent feature erasure loss to deviate from the distribution of clean images. In each iteration, time steps are randomly sampled, and an attention-based erasure loss is established, shifting attention from the region of interest to other regions, thereby disrupting the geometric and visual consistency between the generated multiple views. The two losses are combined to form the final dual-erasure loss, which is then updated using a gradient descent algorithm. This invention's method can simultaneously erase latent features and attention, achieving intellectual property protection.

[0022] The proposed method solves the technical challenge of protecting 3D assets in adversarial attacks on diffusion models, providing an efficient and robust solution to prevent the theft of intellectual property rights for 3D assets. By simultaneously disrupting the latent features and attention mechanisms of the generated image, this method can significantly affect the quality of multi-view images, ensuring that the generated 3D model cannot be reverse-engineered by unauthorized users, thereby protecting the intellectual property rights of original digital content. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a dual erasure method for a multi-view diffusion model disclosed in an embodiment of the present invention; Figure 2 This is a flowchart illustrating another dual erasure method for a multi-view diffusion model disclosed in an embodiment of the present invention. Figure 3 This is a schematic diagram of the visualization results disclosed in the embodiments of the present invention; Figure 4 This is a comparison of the reconstruction results of the method and the comparative method disclosed in the embodiments of the present invention; Figure 5 This is a schematic diagram of the structure of a dual erasure device for a multi-view diffusion model disclosed in an embodiment of the present invention; Figure 6 This is a schematic diagram of another dual erasure device for a multi-view diffusion model disclosed in an embodiment of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] This invention discloses a dual erasure method and apparatus for a multi-view diffusion model. The method includes: acquiring an input image; processing the input image to obtain an adversarial example image; and processing the input image and the adversarial example image to obtain adversarial noise information. This invention extracts latent features from the adversarial examples and designs a latent feature erasure loss to deviate it from the distribution of a clean image. In each iteration, a time step is randomly sampled, and an attention erasure loss is established to shift the attention of the region of interest to other regions, thereby disrupting the geometric and visual consistency between the generated multi-views. The two losses are combined to form the final dual erasure loss, which is then updated using a gradient descent algorithm. This invention can simultaneously erase latent features and attention, achieving intellectual property protection. Detailed descriptions follow.

[0029] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating a dual erasure method for a multi-view diffusion model disclosed in an embodiment of the present invention. Figure 1The described dual erasure method for multi-view diffusion models is applied in the fields of 3D ensemble reconstruction and adversarial learning technologies, and the embodiments of this invention are not limited thereto. Figure 1 As shown, the dual erasure method for multi-view diffusion models can include the following operations: S1, Obtain the input image; S2, process the input image to obtain adversarial sample images; S3, process the input image and the adversarial sample image to obtain adversarial noise information; S4. Using the anti-noise information, the input image is processed to obtain a double erasure result.

[0030] Optionally, the method for generating adversarial example images is as follows: 1) Extract key features from the input image that have a significant impact on the model's decision-making. The input image I first passes through a feature extraction network consisting of convolutional layers, pooling layers, and activation functions, and is abstracted layer by layer into a high-dimensional feature vector f(I). This feature vector captures key semantic information in the image, such as edges, textures, and object parts, providing a basis for subsequent classification decisions.

[0031] The feature vector f(I) is fed into a fully connected layer (or an equivalent 1×1 convolutional layer), and undergoes a linear transformation using a learnable weight matrix w and a bias vector b to obtain the classification confidence function. , This is the weight matrix for category c. As a bias, the softmx function normalizes the predicted values ​​for all categories. The value range is (0,1).

[0032] right Processing is performed to obtain key features. : for The pixel in the i-th row, j-th column, and ch-th channel is... The attention weights for the ch-th channel are learned through the network. Standard deviation Gaussian kernel, This is a convolution operation.

[0033] 2) Process the key features to obtain the image mask; A first threshold and a second threshold are preset. Pixels with key features greater than the first threshold are marked as foreground, and pixels with key features less than the second threshold are marked as background. The foreground is then processed to obtain an image mask.

[0034] 3) Using the image mask, process the input image to obtain an important image region mask. Masks for unimportant image regions; 4) Masking the important image regions The image is processed to obtain a perturbed image; Initialize adversarial samples and gradient ; right (k is the number of scaled replicas) Execute Calculate the gradient of the i-th replica , For real labels, For the i-th copy, Let be the loss function between the i-th copy and the true label; in, The probability of the model predicting a positive class; Update gradient: Push adversarial examples toward gradient ascent: in, The initial learning rate (step size) controls the magnitude of adversarial example updates. for The first moment, for The second moment, It is a very small positive number to prevent the denominator from being 0 and to ensure numerical stability.

[0035] Remove perturbations that are not part of the key feature mask. return For perturbed images 5) Integrate the perturbation image and the non-important image region to obtain the adversarial sample image.

[0036] Optionally, processing the input image to obtain adversarial sample images includes: S21, Obtain initialization noise information; S22, the initial noise information and the input image are processed to obtain adversarial sample images.

[0037] Optionally, the processing of the input image and the adversarial sample image to obtain adversarial noise information includes: S31, Process the input image and the adversarial sample image to obtain the feature erasure loss; S32, process the input image to obtain attention erasure loss; S33, integrate the feature erasure loss and the attention erasure loss to obtain the total erasure loss; S34, using the gradient descent method, the total erasure loss is processed to obtain anti-noise information.

[0038] Optionally, the processing of the input image and the adversarial example image to obtain the feature erasure loss includes: S311, Process the input image to obtain input image features. ; S312, Process the adversarial sample image to obtain adversarial sample image features. ; S313, for the input image features and the adversarial sample image features The feature erasure loss is obtained by processing the feature.

[0039] Optionally, processing the input image to obtain the attention erasure loss includes: S321, Process the input image to obtain the foreground mask of the input image. ; S322, Foreground mask of the input image The process is performed to obtain the attention mask. ; S323, regarding the attention mask Processing is performed to remove the loss of attention; The expression for the attention erasure loss is: in, To erase the loss of attention, for The first in line, number Column elements; For attention graph The first in line, number Column elements, , express The function, where T is the transpose.

[0040] Optionally, the input image is processed to obtain the foreground mask of the input image. ,include; S3211, Encode the adversarial sample image to obtain the adversarial sample image features; S3212, Using the Unet model, the adversarial sample image features are processed to obtain denoised adversarial sample image features; S3213, Process the features of the denoised adversarial sample image to obtain the foreground mask of the input image. .

[0041] Optionally, the attention mask The process involves attention erasure loss, including: S3231, regarding the attention mask The process yields self-attention masks, multi-view attention masks, and cross-domain attention masks. S3232, Process the self-attention mask to obtain a self-attention map; S3233, Process the multi-view attention mask to obtain a multi-view attention map; S3234, Process the cross-domain attention mask to obtain a cross-domain attention map; S3235, Process the self-attention map, the multi-view attention map, and the cross-domain attention map to obtain the attention erasure loss.

[0042] As can be seen, this invention extracts latent features from adversarial examples and designs a latent feature erasure loss to deviate them from the distribution of clean images. In each iteration, time steps are randomly sampled, and an attention-based erasure loss is established, shifting attention from the region of interest to other regions, thereby disrupting the geometric and visual consistency between the generated multi-views. The two losses are combined to form the final dual-erasure loss, which is then updated using a gradient descent algorithm. This invention's method can simultaneously erase latent features and attention, achieving intellectual property protection.

[0043] Example 2 Please see Figure 2 , Figure 2 This is a flowchart illustrating another dual-erasure method for a multi-view diffusion model disclosed in an embodiment of the present invention. Figure 2 The described dual erasure method for multi-view diffusion models is applied in the fields of 3D ensemble reconstruction and adversarial learning technologies, and the embodiments of this invention are not limited thereto. Figure 2As shown, the dual erasure method for multi-view diffusion models can include the following operations: 1. Problem Definition Unlike two-dimensional diffusion models that generate a single image, MVDM aims to generate multiple images simultaneously from different views, represented as... Specifically, given an input image and a set of camera poses , It can be represented as: It comes from The distribution of images in different domains. MVDMs trained a model. Generate images with geometric and visual consistency. (Model) It includes an encoder E, which takes the input image as input. Mapping to a latent feature space Then, denoising is performed using UNet with various attention mechanisms. Simultaneously, a pre-trained CLIP model generates the input image. Embedding, camera pose This guides the desaturation process. Finally, the decoder D maps the latent feature vectors back to the initial space. . From Different domains The conditional distribution of images d generated from different perspectives is used to describe the generation rules of multi-domain, multi-view images. Indicates from Different domains A set of images generated from different views, where d is the final image output by the decoder. CLIP is a pre-trained multimodal model used to generate the input image. x The embedding provides semantic-level conditional guidance. The number of viewpoints.

[0044] 2. Dual erasure of latent features and attention To destroy the internal features of each generated multi-view image, a feature erasure loss is introduced in the following formula, which leads to adversarial examples. The latent features deviate from the clean image Potential characteristics.

[0045] As mentioned above, the geometric and visual consistency among the generated multi-view images is attributed to various attention mechanisms in MVDM, thus eliminating attention to regions of interest by reducing the corresponding attention scores. Representative attention mechanisms used in MVDM include self-attention, multi-view attention, and cross-domain attention. The attention map can be calculated as follows: Where Q is the query matrix, obtained by linear transformation of the input sequence, K is the key matrix, obtained by linear transformation of the input sequence, and d is the dimension of the vector (i.e. the length of each vector), used to scale the dot product to prevent the inner product from being too large and causing the Softmax output to be too saturated. Generally, regions with higher attention scores are more important for enhancing consistency across multi-view images. Considering that low-dimensional attention maps contain more semantic information, all lowest-dimensional attention maps are aggregated and averaged, then erased. The specific loss function is as follows: in It is a binary matrix mask representing the position of the foreground in the attention map. For the various attention mechanisms used in MVDM, and The dimensions are completely different. Specifically, the self-attention mechanism aims to establish internal correlations between different regions of each single-view and single-domain image. Therefore, its and and Having the same size, where Indicates batch size, Indicates the sequence length. Indicates the hidden size. Used for self-attention. The dimension is The multi-view attention mechanism aims to establish extrinsic correlations between different regions of a multi-view image. Therefore, the original... The dimension was first reshaped into , and then repeat Second To match The dimension of attention. Attention graph used for multi-view attention. The dimension is . This represents a product operation; similarly, cross-domain attention mechanisms aim to establish external correlations between different regions across multiple domain images. Therefore, The dimensions are converted The dimension of the attention map m used for self-attention is... .

[0046] In order to match The dimension is first determined by extracting the foreground mask from the input image. To calculate Then, adjust it to and reshape it into To obtain the foreground mask for each row, and repeat This is repeated several times to obtain the foreground mask for each view and domain. Finally, it is repeated further, depending on the different attention mechanisms, respectively. , , It is used for self-attention, multi-view attention, and cross-domain attention.

[0047] In summary, and Finally, these are combined to form a double-erasing loss. PGD (projected gradient descent) is used to update the perturbation.

[0048] The method of this invention was compared with two state-of-the-art (SOTA) attack methods, AdvDM and WAE, designed for single-image generation tasks. WAE uses feature loss to drive latent features of the input image to a specific watermark image, while AdvDM utilizes diffusion loss to disrupt the distribution of latent features. To compare the attack performance of different methods, AdvDM, WAE, and the proposed method were executed on Zero123++ and Wonder3D, respectively. The results are shown in Table 1. It can be seen that WAE and AdvDM only result in small differences in SSIM, LPIPS, and CD scores, while the method of this invention produces significant attack effects on all metrics. Specifically, the method of this invention reduces SSIM to 0.655, increases LPIPS to 0.405, and increases CD to 0.2155 on Zero123++. These results demonstrate that the method of this invention has better attack performance than previous work in terms of the quality of the generated multi-view images and geometrically reconstructed shapes.

[0049] Table 1. Objective comparison of attack results using different methods Figure 3 The subjective comparison of the different methods is further illustrated. The upper part shows the color image generated by Zero123++. For each attack method, six different views of the generated image are presented. It can be seen that the contrasting methods only cause slight changes in surface texture, while the contours remain intact. In contrast, the method of this invention generates completely different features and contours because the proposed double erase attack can significantly disrupt the geometric and visual consistency between the multi-view images.

[0050] In addition to the comparison of attack performance, Figure 3Furthermore, the transferability of the method of the present invention to other models and its robustness against defensive methods are demonstrated. It can be seen that the method of the present invention possesses good transferability and robustness.

[0051] The implementation steps of this invention are as follows: Input: Multi-view diffusion model, including encoder UNet ,image , mask Disturbance range Output: Anti-noise Step 1. Randomly initialize noise and calculate the features of the clean image. Step 2. Forepochinrange(0, ) Step 3. Random sampling time steps t Step 4. Step 5. Step 6. Calculate feature erasure loss Step 7. Capture Attention Step 8. Calculation Step 9. Calculate attention erasure loss Step 10. Calculate the total loss and update the noise. Step 11. Set the noise limit within the range Inside Step 12. Return to final noise Example 3 This embodiment relates to the fields of 3D geometric reconstruction and adversarial machine learning, specifically addressing the intellectual property (IP) issues in protecting 3D assets generated using Multi-View Diffusion Models (MVDM). MVDM is a generative model widely used to rapidly reconstruct 3D geometry from 2D multi-view images. These models are widely used in video games, virtual reality, and digital content creation, capable of accurately generating consistent multi-view images and quickly representing 3D objects. However, with the widespread application of MVDM, intellectual property infringement has become increasingly prominent. Malicious users can use publicly available 2D images to imitate products or designs without authorization, or even completely steal 3D assets. Therefore, how to effectively protect 3D assets and prevent unauthorized copying has become an urgent problem to be solved.

[0052] Traditional adversarial machine learning techniques are often used to address intellectual property protection issues in image generation tasks. However, these methods primarily focus on single-image generation, neglecting the geometric and visual consistency requirements of the generated multi-view images. Previous adversarial methods mainly disrupt image features but fail to adequately consider the importance of geometric consistency between images in multi-view generation tasks. This embodiment proposes a novel adversarial attack method against MVDM, specifically targeting the protection of 3D assets. By disrupting the intrinsic features and geometric consistency of the generated multi-view images, it effectively prevents illegal 3D asset copying. This method introduces a dual eraser attack of latent features and attention, which not only disrupts the intrinsic feature distribution of the generated images but also destroys the geometric consistency between multi-view images.

[0053] This embodiment employs a dual-erasure attack method—simultaneously perturbing latent features and attention mechanisms—aimed at significantly reducing 3D reconstruction quality and providing robust protection for 3D assets. Compared to existing single-image adversarial attack methods, this embodiment's method can more effectively disrupt multi-view attacks during the 3D asset generation process. Figure 1 Consistency. This method also possesses strong defensive capabilities, exhibiting good adaptability to existing defense techniques, and is applicable to different MVDM models. It can protect 3D assets from unauthorized copying in practical applications. Comparative experiments show that the method in this embodiment outperforms existing techniques in terms of attack effectiveness, particularly in terms of the degree of damage to image quality and geometric reconstruction quality, significantly improving attack effectiveness.

[0054] like Figure 4 As shown in the second line, the previous attack method (WAE) is directly transferred to MVDM, causing some chaotic effects by disrupting the distribution of latent features, but the outline remains intact because the consistency between multi-view images is not disturbed. In addition to perturbing the distribution of latent features, we claim that attacking the attention mechanism in MVDM can effectively disrupt the geometric and visual consistency of the generated multi-view images. This invention fully considers various attention mechanisms in MVDM, including self-attention, multi-view attention, and cross-domain attention mechanisms, and then proposes the same attention erasure loss to reduce attention to the region of interest. In this way, the attention of MVDM will shift from the region of interest (i.e., the foreground region) to the background region, thereby disrupting the consistency between the generated multi-view images. Furthermore, an additional feature erasure loss is incorporated to deviate the descendant features from the original distribution to achieve the proposed double erasure attack. Experimental object datasets on two state-of-the-art MVDM datasets and the Google ScannedObjects dataset demonstrate that the method of this invention achieves superior performance in terms of attack effectiveness, transferability, and robustness of the defense method.

[0055] Figure 4This section compares the reconstruction results of the method in this embodiment with those of existing methods. First line: Without protection, the 3D geometry can be accurately reconstructed. Second line: The method's WAE perturbation reveals latent features, leading to cluttered content, but the reconstructed outline remains intact. Third line: The method of this invention can simultaneously erase latent features and attention, significantly reducing the quality of 3D reconstruction.

[0056] Example 4 Please see Figure 5 , Figure 5 This is a schematic diagram of a dual erasure device for a multi-view diffusion model disclosed in an embodiment of the present invention. Figure 5 The described dual erasure device for multi-view diffusion models is applied in the fields of 3D ensemble reconstruction and adversarial learning technologies, and the embodiments of this invention are not limited thereto. Figure 5 As shown, the dual erasure device for multi-view diffusion models may include the following operations: S301, Input Image Acquisition Module, used to acquire input images; S302, Adversarial sample image generation module, used to process the input image to obtain adversarial sample images; S303, Adversarial noise information generation module, used to process the input image and the adversarial sample image to obtain adversarial noise information; S304, Dual Erasure Module, is used to process the input image using the anti-noise information to obtain a dual erasure result.

[0057] Example 5 Please see Figure 6 , Figure 6 This is a schematic diagram of another dual erasure device for a multi-view diffusion model disclosed in an embodiment of the present invention. Figure 6 The described dual erasure device for multi-view diffusion models is applied in the fields of 3D ensemble reconstruction and adversarial learning technologies, and the embodiments of this invention are not limited thereto. Figure 6 As shown, the dual erasure device for multi-view diffusion models may include the following operations: Memory 401 storing executable program code; Processor 402 coupled to memory 401; The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the dual erasure method for the multi-view diffusion model described in Embodiments 1, 2 and 3.

[0058] Example 6 This invention discloses a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program enables a computer to perform the steps in the dual erasure method for a multi-view diffusion model described in Embodiments 1, 2, and 3.

[0059] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0060] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0061] Finally, it should be noted that the dual erasure method and apparatus for multi-view diffusion models disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dual erasure method for multi-view diffusion models, characterized in that, The method includes: S1, Obtain the input image; S2, process the input image to obtain adversarial sample images; S3, process the input image and the adversarial sample image to obtain adversarial noise information; S4. Using the anti-noise information, the input image is processed to obtain a double erasure result.

2. The dual erasure method for multi-view diffusion models according to claim 1, characterized in that, The process of processing the input image to obtain adversarial sample images includes: S21, Obtain initialization noise information; S22, the initial noise information and the input image are processed to obtain adversarial sample images.

3. The dual erasure method for multi-view diffusion models according to claim 1, characterized in that, The process of processing the input image and the adversarial sample image to obtain adversarial noise information includes: S31, Process the input image and the adversarial sample image to obtain the feature erasure loss; S32, process the input image to obtain attention erasure loss; S33, integrate the feature erasure loss and the attention erasure loss to obtain the total erasure loss; S34, using the gradient descent method, the total erasure loss is processed to obtain anti-noise information.

4. The dual erasure method for multi-view diffusion models according to claim 3, characterized in that, The process of processing the input image and the adversarial sample image to obtain the feature erasure loss includes: S311, Process the input image to obtain input image features. ; S312, Process the adversarial sample image to obtain adversarial sample image features. ; S313, for the input image features and the adversarial sample image features The feature erasure loss is obtained by processing the feature.

5. The dual erasure method for multi-view diffusion models according to claim 3, characterized in that, The process of processing the input image to obtain the attention erasure loss includes: S321, Process the input image to obtain the foreground mask of the input image. ; S322, Foreground mask of the input image The process is performed to obtain the attention mask. ; S323, regarding the attention mask Processing is performed to remove the loss of attention; The expression for the attention erasure loss is: in, To erase the loss of attention, for The first in line, number Column elements; For attention graph The first in line, number Column elements, , express The function, where T is the transpose.

6. The dual erasure method for multi-view diffusion models according to claim 5, characterized in that, The input image is processed to obtain the foreground mask of the input image. ,include; S3211, Encode the adversarial sample image to obtain the adversarial sample image features; S3212, Using the Unet model, the adversarial sample image features are processed to obtain denoised adversarial sample image features; S3213, Process the features of the denoised adversarial sample image to obtain the foreground mask of the input image. .

7. The dual erasure method for multi-view diffusion models according to claim 5, characterized in that, The attention mask The process involves attention erasure loss, including: S3231, regarding the attention mask The process yields self-attention masks, multi-view attention masks, and cross-domain attention masks. S3232, Process the self-attention mask to obtain a self-attention map; S3233, Process the multi-view attention mask to obtain a multi-view attention map; S3234, Process the cross-domain attention mask to obtain a cross-domain attention map; S3235, Process the self-attention map, the multi-view attention map, and the cross-domain attention map to obtain the attention erasure loss.

8. A dual erasure device for multi-view diffusion models, characterized in that, The device includes: The input image acquisition module is used to acquire the input image; An adversarial sample image generation module is used to process the input image to obtain an adversarial sample image; An adversarial noise information generation module is used to process the input image and the adversarial sample image to obtain adversarial noise information; The dual erasure module is used to process the input image using the anti-noise information to obtain a dual erasure result.

9. A dual erasure device for multi-view diffusion models, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the dual erasure method for a multi-view diffusion model as described in any one of claims 1-7.

10. A computer-storable medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked, are used to execute the dual erasure method for a multi-view diffusion model as described in any one of claims 1-7.

Images