Adversarial texture generation method and system applied to depth estimation model

Abstract

The invention discloses an adversarial texture generation method and system applied to a depth estimation model. The method comprises the following steps: generating an initial adversarial texture pattern based on a potential vector driving pattern generator; performing up-sampling processing on the initial confrontation texture pattern to obtain an up-sampling texture pattern; mapping the up-sampling texture pattern to a specified area of the object model, and performing image rendering on the mapped object model M by using a rendering engine to generate an object image; performing background synthesis on the object image to generate a complete scene attack image; inputting the complete scene attack image into the depth estimation model to obtain the predicted depth of the image at the current view angle; and designing a loss function for optimization based on the predicted depth and the reference depth. According to the scheme, based on the potential vector driven generator, texture mapping and background synthesis processing, the method is not limited to a two-dimensional image task any more, the training efficiency is improved, and the diversity and stability of generated textures are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent perception systems, and in particular to a method and system for generating adversarial textures for depth estimation models. Background Art

[0002] In recent years, with the rapid development of cutting-edge technologies such as autonomous driving, intelligent monitoring, and augmented reality, deep learning-based 3D perception models have gained widespread application in practical systems. Depth estimation models, which accurately calculate the depth information of relevant target objects in image frames, play a crucial role in various applications of intelligent perception models. For example, autonomous driving models require real-time calculation of the distance to objects such as pedestrians and vehicles in images. Depth estimation models precisely meet this requirement, leading to their widespread use in perception subsystems.

[0003] However, relevant research has shown that depth estimation models are highly vulnerable to adversarial attacks, and their robustness is receiving increasing attention. To improve the overall security and stability of the system, training the model through interference testing is crucial. Currently, some research has focused on generating adversarial textures designed to occlude people or vehicles, attempting to mislead image classification or object detection models, thereby achieving physical attacks.

[0004] However, it must be pointed out that these existing methods have many obvious limitations in practical applications. On the one hand, most current methods are designed primarily for two-dimensional image tasks (such as classification or detection), and lack effective attack methods for three-dimensional perception models, especially depth estimation models. On the other hand, existing texture generation processes mostly rely on directly optimizing pixel values ​​and lack the ability to express structure, which makes it difficult to achieve controllable texture generation and expand its diversity. Summary of the Invention

[0005] In order to solve the above technical problems, the present application provides a method and system for generating adversarial textures for a depth estimation model.

[0006] In a first aspect, the present application provides a method for generating adversarial textures for a depth estimation model:

[0007] Generate an initial adversarial texture pattern t based on a latent vector-driven pattern generator p ;

[0008] The initial adversarial texture pattern t p Perform upsampling processing to convert the initial adversarial texture pattern t p The resolution is expanded to the target resolution to obtain the upsampled texture pattern

[0009] Get the object model M, and upsample the texture pattern Mapping to the designated area of ​​the object model M to replace the original texture of the object model M; and using the rendering engine to render the mapped object model M to generate an object image O with adversarial characteristics. adv ;

[0010] Acquire a natural background image for use in the object image O adv Perform background synthesis to generate a complete scene attack image x adv ;

[0011] The complete scene attack image x adv Input the depth estimation model to obtain the predicted depth of the image at the current viewing angle;

[0012] Based on the predicted depth and reference depth, a first-stage loss function is designed to quantify the attack effect; and the pattern generator is continuously optimized with the goal of maximizing the attack effect until the requirements are met.

[0013] By adopting the above technical solution, compared with the traditional pixel-level direct optimization method, this solution is based on a latent vector-driven generator and uses low-dimensional latent vectors for texture modeling, which not only improves training efficiency, but also enhances the diversity and stability of generated textures. At the same time, based on texture mapping and background synthesis processing, it is better suitable for attack training of three-dimensional perception models and is no longer limited to two-dimensional image tasks.

[0014] Optionally, the up-sampled texture pattern The designated area mapped to the object model M includes:

[0015] First, the UV coordinates and target area mask of each vertex or facet are extracted according to the topological structure of the object model M; the target area to which the texture needs to be applied is determined based on the area mask, and the three-dimensional model surface where the target area is located is matched with the two-dimensional texture space based on the UV coordinates, and the texture map is cropped or masked to ensure that the texture accurately covers the target area.

[0016] By adopting the above technical solution, accurate mapping of two-dimensional texture to three-dimensional object model is guaranteed, which is conducive to improving the generation quality of adversarial texture.

[0017] Optionally, the rendering engine is used to render the mapped object model M to generate an object image O having adversarial characteristics. adv include:

[0018]

[0019] In the formula, R represents the rendering function, M is the object model, and M mask The area mask for the texture map.

[0020] Optionally, the object image O adv Perform background synthesis to generate a complete scene attack image x adv include:

[0021] Based on the foreground transparency mask m, the object image O adv It is mixed with the natural background image at the pixel level to form a complete scene attack image x adv :

[0022] x adv =m·O adv +(1-m)·b

[0023] Among them, b is the natural background image, and m is the foreground transparency mask, which is used to control the fusion ratio of the object image and the natural background image.

[0024] Optionally, the first-stage loss function designed to quantify the attack effect based on the predicted depth and the reference depth includes:

[0025] Calculate the mean square error L between the predicted depth and the reference depth adv ;

[0026] The mean square error L adv Maximize the first stage loss function L for the goal stage1 :

[0027] L stage1 =max(L adv )

[0028] The first stage loss function L stage1 Only the depth estimation error term is included, which is used to directly improve the interference ability of the generated complete scene attack image on the pattern generator.

[0029] Optionally, after the pattern generator is optimized, the method further includes:

[0030] Fixed the pattern generator parameters and optimized the latent vector and its perturbation terms;

[0031] Design a second-stage loss function L to quantify attack effectiveness, color printability, and texture smoothness stage2 :

[0032] L stage2 =L adv +αL smooth +βL nps

[0033]

[0034] Among them, L adv Used to measure the depth estimation deviation and reflect the attack effect; L smooth Indicates the loss of texture smoothness; L nps represents the color printability loss; α and β are used to adjust the texture smoothness loss L smooth and color printability loss L nps For the second stage loss function L stage2 The hyperparameters of the degree of influence; is the pixel vector at position (i, j); c k is the kth color vector from the set of printable colors.

[0035] In a second aspect, the present application provides an adversarial texture generation system applied to a depth estimation model:

[0036] A pattern generation module is used to generate an initial adversarial texture pattern t based on a latent vector driven pattern generator. p ; and the initial confrontation texture pattern t p Perform upsampling processing to convert the initial adversarial texture pattern t p The resolution is expanded to the target resolution to obtain the upsampled texture pattern

[0037] Mapping rendering module, used to obtain the object model M, the upsampled texture pattern Mapping to the designated area of ​​the object model M to replace the original texture of the object model M; and using the rendering engine to render the mapped object model M to generate an object image O with adversarial characteristics. adv ;

[0038] Background synthesis module, used to obtain a natural background image for the object image O adv Perform background synthesis to generate a complete scene attack image x adv ;

[0039] A processing optimization module is used to convert the complete scene attack image x adv The depth estimation model is input to obtain the predicted depth of the image at the current viewing angle. Based on the predicted depth and the reference depth, a first-stage loss function is designed to quantify the attack effect. The pattern generator is continuously optimized with the goal of maximizing the attack effect until the requirements are met.

[0040] Optionally, the mapping rendering module is used to first extract the UV coordinates and target area mask of each vertex or facet according to the topological structure of the object model M; determine the target area to which the texture is to be applied according to the area mask, correspond the three-dimensional model surface where the target area is located to the two-dimensional texture space based on the UV coordinates, and crop or mask the texture map to ensure that the texture accurately covers the target area;

[0041] The rendering engine is used to render the object model M after the mapping is completed to generate an object image O with adversarial characteristics. adv include:

[0042]

[0043] In the formula, R represents the rendering function, M is the object model, and M mask The area mask for the texture map.

[0044] Optionally, the background synthesis module is used to transform the object image O into adv It is mixed with the natural background image at the pixel level to form a complete scene attack image x adv :

[0045] x adv =m·O adv +(1-m)·b

[0046] Among them, b is the natural background image, and m is the foreground transparency mask, which is used to control the fusion ratio of the object image and the natural background image.

[0047] Optionally, the processing optimization module is used to calculate the mean square error L between the predicted depth and the reference depth adv ;

[0048] The mean square error L adv Maximize the first stage loss function L for the goal stage1 :

[0049] L stage1 =max(L adv )

[0050] The first stage loss function L stage1 Only the depth estimation error term is included to directly improve the interference ability of the generated complete scene attack image on the pattern generator;

[0051] After the pattern generator is optimized, the processing optimization module is further configured to:

[0052] Fixed the pattern generator parameters and optimized the latent vector and its perturbation terms;

[0053] Design a second-stage loss function L to quantify attack effectiveness, color printability, and texture smoothness stage2 :

[0054] L stage2 =L adv +αL smooth +βL nps

[0055]

[0056] Among them, L adv Used to measure the depth estimation deviation and reflect the attack effect; L smooth Indicates the loss of texture smoothness; L nps represents the color printability loss; α and β are used to adjust the texture smoothness loss L smooth and color printability loss L nps For the second stage loss function L stage2 The hyperparameters of the degree of influence; is the pixel vector at position (i, j); c k is the kth color vector from the set of printable colors.

[0057] In a third aspect, the present application provides a computer device comprising a processor, a memory, and a communication bus, wherein the communication bus is used to realize a communication connection between the processor and the memory, and the processor is used to execute a computer program stored in the memory to implement the adversarial texture generation method applied to a depth estimation model as described in any one of the above items.

[0058] In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program; the computer program can be executed by a processor to implement the adversarial texture generation method for a depth estimation model as described in any one of the above

[0059] By adopting the above technical solution, a computer program carrier for an adversarial texture generation method applied to a depth estimation model is provided.

[0060] In summary, this application has at least the following beneficial technical effects:

[0061] 1. Compared with traditional pixel-level direct optimization methods, this solution is based on a latent vector-driven generator and uses low-dimensional latent vectors for texture modeling. This not only improves training efficiency, but also enhances the diversity and stability of generated textures. Furthermore, based on texture mapping and background synthesis processing, it is better suited for attack training of 3D perception models, no longer limited to 2D image tasks.

[0062] 2. To achieve physically feasible attacks, this solution introduces attack effect loss, color printability loss, and texture smoothness loss during training. This ensures that the generated textures are both aggressive and highly natural, avoiding being easily detected by the visual system or the human eye. At the same time, they are adaptable to printing or projection equipment, ensuring realistic attack effects after deployment.

[0063] 3. Unlike existing methods that only design attacks at the 2D image level, this solution fully integrates 3D object models, texture mapping, perspective rendering, and depth estimation models to form an end-to-end differentiable training process. This can more realistically simulate the effects of attacks in physical environments and has greater practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of a framework of an adversarial texture generation system provided in an embodiment of the present application;

[0065] Figure 2 A schematic diagram of a process for generating an adversarial texture according to an embodiment of the present invention;

[0066] Figure 3 A schematic diagram of a fully convolutional pattern generator architecture provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of the structure of an adversarial texture generation system provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of the structure of a computer device for implementing an adversarial texture generation method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0070] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include plural expressions, unless the context clearly indicates otherwise. It should also be understood that the terms "and / or" used in this application refer to and include any or all possible combinations of one or more listed items. The term "exemplary" means "serving as an example, embodiment or illustrative", and any embodiment described here as "exemplary" is not necessarily interpreted as being superior to or better than other embodiments. The terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0071] like Figure 1 As shown, an embodiment of the present invention proposes a method and system for generating adversarial textures for a depth estimation model, the core idea of ​​which is to perform structured modeling of the target texture through a generator driven by a latent vector, and to combine three-dimensional model rendering, natural background synthesis and depth estimation model to jointly construct an end-to-end optimizable training process, thereby realizing an adversarial texture attack with physical feasibility. This scheme misleads the prediction results of the depth estimation model in specific areas (such as human clothing and vehicle appearance) by generating adversarial textures, and has good transferability and physical feasibility. This scheme adopts a two-stage training mechanism: the first stage trains the generator to generate the initial adversarial texture through a random latent vector; the second stage further optimizes the latent vector on the basis of a fixed generator structure to enhance the attack effect, while ensuring the printability and structural continuity of the generated texture.

[0072] The following is a detailed description of the specific process. Figure 2 , provides an adversarial texture generation method for depth estimation model, mainly including:

[0073] S102: Generate an initial adversarial texture pattern t based on the latent vector driven pattern generator p .

[0074] See Figure 3In this step, a latent vector driven pattern generator model is constructed to achieve structurally controllable adversarial texture generation. First, a latent vector z is obtained through random initialization. The role of this latent vector is to introduce randomness, so that the pattern generator can generate diverse and non-deterministic outputs; a low-resolution texture pattern with preliminary attack capability (i.e., the initial adversarial texture pattern t p ). This process can be formally expressed as:

[0075] t p =G(z)

[0076] where G(·) is the pattern generator and z is the latent vector.

[0077] like Figure 3 As shown, this scheme uses a fully convolutional generator architecture to map the low-dimensional latent vector z to the initial adversarial texture pattern t p The latent vector is Where B represents the batch size, C is the number of channels, H and W represent the spatial dimensions respectively. The pattern generator G gradually upsamples the latent vector through a series of convolutional layers and transposed convolutional layers to generate a three-channel texture pattern t p It is worth noting that the pattern generator network uses a convolutional layer structure and uses a zero-padding strategy to ensure the continuity of spatial transformation, thereby effectively suppressing the introduction of boundary artifacts. Of course, in the optional embodiments of this application, the pattern generator architecture can be flexibly selected and is not limited here.

[0078] S104: Initial adversarial texture pattern t p Perform upsampling to convert the initial adversarial texture pattern t p The resolution is expanded to the target resolution to obtain the upsampled texture pattern

[0079] In order to meet the resolution requirements of the 3D object model, an upsampling module is used to expand the low-resolution texture to the target resolution. This process can be formally expressed as:

[0080]

[0081] Here, "Expand" represents the upsampling operation; this expansion process not only preserves latent features but also enhances texture continuity, ensuring no noticeable splicing or misalignment when mapped to a 3D surface. This upsampling operation provides a texture foundation for subsequent rendering and attack effects, and is differentiable, enabling end-to-end optimization.

[0082] S106: Obtain the object model M and upsample the texture pattern Mapping to the specified area of ​​the object model M to replace the original texture of the object model M; and using the rendering engine to render the mapped object model M to generate an object image O with adversarial characteristics adv .

[0083] First, the UV coordinates and target area mask of each vertex or facet are extracted according to the topological structure of the object model M. The target area to which the texture needs to be applied is determined based on the area mask. The three-dimensional model surface where the target area is located is matched with the two-dimensional texture space based on the UV coordinates. The texture map is cropped or masked to ensure that the texture accurately covers the target area.

[0084] Specifically, we first need to thoroughly analyze the topology of the object model M. This topology describes the connections and organization between vertices and facets within the model. By studying the topology, we can clearly understand the model's geometric characteristics and surface morphology, laying the foundation for subsequent extraction of UV coordinates and target region masks.

[0085] After determining the model's topology, extract the corresponding UV coordinates for each vertex or face in the model. UV coordinates are a two-dimensional coordinate system used to describe how textures are mapped onto a three-dimensional model surface. The U and V coordinates correspond to the horizontal and vertical directions in the two-dimensional texture space, respectively, similar to the x and y coordinates in a plane coordinate system. By extracting the UV coordinates, the position of each vertex or face in the two-dimensional texture space can be determined, thereby achieving accurate texture mapping.

[0086] The target region mask identifies specific areas on the model surface where textures are applied. Target regions can be determined based on specific requirements or rules, such as color, shape, or semantic information. By analyzing and marking the model surface, a target region mask is generated. This mask serves as the basis for subsequent texture mapping, ensuring that textures are applied only to the designated areas.

[0087] Based on the extracted target region mask, a detailed interpretation is performed. The mask is typically represented as a binary image, where each pixel value indicates whether the location belongs to the target region. By traversing the mask image, the specific location and extent of the target region on the model surface can be determined.

[0088] By combining the model's topology and UV coordinates, the target area can be accurately positioned on the 3D model surface. The UV coordinate range of each vertex or patch within the target area can be calculated to determine its corresponding position in the 2D texture space, thus achieving precise positioning of the target area.

[0089] Based on the extracted UV coordinates, a mapping relationship is established between the 3D model surface and the 2D texture space. This mapping relationship describes how each point on the model surface corresponds to a point in the 2D texture space. By establishing this mapping relationship, the geometric information of the 3D model surface can be converted into coordinate information in the 2D texture space, providing a basis for accurate texture coverage.

[0090] After establishing the mapping relationship, the 3D model surface where the target area is located is converted to a 2D texture space. The UV coordinates of each vertex or patch within the target area are transformed and mapped to the corresponding position in the 2D texture space. This allows the target area on the 3D model surface to be visualized in the 2D texture space, facilitating subsequent texture processing.

[0091] Based on the location and extent of the target area in the 2D texture space, the texture image is clipped or masked. Clipping removes the portion of the texture image that exceeds the target area, retaining only the texture portion corresponding to the target area. Masking blocks the unnecessary portion of the texture image so that it does not appear on the model surface. By clipping or masking, you can ensure that the texture image only contains the texture information required for the target area, avoiding texture redundancy and waste.

[0092] After cropping or masking, you may need to make adjustments to the texture to ensure it accurately covers the target area. For example, you can scale, rotate, and translate the texture to match the shape and size of the target area. You can also adjust the texture's color, brightness, contrast, and other parameters to enhance the texture's visual quality.

[0093] After the above processing, the adjusted texture map is applied to the target area on the 3D model surface. By mapping each pixel value in the texture map to the UV coordinates of each vertex or patch within the target area, the texture information is mapped onto the model surface. This allows for accurate texture effects to be displayed on the model surface, achieving the mapping of a 2D texture to the 3D model surface.

[0094] After the texture mapping is completed, the model is rendered using a rendering engine based on perspective projection to generate an object image with adversarial characteristics. adv The process can be expressed as:

[0095]

[0096] Among them, R represents the rendering function, M is the three-dimensional object model, and M mask The target area mask for the texture map. The image generated by this process will serve as the basis for the subsequent attack input.

[0097] Rendering engines based on perspective projection simulate the way the human eye perceives objects. They project objects onto a two-dimensional image plane based on their position, orientation, and size in three-dimensional space. Perspective projection is characterized by objects appearing larger when closer to the viewer and smaller when farther away.

[0098] The rendering function R is the core of the entire rendering process. It receives the adversarial texture, the 3D object model M and the target area mask Mmask of the texture mapping as input, performs scene construction, lighting calculation, projection transformation and rasterization processing, and renders the object image O with adversarial characteristics. adv It should be understood that the rendering process of the rendering function R is not the focus of the present invention, and the embodiments of the present invention are not limited to which rendering function is used, as long as rendering can be achieved.

[0099] The rendering engine based on perspective projection can simulate real-world lighting and perspective effects, generating more realistic images. This makes the attack scenario closer to reality, improving the practicality and credibility of the attack.

[0100] In an optional embodiment of the present application, in order to enhance the diversity and generalization ability of the training data, multiple perspectives and multiple lighting conditions may be used for image rendering for each background image.

[0101] S108: Acquire a natural background image for use in the object image O adv Perform background synthesis to generate a complete scene attack image x adv .

[0102] In order to improve the adaptability of adversarial texture in the actual environment, this step will render the object image O adv The image is then fused with the real background image to generate a complete attack image. This synthetic image retains the interference effect of the adversarial texture in the portrait area while incorporating natural scene information, making the overall image more realistic. The synthesis operation is based on the transparency mask m, which is a pixel-wise weighted blend of the object image and the background image. The formula is as follows:

[0103] x adv =m·O adv +(1-m)·b

[0104] Among them, x adv is the synthetic image (i.e., the complete scene attack image), b is the natural background image, and m is the foreground transparency mask, which is used to control the fusion ratio of the object image and the natural background image. This synthetic image is used as the complete attack sample and input into the subsequent depth estimation model for evaluation.

[0105] S110: attack the complete scene image x advInput the depth estimation model to obtain the predicted depth of the image at the current viewing angle; and input the object model M into the depth estimation model to obtain the reference depth.

[0106] The goal of this step is to evaluate the effect of the generated complete scene attack image on the depth estimation model. Specifically, the complete scene attack image synthesized in the previous step is input into a pre-trained depth estimation model to obtain a predicted depth map of the image at the current viewing angle. At the same time, for comparison purposes, the original image (i.e., the object model M) without the added adversarial texture is also used as input to obtain a reference depth map under normal conditions.

[0107] S112: Based on the predicted depth and reference depth, a first-stage loss function is designed to quantify the attack effect; and the pattern generator is continuously optimized with the goal of maximizing the attack effect until the requirements are met.

[0108] In order to evaluate the ability to counter the interference of texture more focusedly, the region mask M on the object model can be used mask (For example, clothing or skin areas), the difference between the two depth maps is compared only in the area covered by the mask. In an optional embodiment of the present application, in order to simulate the state where the target area "disappears" in the depth map after a successful attack, we uniformly set the reference depth of the area to 0, that is, regard it as the background. This setting clarifies the core goal of the attack: to make the model completely unable to perceive the true depth of the target object. Traditional methods often use real depth as a reference, which often only causes local interference and makes it difficult to achieve an overall "disappearance" effect. In contrast, by directly setting the reference depth to 0, we can guide the model to output a depth value close to the background in the mask area, thereby more effectively improving the concealment and disturbance intensity of the attack. This error indicator will also participate in the optimization process as an aggressive loss term in subsequent training. The attack loss can be defined as the mean square error (MSE) in the mask area:

[0109] L adv =MSE(D(x adv ) -1 ⊙m,0)

[0110] where MSE(·,·) is the mean square error between the two variables, D is the depth estimation model, and m is the foreground mask.

[0111] In an optional embodiment of the present application, based on the predicted depth and the reference depth, a first-stage loss function is designed to quantify the attack effect:

[0112] Calculate the mean square error L between the predicted depth and the reference depth adv ;

[0113] The mean square error L advMaximize the first stage loss function L for the goal stage1 :

[0114] L stage1 =max(L adv )

[0115] The first stage loss function L stage1 Only the depth estimation error term is included to directly improve the interference ability of the generated complete scene attack image on the pattern generator.

[0116] Existing methods often ignore physical feasibility constraints, such as texture continuity, printability, color renderability, etc., which will seriously affect the operability and effect stability of the attack in actual deployment. To this end, in an optional embodiment of the present invention, a two-stage optimization scheme is provided. In the first stage, only the generator is optimized, and the focus is on maximizing the attack effect; in the second stage, the parameters of the optimized pattern generator are fixed, and only the potential vector and its perturbation terms are optimized. By introducing three joint optimizations: aggressiveness loss, color printability loss, and texture smoothness loss. The purpose of this is to further refine the adversarial and physical feasibility of the generated texture while maintaining the stability of the texture structure, so that the texture is suitable for printing and projection in real physical scenes.

[0117] Specifically, after the optimization of the pattern generator is completed (reaching the number of iterations or reaching the set threshold of the mean square error), a multi-objective loss function is constructed to guide the entire adversarial texture training process. The loss function comprehensively considers three aspects: attack effectiveness, physical feasibility, and texture quality, to ensure that the generated texture can not only mislead the depth model, but also have good printing and application performance. First, in order to measure the attack effect, the offensive loss is introduced, which is used to minimize the depth estimation difference between the attack image and the normal image in the target area. Secondly, in order to improve the physical printability of the texture, the color printability loss is introduced. This item constrains the texture color to be close to a set of printable colors, thereby ensuring that the generated texture will not have obvious color difference or unrenderable color problems when actually printed. Finally, the texture smoothness loss is introduced to limit the drastic changes in gradients between adjacent pixels in the texture, thereby enhancing the continuity and naturalness of the pattern. This loss helps to improve visual realism and reduce rendering artifacts caused by sudden changes in texture boundaries.

[0118] L stage2 =L adv +αL smooth +βL nps

[0119]

[0120] Among them, L advUsed to measure the depth estimation deviation and reflect the attack effect; L smooth Indicates the loss of texture smoothness; L nps represents the color printability loss; α and β are used to adjust the texture smoothness loss L smooth and color printability loss L nps For the second stage loss function L stage2 The values ​​of the two directly affect the optimization direction and final effect of the attack texture; is the pixel vector at position (i, j); c k is the kth color vector from the set of printable colors. This loss function guides the adjustment of the latent variable z during the second-stage optimization process, ensuring that the generated adversarial textures are both aggressive and physically realistic and natural.

[0121] Through the joint optimization strategy of the above two stages, this scheme can generate adversarial texture images with both attack capabilities and physical realization capabilities, which is suitable for the security research and testing of depth estimation models in scenarios such as autonomous driving and monitoring.

[0122] On the basis of the above method embodiment, this embodiment further provides an adversarial texture generation system applied to a depth estimation model, so as to implement the specific process of the above method.

[0123] refer to Figure 4 , the system comprises:

[0124] The pattern generation module 41 is used to generate an initial adversarial texture pattern t based on the potential vector driving pattern generator. p ; and the initial adversarial texture pattern t p Perform upsampling to convert the initial adversarial texture pattern t p The resolution is expanded to the target resolution to obtain the upsampled texture pattern

[0125] Mapping rendering module 42, used to obtain the object model M, upsample the texture pattern Mapping to the specified area of ​​the object model M to replace the original texture of the object model M; and using the rendering engine to render the mapped object model M to generate an object image O with adversarial characteristics adv ;

[0126] The background synthesis module 43 is used to obtain a natural background image for use in the object image O adv Perform background synthesis to generate a complete scene attack image x adv ;

[0127] The processing optimization module 44 is used to convert the complete scene attack image xadv The depth estimation model is input to obtain the predicted depth of the image at the current viewing angle. Based on the predicted depth and the reference depth, a first-stage loss function is designed to quantify the attack effect. The pattern generator is continuously optimized with the goal of maximizing the attack effect until the requirements are met.

[0128] In an optional embodiment, the mapping rendering module 42 is configured to first extract the UV coordinates and target region mask of each vertex or facet based on the topological structure of the object model M; determine the target region to which the texture is to be applied based on the region mask, map the three-dimensional model surface where the target region is located to the two-dimensional texture space based on the UV coordinates, and crop or mask the texture map to ensure that the texture accurately covers the target region;

[0129] The rendering engine is used to render the object model M after the mapping is completed to generate an object image O with adversarial characteristics. adv include:

[0130]

[0131] In the formula, R represents the rendering function, M is the object model, and M mask The area mask for the texture map.

[0132] In an optional embodiment, the background synthesis module is used to transform the object image O into the foreground transparency mask m. adv Mixed with the natural background image at pixel level weighted to form a complete scene attack image x adv :

[0133] x adv =m·O adv +(1-m)·b

[0134] Among them, b is the natural background image, and m is the foreground transparency mask, which is used to control the fusion ratio of the object image and the natural background image.

[0135] In an optional embodiment, the processing optimization module 44 is configured to calculate the mean square error L between the predicted depth and the reference depth. adv ;

[0136] The mean square error L adv Maximize the first stage loss function L for the goal stage1 :

[0137] L stage1 =max(L adv )

[0138] The first stage loss function L stage1Only the depth estimation error term is included to directly improve the interference ability of the generated complete scene attack image on the pattern generator;

[0139] After optimizing the pattern generator, the processing optimization module 44 is further configured to:

[0140] Fixed the pattern generator parameters and optimized the latent vector and its perturbation terms;

[0141] Design a second-stage loss function L to quantify attack effectiveness, color printability, and texture smoothness stage2 :

[0142] L stage2 =L adv +αL smooth +βL nps

[0143]

[0144] Among them, L adv Used to measure the depth estimation deviation and reflect the attack effect; L smooth Indicates the loss of texture smoothness; L nps represents the color printability loss; α and β are used to adjust the texture smoothness loss L smooth and color printability loss L nps For the second stage loss function L stage2 The hyperparameters of the degree of influence; is the pixel vector at position (i, j); c k is the kth color vector from the set of printable colors.

[0145] The various variations and specific examples of the methods provided in the above embodiments are also applicable to the system of this embodiment. Through the above detailed description of the adversarial texture generation method, those skilled in the art can clearly understand the system implementation method of this embodiment. For the sake of brevity, it will not be described in detail here.

[0146] The present invention proposes an adversarial texture generation method and system with excellent versatility and physical feasibility, suitable for a variety of application scenarios related to 3D perception and artificial intelligence security. The generated adversarial texture can be deployed in the real world on the surfaces of target objects such as human figures and vehicles, thereby interfering with and testing perception systems by misleading depth estimation models. Key application areas include, but are not limited to:

[0147] 1. Safety testing and evaluation of perception models in autonomous driving systems

[0148] Autonomous vehicles rely on monocular or multi-camera vision for scene modeling and depth estimation. This method can be used to simulate attack scenarios. By applying generated adversarial textures to pedestrian models, the robustness and recognition capabilities of autonomous driving systems under physical attacks can be tested, aiding the development of safer and more reliable perception modules.

[0149] 2. Research on stealth and misdirection in intelligent monitoring and security systems

[0150] In security video surveillance systems, depth estimation models are often used for humanoid target detection and distance judgment. The physical adversarial texture generated by this method can achieve a camouflage effect, preventing the system from misestimating the target's distance or shape. This approach can be used to study the adversarial robustness and recognition error margins of visual security systems.

[0151] 3. Generate general adversarial examples that can be extended to other perceptual models

[0152] In addition to depth estimation, the generation-optimization mechanism of the present invention can also be extended to other visual tasks such as attack target detection and human posture estimation, providing a structurally unified means of generating physical attack samples for multi-class perception models.

[0153] In order to better execute the program of the above method, the embodiment of the present application also provides a computer device, see Figure 5 , including a processor 51, a memory 52 and a communication bus 53 for realizing communication connection between the processor 51 and the memory 52.

[0154] Computer devices may be implemented in various form factors, including devices such as mobile phones, tablet computers, PDAs, laptop computers, and desktop computers.

[0155] The memory can be used to store instructions, programs, codes, code sets, or instruction sets. The memory can include a program storage area and a data storage area. The program storage area can store instructions for implementing an operating system, instructions for at least one function, and instructions for implementing the adversarial texture generation method provided in the above embodiment, etc. The data storage area can store data involved in the adversarial texture generation method provided in the above embodiment, etc.

[0156] The processor may include one or more processing cores. The processor calls the data stored in the memory by running or executing the instructions, programs, code sets or instruction sets stored in the memory, performs the various functions of the present application and processes data. The processor can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller and a microprocessor. It is understandable that for different devices, the electronic device for realizing the above-mentioned processor function can also be other, and the embodiments of the present application are not specifically limited.

[0157] The present invention provides a computer-readable storage medium, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, among other media capable of storing program code. The computer-readable storage medium stores a computer program capable of being loaded by a processor and executing the adversarial texture generation method of the aforementioned embodiment.

[0158] An embodiment of the present application also provides a computer program product, which includes a computer program tangibly embodied on a readable medium, wherein the computer program includes program code for executing any adversarial texture generation method in the embodiments of the present application. The computer program can be downloaded and installed over the network and / or installed from a removable medium (such as a disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc.).

[0159] The above embodiments are merely a detailed introduction to the technical solutions of the present application. However, the description of the above embodiments is only intended to help understand the method and core concept of the present application and should not be construed as limiting the present application. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application.

Claims

1. A method for generating adversarial textures for a depth estimation model, characterized in that: The adversarial texture generation method comprises: Generate an initial adversarial texture pattern t based on a latent vector-driven pattern generator p ; The initial adversarial texture pattern t p Perform upsampling processing to convert the initial adversarial texture pattern t p The resolution is expanded to the target resolution to obtain the upsampled texture pattern Get the object model M, and upsample the texture pattern Mapping to the designated area of ​​the object object model M to replace the original texture of the object object model M; and using the rendering engine to render the mapped object object model M to generate an object object image O adv ; Acquire a natural background image for use in the object image O adv Perform background synthesis to generate a complete scene attack image x adv ; The complete scene attack image x adv Input the depth estimation model to obtain the predicted depth of the image at the current viewing angle; Based on the predicted depth and reference depth, a first-stage loss function is designed to quantify the attack effect; and the pattern generator is continuously optimized with the goal of maximizing the attack effect until the requirements are met.

2. The adversarial texture generation method according to claim 1, wherein: The up-sampled texture pattern The designated area mapped to the object model M includes: First, the UV coordinates and target area mask of each vertex or facet are extracted according to the topological structure of the object model M; the target area to which the texture needs to be applied is determined based on the area mask, and the three-dimensional model surface where the target area is located is matched with the two-dimensional texture space based on the UV coordinates, and the texture map is cropped or masked to ensure that the texture accurately covers the target area.

3. The adversarial texture generation method according to claim 2, wherein: The rendering engine is used to render the object model M after the mapping is completed to generate an object image O with adversarial characteristics. adv include: In the formula, R represents the rendering function, M is the object model, and M mask The area mask for the texture map.

4. The adversarial texture generation method according to claim 1, wherein: The object image O adv Perform background synthesis to generate a complete scene attack image x adv include: Based on the foreground transparency mask m, the object image O adv It is mixed with the natural background image at the pixel level to form a complete scene attack image x adv : x adv =m·O adv +(1-m)·b Among them, b is the natural background image, and m is the foreground transparency mask, which is used to control the fusion ratio of the object image and the natural background image.

5. The adversarial texture generation method according to any one of claims 1 to 4, characterized in that: The first-stage loss function designed to quantify the attack effect based on the predicted depth and the reference depth includes: Calculate the mean square error L between the predicted depth and the reference depth adv ; The mean square error L adv Maximize the first stage loss function L for the goal stage1 : L stage1 =max(L adv ) The first stage loss function L stage1 Only the depth estimation error term is included, which is used to directly improve the interference ability of the generated complete scene attack image on the pattern generator.

6. The adversarial texture generation method according to claim 5, wherein: After the pattern generator is optimized, the method further includes: Fixed the pattern generator parameters and optimized the latent vector and its perturbation terms; Design a second-stage loss function L to quantify attack effectiveness, color printability, and texture smoothness stage2 : L stage2 =L adv +αL smooth +βL nps Among them, L adv Used to measure the depth estimation deviation and reflect the attack effect; L smooth Indicates the loss of texture smoothness; L nps represents the color printability loss; α and β are used to adjust the texture smoothness loss L smooth and color printability loss L nps For the second stage loss function L stage2 The hyperparameters of the degree of influence; is the pixel vector at position (i, j); c k is the kth color vector from the set of printable colors.

7. An adversarial texture generation system for a depth estimation model, characterized in that: include: A pattern generation module is used to generate an initial adversarial texture pattern t based on a latent vector driven pattern generator. p ; And the initial adversarial texture pattern t p Perform upsampling processing to convert the initial adversarial texture pattern t p The resolution is expanded to the target resolution to obtain the upsampled texture pattern Mapping rendering module, used to obtain the object model M, the upsampled texture pattern Mapping to a designated area of ​​the object model M to replace the original texture of the object model M; The rendering engine is used to render the mapped object model M to generate an object image O with adversarial characteristics. adv ; Background synthesis module, used to obtain a natural background image for the object image O adv Perform background synthesis to generate a complete scene attack image x adv ; A processing optimization module is used to convert the complete scene attack image x adv The depth estimation model is input to obtain the predicted depth of the image at the current viewing angle. Based on the predicted depth and the reference depth, a first-stage loss function is designed to quantify the attack effect. The pattern generator is continuously optimized with the goal of maximizing the attack effect until the requirements are met.

8. The adversarial texture generation system according to claim 7, wherein: The mapping rendering module is used to first extract the UV coordinates and target area mask of each vertex or facet according to the topological structure of the object model M; determine the target area to which the texture is to be applied based on the area mask, correspond the three-dimensional model surface where the target area is located with the two-dimensional texture space based on the UV coordinates, and crop or mask the texture map to ensure that the texture accurately covers the target area; The rendering engine is used to render the mapped object model M to generate an object image O with adversarial characteristics. adv include: In the formula, R represents the rendering function, M is the object model, and M mask The area mask for the texture map.

9. The adversarial texture generation system according to claim 7, wherein: The background synthesis module is used to transform the object image O into adv It is mixed with the natural background image at the pixel level to form a complete scene attack image x adv : x adv =m·O adv +(1-m)·b Among them, b is the natural background image, and m is the foreground transparency mask, which is used to control the fusion ratio of the object image and the natural background image.

10. The adversarial texture generation system according to claim 7, wherein: The processing optimization module is used to calculate the mean square error L between the predicted depth and the reference depth adv ; The mean square error L adv Maximize the first stage loss function L for the goal stage1 : L stage1 =max(L adv ) The first stage loss function L stage1 Only the depth estimation error term is included to directly improve the interference ability of the generated complete scene attack image on the pattern generator; After the pattern generator is optimized, the processing optimization module is further configured to: Fixed the pattern generator parameters and optimized the latent vector and its perturbation terms; Design a second-stage loss function L to quantify attack effectiveness, color printability, and texture smoothness stage2 : L stage2 =L adv +αL smooth +βL nps Among them, L adv Used to measure the depth estimation deviation and reflect the attack effect; L smooth Indicates the loss of texture smoothness; L nps represents the color printability loss; α and β are used to adjust the texture smoothness loss L smooth and color printability loss L nps For the second stage loss function L stage2 The hyperparameters of the degree of influence; is the pixel vector at position (i, j); c k is the kth color vector from the set of printable colors.

Images