Multi-view collaborative adversarial texture generation method and device based on deformable mesh
By adopting a multi-view collaborative adversarial texture generation method based on deformable meshes, the problems of gradient sparsity and multi-view optimization conflict in 3D adversarial texture generation are solved, generating adversarial textures with good visual continuity and improving the robustness of attacks and computational efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUN YAT SEN UNIV
- Filing Date
- 2026-02-10
- Publication Date
- 2026-06-12
AI Technical Summary
Existing 3D adversarial texture generation methods suffer from gradient sparsity leading to update fragmentation and multi-view optimization conflicts, making it difficult to generate adversarial textures with good visual continuity.
A multi-view collaborative adversarial texture generation method based on deformable mesh is adopted. By initializing global meta-parameters, constructing a mapping function, and performing gradient descent updates and state reset operations on local parameters, the target adversarial texture is generated.
The generated textures maintain a high degree of continuity in geometry and color, exhibiting stronger attack transferability and robustness, reducing computational complexity and memory usage, and supporting higher resolution adversarial texture generation.
Smart Images
Figure CN122199772A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision, and more particularly to a method and apparatus for generating multi-view cooperative adversarial textures based on deformable meshes. Background Technology
[0002] In the fields of computer vision and artificial intelligence security, current adversarial texture generation for 3D objects mainly adopts pixel-level optimization methods: directly using each pixel in the texture map as a learnable parameter and updating the pixel value through backpropagation.
[0003] However, the above methods suffer from problems such as gradient sparsity leading to update fragmentation and multi-view optimization conflicts causing convergence difficulties.
[0004] Because 3D objects are only visible from a specific viewpoint in a single rendering, the gradients backpropagated to the texture map can only cover a very small portion of the texture surface, a phenomenon known as "gradient sparsity." Directly optimizing pixels or fixed points leads to discrete and fragmented texture updates, resulting in textures prone to noise, holes, and tearing, lacking the visual continuity of the physical world.
[0005] A robust adversarial texture needs to fool all possible viewpoints simultaneously. However, the gradient update directions generated by different viewpoints often conflict (e.g., a front view requires a certain area to be brightened, while a side view requires it to be darkened). Traditional joint training or simple alternating optimization strategies can easily lead to the model parameters oscillating wildly between local optima in different viewpoints, making it difficult to converge to a globally acceptable solution. This results in a sharp decline in the attack effectiveness of the generated texture under unseen viewpoints. Summary of the Invention
[0006] The following is an overview of the topics described in detail in this article.
[0007] The purpose of this application is to at least partially solve one of the technical problems existing in the related technologies. The embodiments of this application provide a method and apparatus for generating multi-view cooperative adversarial textures based on deformable meshes to solve the texture continuity problem.
[0008] An embodiment of the first aspect of this application provides a multi-view cooperative adversarial texture generation method based on deformable meshes, comprising:
[0009] Obtain the initial 3D object surface texture;
[0010] Initialize global meta-parameters, which include a content tensor and a flow field tensor. The content tensor represents the color primitives of the texture, and the flow field tensor represents the geometric deformation field of the texture.
[0011] Based on the global meta-parameters, flow field upsampling, mesh deformation processing, and differentiable sampling processing are performed to construct a mapping function based on a deformable mesh. The mapping function is used to map the initial 3D object surface texture into an adversarial texture.
[0012] The mapping function is updated by gradient descent of local parameters, the global parameters are moved according to the direction of the cumulative vector from the starting local parameters to the ending local parameters, and the state is reset to adjust the parameters of the mapping function to the optimal parameters, thereby obtaining the target mapping function.
[0013] The initial 3D object surface texture is mapped to a target adversarial texture according to the target mapping function.
[0014] According to certain embodiments of the first aspect of this application, the expected loss of the mapping function is: ;in, It is the total loss. Distribution from perspective From the perspective of mid-sampling, It is the detector's adversarial loss. It is the rendering function. It is a mapping function. These are the parameters of the mapping function. It is a hyperparameter that balances adversarial loss and regularization loss. It is a regular loss.
[0015] According to certain embodiments of the first aspect of this application, the flow field upsampling process includes: amplifying the flow field tensor to a target resolution using bilinear interpolation to obtain a pixel-level offset field.
[0016] According to certain embodiments of the first aspect of this application, the mesh deformation processing includes:
[0017] Construct a standard normalized base mesh, the resolution of which is the target resolution, and each node of the standard normalized base mesh corresponds to the pixel coordinates on the target texture.
[0018] The offset field is superimposed onto the standard normalized base grid to obtain the deformed sampling grid;
[0019] Wherein, the integer pixel coordinates on the target texture The corresponding continuous floating-point coordinates in the sampling grid Represented as: In the formula, Indicates the sampling grid. Coordinates Normalized base coordinates Coordinates The geometric offset.
[0020] According to certain embodiments of the first aspect of this application, the differentiable sampling process includes: calculating the pixel values of the target adversarial texture using a differentiable bilinear interpolation method;
[0021] The pixel values of the target adversarial texture are represented as follows: In the formula, For a moment, Indicates the target adversarial texture in coordinates pixel values, Represents a content tensor. Representing coordinates The set of discrete neighborhoods in the coordinate system of the content tensor. This represents the bilinear interpolation kernel function.
[0022] According to certain embodiments of the first aspect of this application, the gradient descent update operation of the local parameters includes:
[0023] Temporary adversarial textures are generated using the mapping function;
[0024] From the data stream sampling perspective, the temporary adversarial texture is mapped onto the surface of a three-dimensional object and rendered to obtain a rendered image based on the sampling perspective;
[0025] The gradient of the loss function with respect to local parameters is calculated based on the rendered image, and the gradient is backpropagated from the pixel to the content tensor and the flow field tensor.
[0026] The local parameters are updated using a local optimizer to obtain the endpoint local parameters.
[0027] According to certain embodiments of the first aspect of this application, the cumulative vector from the starting local parameter to the ending local parameter is represented as: In the formula, For cumulative vectors, Local parameters at the starting point These are local parameters for the endpoint.
[0028] According to certain embodiments of the first aspect of this application, the state reset operation includes: overwriting the value of the local parameter with the value of the moved global parameter.
[0029] According to a second aspect of this application, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-view cooperative adversarial texture generation method based on deformable mesh as described in the first aspect of this application.
[0030] According to a third aspect of this application, a computer storage medium stores computer-executable instructions for performing a multi-view cooperative adversarial texture generation method based on deformable meshes as described in an embodiment of the first aspect of this application.
[0031] The above scheme has at least the following beneficial effects: Utilizing the bilinear upsampling mechanism of the deformable mesh generation module, low-resolution parameter updates can smoothly radiate to high-resolution texture regions. It fundamentally overcomes the sparsity of single-view gradients, eliminating noise and texture tearing, and the generated textures maintain high continuity in both geometry and color. Through a two-layer optimization strategy, it effectively extracts the consensus direction between gradients from different views, mitigating the randomness and conflicts of single-view optimization, enabling the generated adversarial textures to exhibit stronger attack transferability and robustness under unseen test views. The mesh interpolation-based generation method reduces computational complexity, significantly decreasing memory usage and training time, and supports the generation of higher-resolution adversarial textures. Attached Figure Description
[0032] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.
[0033] Figure 1 This is a step diagram of a multi-view cooperative adversarial texture generation method based on deformable meshes;
[0034] Figure 2 This is a step diagram of the gradient descent update operation for local parameters;
[0035] Figure 3 This is a flowchart of a two-level loop optimization strategy. Detailed Implementation
[0036] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0037] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0038] In the fields of computer vision and artificial intelligence security, current adversarial texture generation for 3D objects mainly adopts pixel-level optimization methods: directly using each pixel in the texture map as a learnable parameter and updating the pixel value through backpropagation.
[0039] However, the above methods suffer from problems such as gradient sparsity leading to fragmented updates and multi-view optimization conflicts causing convergence difficulties.
[0040] Because 3D objects are only visible from a specific viewpoint in a single rendering, the gradients backpropagated to the texture map can only cover a very small portion of the texture surface, a phenomenon known as "gradient sparsity." Directly optimizing pixels or fixed points leads to discrete and fragmented texture updates, resulting in textures prone to noise, holes, and tearing, lacking the visual continuity of the physical world.
[0041] A robust adversarial texture needs to fool all possible viewpoints simultaneously. However, the gradient update directions generated by different viewpoints often conflict (e.g., a front view requires a certain area to be brightened, while a side view requires it to be darkened). Traditional joint training or simple alternating optimization strategies can easily lead to the model parameters oscillating wildly between local optima in different viewpoints, making it difficult to converge to a globally acceptable solution. This results in a sharp decline in the attack effectiveness of the generated texture under unseen viewpoints.
[0042] To address the above issues, embodiments of this application provide a method and apparatus for generating multi-view cooperative adversarial textures based on deformable meshes.
[0043] The embodiments of this application will be further described below with reference to the accompanying drawings.
[0044] Reference Figure 1 A multi-view cooperative adversarial texture generation method based on deformable meshes includes the following steps:
[0045] Step S100: Obtain the initial 3D object surface texture;
[0046] Step S200: Initialize global meta-parameters; global meta-parameters include content tensor and flow field tensor. The content tensor represents the color primitives of the texture, and the flow field tensor represents the geometric deformation field of the texture.
[0047] Step S300: Perform flow field upsampling, mesh deformation processing, and differentiable sampling processing based on global meta-parameters to construct a mapping function based on a deformable mesh; the mapping function is used to map the initial 3D object surface texture to the target adversarial texture;
[0048] Step S400: Perform gradient descent update operation on the local parameters of the mapping function, move the global parameters according to the direction of the cumulative vector from the starting local parameters to the ending local parameters, and perform state reset operation to adjust the parameters of the mapping function to the optimal parameters, thereby obtaining the target mapping function;
[0049] Step S500: Map the initial 3D object surface texture to the target adversarial texture according to the target mapping function.
[0050] The core logic of this method lies in constructing a differentiable texture generation mapping function. The low-dimensional parameter space is mapped to a high-dimensional texture space; the low-dimensional parameters are iteratively updated through a two-layer loop optimization strategy, and finally a robust adversarial texture is obtained through the mapping function. .
[0051] The goal is to find an adversarial texture. This allows it to mislead the target detector after being attached to the surface of a 3D object. To overcome the curse of dimensionality and gradient sparsity of pixel-level optimization, instead of directly optimizing texture pixels, a set of generation parameters is optimized. .
[0052] Let the initial 3D object surface texture be... Define the texture generation mapping function as follows: The optimization objective is to minimize the distribution of views. The expected loss is as follows: ;in, It is the total loss. Distribution from perspective From the perspective of mid-sampling, It is the detector's adversarial loss. It is the rendering function. It is a mapping function. These are the parameters of the mapping function. It is a hyperparameter that balances adversarial loss and regularization loss. It is a regular loss.
[0053] To construct the mapping function, a set of low-resolution global meta-parameters is first initialized. .
[0054] Global meta-parameters include content tensors and flow field tensor The content tensor represents the color primitives of the texture, and the flow field tensor represents the geometric deformation field of the texture. Among these, and Used to control grid resolution, for example and It can be set to 64, which is much smaller than the target texture resolution. and ,For example and It can be set to 800.
[0055] In the initial state, Can be randomly initialized. It is initialized as a zero tensor. This low-dimensional design ensures the compactness of the parameter space, which is the foundation for subsequent smooth texture generation.
[0056] At any time during training Given the current parameter ,in For a moment The low-resolution content tensor, with a size of ; For a moment The low-resolution flow field tensor, with a size of .
[0057] Based on global meta-parameters, flow field upsampling, mesh deformation processing, and differentiable sampling are performed to construct a mapping function based on a deformable mesh. The goal is to generate a target resolution (size...). adversarial textures ,Right now .
[0058] Flow field upsampling processing specifically involves: using bilinear interpolation operations. low-resolution flow field tensor Magnify to the target resolution to obtain the pixel-level offset field. .
[0059] Mesh deformation processing specifically involves: constructing a standard normalized base mesh. The resolution of the standard normalized base mesh is the target resolution, and each node of the standard normalized base mesh corresponds to the pixel coordinates on the target texture. ; to offset the field Superimposed onto the standard normalized base grid Obtain the sampled mesh after deformation ; where the integer pixel coordinates on the target texture The corresponding continuous floating-point coordinates in the sampling grid Represented as: In the formula, Indicates the sampling grid. Coordinates Normalized base coordinates Coordinates The geometric offset.
[0060] Differentiable sampling processing specifically involves using a differentiable bilinear interpolation method to calculate the pixel values of the target adversarial texture.
[0061] Due to the calculation Sampling coordinates in Typically, these are floating-point numbers (not integers) and cannot be directly used as indices to read the content tensor. The discrete pixel values are used. Therefore, a differentiable bilinear interpolation method is employed to calculate the final pixel values.
[0062] Define sampling coordinates In content tensor Discrete neighborhood set in coordinate system For the four integer grid points closest to this coordinate, we have: ;in, This indicates rounding down. This indicates rounding up to the nearest integer.
[0063] Target adversarial texture in position The pixel value at a given location is the weighted sum of the values of its four neighboring pixels, and we have: In the formula, For a moment, Indicates the target adversarial texture in coordinates pixel values, This represents a content tensor, specifically a content tensor in integer coordinates. The RGB color value at that location; Representing coordinates The set of discrete neighborhoods in the coordinate system of the content tensor; The bilinear interpolation kernel function is represented as: This function calculates the weight based on the distance between the sampling point and its neighboring grid points. The closer the distance, the greater the weight; if the distance exceeds one unit, the weight is 0.
[0064] Through step S300, parameters Tiny changes are smoothly magnified into texture. The global changes. When backpropagation goes through this step, the gradient of a single pixel will diffuse backward along the interpolation formula to... and Multiple neighboring nodes effectively solve the sparsity problem of single-view gradients.
[0065] Faced with infinite and conflicting multi-view optimization objectives, a single gradient descent attempt struggles to find a generalizable optimal solution. Traditional gradient aggregation methods simply average gradients from different views, which can easily lead to cancellation when gradient directions differ significantly, resulting in low convergence efficiency. A two-layer optimization strategy guides the parameter space towards a region where gradient directions from all views are as consistent as possible. Related analysis shows that by conducting K consecutive optimization steps within a local area and then adjusting global parameters based on the results, the inner product (or cosine similarity) of gradient directions in different optimization steps can be implicitly maximized. This means the model is encouraged to learn a "universal" initialization point from which parameters can be updated in similar and effective directions regardless of the viewpoint. This characteristic is crucial for generating adversarial textures that maintain robustness across multiple views.
[0066] Reference Figure 3 The two-level loop optimization strategy includes an inner loop optimization step and an outer loop optimization step.
[0067] The inner loop optimization step focuses on exploring the continuous optimization trajectory of local parameters. In the first... At the start of a major iteration, the parameters are copied first, including the global meta-parameters. Make a copy as a local parameter Then, K consecutive gradient descent updates are performed.
[0068] The inner loop optimization step implements the gradient descent update operation of local parameters.
[0069] Reference Figure 2 The gradient descent update operation for local parameters includes the following steps:
[0070] Step S411: Generate a temporary adversarial texture using a mapping function;
[0071] Step S412: From the data stream sampling perspective, the temporary adversarial texture is mapped onto the surface of the 3D object and rendered to obtain the rendered image.
[0072] Step S413: Calculate the gradient of the loss function with respect to local parameters based on the rendered image, and backpropagate the gradient from the pixel to the content tensor and the flow field tensor.
[0073] Step S414: Update the local parameters using the local optimizer to obtain the endpoint local parameters.
[0074] Specifically, the current temporary adversarial texture is generated through a mapping function. Read random viewpoint data and sample a batch of viewpoints from the data stream. Based on the sampling perspective, flow field upsampling and texture interpolation are performed using a deformable mesh. The temporary adversarial texture is then mapped onto the surface of the 3D object and rendered to generate a rendered image. The gradient of the loss function with respect to local parameters is calculated based on the rendered image. Thanks to the differentiability of the generation process, gradients are backpropagated from pixels to the content tensor and the flow field tensor. Gradient descent updates local parameters using a local optimizer to obtain the final local parameters.
[0075] During the process of updating local parameters, an exploration trajectory is formed: ;in, The learning rate is the inner loop rate. After K steps, the endpoint parameters of the local exploration are obtained. .
[0076] After the inner loop exploration ends, the local parameters This represents a local optimum for the current view sequence, but its optimization direction may be affected by specific disturbances of the current view sequence. In this case, the system performs aggregation and correction operations in the outer loop to transform the local exploration results into robust updates to the global meta-parameters.
[0077] The outer loop optimization steps implement the movement of global parameters and state reset operations based on the direction of the cumulative vector from the starting local parameters to the ending local parameters.
[0078] The cumulative vector from the starting local parameters to the ending local parameters is calculated as follows: In the formula, For cumulative vectors, Local parameters at the starting point The endpoint local parameters. This vector represents the average gradient manifold direction aggregated by K steps of local exploration.
[0079] Using meta-learning rate Moving the global meta-parameter one step along the aggregation direction to update the global parameter, we have: This formula is mathematically equivalent to performing vector smoothing on the gradient directions of multiple perspectives, extracting the common optimization direction of each perspective.
[0080] After the global parameters are updated, a critical state reset operation is performed: the local parameters are reset. The value is forcibly overwritten with the updated global meta-parameter. Simultaneously, the internal state of the inner loop optimizer (including the first and second angular momentum) is completely cleared. This reset operation cuts off the optimization inertia of the previous view sequence, forcing the model to re-accumulate momentum and explore new local trajectories in the next inner loop with the latest global meta-parameters as the origin, ensuring that the model can converge to the global optimum that takes into account the effects of multi-view attacks.
[0081] Repeat the inner and outer loops as described above until convergence is achieved, thus obtaining the final globally optimal parameters. The target mapping function is obtained based on the globally optimal parameters.
[0082] By calling the target mapping function, the final target adversarial texture can be obtained: .
[0083] In this embodiment, the multi-view collaborative adversarial texture generation method has the technical effects of significantly enhancing texture visual continuity, improving robustness against multi-view attacks, and optimizing computational efficiency and video memory.
[0084] By utilizing the bilinear upsampling mechanism of the deformable mesh generation module, low-resolution parameter updates can be smoothly radiated to high-resolution texture regions. This fundamentally overcomes the sparsity of single-view gradients, eliminates noise and texture tearing common in traditional methods, and generates textures that maintain high continuity in both geometry and color.
[0085] By introducing a two-layer optimization strategy of "inner loop exploration - outer loop aggregation," the consensus direction between gradients from different perspectives can be effectively extracted, mitigating the randomness and conflicts inherent in single-view optimization. The generated adversarial textures are not only effective under the training perspective but also exhibit stronger attack transferability and robustness under unseen testing perspectives.
[0086] The generation method based on grid interpolation reduces computational complexity, greatly reduces memory usage and training time, and supports the generation of higher resolution adversarial textures.
[0087] This application also provides an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-described multi-view cooperative adversarial texture generation method based on deformable meshes.
[0088] This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.
[0089] The processor can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory can be implemented using read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory, and the processor calls and executes the data storage method or data reading method of the embodiments of this application.
[0090] Input / output interfaces are used to implement information input and output; communication interfaces are used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); the bus transmits information between various components of the device (such as processor, memory, input / output interfaces and communication interfaces); among them, the processor, memory, input / output interfaces and communication interfaces are connected to each other within the device through the bus.
[0091] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described multi-view cooperative adversarial texture generation method based on deformable meshes.
[0092] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0093] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0094] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.
[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0096] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.
[0097] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0098] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0099] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0100] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0102] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0103] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.
Claims
1. A multi-view cooperative adversarial texture generation method based on deformable meshes, characterized in that, include: Obtain the initial 3D object surface texture; Initialize global meta-parameters, which include a content tensor and a flow field tensor. The content tensor represents the color primitives of the texture, and the flow field tensor represents the geometric deformation field of the texture. Based on the global meta-parameters, flow field upsampling, mesh deformation processing, and differentiable sampling processing are performed to construct a mapping function based on a deformable mesh. The mapping function is used to map the initial 3D object surface texture into an adversarial texture. The mapping function is updated by gradient descent of local parameters, the global parameters are moved according to the direction of the cumulative vector from the starting local parameters to the ending local parameters, and the state is reset to adjust the parameters of the mapping function to the optimal parameters, thereby obtaining the target mapping function. The initial 3D object surface texture is mapped to a target adversarial texture according to the target mapping function.
2. The multi-view cooperative adversarial texture generation method based on deformable mesh according to claim 1, characterized in that, The expected loss of the mapping function is ;in, It is the total loss. Distribution from perspective From the perspective of mid-sampling, It is the detector's adversarial loss. It is the rendering function. It is a mapping function. These are the parameters of the mapping function. It is a hyperparameter that balances adversarial loss and regularization loss. It is a regular loss.
3. The multi-view cooperative adversarial texture generation method based on deformable mesh according to claim 1, characterized in that, The flow field upsampling process includes: using bilinear interpolation to amplify the flow field tensor to the target resolution to obtain a pixel-level offset field.
4. The multi-view cooperative adversarial texture generation method based on deformable mesh according to claim 3, characterized in that, The mesh deformation processing includes: Construct a standard normalized base mesh, the resolution of which is the target resolution, and each node of the standard normalized base mesh corresponds to the pixel coordinates on the target texture. The offset field is superimposed onto the standard normalized base grid to obtain the deformed sampling grid; Wherein, the integer pixel coordinates on the target texture The corresponding continuous floating-point coordinates in the sampling grid Represented as: In the formula, Indicates the sampling grid. Coordinates Normalized base coordinates Coordinates The geometric offset.
5. The multi-view cooperative adversarial texture generation method based on deformable mesh according to claim 4, characterized in that, The differentiable sampling process includes: calculating the pixel values of the target adversarial texture using a differentiable bilinear interpolation method; The pixel values of the target adversarial texture are represented as follows: In the formula, For a moment, Indicates the target adversarial texture in coordinates pixel values, Represents a content tensor. Representing coordinates The set of discrete neighborhoods in the coordinate system of the content tensor. This represents the bilinear interpolation kernel function.
6. The multi-view cooperative adversarial texture generation method based on deformable mesh according to claim 1, characterized in that, The gradient descent update operation for the local parameters includes: Temporary adversarial textures are generated using the mapping function; From the data stream sampling perspective, the temporary adversarial texture is mapped onto the surface of a three-dimensional object and rendered to obtain a rendered image based on the sampling perspective; The gradient of the loss function with respect to local parameters is calculated based on the rendered image, and the gradient is backpropagated from the pixel to the content tensor and the flow field tensor. The local parameters are updated using a local optimizer to obtain the endpoint local parameters.
7. The multi-view cooperative adversarial texture generation method based on deformable mesh according to claim 6, characterized in that, The cumulative vector from the starting local parameters to the ending local parameters is represented as: In the formula, For cumulative vectors, Local parameters at the starting point These are local parameters for the endpoint.
8. The multi-view cooperative adversarial texture generation method based on deformable mesh according to claim 7, characterized in that, The state reset operation includes: overwriting the values of the local parameters with the values of the moved global parameters.
9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the multi-view cooperative adversarial texture generation method based on deformable mesh as described in any one of claims 1 to 8.
10. A computer storage medium, characterized in that, The device stores computer-executable instructions for performing the multi-view cooperative adversarial texture generation method based on deformable meshes as described in any one of claims 1 to 8.