Generating partitioned images using differentiable mesh generation pipelines

Abstract

The present disclosure relates to systems, methods, and non-transitory computer readable media that provide a differentiatable partitioning system that uses fully automated generative methods of machine learning and text guidance to generate aesthetically reasonable, periodic, and partitioned non-square images. That is, given a textual description of an object and a symmetric pattern of 2D planes, the system produces a textured 2D mesh that is visually similar to the textual description, follows geometric rules that ensure that it can be used to block the plane, and contains only foreground objects. In effect, the disclosed system generates a rationalized textured 2D triangular mesh that visually matches the text input and optimizes the texture and shape of the mesh and satisfies overlap conditions and partitioning conditions. Using the described methods, a differential partitioning system may generate a grid such that edges and vertices are aligned between repeatable instances of the grid.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to generating separable patchable images using a differentiable mesh generation pipeline. Background Art

[0002] Advances in computing devices and computer design applications have led to a variety of innovations in computer image creation and editing software. For example, image design systems have been developed that provide tools for creating and integrating variations of design elements within digital images. Some computer design applications can manipulate digital images as part of a design process to create digital images composed of repeating design elements. However, due in part to the inherent complexity of such image manipulation, it can be difficult for computer systems to manipulate the geometry and appearance of individual non-square objects in a way that combines individual non-square objects into blocks in an interdependent and visually consistent manner. Therefore, when creating, editing, and manipulating divisible digital images, the prior art exhibits many shortcomings in terms of flexibility, accuracy, and computational efficiency. Summary of the invention

[0003] One or more embodiments provide benefits and / or solve one or more of the foregoing or other problems in the art using systems, methods, and non-transient computer-readable storage media, which provide systems for creating and modifying digital depictions of a subject so as to provide a periodic, repeatable, and reasonable design that can be divided into blocks in a two-dimensional space. Specifically, in one or more implementations, the disclosed system optimizes both the geometry and color of a 2D mesh so as to generate a non-square block with the shape and appearance of a desired subject (e.g., without additional background details). For example, the disclosed system performs geometric optimization of the block by generating an unconstrained, differentiable parameterization of the space of all possible separable block shapes within a symmetry group. In one or more embodiments, the disclosed system utilizes dynamic edge weights and texture images of a mesh generation model as optimizable parameters to generate a separable texture mesh. Specifically, the disclosed system can utilize the dynamic edge weights of the mesh generation model to generate a mesh that satisfies an overlap condition (i.e., non-overlapping edges and vertices) and a separable block boundary condition (i.e., the boundaries of the edges can fit together in a repeatable pattern). In addition, the disclosed system utilizes a differentiable renderer to generate a rendered image from a mesh and a texture image. In some cases, the disclosed system utilizes a trained image diffusion model and determines a loss between a rendered image and a representation of a desired subject to update dynamic edge weights for the mesh and texture image. By iteratively updating the dynamic edge weights and texture image based on a measure of this loss, in one or more implementations, the disclosed system produces a plausible and visually appealing separable block digital design that reflects the desired subject. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The present disclosure will describe one or more example implementations of systems and methods with additional specificity and detail by referring to the accompanying drawings. The following paragraphs briefly describe these drawings, in which:

[0005] Figure 1 A schematic diagram illustrating an example environment of a differentiable partitioning system according to one or more embodiments is illustrated;

[0006] Figure 2 illustrates an example overview of generating a separable patchable image using a differentiable patching system according to one or more embodiments;

[0007] Figure 3 An example of generating a separable image using a grid, a texture image, and a loss function according to one or more embodiments is illustrated;

[0008] Figure 4 Illustrated is mapping of a separable block image using separable block boundary conditions according to one or more embodiments;

[0009] Figure 5 illustrates utilizing edge weights and vertices to generate a mesh with non-overlapping boundary edges according to one or more embodiments;

[0010] Figure 6 Illustrated is the generation of a separable tileable image using an input according to one or more embodiments;

[0011] Figure 7 illustrates tiles created from translated and rotated copies of a single textured 2D mesh according to one or more embodiments;

[0012] Figure 8 illustrates tiles created by translated and rotated copies of a single textured 2D mesh without incorporating excessive background colors in accordance with one or more embodiments;

[0013] Fig. 9 illustrates creating a tile using two tile shapes for two different objects according to one or more embodiments;

[0014] Fig.10 illustrates a schematic diagram of a differentiable blocking system according to one or more embodiments;

[0015] Figure 11-Figure 12 A flow chart illustrating a series of actions for generating a separable block grid according to one or more embodiments; and

[0016] Fig.13 A block diagram of an example computing device is illustrated in accordance with one or more embodiments. DETAILED DESCRIPTION

[0017] The present disclosure describes one or more embodiments of a differentiable blocking system that utilizes a text-guided, fully automatic generative approach to generate aesthetically plausible, periodic, and blockable non-square images using machine learning. For example, given a desired text description of an object and a desired symmetrical pattern of a 2D plane, the system generates a textured 2D mesh that is visually similar to the text description, follows geometric rules that ensure it can be used to block the plane, and contains only foreground objects (to produce a blocking effect similar to Escher's work). In contrast, conventional blocking systems have many technical deficiencies in flexibility, accuracy, and computational efficiency when creating blockable graphical objects.

[0018] In particular, many conventional blocking systems lack flexibility / functionality. For example, many conventional systems simply use a fixed set of blocks that are aligned in different ways. To illustrate, many conventional systems utilize trivial separable block images, such as squares (or rectangles) or fixed separable block shapes whose boundaries cannot be modified to fit a desired object (e.g., wallpaper). Specifically, the goal of conventional systems is to convert existing shapes into blocks, rather than to obtain a combination of geometry and texture optimization (e.g., via a perceptual loss or another loss metric).

[0019] Conventional systems are also not flexible enough when attempting to modify image geometry to create tileable images. For example, some conventional systems define tile polygons consisting of a fixed, small set of tile vertices, and divide the edges with shape vertices. However, these systems only define tile polygons and cannot directly ensure that shape vertices (which are necessary to achieve more detailed shapes) form boundaries without self-overlapping. It is worth noting that the goal of these methods is to convert existing shapes into tiles. In fact, none of these conventional systems provide the flexibility to optimize a tile for a specific perceptual appearance (optimizing both the texture and the actual shape of the tile) while ensuring that it tiles the plane.

[0020] Traditional blocking systems also have many technical disadvantages regarding computational efficiency when generating separable block mesh embeddings. For example, some traditional energy-based methods for ensuring overlap-free mesh embeddings provide barrier terms that prevent triangle inversions and boundary self-overlap. However, the barrier terms exhibit extremely large magnitudes and produce large gradients, which interfere with the calculation of the loss metric. In addition, these traditional energy-based methods require performing line searches as well as boundary collision checks, which significantly increases the running time. Other systems attempt to generate separable block mesh embeddings through convexification methods, which restrict the space of possible embeddings, which requires computationally slower optimization techniques.

[0021] Furthermore, conventional systems are also inaccurate. In fact, conventional systems often generate digital images that have gaps or large background areas between blockable shapes. For example, conventional systems optimize the color of a square image, which results in an image that contains an object in the foreground, but also has a background, thereby making the blocking effect of the entire square image negligible. As a result, conventional systems often inaccurately generate images that fail to align the boundaries that define a specific foreground subject in a permanently repeatable manner.

[0022] In contrast, in some embodiments, a differentiable blocking system accepts text input (or other input indicative of a theme) and generates a periodic, repeatable, and plausible textured 2D triangular mesh that visually matches the text input and optimizes the texture as well as the shape of the mesh. As mentioned, in one or more embodiments, the differentiable blocking system ensures that mesh edges and vertices satisfy overlap conditions and separable blocking conditions. Specifically, in one or more implementations, the disclosed system optimizes both the geometry and texture of the 2D mesh to generate repeatable blocks having the shape and appearance of a desired theme (e.g., of different geometries), and close to no additional background details. For example, the disclosed system performs geometric optimization of blocks by generating an unconstrained, differentiable parameterization of the space of all possible separable blocking shapes within a symmetry group.

[0023] For example, the differentiable blocking system utilizes dynamic edge weights of the mesh generation model to ensure non-overlapping boundary edges and non-overlapping internal edges. Specifically, the differentiable blocking system satisfies the separable blocking boundary condition so that the mesh can be repeated infinitely, where the boundaries are defined by edges and vertices that align between all instances of the mesh. In one or more embodiments, the differentiable blocking system utilizes dynamic edge weights to generate a mesh that satisfies the overlapping condition and the separable blocking boundary condition.

[0024] In addition, in one or more embodiments, the differentiable blocking system utilizes a dynamic texture image and a differentiable renderer to create a rendered image from the generated mesh. For example, the differentiable blocking system generates a rendered image and compares the rendered image to a representation of a desired subject of the separable image. For example, the differentiable blocking system utilizes a diffusion neural network to generate a digital image from a text input describing a desired subject of the separable image. The differentiable blocking system determines a measure of loss between the rendered image and the visual representation of the desired subject. In one or more implementations, the differentiable blocking system uses the measure of loss to update the dynamic edge weights of the mesh generation model and / or the texture image to iteratively generate a feasible separable image that accurately reflects the subject of the input text.

[0025] Thus, in one or more embodiments, the disclosed system utilizes a mesh and a texture image as optimizable parameters to generate a separable textured mesh using a differentiable renderer. In some cases, the differentiable blocking system randomizes the background color of a dynamic texture image generated by the differentiable renderer to accurately determine a measure of the loss between the rendered image and the vector representation of the text input. In some embodiments, the differentiable blocking system utilizes a learning rate to update the dynamic edge weights and utilizes a second learned weight to update the dynamic texture image. Specifically, the differentiable blocking system utilizes gradient optimization to modify the dynamic edge weights of a differentiable mesh generation model, thereby updating the dynamic edge weights in a progressive manner. Using the described method, the differentiable blocking system generates a mesh such that edges and vertices are aligned between repeatable instances of the mesh.

[0026] As described above, embodiments of differentiable blocking systems can provide multiple advantages over conventional blocking systems. For example, differentiable blocking systems can provide increased operational flexibility. Compared to conventional systems, given a textual prompt (or other input indicating a subject), a differentiable blocking system can provide a textured grid that visually matches the text, consists (only) of the desired objects with little to no background, and is blockable. In fact, compared to conventional systems that use a set of fixed blocks and constrained shapes, a differentiable blocking system can generate non-square geometric shapes that can be repeated indefinitely, mapping a plane without any gaps or overlaps and containing the desired foreground objects.

[0027] In addition, differentiable block systems can provide improved computational efficiency by forming a highly non-convex restricted space of possible grid configurations into an unconstrained differentiable representation that satisfies the conditions of non-overlapping boundaries and is not subject to self-overlapping constraints. This approach enables differentiable block systems to apply unconstrained gradient-based optimization while also ensuring that the grid is a valid block. Unlike traditional methods that are used to limit the possible embedding space that produces large gradients, differentiable block systems can generate non-overlapping grids while imposing periodic boundary constraints by utilizing a computationally efficient gradient-based perceptual loss optimization. In fact, differentiable block systems can utilize computationally efficient generative methods for blocks formed by quotients of symmetric groups (e.g., generated by repeatedly applying a finite set of transformations to a block).

[0028] In addition, in one or more implementations, the differentiable patching system improves accuracy by generating separable patch images that reflect the content of a specific input modality. For example, the differentiable patching system generates separable patch images with complex, non-trivial boundaries that align to form repeating patterns. Thus, the differentiable patching system avoids gaps or background regions where the boundaries of the separable patch images cannot be accurately aligned with foreground objects / subjects. Furthermore, by utilizing a metric of loss and a differentiable pipeline, the differentiable patching system generates separable patch images that accurately reflect specific input cues that describe the desired subject matter of the separable patch images.

[0029] Additional details about the differentiable partitioning system will now be provided with reference to the accompanying drawings. For example, Figure 1 A schematic diagram of an exemplary system environment ("environment") 100 in which a differentiable partitioning system 106 operates is illustrated. Figure 1 As illustrated in , environment 100 includes server device(s) 102 , network 108 , and client device(s) 110 .

[0030] although Figure 1 Although the environment 100 is depicted as having a particular number of components, the environment 100 can have any number of additional or alternative components (e.g., any number of servers, client devices, or other components that communicate with the differentiable partitioning system 106 via the network 108). Figure 1 A particular arrangement of server device(s) 102, network 108, and client device(s) 110 is illustrated, but various additional arrangements are possible.

[0031] Server device(s) 102, network 108, and client device(s) 110 are directly or indirectly (e.g., via a combination of Fig.13 In addition, the server device(s) 102 and the client device(s) 110 may include one of a variety of computing devices (including, for example, a combination of Fig.13 One or more computing devices discussed in more detail).

[0032] like Figure 1As illustrated in , environment 100 includes (multiple) server devices 102. (Multiple) server devices 102 generate, track, store, process, receive, and transmit electronic data, including segmentable images. For example, (multiple) server devices 102 receive or monitor interactions between (multiple) client devices 110. In some implementations, (multiple) server devices 102 transmit content to (multiple) client devices 110 to cause (multiple) client devices 110 to display content associated with the segmentable images. For example, (multiple) server devices 102 can present digital grids and / or segmentable images to (multiple) client devices 110, and display the digital grids and / or segmentable images on (multiple) client devices 110, wherein the displayed digital grids and / or segmentable images correspond to system needs (e.g., providing segmentable images for display via (multiple) client applications 112).

[0033] Additionally, the server device(s) 102 include all or a portion of the differentiable tile system 106. For example, the differentiable tile system 106 operates on the server device(s) 102 to access digital content (including meshes and / or separable tile images), determine changes in the digital content, and provide the location of the content changes to the client device(s) 110. In one or more embodiments, the differentiable tile system 106 generates and displays the digital mesh and / or separable tile image based on the modification of the edge weights for the digital mesh via the server device(s) 102. Fig.10 hereinafter describes example components of the differentiable blocking system 106 .

[0034] In addition, if Figure 1 As shown in , the illustrated system includes (multiple) client devices 110. In some embodiments, (multiple) client devices 110 include, but are not limited to, a mobile device (e.g., a smart phone, a tablet), a laptop computer, a desktop computer, or another type of computing device, including the following reference Fig.13 Those explained. Some embodiments of the (multiple) client devices 110 are operated by users to perform various functions via corresponding (multiple) client applications 112, in the generation and modification of segmentable images. The (multiple) client devices 110 include one or more applications (e.g., (multiple) client applications 112) that can access, edit, modify, store and / or provide digital image content for display. For example, in some embodiments, the (multiple) client applications 112 include software applications installed on the (multiple) client devices 110. However, in other cases, the (multiple) client applications 112 include web browsers or other applications that access software applications hosted on the (multiple) server devices 102.

[0035] Differentiable partitioning system 106 may be implemented in whole or in part by various elements of environment 100. In practice, Figure 1 As shown in , the differentiable tile system 106 can be implemented with respect to (multiple) server devices 102 and (multiple) client devices 110. In certain embodiments, the differentiable tile system 106 on the (multiple) client devices 110 includes a web application, a native application (e.g., a mobile application, a desktop application, a plug-in application, etc.) installed on the (multiple) client device 110, or a cloud-based application, wherein part of the functionality is performed by the (multiple) server device 102.

[0036] In additional or alternative embodiments, the differentiable chunking system 106 on the client device(s) 110 represents and / or provides the same or similar functionality as described herein in conjunction with the differentiable chunking system 106 on the server device(s) 102. In some implementations, the differentiable chunking system 106 on the server device(s) 102 supports the differentiable chunking system 106 on the client device(s) 110.

[0037] In some embodiments, the differentiable chunking system 106 includes a web hosting application that allows client device(s) 110 to interact with content and services hosted on server device(s) 102. To illustrate, in one or more implementations, client device(s) 110 access a web page or computing application supported by server device(s) 102. Client device(s) 110 provide input (e.g., selected content items) to server device(s) 102. In response, the differentiable chunking system 106 on server device(s) 102 generates / modifies digital content. Server device(s) 102 then provides the digital content to client device(s) 110.

[0038] In some embodiments, although Figure 1 1, but environment 100 has a different arrangement of components and / or has a completely different number or set of components. For example, in some embodiments, client device(s) 110 bypass network 108 and communicate directly with server device(s) 102. As another example, environment 100 includes a third-party server that includes a content server and / or a data collection server.

[0039] As previously mentioned, in one or more embodiments, the differentiable blocking system 106 creates and modifies a digital depiction of a subject to provide a periodic, repeatable, and plausible design that is blockable in two-dimensional space. For example, Figure 2 is an example overview of using the differentiable blocking system 106 to generate a separable block image according to one or more embodiments. Figure 2Additional details of the various actions.

[0040] like Figure 2 As shown in , the differentiable chunking system 106 receives text input 210 and generates a chunkable image 250. In one or more embodiments, the text input 210 refers to a given text prompt that includes a text description of a concept or object. In addition, the text input 210 refers to a subject (e.g., an object or concept) as a visually identifiable entity. A subject represents a concept, item, element, or object that can be identified by its visual characteristics and attributes. The range of subjects utilized by the differentiable chunking system 106 ranges from everyday items such as flowers, books, and fruits to more complex entities such as animals, vehicles, and people. As shown, in some embodiments, the text input 210 includes a subject and an adjective, such as a text input 210 of a "Dalmatian dog". In certain embodiments, the text input 210 may include additional information, such as instructions for a desired visual style and other context or constraints related to the subject.

[0041] In addition, the tileable image 250 refers to a visual representation (e.g., an image or grid) that can be repeated or tiled. Specifically, a "tileable" image includes a non-square image or grid that can be iteratively repeated so that the boundaries align in a repeatable pattern (e.g., without showing any visible seams or interruptions, thereby creating the illusion of a continuous and uninterrupted image). For example, when complete coverage of a 2D plane is to be achieved, the tileable image can be replicated without any overlap between any two copies of the tile (including self-overlap of the tile with itself) and without space between any two copies of the tile.

[0042] like Figure 2As illustrated in , the differentiable blocking system 106 utilizes the text input 210 to determine an object (e.g., a subject) used to generate a textured 2D triangular mesh (e.g., mesh 230). For example, the differentiable blocking system 106 generates a textured 2D triangular mesh that is visually similar to the object indicated by the text input 210. As shown, the differentiable blocking system 106 utilizes the text input 210 of "Dalmatian dog" to generate a separable image 250 that is similar to a Dalmatian dog and is separable. In fact, using the trained diffusion model, the differentiable blocking system 106 optimizes the shape of the object indicated by the text input 210 while respecting the constraint that the shape of the object is separable. In addition, the differentiable blocking system 106 optimizes the texture (e.g., visual appearance) and the actual shape (e.g., geometry) of the block while maintaining its blocking ability. The differentiable blocking system 106 optimizes the geometry and texture of the image. In fact, this approach avoids optimizing color / texture over the square image, which results in an image that contains objects in its foreground, but also has background, making the blocking effect of the entire square image negligible. For example, the differentiable blocking system 106 exploits the interaction between geometry and texture to enable the differentiable blocking system 106 to determine a separable blockable image that contains little or no background.

[0043] As further shown, the differentiable blocking system 106 utilizes the symmetry groups 220 to optimize the shape. Specifically, the differentiable blocking system 106 utilizes the symmetry groups 220, which indicate different isometries of the blocks for the 2D plane. For example, the differentiable blocking system 106 utilizes a generative approach to blocks formed by quotients of symmetry groups by repeatedly applying a finite set of transformations to a block. In this way, the differentiable blocking system 106 can generate blocks that cover almost all Euclidean blocks (e.g., such as blocks generated by Escher).

[0044] Specifically, in one or more implementations, the differentiable blocking system 106 optimizes the shape of the mesh while maintaining its ability to block a plane without two copies of the mesh overlapping each other. In addition, the differentiable blocking system 106 generates the mesh 230 so that it has no self-overlap (e.g., the boundaries have no self-overlap and no triangles of the mesh are inverted). In fact, the differentiable blocking system 106 converts the non-convex restricted space of possible mesh configurations that can be blocked and have no overlap into an unconstrained differentiable representation of the mesh. Specifically, the differentiable blocking system 106 maps the triangular mesh bijectively into a 2D shape that can block a plane by modifying the edge weights of the mesh.

[0045] As further shown, the differentiable blocking system 106 uses a dynamic texture image 240 (e.g., an optimizable texture image) to texture the mesh 230 embedding. The dynamic texture image 240 indicates the visual appearance of the separable blocking image 250. Specifically, the differentiable blocking system 106 wraps or maps the dynamic texture image 240 to the mesh 230 to give the mesh 230 a visual appearance (e.g., color, hue, shading) that reflects the input prompt. As discussed in more detail below, the differentiable blocking system 106 utilizes a renderer (e.g., a differential renderer) to generate a rendered image from the mesh 230 and the dynamic texture image 240. The differentiable blocking system 106 utilizes the rendered image to modify the mesh 230 and the dynamic texture image 240 to align with the text input 210. Therefore, based on a measure of the loss between the rendered image and the representation of the text input, the differentiable blocking system 106 iteratively modifies the mesh 230 (according to a particular symmetry group) and the dynamic texture image 240.

[0046] As shown, the differentiable blocking system 106 determines a non-square geometric block (e.g., grid 230) that can be repeated infinitely without gaps or overlaps to map the plane, wherein the block contains only the desired foreground object. In addition, the differentiable blocking system 106 generates the blocks so that the boundary of each block is the outline of the desired object. In this way, the differentiable blocking system 106 generates a separable block image 250 that is a reasonable, separable representation of the desired object. For example, the differentiable blocking system 106 generates a separable block image 250 that is geometrically a pattern that follows a specified layout, seamlessly connects to itself, and repeats continuously infinitely. In addition, the differentiable blocking system 106 generates a separable block image 250 that is perceptually a reasonable, separable representation of the desired object while also presenting visual appeal.

[0047] As previously mentioned, in one or more embodiments, the differentiable tiling system 106 utilizes a differentiable renderer to create and modify digital depictions of a subject. Figure 3 An example of differentiable rendering of a separable patch image generated using a mesh, a texture image, and a loss function according to one or more embodiments is illustrated.

[0048] As shown, the mesh generation model 302 utilizes dynamic edge weights 310 in conjunction with symmetry groups 320. Specifically, the mesh generation model 302 utilizes symmetry groups 320 to generate a mesh that can be copied and shifted to cover a plane without any overlap between copies of the mesh. In the context of the differentiable blocking system 106, dynamic edge weights refer to changing or adaptive weights corresponding to connections between nodes in a mesh. For example, the dynamic edge weights 310 represent weights or strengths between nodes / vertices that are utilized to generate the resulting geometry of the mesh. For example, the mesh generation model 302 utilizes a generative method to generate blocks formed by quotients of symmetry groups 320 (e.g., generated by repeatedly applying a finite set of transformations to a block), which generative method utilizes dynamic edge weights 310 and symmetry groups 320.

[0049] To illustrate, by modifying the dynamic edge weights of the Laplacian operator used to modify the symmetry group 320, the mesh generation model 302 utilizes a differentiable mesh generation model and can represent any separable block mesh configuration (and only valid separable block configurations) for a certain selection of dynamic edge weights 310. The differentiability of the mesh generation model 302 allows the differentiable blocking system 106 to utilize gradient-based optimization techniques and backpropagation to update the dynamic edge weights based on a measure of loss. Therefore, a differentiable mesh generation model refers to a model that utilizes continuous functions where the gradient (or derivative) exists within the domain (e.g., for applying gradient optimization techniques). In addition, because the vertices of the mesh 330 are differentiable with respect to the dynamic edge weights 310, the mesh generation model 302 obtains an unconstrained, differentiable representation of all possible valid block geometries corresponding to the symmetry group 320. In one or more embodiments, the mesh generation model 302 utilizes symmetry groups, such as the seventeen discrete symmetry groups (e.g., wallpaper groups) described in The Symmetries of Things (2008) by John HConway, Heidi Burgiel, and Chaim Goodman-Strauss, which is incorporated herein by reference in its entirety. Depending on the embodiment, the differentiable blocking system 106 may utilize a subset of these symmetry groups (or a different symmetry group).

[0050] As further shown, the grid generation model 302 determines a grid 330 that satisfies an overlapping condition and a divisible boundary condition (e.g., divisible). As used herein, an overlapping condition for a divisible object refers to a grid property that the parts of the grid (e.g., at the edge) do not overlap or intersect. Therefore, when an object is divided or repeated to cover a larger area, each instance fits together without any visual inconsistency or overlap. In addition, a divisible boundary condition refers to a property that allows an object to repeat or replicate itself seamlessly with aligned boundaries (e.g., repeated on a plane without producing an obvious seam or interruption at the boundary). For example, a boundary condition ensures that an indication divisible object can be placed adjacent to its copy in a manner that maintains continuity and coherence at the boundary (to provide a seamless divisible pattern or texture). For illustration, the differentiable blocking system 106 generates a grid 330 that can be copied to a new position to achieve complete coverage of a 2D plane, without any overlap between any two copies of the grid 330 and without any self-overlap of the grid 330 and itself. In practice, the differentiable blocking system 106 generates a grid 330 that satisfies these two constraints and is blockable. Figure 4 and Figure 5 Provides additional details on overlap conditions and separable boundary conditions.

[0051] To represent the textured patch, the differentiable blocking system 106 selects an initial mesh. The differentiable blocking system 106 can utilize any valid patch as a possible mesh, but is not limited thereto (e.g., a regular triangulation of a square or diamond). The differentiable blocking system 106 uses the initial patch vertex coordinates as the UV coordinates of the vertices (to wrap the 3D surface onto a flat 2D plane while preserving spatial relationships), which the differentiable blocking system 106 keeps static throughout the optimization process. Specifically, the differentiable blocking system 106 optimizes the mesh 330 geometry by modifying edge weights to determine the vertices of the mesh 330.

[0052] For example, to represent an infinitely repeating partitioning of the mesh 330, the differentiable partitioning system 106 utilizes a single tile T, which is represented as a 2D triangular mesh having vertices V and triangles T. The differentiable partitioning system 106 utilizes a fixed, finite set of isometric 2D transformations (e.g., defined by the symmetry group 320), represented as periodic boundary conditions, that define the desired partitioning pattern. For example, when repeatedly applied to the mesh 330, the transformation defined by the symmetry group 320 copies the mesh 330 to new locations to achieve complete coverage of the 2D plane without any overlap between any two copies of the mesh 330 and without any self-overlap of the mesh with itself. In effect, the differentiable partitioning system 106 generates a mesh 330 that satisfies both constraints and is partitionable.

[0053] In addition, the differentiable blocking system 106 uses the optimizable texture image to texture the mesh embedding. Specifically, the differentiable blocking system 106 uses the dynamic texture image 340 to assign a color (or grayscale) to each point on the block via a predetermined UV coordinate. For example, the differentiable blocking system 106 wraps or maps the dynamic texture image 340 to the mesh 330 to give the mesh 230 a visual appearance (e.g., color, hue, shade) that reflects the input prompt. In addition, the differentiable blocking system 106 uses the UV mapping to texture the mesh to assign two-dimensional coordinates to the vertices of the mesh 330, thereby using the dynamic texture image 340 to determine how the texture will be applied to the surface of the mesh 330. In some embodiments, the differentiable blocking system 106 optimizes the value of the dynamic texture image 340 to adjust and enhance the color, contrast, brightness and other visual attributes of the dynamic texture image 340. As mentioned, the differentiable blocking system 106 uses the dynamic texture image 340 to generate a texture representing a block of a desired object, and wherein the boundary of each block is the outline of the object.

[0054] As further shown, the differentiable blocking system 106 uses a renderer 350 to render an image. Specifically, the differentiable blocking system 106 generates a rendered image 360 ​​from the text input 396 using the renderer 350. For example, the renderer includes a computer-implemented model that combines a texture image and a mesh to generate a rendering (e.g., a rendered image). For example, the renderer may include a computer-implemented algorithm that simulates the way light interacts with surfaces, materials, and objects to produce a 2D image that realistically and / or stylistically depicts the rendered image 360 ​​based on the text input 396. For example, the differentiable blocking system 106 combines the texture image 340 and the mesh 330 to generate the rendered digital image 360 ​​(e.g., mapping the texture image 340 to the mesh 330 using UV coordinates).

[0055] In some embodiments, the renderer 350 is a differentiable renderer. For example, a differentiable renderer is a renderer that allows gradients to be determined and propagated through an image. For illustration, a differentiable renderer models rendering through a continuous function so that derivatives (or gradients) can be determined and used for optimization. For example, in some implementations, the differentiable blocking system 106 utilizes a differentiable renderer described by Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila in High-Performance Differentiable Rendering, ACM Transactions on Graphics 39, 6 (2020), the entire contents of which are incorporated herein by reference. The differentiable blocking system 106 can utilize a variety of different differentiable renderers.

[0056] Thus, by utilizing a differentiable renderer, the differentiable blocking system 106 can use a gradient-based optimization algorithm (e.g., gradient descent) to optimize parameters to produce the best fit between the rendered image 360 ​​and the target image (based on the given text input 396). As mentioned, utilizing a gradient-based optimization with a perceptual loss, the differentiable blocking system 106 jointly optimizes the mesh 330 geometry via its vertices and jointly optimizes its color via the dynamic texture image 340. The renderer 350 ensures that the color and shape cooperate with each other to produce blocks that contain only the desired foreground image (this avoids the simplistic scheme of blocking square blocks of an image of an object with background color).

[0057] As further shown, in order to optimize the appearance of the rendered image 360 ​​to match a given text input 396, the differentiable blocking system 106 utilizes a loss function 370 to iteratively determine a metric of loss (e.g., a differentiable loss). Specifically, the differentiable blocking system 106 determines the loss using a perception module or directly based on the geometry of the mesh. For example, in some embodiments, the differentiable blocking system 106 utilizes a pre-trained diffusion model to define the loss. In this way, the differentiable blocking system 106 utilizes text guidance from the diffusion model to optimize the appearance of the mesh 330 to match the text input 396. To illustrate, in some embodiments, the differentiable blocking system 106 injects the rendered image 360 ​​into the diffusion process by adding noise to the rendered image 360 ​​and then applying a denoising step (conditioned on the expected text input 396).

[0058] In particular, during denoising, the differentiable chunking system 106 utilizes the embedding model to generate a representation (e.g., a vector representation) of the text input 396 (or different input prompts). The differentiable chunking system 106 also utilizes a first step of a trained diffusion neural network to generate a partially denoised image representation from a noisy representation (e.g., from a noisy representation of a rendered image 360) conditioned on the vector representation of the text input 396. The differentiable chunking system 106 can iteratively utilize additional steps (e.g., a second step, a third step, up to the thousandth step) of the diffusion neural network conditioned on the vector representation of the text input to generate a representation (e.g., an image representation) of the text input 396. The differentiable chunking system 106 can utilize various generative models (e.g., GANs, diffusion models, or other generative machine learning models) to generate representations of the input prompts and determine a measure of loss.

[0059] Thus, the differentiable blocking system 106 obtains a modified version of the input image (e.g., the modified rendered image 360) that is more relevant to the given text input 396. In addition, the differentiable blocking system 106 generates a loss as the difference between the rendered image 360 ​​and the modified rendered image via a loss function 370. The differentiable blocking system 106 can utilize a variety of different loss functions. For example, the differentiable blocking system 106 can utilize a loss function such as mean squared error loss (MSE), mean absolute error loss, binary cross entropy loss, categorical cross entropy loss, sparse categorical cross entropy loss, hinge loss, Huber loss, and / or Kullback-leibler divergence.

[0060] In one or more implementations, the differentiable block system 106 utilizes a generative approach to generate an image from a given text prompt. Specifically, the differentiable block system 106 transforms a trained diffusion model into a prior model that can be used as a loss in a traditional optimization setting. In this way, the differentiable block system 106 optimizes the shape of the object based on the text, thereby respecting the constraint that the shape should be separable. As mentioned, the differentiable block system 106 can generate separable images using a variety of different loss functions or losses. In one or more embodiments, the differentiable block system 106 utilizes fractional distillation sampling, as described in DreamFusion: Text-to-3D using 2D Diffusion (2022) by Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall, which is incorporated herein by reference in its entirety.

[0061] As further shown, the differentiable blocking system 106 modifies the dynamic texture image 340 and the dynamic edge weights 310 based on the metric of loss 370. For example, because the pipeline utilized to determine the metric of loss is differentiable, the differentiable blocking system 106 can utilize back propagation and / or gradient optimization to modify the dynamic texture image 340 and the dynamic edge weights 310 to a more optimal solution. After various iterations, the dynamic texture image 340 and the dynamic edge weights 310 can converge to generate a separable blocking image that is accurately aligned with the input prompt.

[0062] like Figure 3 As shown in , in one or more implementations, the differentiable blocking system 106 uses different rates for geometry optimization (first learning rate 380) and texture optimization (second learning rate 390). Specifically, the differentiable blocking system 106 uses the learning rate to set a hyperparameter that determines the step size of the mesh generation model 302 and the renderer 350 to adjust its internal parameters during the training process. Using this method, the differentiable blocking system 106 can control the magnitude of the parameter adjustment in each iteration.

[0063] In effect, by modifying the first learning rate 380 and the second learning rate 390, the differentiable blocking system 106 controls how quickly or slowly the mesh generation model 302 and the renderer 350 converge to a desired solution. In some embodiments, the differentiable blocking system 106 utilizes the same learning rate for the first learning rate 380 and the second learning rate 390. In these embodiments, the differentiable blocking system 106 generates the final texture and geometry at the same rate to determine the textured mesh. In one or more embodiments, the differentiable blocking system 106 independently controls the convergence rate of the mesh generation model 302 and the renderer 350 by utilizing the first learning rate 380 and the second learning rate 390. For illustration, the differentiable blocking system 106 utilizes a higher learning rate (e.g., 0.1) for geometry optimization and a lower learning rate (e.g., 0.01) for texture optimization. In these embodiments, by setting a higher learning rate for geometry optimization than for texture optimization, the differentiable blocking system 106 forces the final texture to appear more slowly and gives the optimized geometry time to form. In this way, given a text cue, the differentiable patching system 106 produces a texture mesh that visually matches the text, includes the desired object with little to no background, and is patchable.

[0064] As mentioned, the differentiable blocking system 106 maps the separable blocking image to create blocks using translated and rotated copies of a single textured 2D mesh. Figure 4 Mapping of a separable block image using separable block boundary conditions according to one or more embodiments is illustrated.

[0065] Figure 4A differentiable blocking system 106 is illustrated that utilizes symmetry groups to map translated and rotated copies of a mesh to blocks. Specifically, the differentiable blocking system 106 utilizes a symmetry group that represents a set of all symmetric transformations that preserve geometric properties of the mesh. For example, the differentiable blocking system 106 utilizes a symmetry group that includes a set of isometric affine transformations (rotations, translations, and reflections). In other words, the differentiable blocking system 106 uses symmetry groups to define a set of transformations that, when applied to the mesh, result in an indistinguishable configuration in a new position.

[0066] To illustrate, the differentiable block system 106 utilizes blocks corresponding to the symmetry group G, and formally, the blocks are The differentiable block system 106 moves each isometric transformation g∈G to a block by applying it to the copy Each group has a small, finite set of transform generation, which, when combined and inverted, generates the entire group. Differentiable blocking system 106 uses the generation set and its inversion to define boundary conditions for the block. Figure 4 As shown in the example of , the transformation generation set may include two 90 degree rotations. Specifically, block 410 is performed by isometric g i and is mapped to its own copy 420, equidistant g i is a member of the symmetry group that describes the symmetry of the desired partition. By g i Exactly map to another boundary vertex Vertices A, B, and C are fixed rotation axes.

[0067] In addition, the differentiable blocking system 106 determines necessary and sufficient conditions to ensure that the block is blockable for a given group G, as well as determines the blockability boundary conditions. Specifically, the differentiable blocking system 106 considers a valid block T and its copies that are directly adjacent to it. one of them (for example, their boundaries are Figure 4 ). This alignment means that some of the boundary vertices With copied blocks The boundary vertices in Alignment: where the transformation is denoted gi, with the subscript indicating that it is a particular member of the group G. Its corresponding vertex in the shifted block Therefore, each boundary vertex Each has a unique corresponding boundary vertex It is aligned with it in some replica. These correspondences can be assigned to a given mesh by choosing a known canonical vertex placement that is separable and inferring correspondences between vertices for that placement. Furthermore, a mesh T is a valid block with respect to a group G if it satisfies the following two necessary and sufficient block conditions: (1) All boundary vertices satisfy the separable block boundary condition and (2) the grid does not overlap itself. The differentiable block system 106 is constructed by considering each g i is a fixed affine transformation that incorporates the separable block boundary condition (1) into the optimization, and thus, Requirements: is a linear equation.

[0068] It is worth noting that merely satisfying the tile boundary condition (1) does not guarantee that the tile is valid, as it is still possible to have overlaps (and not satisfy the overlap condition). Furthermore, according to condition (2), self-overlaps may occur not only due to the boundary itself overlapping, but also due to inverted triangles in the interior of the mesh overlapping other triangles. These may lead to entangled configurations that cannot be interpreted as defining a correct planar shape and cause serious rendering problems that prohibit the use of visual guidance via neural networks. To ensure that the mesh represents a valid tile, the differentiable tile system 106 ensures that condition (2) discussed above is also satisfied during optimization. Figure 5 Utilizing dynamic edge weights and vertices to ensure non-overlapping border edges is illustrated in accordance with one or more embodiments.

[0069] It is worth noting that since the space of non-overlapping configurations of a given triangulation is highly non-convex, the differentiable blocking system 106 does not need to directly enforce the constraint that the mesh does not have self-overlaps. Specifically, in some implementations, the differentiable blocking system 106 utilizes a method that receives a disk topology triangular mesh and determines the solution of a linear system that satisfies the exact periodic boundary requirements of condition (1) and positions the vertices of the mesh so that it is non-overlapping and satisfies condition (2). Specifically, the differentiable blocking system 106 solves the discrete Laplace equation, placing each vertex so that it is a weighted average of its neighbors and satisfies:

[0070]

[0071] in Yes i The adjacent vertices (consider Figure 4 The periodicity of the boundary in the middle), and W = [w ij ] is a predetermined positive dynamic edge weight assigned to each edge (i, j). In some embodiments, the differentiable blocking system 106 uses weights derived from the cotangent Laplacian operator and keeps them positive.

[0072] In fact, the differentiable block system 106 defines Figure 4 and Figure 5 The linear system described (for example, using the equation and ) to generate a solution that produces final vertex positions, which are proven to be non-overlapping. Specifically, the differentiable blocking system 106 provides a way to parameterize the entire space of valid block shapes. That is, the dynamic edge weights W, i.e., the coefficients of the Laplacian matrix, can be directly optimized to control the solution of the system and in fact accurately parameterize the space of all possible block shapes. Specifically, if there are dynamic edge weights W assigned to the directed edges of the mesh that achieve a specific placement (e.g., the solution of the linear system described above) relative to the dynamic edge weights W that lead to those specific vertices V, then the placement of the vertex V is a valid block.

[0073] Therefore, the choice of W results in a separable block configuration of the grid. For illustration, assume that the grid is separable. Then, by the necessary and sufficient blocking conditions, the differentiable blocking system 106 determines that V satisfies And the configurations have no overlap. For a mesh with no overlap, each vertex lies in the convex hull of its neighbors (taking into account the periodicity of the boundaries) and thus the differentiable blocking system 106 expresses the positive barycentric coordinates of its neighbors as And because b j The sum is 1, so (For example, b j It is exactly the dynamic edge weight w we are looking for. ij ). With these dynamic edge weights W, the vertex V also satisfies and is the solution of the linear system. In addition, this approach produces an unconstrained differentiable parameterization of efficient non-overlapping patches. Therefore, the differentiable patching system 106 provides the above parameterization and uses the dynamic edge weights W as optimizable parameters to generate a patchable image. In some embodiments, by global rotation R of angle θ θ Apply to the resulting vertex placement R θ (V) and adding θ as a variable to be optimized, the differentiable patching system 106 also optimizes a non-rotationally invariant perceptual loss. In one or more embodiments, the differentiable patching system 106 parameterizes the entire space of valid patch shapes using an approach similar to that used for Orbifold Tutte Embeddings described by Noam Aigerman and Yaron Lipman in Orbifold Tutte Embeddings (2015) and HyperbolicOrbifold Tutte Embeddings (2016), the entire contents of which are incorporated herein by reference.

[0074] In one or more embodiments, differentiable blocking system 106 creates a separable blocking image comprising a separable blocking grid. Figure 6 Utilizing input to generate a separable grid according to one or more embodiments is illustrated.

[0075] like Figure 6 As shown in FIG. 6 , the mesh generation model 602 utilizes dynamic edge weights 610 in conjunction with symmetry groups 620 to generate a mesh 630. Figure 3 The described method, by modifying the dynamic edge weights of the Laplacian operator used to generate the mesh according to the symmetry groups in the symmetry group 620, the mesh generation model 602 can represent any separable block mesh configuration (and only valid separable block configurations) for certain choices of dynamic edge weights 610. Specifically, the mesh generation model 602 assigns new positions to the vertices of the mesh 630 through the solution of a linear system to ensure valid blocks that can block the plane without self-overlap. In addition, because the vertices of the mesh are differentiable with respect to the dynamic edge weights 610, the differentiable blocking system 106 obtains an unconstrained differentiable representation (e.g., mesh 630) of all possible valid block geometries corresponding to the symmetry group 620.

[0076] As further shown, the mesh generation model 602 determines a mesh 630 that satisfies the conditions of non-overlapping and blockable. As discussed, the differentiable blocking system 106 can utilize any valid block as an initial mesh (e.g., a triangulation of a square, diamond, or other shape. In addition, the differentiable blocking system 106 optimizes the mesh geometry by modifying dynamic edge weights to determine the vertices of the mesh 630. The differentiable blocking system 106 utilizes a fixed finite set of isometric 2D transformations (e.g., utilizing a symmetry group in the symmetry group 620) represented as periodic boundary conditions, which defines the desired blocking pattern. For example, the differentiable blocking system 106 repeatedly applies transformations according to a symmetry group in the symmetry group 620 to the mesh and repositions the vertices of the mesh to achieve complete coverage of the 2D plane without any overlap between any two copies of the mesh 630 and without any self-overlap of the mesh with itself.

[0077] like Figure 6As further shown in , the differentiable tiling system 106 receives an input 640 identifying a separable subject. For example, as shown, the differentiable tiling system 106 receives an input 640 identifying a sketch of a graphical object such as a facial image. In one or more embodiments, the differentiable tiling system 106 may receive other inputs identifying a separable subject, such as a photo, video, text, image, and / or voice input. As shown, the differentiable tiling system 106 utilizes a loss function 650, including utilizing a geometric loss to determine a loss between the mesh 630 and the input 640, and updating the mesh 630 parameters based on the appearance of the mesh 630 matching the appearance of the input 640.

[0078] To illustrate, the differentiable patching system 106 can generate a representation (e.g., a geometric representation or some other embedded representation) from the input 650 and compare the representation to the mesh 360. Specifically, the loss function 650 measures the difference or distance to evaluate the similarity or dissimilarity between the mesh 630 and the input 640. The differentiable patching system 106 can then utilize the loss function 650 to modify the edge weights 610 of the mesh generation model 602. By iteratively modifying the edge weights 610 of the mesh generation model 602, the mesh generation model iteratively optimizes the mesh to align with the input 640. In this way, the differentiable patching system 106 optimizes the appearance of the mesh 630 to match the input 640 and generates an output 660 of a separable patchable mesh. Specifically, given the input 640, the differentiable patching system 106 produces a mesh that geometrically matches the input 640, that consists of the desired object (with little background), and that is separable.

[0079] As mentioned, the differentiable blocking system 106 generates a grid that is utilized to create the blocking. Figure 7 Illustrated are tiles created from translated and rotated copies of a single textured 2D mesh in accordance with one or more embodiments.

[0080] Specifically, the tiles are created by combining translations and rotations of a single textured 2D mesh generated by the differentiable tile system 106. As shown, given a text prompt (e.g., "Dalmatian dog," "Chimpanzee," or "Dragon") and a selection of a plane symmetry group, the differentiable tile system 106 automatically generates a textured, non-square 2D mesh that matches the input prompt and is used as a tile. For example, the differentiable tile system 106 can provide a graphical user interface with various interface elements. These interface elements can include an input element (e.g., for selecting a text prompt or another input prompt) and an additional input element for selecting a specific symmetry group (e.g., a drop-down menu for selecting one of seventeen different symmetry groups). Based on these inputs, the differentiable tile system 106 can generate a separable tileable image and fill the two-dimensional plane with a translation instance of the separable tileable image. Specifically, the separable tileable image has no self-overlapping and can be shifted by the symmetry group to cover the entire plane without any overlap between the tiles. As shown, the differentiable tile system 106 can use different colors for different copies of the mesh to visually distinguish the copies.

[0081] Notably, each input prompt leaves a great deal of freedom in selecting a particular visual effect (e.g., size, shape, style, perspective, number of subjects), and the differentiable tiling system 106 employs a variety of strategies to create a mesh for the desired object, such as rendering the output in a wide shot, close-up, full body, detailed view, and / or overall view. For example, in Figure 7 In the video, the Dalmatian is shown with a wide angle shot to reduce the area occupied by the dog's back and tail. Figure 7 In another example shown in , in the case of chimpanzees, a photo of an object is requested in the input prompt, and the differentiable patching system 106 restricts the optimization to a realistic visualization space with more convex wheel shapes, or truncates less salient parts of the shape to avoid destroying realism. Figure 7 In another example shown in , in the case of a dragon, the differentiable tiling system 106 creates tiles that incorporate a truncated version of the dragon, which emphasizes the salient features of the dragon's head and tail.

[0082] As mentioned, the differentiable blocking system 106 creates blocks consisting of foreground objects with little or no extraneous background. Figure 8 The illustration shows tiles created by translating and rotating copies of a single textured 2D mesh without incorporating excessive background colors according to one or more embodiments. Specifically, by placing a rendered image (e.g., Figure 3360 in the rendered image 360), the differentiable blocking system 106 produces a reasonable block 810 of the "Chinese dragon's head" that is blockable to cover the entire plane without including the additional background color. In contrast, if the optimized background color is fixed (e.g., it is fixed to white), then in some cases, the result may depict a less believable result, as shown in block 820. In practice, the differentiable blocking system 106 utilizes the randomized background color of the rendered image to create a reasonable separable blocking grid that fills the blocking area with the foreground object, such as illustrated in block 810. Therefore, the differentiable blocking system 106 randomizes the background color to utilize the loss function (e.g., Figure 3 The loss function 370 in generates a block, the loss function excludes the background from its determination and modifies the geometry of the foreground object. It is worth noting that in this way, the differentiable blocking system 106 has more freedom in how to texture the block and modify the geometry of the separable block grid.

[0083] In one or more embodiments, differentiable chunking system 106 optimizes for two chunk shapes for two different input cues (or a single input cue with two themes). Fig. 9 The diagram illustrates creating a patch using two patch shapes for two different subjects according to one or more embodiments. Specifically, by having each shape contribute to a portion (e.g., or half) of the underlying mesh, the differentiable patch system 106 simultaneously optimizes the two patch shapes for two different cues. For example, the differentiable patch system 106 splits the mesh into two parts, each with a different cue, and calculates the loss for each input (e.g., each part of the mesh) separately, thereby producing two different objects with clearly separated contours within the patch. Fig. 9 As shown in , the differentiable partitioning system 106 creates a partitionable mesh 910 including "scuba diver" and "tropical fish". Fig. 9 As shown in , the differentiable blocking system 106 creates a separable blocking mesh 920 that includes “person doing yoga in tree pose” and “tree.” In some embodiments, the differentiable blocking system 106 renders each portion of the separable blocking mesh in a different color and texture to visually outline each object.

[0084] Now turn to Fig.10 , additional details will now be provided regarding the various components and capabilities of the differentiable blocking system 106. Specifically, Fig.10 The diagram shows a computer 1000 (eg, Figure 1 The differentiable chunking system 106 is implemented by one of the (multiple) server devices 102 and / or (multiple) client devices 110 discussed above. In addition, the differentiable chunking system 106 is also part of the digital content management system 104. Fig.10 As shown in , the differentiable partitioning system 106 includes, but is not limited to, an input manager 1002 , a symmetry group manager 1004 , a partitionable partitioning grid manager 1006 , and a data storage manager 1008 .

[0085] As just mentioned, and as Fig.10 , the differentiable blocking system 106 includes an input manager 1002. In one or more embodiments, the input manager 1002 retrieves or determines input including a subject for a separable blocking image. As mentioned, the input manager 1002 manages the use of the input to determine an object (e.g., a subject) for generating a textured 2D triangle mesh that is visually similar to the object indicated by the input. In addition, the input manager 1002 manages the input, which may include text input, photo input, video input, voice input, image input, and / or other types of input representing an object.

[0086] In addition, if Fig.10 As shown in , the differentiable blocking system 106 includes a symmetry group manager 1004. The differentiable blocking system 106 utilizes a selection of plane symmetry groups to generate a textured non-square 2D mesh that matches an input prompt. Specifically, the differentiable blocking system 106 utilizes the symmetry group manager 1004 to manage and utilize the symmetry groups to generate a mesh that can be copied and shifted to cover the entire plane without any overlap between copies of the mesh. For example, the symmetry group manager 1004 utilizes a generative method to generate tiles formed by quotients of symmetry groups (e.g., generated by repeatedly applying a finite set of transformations to a tile). In one or more embodiments, the symmetry group manager 1004 utilizes symmetry groups, each symmetry group comprising a set of discrete isometric affine transformations.

[0087] like Fig.10 As further shown in , the differentiable blocking system 106 includes a separable block mesh manager 1006. Specifically, the differentiable blocking system 106 generates and manages a textured 2D triangle mesh that is visually similar to an object indicated by an input prompt. In one or more embodiments, using a trained diffusion model, the separable block mesh manager 1006 optimizes the shape of the object indicated by the input while respecting the constraint that the shape of the object is separable. In addition, the separable block mesh manager 1006 optimizes the texture of the object as well as the actual shape while maintaining its block capability. For example, the separable block mesh manager 1006 uses the interaction between geometry and texture to determine a separable block image that contains little or no background.

[0088] In addition, as shown, the differentiable tiling system 106 includes a data storage manager 1008. Specifically, the data storage manager 1008 (implemented by one or more memory devices) stores digital content used by the differentiable tiling system 106, including input, texture images, symmetry groups, meshes, and rendered images. The data storage manager 1008 facilitates the creation of a separable tiling mesh by the differentiable tiling system 106. As shown, the data storage manager 1008 merges a mesh generation model 1008a and a differential renderer model 1008b. The differentiable tiling system 106 utilizes gradient optimization to modify the dynamic edge weights of the mesh generation model 1008a, thereby updating the dynamic edge weights in a progressive manner. The mesh generation model 1008a generates a mesh so that edges and vertices are aligned between repeatable instances of the mesh. The differentiable tiling system 106 utilizes a differential renderer model 1008b to generate a rendered image from the mesh and the dynamic texture image. The differentiable tiling system 106 utilizes the rendered image to modify the mesh and the dynamic texture image to reflect the text input.

[0089] Each of the components 1002-1008 of the differentiable block system 106 may include software, hardware, or both. For example, the components 1002-1008 may include one or more instructions stored on a computer-readable storage medium and executable by a processor of one or more computing devices (such as a client device or a server device). When executed by one or more processors, the computer-executable instructions of the differentiable block system 106 may cause (multiple) computing devices to perform the methods described herein. Alternatively, the components 1002-1008 may include hardware, such as a dedicated processing device for performing a specific function or group of functions. Alternatively, the components 1002-1008 of the differentiable block system 106 may include a combination of computer-executable instructions and hardware.

[0090] In addition, the components 1002-1008 of the differentiable block system 106 can be implemented, for example, as one or more operating systems, one or more independent applications, one or more modules of an application, one or more plug-ins, one or more library functions or functions that can be called by other applications, and / or a cloud computing model. Therefore, the components 1002-1008 of the differentiable block system 106 can be implemented as independent applications, such as desktop or mobile applications. In addition, the components 1002-1008 of the differentiable block system 106 can be implemented as one or more web-based applications hosted on a remote server. Alternatively or alternatively, the components 1002-1008 of the differentiable block system 106 can be implemented in a set of mobile device applications or "apps". For example, in one or more embodiments, the differentiable block system 106 may include a digital software application or operate in conjunction with a digital software application, such as: PHOTOSHOP, PHOTOSHOP ELEMENTS, ILLUSTRATOR, LIGHTROOM, INDESIGN, XD, FRESCO, PROTEUS, EXPRESS, ACROBATPRO DC or FigMA. Foregoing are registered trademarks or trademarks of Adobe Systems Incorporated in the United States and / or other countries.

[0091] refer to Figure 1-Figure 10 , the corresponding text and examples provide a variety of different methods, systems, devices and non-transitory computer-readable media of the differentiable partitioning system 106. In addition to the foregoing, one or more embodiments may also be described in terms of flowcharts including actions for achieving specific results, such as Figure 11-Figure 12 as shown in . Figure 11-Figure 12 The actions shown in the description may be performed in combination with more or fewer actions. Furthermore, the actions may be performed in a different order. In addition, the actions described herein may be repeated or performed in parallel with each other, or with different instances of the same or similar actions. A non-transitory computer-readable medium may include instructions that, when executed by one or more processors, cause a computing device to perform Figure 11-Figure 12 In some embodiments, the system can be configured to perform Figure 11-Figure 12 Alternatively, Figure 11-Figure 12 The actions may be performed as part of a computer-implemented method.

[0092] Fig.11 A flow chart illustrating a series of actions 1100 for generating a separable blockable image using the differentiable blocking system 106 according to one or more embodiments. Fig.11 The actions according to one embodiment are illustrated, but alternative embodiments may omit, add, reorder, and / or modify Fig.11 Any action shown in .

[0093] Fig.11A series of example actions 1100 for generating a separable block image using a differentiable block system 106 is illustrated. Specifically, a series of actions 1100 includes an action 1102 for receiving a text input including a subject for the separable block image. As illustrated, a series of actions 1100 may also include an action 1104 for generating a mesh including edges and vertices that satisfy an overlap condition and a separable block boundary condition. Specifically, action 1104 may include generating a mesh including edges and vertices that satisfy an overlap condition and a separable block boundary condition based on dynamic edge weights of a mesh generation model. As illustrated, a series of actions 1100 may also include an action 1106 for generating a rendered image from a mesh and a dynamic texture image. Specifically, action 1106 may include generating a rendered image from a mesh and a dynamic texture image using a renderer. As illustrated, a series of actions 1100 may also include an action 1108 for generating a separable block image by modifying dynamic mesh weights and a dynamic texture image. Specifically, act 1108 may include generating a separable image by modifying dynamic edge weights and a dynamic texture image based on a measure of loss between the rendered image and the vector representation of the text input.

[0094] In addition to the above actions (or as an alternative), a series of actions 1100 of the differentiable blocking system may include generating mesh edges and vertices that meet the overlap condition, including: using the dynamic edge weights of the mesh generation model to generate edges and vertices to ensure non-overlapping boundary edges and non-overlapping internal edges. The series of actions 1100 may also include modifying the dynamic edge weights to generate modified dynamic edge weights. In addition, in one or more embodiments, the series of actions 1100 of the differentiable blocking system 106 includes modifying the dynamic texture image to generate a modified texture image. In addition, in some embodiments, the series of actions 1100 of the differentiable blocking system 106 includes generating a modified mesh that meets the overlap condition and the blockable boundary condition based on the modified dynamic edge weights of the mesh generation model.

[0095] In addition, in one or more embodiments, the differentiable blocking system series of actions 1100 includes utilizing a differentiable renderer, and also includes utilizing the differentiable renderer to generate additional rendered images from the modified mesh and the modified texture image. In addition, in one or more embodiments, the series of actions 1100 of the differentiable blocking system 106 includes generating a separable block image from the additional rendered image. In addition, in some embodiments, the series of actions 1100 of the differentiable blocking system 106 includes generating a mesh so that the mesh can be infinitely repeated using a non-square boundary defined by edges and vertices aligned between all instances of the mesh. In addition, in one or more embodiments, the differentiable blocking system series of actions 1100 includes generating a separable block image by randomizing a background color in a dynamic texture image. In addition, in one or more embodiments, the series of actions 1100 includes utilizing a first learning rate for updating dynamic edge weights and utilizing a second learning rate for updating dynamic texture images, wherein the first learning weights are different from the second learning rate.

[0096] Fig.12 A flow chart illustrating a series of actions 1200 for generating a separable blockable image using the differentiable block system 106 according to one or more embodiments. Fig.12 The actions according to one embodiment are illustrated, but alternative embodiments may omit, add, reorder, and / or modify Fig.12 Any action shown in .

[0097] Fig.12 A series of example actions 1200 for generating a separable blockable image using a differentiable block system 106 is illustrated. Specifically, the series of actions 1200 includes an action 1202 of receiving an input indicating a subject. Specifically, the action 1202 may include receiving an input indicating a subject for a repeatable block. As illustrated, the series of actions 1200 may also include an action 1204 of generating a mesh including edges and vertices and applying an overlap condition and a blockability boundary condition. In particular, the action 1204 may include generating a mesh including edges and vertices based on dynamic edge weights of a mesh generation model, wherein the mesh generation model applies an overlap condition using a linear model and dynamic edge weights so that the edges of the mesh do not overlap, and applies a blockability boundary condition so that the mesh can be infinitely repeated using boundaries defined by edges and vertices aligned between repeated separable block instances of the mesh. As illustrated, the series of actions 1100 may also include an action 1206 of generating a measure of loss by comparing the mesh to a digital representation of the subject. As illustrated, the series of acts 1200 may also include an act 1208 of generating a repeatable block by modifying the dynamic grid weights based on a measure of loss.

[0098] In addition to the above actions (or as an alternative, the series of actions 1200 of the differentiable chunking system may include: receiving input includes receiving text content indicating a topic. Furthermore, in one or more embodiments, the series of actions 1200 of the differentiable chunking system includes: generating a measure of loss includes comparing the text content and the grid. Furthermore, in one or more embodiments, the series of actions 1200 includes: modifying the dynamic edge weights includes modifying the dynamic edge weights of the differentiable grid generation model using gradient optimization. Furthermore, in one or more embodiments, the series of actions 1200 includes: receiving input includes receiving a visual representation of the topic. Furthermore, in one or more embodiments, the series of actions 1200 of the differentiable chunking system includes: generating a measure of loss includes comparing the visual representation and the grid.

[0099] Furthermore, in one or more embodiments, a series of actions 1200 includes: generating mesh edges and vertices to impose overlapping conditions includes using a mesh generation model to generate a mesh to ensure non-overlapping boundary edges. Furthermore, in one or more embodiments, a series of actions 1200 includes: generating a rendered image from the mesh and a dynamic texture image using a differentiable renderer. Additionally, in one or more embodiments, a series of actions 1200 includes: determining a measure of loss by comparing the rendered image to a vector representation of the input. Additionally, in one or more embodiments, a series of actions 1200 includes: generating a separable block image by modifying the dynamic edge weights and the dynamic texture image based on a measure of loss.

[0100] Furthermore, in one or more embodiments, the sequence of actions 1200 includes: receiving input includes receiving a topic for a repeatable block and an additional topic. Furthermore, in one or more embodiments, the sequence of actions 1200 includes: generating an additional measure of loss by comparing the grid to a digital representation of the additional topic. Furthermore, in one or more embodiments, the sequence of actions 1200 includes: updating the grid by modifying the dynamic edge weights based on the measure of loss and the additional measure of loss.

[0101] In addition, in one or more embodiments, a series of actions 1200 includes: receiving text input including a subject for a separable image. In addition, in one or more embodiments, a series of actions 1200 includes: generating a mesh including edges and vertices that satisfy an overlap condition and a separable boundary condition based on dynamic edge weights of a mesh generation model. In addition, in one or more embodiments, a series of actions 1200 includes: generating a rendered image from a mesh and a dynamic texture image using a renderer. In addition, in one or more embodiments, a series of actions 1200 includes: generating a separable image by modifying the dynamic edge weights and the dynamic texture image based on a measure of loss between the rendered image and the vector representation of the text input. In addition, in one or more embodiments, a series of actions 1200 includes: generating a mesh using a mesh generation model includes generating a mesh using a differentiable mesh generation model. In addition, in one or more embodiments, a series of actions 1200 includes: generating a mesh so that edges and vertices are aligned between repeatable instances of the mesh.

[0102] Embodiments of the present disclosure may include or utilize a special-purpose or general-purpose computer including computer hardware, such as, for example, one or more processors and system memory, as discussed in more detail below. Embodiments within the scope of the present disclosure also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Specifically, one or more of the processes described herein may be implemented at least in part as instructions embodied in a non-transient computer-readable medium and executable by one or more computing devices (e.g., any of the media content access devices described herein). In general, a processor (e.g., a microprocessor) receives instructions from a non-transient computer-readable medium (e.g., a memory) and executes the instructions to perform one or more processes, including one or more of the processes described herein.

[0103] Computer readable media can be any available media that can be accessed by a general or special purpose computer system. A computer readable medium that stores computer executable instructions is a non-transient computer readable storage medium (device). A computer readable medium that carries computer executable instructions is a transmission medium. Therefore, by way of example and not limitation, embodiments of the present disclosure may include at least two distinct computer readable media: a non-transient computer readable storage medium (device) and a transmission medium.

[0104] Non-transitory computer-readable storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid-state drives (“SSD”) (e.g., RAM-based), flash memory, phase change memory (“PCM”), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other media that can be used to store desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general or special purpose computer.

[0105] "Network" is defined as one or more data links that enable electronic data to be transmitted between computer systems and / or modules and / or other electronic devices. When information is transmitted or provided to a computer via a network or other communication connection (hardwired, wireless, or a combination of hardwired or wireless), the computer correctly regards the connection as a transmission medium. Transmission media may include networks and / or data links that can be used to carry computer executable instructions or desired program code devices in the form of data structures and can be accessed by general or special computers. The above combinations should also be included in the scope of computer-readable media.

[0106] In addition, upon reaching various computer system components, program code means in the form of computer executable instructions or data structures may be automatically transferred from a transmission medium to a non-transitory computer readable storage medium (device) (or vice versa). For example, a computer executable instruction or data structure received over a network or data link may be buffered in RAM within a network interface module (e.g., "NIC") and then ultimately transferred to the computer system RAM and / or a less volatile computer storage medium (device) at the computer system. Thus, it should be understood that a non-transitory computer readable storage medium (device) may be included in a computer system component that also (or even primarily) utilizes a transmission medium.

[0107] Computer executable instructions include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a specific function or group of functions when executed by a processor. In some embodiments, computer executable instructions are executed by a general-purpose computer to convert a general-purpose computer into a special-purpose computer that implements the elements of the present disclosure. Computer executable instructions can be, for example, binary, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in a language specific to structural features and / or method actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the above-mentioned features or actions. On the contrary, the described features and actions are disclosed as example forms of implementing the claims.

[0108] Those skilled in the art will appreciate that the present disclosure can be practiced in a network computing environment with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablet computers, pagers, routers, switches, etc. The present disclosure can also be practiced in a distributed system environment, where local and remote computer systems linked by a network (or by a hardwired data link, a wireless data link, or by a combination of hardwired and wireless data links) all perform tasks. In a distributed system environment, program modules can be located in local and remote memory storage devices.

[0109] Embodiments of the present disclosure may also be implemented in a cloud computing environment. As used herein, the term "cloud computing" refers to a model for implementing on-demand network access to a shared pool of configurable computing resources. For example, cloud computing may be employed in a market to provide universal and convenient on-demand access to a shared pool of configurable computing resources. A shared pool of configurable computing resources may be quickly configured through virtualization and released with low management effort or service provider interaction, and then scaled accordingly.

[0110] The cloud computing model may consist of various features, such as, for example, on-demand self-service, broad network access, resource pooling, rapid elasticity, measured services, and the like. The cloud computing model may also expose various service models, such as, for example, software as a service (“SaaS”), platform as a service (“PaaS”), and infrastructure as a service (“IaaS”). The cloud computing model may also be deployed using different deployment models, such as private cloud, community cloud, public cloud, hybrid cloud, and the like. Additionally, as used herein, the term “cloud computing environment” refers to an environment in which cloud computing is employed.

[0111] Fig.13 A block diagram of an example computing device that can be configured to perform one or more of the above-described processes is illustrated. It will be appreciated that one or more computing devices, such as computing device 1300, can represent the above-described computing devices (e.g., server device(s) 102, client device(s) 110, and computing device 1300). In one or more embodiments, computing device 1300 can be a mobile device (e.g., a mobile phone, a smart phone, a PDA, a tablet computer, a laptop computer, a camera, a tracker, a watch, a wearable device, etc.). In some embodiments, computing device 1300 can be a non-mobile device (e.g., a desktop computer or another type of client device). In addition, computing device 1300 can be a server device that includes cloud-based processing and storage capabilities.

[0112] like Fig.13 As shown in , computing device 1300 may include one or more processors 1302, memory 1304, storage device 1306, input / output interface 1308 (or "I / O interface 1308"), and communication interface 1310, which may be communicatively coupled via a communication infrastructure (e.g., bus 1312). Fig.13 The computing device 1300 is illustrated in FIG. Fig.13 The components illustrated in the figure are not intended to be limiting. Additional or alternative components may be used in other embodiments. Furthermore, in some embodiments, the computing device 1300 includes more than Fig.13 The components shown in FIG. 1 are fewer than those shown in FIG. 1 . Fig.13 Components of computing device 1300 are shown in .

[0113] In certain embodiments, processor 1302 includes hardware for executing instructions, such as instructions that make up a computer program. As an example and not by way of limitation, to execute instructions, processor(s) 1302 may retrieve (or fetch) instructions from internal registers, internal cache, memory 1304, or storage device 1306 and decode and execute them.

[0114] Computing device 1300 includes memory 1304 coupled to processor(s) 1302. Memory 1304 may be used to store data, metadata, and programs for execution by processor(s). Memory 1304 may include one or more of volatile and non-volatile memory, such as random access memory ("RAM"), read-only memory ("ROM"), solid state disk ("SSD"), flash memory, phase change memory ("PCM"), or other types of data storage. Memory 1304 may be internal memory or distributed memory.

[0115] The computing device 1300 includes a storage device 1306, which includes a storage device for storing data or instructions. As an example and not by way of limitation, the storage device 1306 may include the non-transitory storage medium described above. The storage device 1306 may include a hard disk drive (HDD), a flash memory, a universal serial bus (USB) drive, or a combination of these or other storage devices.

[0116] As shown, computing device 1300 includes one or more I / O interfaces 1308, which are provided to allow a user to provide input (such as user strokes) to computing device 1300, receive output from computing device 1300, and otherwise transfer data to and from computing device 1300. These I / O interfaces 1308 may include a mouse, a keypad or keyboard, a touch screen, a camera, an optical scanner, a network interface, a modem, other known I / O devices, or a combination of such I / O interfaces 1308. The touch screen may be activated with a stylus or a finger.

[0117] The I / O interface 1308 may include one or more devices for presenting output to a user, including, but not limited to, a graphics engine, a display (e.g., a display screen), one or more output drivers (e.g., a display driver), one or more audio speakers, and one or more audio drivers. In some embodiments, the I / O interface 1308 is configured to provide graphical data to the display for presentation to the user. The graphical data may represent one or more graphical user interfaces and / or any other graphical content that may serve a particular implementation.

[0118] The computing device 1300 may also include a communication interface 1310. The communication interface 1310 may include hardware, software, or both. The communication interface 1310 provides one or more interfaces for communication (such as, for example, packet-based communication) between the computing device and one or more other computing devices or one or more networks. As an example and not by way of limitation, the communication interface 1310 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wired-based network, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network (such as WI-FI). The computing device 1300 may also include a bus 1312. The bus 1312 may include hardware, software, or both that connect the components of the computing device 1300 to each other.

[0119] In the foregoing description, the present disclosure has been described with reference to specific exemplary embodiments of the present disclosure. Various embodiments and aspects of the present disclosure are described with reference to the details discussed herein, and the accompanying drawings illustrate various embodiments. The above description and drawings illustrate the present disclosure and should not be construed as limiting the present disclosure. Many specific details are described to provide a thorough understanding of the various embodiments of the present disclosure.

[0120] The present disclosure may be implemented in other specific forms without departing from its spirit or basic characteristics. The described embodiments should be considered to be illustrative and non-restrictive in all aspects. For example, the method described herein may be performed with fewer or more steps / actions, or the steps / actions may be performed in different orders. In addition, the steps / actions described herein may be repeated or performed in parallel with each other, or performed in parallel with different examples of the same or similar steps / actions. Therefore, the scope of the application is indicated by the appended claims rather than the foregoing description. All changes falling within the equivalent meanings and scope of the claims are included within their scope.

Claims

1. A method comprising: receiving a text input including a subject for the segmentable image; Generate a mesh including edges and vertices satisfying overlapping conditions and divisible boundary conditions based on dynamic edge weights of a mesh generation model; generating a rendered image from the mesh and the dynamic texture image using a renderer; as well as The separable image is generated by modifying the dynamic edge weights and the dynamic texture image based on a measure of loss between the rendered image and a vector representation of the textual input.

2. The method according to claim 1, wherein generating mesh edges and vertices satisfying the overlap condition comprises: The edges and the vertices are generated using the dynamic edge weights of the mesh generation model to ensure non-overlapping boundary edges and non-overlapping internal edges.

3. The method according to claim 1, further comprising: Modifying the dynamic edge weight to generate a modified dynamic edge weight; modifying the dynamic texture image to generate a modified texture image; as well as A modified mesh satisfying the overlap condition and the separable boundary condition is generated based on the modified dynamic edge weights of the mesh generation model.

4. The method of claim 3, wherein the renderer is a differentiable renderer and further comprising: generating an additional rendered image from the modified mesh and the modified texture image using the differentiable renderer; as well as The separable tileable image is generated from the additional rendered image.

5. The method of claim 1 , wherein generating the mesh including the edges and the vertices satisfying the separable boundary condition comprises: The mesh is generated such that the mesh can be repeated indefinitely with a non-square boundary defined by the edges and the vertices aligned between all instances of the mesh.

6. The method according to claim 1, further comprising: The separable image is generated by randomizing the background color in the dynamic texture image.

7. The method according to claim 1, further comprising: Using a first learning rate to update the dynamic edge weights; as well as A second learning rate is utilized for updating the dynamic texture image, wherein the first learning weight is different from the second learning rate.

8. A system comprising: Memory components; as well as One or more processing devices, coupled to the memory component, the one or more processing devices configured to perform operations, the operations comprising: receiving input indicating a subject for the repeatable block; A mesh including edges and vertices is generated based on dynamic edge weights of a mesh generation model, wherein the mesh generation model uses a linear model and the dynamic edge weights to apply: overlap condition so that the edges of the meshes do not overlap, and a tileability boundary condition such that the mesh can be repeated indefinitely with boundaries defined by the edges and vertices aligned between repeated tileable instances of the mesh; generating a measure of loss by comparing the grid to a digital representation of the subject; and The repeatable block is generated by modifying the dynamic edge weights based on a measure of the loss.

9. The system of claim 8, wherein generating the mesh using the mesh generation model comprises: The mesh is generated using a differentiable mesh generation model, wherein the dynamic edge weights comprise modifiable parameters of the differentiable mesh generation model.

10. The system of claim 9, wherein: Receiving the input includes: receiving textual content indicating the subject matter; Generating the metric of the loss includes: comparing the text content to the grid; and Modifying the dynamic edge weights includes: modifying the dynamic edge weights of the differentiable mesh generation model using gradient optimization.

11. The system of claim 8, wherein: Receiving the input includes: receiving a visual representation of the subject matter; and Generating the measure of the loss includes comparing the visual representation to the mesh.

12. The system of claim 8, wherein generating mesh edges and vertices to impose the overlap condition comprises: The mesh generation model is used to generate the mesh to ensure that boundary edges do not overlap.

13. The system of claim 8, wherein the operations further comprise: A differentiable renderer is used to generate a rendered image from the mesh and the dynamic texture image.

14. The system of claim 13, wherein the operations further comprise: The measure of the loss is determined by comparing the rendered image to a vector representation of the input.

15. The system of claim 14, wherein the operations further comprise: A separable image is generated by modifying the dynamic edge weights and the dynamic texture image based on a measure of the loss.

16. The system of claim 8, wherein: Receiving the input includes: receiving the subject and additional subjects for the repeatable block; generating an additional measure of loss by comparing the grid to the digital representation of the additional subject matter; and The grid is updated by modifying the dynamic edge weights based on the measure of the loss and the measure of the additional loss.

17. A non-transitory computer-readable medium storing executable instructions, which, when executed by a processing device, cause the processing device to perform operations, the operations comprising: receiving a text input including a subject for the segmentable image; Generate a mesh including edges and vertices satisfying overlapping conditions and divisible boundary conditions based on dynamic edge weights of a mesh generation model; generating a rendered image from the mesh and the dynamic texture image using a renderer; as well as The separable image is generated by modifying the dynamic edge weights and the dynamic texture image based on a measure of loss between the rendered image and a vector representation of the textual input.

18. The non-transitory computer readable medium of claim 17, wherein generating the mesh using the mesh generation model comprises: The mesh is generated using a differentiable mesh generation model.

19. The non-transitory computer readable medium of claim 17, wherein generating the mesh including the edges and the vertices satisfying the separable boundary condition comprises: The mesh is generated such that the edges and vertices are aligned between repeatable instances of the mesh.

20. The non-transitory computer readable medium of claim 17, further comprising: modifying the dynamic edge weight to generate a modified dynamic mesh weight; modifying the dynamic texture image to generate a modified texture image; generating a modified grid satisfying the overlap condition and the divisible boundary condition based on the modified dynamic grid weight of the grid generation model; generating, using a renderer, an additional rendered image from the modified mesh and the modified texture image; as well as The separable tileable image is generated from the additional rendered image.

Images