Learning-based 2D shape packing

Abstract

Multiple UV patches in a 3D model are divided into a primary set and a secondary set based on a patch size threshold. The UV patches in the primary set are grouped into multiple superpatches. Each of the superpatches contains different UV patches from the primary set that are packed together in a predefined shape. The superpatches are assembled together into a first bounding box. The orientation of the superpatches is adjusted to reduce the spacing between them. Gaps between UV patches in the primary set are filled with UV patches from the secondary set.

Description

[Technical Field]

[0001] [Cross-reference of related applications] This application claims priority under U.S. Patent Application No. 18 / 376,324, filed on 3 October 2023, the entirety of which is incorporated herein by reference.

[0002] [Technical field to which the review belongs] This disclosure includes embodiments relating to packing patches derived from a three-dimensional (3D) model in texture space. [Background technology]

[0003] The background art provided herein is intended to provide a general context for this disclosure. The research of the inventors currently named, as well as any descriptions that may not qualify as prior art at the time of filing, are not expressly or implicitly considered prior art to this disclosure.

[0004] 2D irregular shape packing is a necessary step in computer graphics to arrange UV patches of a 3D model within a texture atlas for memory-efficient rendering of the appearance. Because it involves decision-making problems of joining and combining, including the position and orientation of the patches, this problem has a well-known NP-hard complexity. In related solutions, to simplify the problem, heuristic packing orders are assumed, or upstream mesh cuts and UV mappings are modified, which can limit the packing rate or lead to robustness or generality issues. [Overview of the project] [Means for solving the problem]

[0005] Aspects of this disclosure include methods, apparatus, and non-temporary computer-readable storage media for patch packing. In some examples, the apparatus for patch packing includes processing circuits.

[0006] According to one aspect of this disclosure, a method for processing multiple UV patches in a three-dimensional model is provided. In this method, the multiple UV patches are divided into a primary set of UV patches and a secondary set of UV patches based on a patch size threshold. The UV patches of the primary set are grouped into multiple superpatches. Each of the multiple superpatches contains different UV patches from the primary set that are packed together in a predefined shape. The multiple superpatches are assembled together into a first bounding box. The orientation of the multiple superpatches is adjusted to reduce the spacing between the multiple superpatches. The gaps between UV patches in the primary set are filled with UV patches from the secondary set.

[0007] In one example, M subsets are selected from a primary set based on a high-level group selector network (HSN) configured to identify subsets from a primary set to form superpatches of multiple superpatches. Each of the M subsets contains N UV patches, where M is a first positive integer and N is a second positive integer less than M. An estimated area-average packing rate is determined for each of the M subsets. The M subsets are sorted based on the estimated area-average packing rates. L subsets are determined from the M subsets corresponding to L maximum estimated area-average packing rates.

[0008] In one example, the UV patches within each of the L subsets are packed together into their respective subset bounding boxes. An area-average packing rate is determined for each of the L packed subsets. The first of several superpatches is determined from the L packed subsets. The first superpatch corresponds to the largest area-average packing rate among the determined area-average packing rates of the L packed subsets.

[0009] In one aspect, UV patches in a first subset of L subsets are packed into a first subset bounding box. For example, based on a low-level sorter network (LSN) configured to determine the packing order of UV patches in a first subset, the UV patches in a first subset are organized into a connected graph in which already packed patches and patches to be packed in the first subset are linked together. The node features of the UV patches in the first subset are input to a graph attention network (GAT) to obtain the graph features of the UV patches in the first subset. The graph features are converted into corresponding Q values ​​via a multilayer perceptron (MLP) which includes an input layer, an output layer, and one or more hidden layers with stacked neurons. Based on the Q values, the first patch to be packed into an already packed patch is determined.

[0010] In one example, the node features of the UV patches in the first subset are determined based on a fully convolutional network (FCN) in which the UV patches in the first subset are encoded in an F-dimensional latent space, where F is a positive integer.

[0011] In one aspect, the state space is determined based on a low-level attitude network (LPN) configured to determine the attitude of UV patches in a primary set. The state space represents the positional states of already packed patches and patches to be packed in the first subset. An action space is determined for the patches to be packed. The action space represents candidate packing actions for the patches to be packed. Each of the candidate packing actions in the action space is applied to the first patch to be packed. An updated state space is determined corresponding to each of the candidate packing actions applied to the first patch to be packed. A reward value is determined corresponding to each of the updated state spaces associated with the first patch to be packed. Each reward value corresponds to the area-average packing rate associated with each candidate packing action. From the candidate packing actions in the action space, a packing action corresponding to the maximum reward value is determined. The first patch to be packed is packed by adjusting its attitude and distance according to the determined packing action.

[0012] In one example, the determined packing action includes a translation action to reduce the distance between the first patch to be packed and the already packed patch, and a rotation action to adjust the orientation of the first patch to be packed. The determined packing action is applied to the first patch to be packed according to collision-constrained local optimization so that the center of mass (COM) of the first patch to be packed and the COM of the already packed patch are reduced to predetermined values ​​and the first patch to be packed and the already packed patch do not overlap.

[0013] In one example, the packing ratio of a first packed subset of L packed subsets is determined based on the ratio of the area of ​​the first subset to the area of ​​the first bounding box of the first packed subset. The area-average packing ratio associated with a first packed subset of L packed subsets is determined based on the ratio of (i) the sum of the areas of UV patches in the superpatch containing the first packed subset in the primary set to (ii) the sum of the areas of the subset bounding boxes of the superpatch containing the first packed subset in the primary set.

[0014] In one respect, LPNs are trained based on the Q-learning algorithm by maximizing the expected cumulative reward. The expected cumulative reward is defined as follows:

number

[0015] In one aspect, the HSN is trained. To train the HSN, a first estimated area-average packing rate is determined for a first training subset from the primary set, and a second estimated area-average packing rate is determined for a second training subset from the primary set. A first area-average packing rate for the first training subset and a second area-average packing rate for the second training subset are further determined. The HSN is updated via a margin ranking loss, and the margin ranking loss is,

number

number

number

[0016] In one example, the orientation of multiple superpatches is adjusted by rotating and translating them, thereby reducing the spacing between them, and consequently shrinking the first bounding box to a second bounding box. The second bounding box corresponds to a bounding size minimized according to an optimization function so that the multiple superpatches within the second bounding box do not overlap.

[0017] According to other aspects of this disclosure, an apparatus is provided. The apparatus includes a processing circuit. The processing circuit may be configured to perform one or a combination of methods for packing patches derived from a 3D model in the same texture space.

[0018] Aspects of this disclosure also provide a non-temporary computer-readable medium that, when executed by at least one processor, stores instructions causing at least one processor to execute one or a combination of methods for packing patches derived from a 3D model in the same texture space.

[0019] Further features, properties, and various advantages of the disclosed subject matter will become clearer from the detailed description and accompanying drawings below. [Brief explanation of the drawing]

[0020] [Figure 1] This is a schematic diagram of a pipeline for packing patches derived from a 3D model in the same texture space, according to several embodiments.

[0021] [Figure 2] This is a schematic diagram of the action space according to several embodiments.

[0022] [Figure 3] This figure shows the formation of an alpha shape according to several embodiments.

[0023] [Figure 4] This is a schematic diagram illustrating patch squeezing and gap removal according to several embodiments.

[0024] [Figure 5] This figure shows an exemplary 3D model dataset in several embodiments.

[0025] [Figure 6] This figure shows a comparison of the first exemplary packing rate between the method of this disclosure and related examples according to several embodiments.

[0026] [Figure 7]This figure shows a second exemplary packing rate comparison between the method of this disclosure and related examples according to several embodiments.

[0027] [Figure 8] This figure shows an exemplary comparison of average packing times between relevant examples of several embodiments and the method of the present disclosure.

[0028] [Figure 9] This figure shows a comparison of a third exemplary packing rate between related examples of several embodiments and the method of the present disclosure.

[0029] [Figure 10] This flowchart outlines an exemplary process for packing patches according to some embodiments of the present disclosure.

[0030] [Figure 11] This is a schematic diagram of a computer system according to one embodiment. [Modes for carrying out the invention]

[0031] In this disclosure, UV unwrapping may include the process of mapping a 3D model onto a 3D surface. UV patches / charts may include portions of a 3D model cut from (or derived from) a full model. UV packing may include the process of placing UV patches in the same texture space, for example, as densely as possible.

[0032] In related examples, open-source software such as Blender and Xatlas may be applied to pack patches. For example, Blender and Xatlas can combine multiple heuristic methods to pack UV charts together. In related examples, a heuristic method for packing UV charts may also be provided. The heuristic method may generate a Multi-Chart Geometry Image (MCGI). The heuristic method may calculate the top horizon of an already packed UV chart. Given a new UV chart, the method may heuristically check all horizontal positions on the top horizon. The heuristic method may then select the lowest position on the top horizon. In related examples, a modified version of the heuristic method may also be provided. In the modified heuristic method, each new UV chart may be positioned in several orientations. According to the modified heuristic method, a new heuristic scoring function may be provided. The new heuristic scoring function may measure the wasted area and select an arrangement to minimize the wasted area. Furthermore, the UV chart can wrap around the texture image boundary. No Fit Polygon (NFP) is a basic geometric algorithm that can be used to find all possible collision-free positions for placing UV patches around an existing set of patches. The NFP algorithm may not work alone to complete UV packing tasks. For example, UV patches may need to be ordered so that they are placed one by one.

[0033] This disclosure provides a learning-assisted 2D irregular shape packing method. This method can achieve high packing quality with fewer requirements from the input. A subset of UV patches is iteratively selected and grouped into predefined shapes, such as roughly rectangular superpatches, which can inherently reduce the problem for bin packing. Based on the roughly rectangular superpatches, joint optimization is used to further improve the packing rate. To efficiently handle large problem instances (or large packing instances) with hundreds of patches, a deep neural policy can be trained to predict roughly rectangular patch subsets and determine the relative poses between subsets, which can lead to linear time scaling with respect to the number of patches. In this disclosure, the effectiveness of the provided method is demonstrated on three datasets for UV packing, and the provided method exhibits higher packing rates than several widely used related examples at competitive computational speeds.

[0034] Given a set of UV patches, the algorithm of the provided method can find the position and orientation of each patch. Thus, the patches may not collide, and the bounding boxes of the patches can be minimized. An exemplary bounding box in which the patches are packed can be shown in Figure 9.

[0035] In this disclosure, the provided method may take a set of UV patches as input. Each patch may be represented, for example, as an irregular, planar, manifold triangular mesh in a reference local frame. Any number of holes may be allowed in each UV patch. The output of the algorithm of the provided method may be a set of rigid body transformations. Each rigid body transformation may correspond to a respective patch. Thus, after transformation, the UV patches can be densely packed in a collision-free manner into a rectangular texture region in a reference global frame. The pipeline of the provided method may be shown in Figure 1.

[0036] As shown in Figure 1, the pipeline (100) may begin by filtering the input patch (102) into separate sets of patches, such as a primary set and a secondary set. Filtering may divide the input patch (102) into multiple sets based on size, such as by using one or more size thresholds. In one example, filtering the input patch (102) divides the input patch into a tiny (or secondary) set (104) and a non-tiny (or primary) set (106), the non-tiny set (106) may be processed in two steps in the first stage (100A). The tiny and non-tiny patches may be determined according to a predefined size threshold. In one example, the predefined size threshold may be a pixel value between 1 pixel and 5 pixels.

[0037] In the first stage (100A), the first step may iteratively query a high-level group selector network (HSN) (108) to generate a subset of patches, such as subset (110). The second step then queries a low-level sorter network (LSN) (112) to determine the order in which the patches should be packed into the subset. For example, the packing order of the patches in subset (110) may be determined by the LSN (112). The patches in the subset may be forwarded to a low-level attitude network (LPN) (114) in the determined order to determine the attitude of each patch. These two steps may group a subset of patches, such as subset (110), into a superpatch having a predefined shape. For example, subset (110) may be grouped into a roughly rectangular superpatch (116), which can then be inserted back into a non-miniature set (106). The first and second steps may be repeated to hierarchically group the patches in the non-minimal set (106). After the first stage (100A), the patches in the non-minimal set (106) may be grouped into superpatches with a predefined shape. For example, the non-minimal set (106) may be grouped into a roughly rectangular superpatch (118). In the second stage (100B), an algorithm such as a heuristic bin packing algorithm may be applied to assemble all the superpatches (118) into a first bounding box (120). Optimizations such as joint optimization may be applied to locally squeeze the patches within the first bounding box (120). For example, the spacing between patches may be squeezed to a predefined value. As shown in Figure 1, a patch (or non-micropatch) within the first bounding box (120) can be squeezed into a second bounding box (122) which is smaller than the first bounding box (120). Finally, gaps between non-micropatch within the second bounding box (122) can be filled using micropatch from the microset (104).For example, as shown in Figure 1, a micropatch (128) can be filled in the gap between non-micropatches (124) and non-micropatches (126).

[0038] In this disclosure, given a set of UV patches such as input patch (102), minute patches (e.g., patch (128)) can be filtered out first, and non-minute patches (e.g., patch (124)) can be processed in two stages (e.g., (100A) and (100B)). The first stage (e.g., (100A)) may aim to convert the non-minute patches (e.g., subset (110)) into a substantially rectangular superpatch.

number

[0039] The above process in the first stage (100A) is:

number

[0040] In this disclosure, a high-level selector network (e.g., HSN) is applied before low-level policies (e.g., LSN and LPN), although the low-level policies may be trained first due to data dependencies. The methods provided in this disclosure may be described in terms of the order of training.

[0041] Given a subset of H patches selected by HSN (e.g., HSN(108)), the first i-1 packed patches form a superpatch P in the global frame i-1 which can be represented as, where p i can be represented as the geometric region of the i-th patch in the local frame. Given p i and P i-1 , the low-level packing algorithm needs to select the translation t [Math.]] and rotation θ such that there is no collision for the packed shape at a high packing density i and rotation θ i needs to be selected. A related packing algorithm considers each patch independently, uniformly samples K rotations, and uses the NFP algorithm to consider possible translations under each rotation. A related packing algorithm has at least [Math.]] complexity, where N is the total number of edges in p i and P i , which can be a major bottleneck of the packing algorithm. Due to the short-sighted nature of the related packing algorithm, the packing density of the related packing algorithm may be suboptimal.

[0042] To address the shortcomings of related packing algorithms, the packing procedure of this disclosure can be modeled in some respects as a Markov decision process (MDP), and the LPN can be trained to maximize the packing rate through reinforcement learning. The LPN provided in this disclosure (e.g., LPN(114)) can identify not only the current patch but also future input patches and may exhibit a small optimality gap. In short, the MDP is a tuple<S,A,τ,r> This can be identified as an action space, which can iteratively model the decision-maker's procedure by observing the current system state in state space S and performing actions in action space A to change the environment (or state) in state space S. The state in state space S can then be updated via a state transition function τ, and the decision-maker can receive a reward r. However, for packing problems, the action space can include all possible patch translations and rotations, which can be difficult to handle for reinforcement learning. Therefore, in the methods provided in this disclosure, the action space may be limited to a small discrete subset, and then local optimizations can be applied to fine-tune the final pose of each patch.

[0043] already packed patch P i-1 For the i-th patch p i During the i-th iteration of the packing procedure in which the LPN can be packed, the current system state s in the state space S is i It is possible to observe (or identify) the current packed patch P. In this disclosure, LPN is the current packed patch P. i-1 And we can observe a set of up to H future patches that should be packed. The set of up to H patches that should be packed is, for example, p i ,···,p i+H-1 It can be expressed as follows: Therefore, the current system state s i teeth,

number

[0044] Single future patch p i Unlike myopia algorithms in related examples where patch p may be considered, this disclosure uses i ,p i+1 ,···,and p i+H-1 The entire ordered sequence of H future patches can be supplied to the LPN network. By supplying the entire ordered sequence of H future patches to the LPN network, the LPN network can be effectively guided to avoid myopia minimums.

[0045] In one respect, each patch can have any geometric shape. Therefore, the current system state s i (P i-1 Each patch in (including) can be rasterized into a 50x50 2D image. For each patch, the center of mass (COM) can be moved to the center of the image. Furthermore, each patch can be encoded into a 432-dimensional latent code using a shared fully convolutional network (FCN). The FCN can be configured to generalize the representative properties (or features) of the patch. Thus, patch p i The characteristics are,

number

number

[0046] Figure 2 is a schematic diagram of the activity space (200). As shown in Figure 2, patch P has already been packed. i-1 and the current patch p i Given, the current patch p i already packed patch P i-1 To pack into a global frame, actions from the action space are moved from the local frame to the global frame. i Coarse rotation θ i , and P i-1 Relative rotation φ i It is necessary to determine this. In one example, the action space (200) is θ i and φ i Each of these can have 16 candidate angles.

number

[0047] s i Observing this, LPN is s in the state space S i Action a in action space A i Policy function to map to

number

[0048] To address the two issues mentioned above, the policy function

number

number

[0049] This disclosure describes the state transition function

number

number

number

[0050] To train the LPN, π LPN This involves mapping the Q-values ​​of all 256 actions in action space A to an MLP (Multilayer Perceptron).

number

number

number

number

[0051] In this disclosure, since the LPN can only pack patches in a given order, the LSN may provide an optimal patch ordering for the LPN to achieve the best (or maximum) packing rate. For example, as shown in Figure 1, the LSN(112) may determine the packing order of patches in a subset (110) for the LPN(114) to operate patch packing. The patch sorting procedure is performed with the same state space S and the same reward signal r as the LSN.<S,A’,τ’,r> It can be modeled as another MDP represented as follows.

number

[0052] Neural networks need to understand the relative relationships between future patches in order to accomplish sorting tasks. Therefore, a graph attention network (GAT) module can be applied. A GAT is configured to transform patch features (e.g., node features) into high-level graph features, making it effective for solving sorting tasks.

number

[0053]

number

[0054]

number

[0055] Figure 3 shows an exemplary alpha shape (304) of the superpatch (302). As shown in Figure 3, the alpha shape (304) can be formed by filling internal gaps such as (306) within the superpatch (302).

[0056]

number

number

[0057]

number

number

number

[0058] As shown in Table 1, the algorithm can start from step 1, and the non-infinite set can be defined as S.

number

[0059] After a first stage, such as the first stage (100A) in FIG. 1, non-minute patches can be grouped into substantially rectangular superpatches. During a second stage (e.g., (100B)), superpatches having a rectangular shape can be assembled using a divide-and-conquer algorithm implemented in a mesh processing library such as the Trimesh library. The Trimesh library may comprise a set of utilities for reading, writing, and manipulating 3D triangular meshes. The superpatches may be assembled into a bounding box, such as the bounding box (120) in FIG. 1.

[0060] When the superpatches are assembled based on a bin packing algorithm, the joint pose of all patches in the superpatch can be adjusted such that the patches can be locally squeezed together via numerical optimization. For example, a bounding box (e.g., (120)) enclosing all M patches is bound(p1,...,p M M) and the numerical optimization can be formulated as shown in equations (8) and (9) below. [Numerical formula] Equation (8) can indicate that the boundary of the bounding box containing all M patches should be minimized, and Equation (9) indicates that any two patches within the bounding box should not overlap.

[0061] Furthermore, a barrier function technique can be applied to solve Equations (8) and (9). Therefore, a collision-free guarantee can be ensured, and it is guaranteed that the bounding box encloses all patches. Although Equations (8) and (9) relate to joint optimization, they are still efficient to solve because rigid motion is allowed for all patches.

[0062] Finally, a set of minute patches, such as a minute set (104), may be applied, which is filtered out at the beginning of the pipeline (100). The small patches in the minute set can be sorted in descending order of area and then fitted into the gaps and holes of the superpatch. In one example, the small patches may be fitted using a scanline algorithm. When a scanline algorithm is applied, the alpha shape of the superpatch can be replaced with the original patches in the superpatch. The original patches undergo the scanline algorithm so that the scanline algorithm can identify potentially useful gaps and holes.

[0063] Figure 4 shows multiple input patches packed at different stages. As shown in Figure 4, after the first stage, non-miniature patches of the input patch can be grouped into superpatches. Gaps, such as gap (402), may exist between non-miniature patches. After joint optimization, non-miniature patches can be squeezed and gaps can be reduced. After filling the gaps, miniature patches can be fitted into the gaps, achieving a packing rate of 76.5%. Referring further to Figure 4, in the baseline without filtering, miniature patches are not removed by filtering, and both miniature and non-miniature patches are transferred to the HSN, LSN, and LPN for patch packing. Thus, a packing rate of 71.9% can be achieved.

[0064] This disclosure describes how networks (e.g., HSN, LSN, and LPN) can be trained. For example, a training experiment may be performed on a computer with a 3.60GHz Intel E5-1650 12-core CPU and 32 GB of RAM. The training algorithm may be implemented via PyTorch. PyTorch can be a machine learning framework based on the Torch library. Based on PyTorch, GAT may be implemented. To train an LPN, DDQN is 10 6 This was applied to the experience buffer size of the transition tuple. In one example, to populate the experience buffer, H=4 was roughly 8×10 4A random packing problem is sampled. π LPN was updated using 2×10 4 epochs of stochastic gradient descent (SGD). The same procedure was applied to the training of π LSN . Both the learning rate of LPN and the learning rate of LSN were set to 10 -4 . HSN was trained on the collected dataset of 6×10 4 H patch subsets with precomputed ground-truth packing rates. In one example, HSN was updated using 500 epochs with a learning rate of 10 -3 and a batch size of 256. For each dataset, 70% of the data was used for training and the remainder for testing.

[0065] [[Mathematical Expression]] The numerical optimization in Equations (1), (2), (8), and (9) is implemented in C++, the maximum allowable iterations are set, and the initial step sizes are 10 3 and 10 -4 are set respectively. In super patch assembly, the aspect ratio between the width and height of the texture image area ranged from 1 to 2. To select an appropriate aspect ratio, the bin packing algorithm was run 10 times with different aspect ratios, and the one with the highest packing rate was selected.

[0066] Datasets were defined for packing patches. The methods provided in this disclosure were evaluated on three datasets of 2D UV patches obtained from UV unwrapping of 3D models using XAtlas. XAtlas may generate mutated patches with zero or negative regions, which were removed from the datasets. As shown in Figure 5, the datasets include a building dataset. The building dataset contains 86 artificial 3D building models, mostly with sharp features. Each building model yields 5 to 131 patches. The database includes an organic dataset. The organic database contains 81 3D organic models, mostly with sharp features. Each 3D organic model yields 9 to 200 patches. The database further includes a general dataset containing 221 3D general models from Thingi10k, each 3D general model yielding 4 to 200 patches.

[0067] In this disclosure, the methods provided are compared with three relevant examples. The first relevant example is an NFP-based packing technique that combines two heuristic methods, namely the maximum packing ratio and the lowest center of mass. Thus, the first relevant example may be referred to as NFP-Heuristic. Given a list of patches, the NFP heuristic first sorts all patches in descending order of area and then packs each patch sequentially. For a new patch, the NFP-Heuristic considers 16 rotations of the new patch and calculates the NFP for each rotation using the Minkowski sum to find a collision-free translation. Finally, the orientation that yields the highest packing ratio is selected. If two orientations yield the same packing ratio, the one with the lower center of mass position is selected. The NFP was calculated using a highly optimized shrinking convolution algorithm implemented in the Computational Geometry Algorithm Library (CGAL). A second relevant example is the packer algorithm implemented in the open-source software XAtlas, which implements aggressive simplification and acceleration techniques, allowing the packing algorithm to scale to problems with hundreds or thousands of patches. For example, XAtlas uses voxelized patches instead of piecewise linear patches, thereby allowing the use of a scanline algorithm instead of precise NFP calculations. A third relevant example was a method for generating multi-chart geometry images (MCGI) using Python. The main difference between XAtlas and MCGI lies in the heuristic approach, with XAtlas maximizing the packing rate and MCGI minimizing wasted area. A comparison of the provided methods with relevant examples can be seen in Table 2. [Table 2] As shown in Table 2, the packing rates (Min|Max|Avg) are summarized for the provided methods and related examples.

number

number

[0068] For each dataset, the packing rates of all algorithms for the test questions can be profiled. The profiled packing rates can be summarized in Table 2. As shown in Table 2, the algorithms provided in this disclosure consistently perform 5% to 10% better than relevant examples. To further justify the generality of the provided methods, the networks of the provided methods (e.g., HSN, LPN, and LSN) are used for general datasets.

number

[0069] This disclosure further explores ablation studies. For example, it analyzes aspects of learning support techniques in the provide method. First, the accuracy of the HSN was profiled and measured by the percentage of correctly ranked patch pairs. The HSN in the provide method achieves accuracy of 90.8%, 86.9%, and 84.6% for the building, organic, and general test sets, respectively. The trained HSN in this disclosure achieves high ranking accuracy for the building dataset, but lower accuracy for the organic and general datasets, where the patch shapes are more complex than those in the building dataset. Next, the low-level π in the provide method LPN and π LSN The packing rates were highlighted (or evaluated) separately. In one example, a random subset of H patches was sampled, and LL in the provided method was applied to patch packing. The LL in this disclosure was compared to relevant examples, and the averaged results over 2500 random problems are summarized in Table 3. As shown in Table 3, the deep reinforcement learning (DRL) based packing policy in the provided method still performs better than the relevant examples for smaller packing problems with H patches, which validates the need to use a learned packing policy as a low-level packer.

[0070] Furthermore, we compared the packing rates of LL under different horizons H. For example, four low-level algorithms were trained at H=2,...,5, and we found that the packing rates across 2500 random problems varied from 74.1%, 77.1%, 77.6%, and 77.0%, respectively. The provided method exhibited the worst performance at H=2, and the low-level policy of the provided method became myopic,

number

[0071] Table 3 shows a comparison of the average packing rates between the provided method's LL and related examples across 2500 random problems in H patches. Table 4 shows a comparison of the packing rates of algorithmic variations based on a typical dataset. [Table 3] [Table 4]

[0072] This disclosure provides a comparison of computational costs. The provided method achieves a better packing rate, but its computational efficiency may be lower than XAtlas due to repeated network evaluations. For a typical dataset, the average packing times for XAtlas, MCGI, NFP, and the provided method are 1.81 seconds, 33.52 seconds, 93.62 seconds, and 37.76 seconds, respectively. A breakdown of the provided method's performance on a typical dataset can be summarized in Table 5. As shown in Table 5, the computational bottleneck lies in scanline-based hole filling implemented in Python, including nested loops. The provided method could be accelerated, for example, if the scanline algorithm were implemented in native C++. The scalability of the provided method was evaluated when dealing with a large UV packing problem. For example, patches from several 3D models were combined, and each algorithm in the provided method was used to pack all patches into a single texture. We created datasets containing packing instances with 50, 100, 150, 200, 250, and 300 patches. We further implemented the provided method and related examples on the created databases. Figure 8 plots the computational overhead against the number of patches. As shown in Figure 8, the cost of NFP is much higher than that of the other algorithms due to the hyperlinear increase in computational complexity when calculating the Minkowski sum. By introducing a high-level selection policy, the provided method can scale linearly with respect to the number of patches, although it may be slower than XAtlas. Figure 9 shows a comparative example where 784 charts segmented from six animal chess pieces were packed into a single atlas. The packing times for MCGI, XAtlas, NFP, and the provided method are 180.68 seconds, 4.68 seconds, 2966.63 seconds, and 278.87 seconds, respectively. Thus, the provided method can achieve a better packing rate than NFP while requiring significantly fewer computational resources. [Table 5]

[0073] In this disclosure, user-controlled aspect ratios were evaluated. By default, the optimal aspect ratio of the texture image was sought to maximize the packing ratio. However, the provided method can be easily adapted to support user-specified packing ratios by transferring the user-specified packing ratio to the bin packing procedure. For comparison with XAtlas, experiments were conducted with an aspect ratio of 1. The results of these experiments are

number

number

[0074] This disclosure provides a learning-assisted shape packing algorithm for UV patches, where shape packing can be performed on one or more irregular shapes. For three datasets with varying topological and geometric properties, the provided algorithm can achieve a 5%–10% improvement in packing rate compared to algorithms provided by XAtlas, NFP, and MCGI. The provided algorithm can handle problem instances (or packing instances) with up to several hundred patches within an acceptable computational overhead for offline packing. By optimizing rigid body transformations for patches, the provided method respects input UV patch shapes and parameterizations that can be readily integrated into existing UV unwrapping pipelines.

[0075] Figure 10 shows a flowchart outlining a process (1000) for sound simulation according to one embodiment of the present disclosure. The process starts at (S1001) and proceeds to (S1010).

[0076] In (S1010), multiple UV patches are divided into a primary set of UV patches and a secondary set of UV patches based on a patch size threshold. For example, as shown in Figure 1, the input patch (102) may be filtered into a primary set (or non-minor set) (106) and a secondary set (or minor set) (104).

number

[0077] In (S1020), the primary set of UV patches is grouped into a plurality of superpatches. Each of the plurality of superpatches contains a different UV patch from the primary set that is packed together in a predefined shape. An exemplary embodiment of step (S1020) is shown in Figure 1, in which the non-microset (106) of UV patches is grouped into a plurality of superpatches (118) based on HSN, LSN, and LPN.

[0078] In (S1030), multiple superpatches are assembled together into a first bounding box. For example, as shown in Figure 1, multiple superpatches (108) are assembled into a bounding box (120) based on a heuristic bin packing algorithm.

[0079] In (S1040), the orientations of multiple superpatches are adjusted to reduce the spacing between them. For example, as shown in Figure 1, the orientations of the patches within the first bounding box (120) are adjusted based on joint optimization. Therefore, the spacing between multiple superpatches is reduced.

[0080] In (S1050), the gaps between UV patches in the primary set are filled with UV patches in the secondary set. For example, as shown in Figure 1, a minute patch such as patch (128) fills the gaps between UV patches in the non-minute set.

[0081] In one example, M subsets are selected from a primary set based on an HSN configured to identify subsets from a primary set to form superpatches of multiple superpatches. Each of the M subsets contains N UV patches, where M is a first positive integer and N is a second positive integer less than M. An estimated area-average packing rate is determined to be associated with each of the M subsets. The M subsets are sorted based on the estimated area-average packing rates. L subsets are determined from the M subsets corresponding to L maximum estimated area-average packing rates.

[0082] In one example, the UV patches within each of the L subsets are packed together with their respective subset bounding boxes. An area-average packing rate is determined for each of the L packed subsets. The first of several superpatches is determined from the L packed subsets. The first superpatch corresponds to the largest area-average packing rate among the determined area-average packing rates of the L packed subsets.

[0083] In one aspect, UV patches in a first subset of L subsets are packed into a first subset bounding box. For example, based on an LSN configured to determine the packing order of UV patches in the first subset, the UV patches in the first subset are organized into a linkage graph in which already packed patches and patches to be packed in the first subset are connected to each other. The node features of the UV patches in the first subset are input to a GAT to obtain the graph features of the UV patches in the first subset. The graph features are converted into corresponding Q values ​​via an MLP that includes input and output layers, as well as one or more hidden layers with stacked neurons. Based on the Q values, the first patch to be packed into the already packed patch is determined.

[0084] In one example, the node features of the UV patches in the first subset are determined based on an FCN into which the UV patches in the first subset are encoded into an F-dimensional latent space, where F is a positive integer.

[0085] In one aspect, a state space is determined based on an LPN configured to determine the orientation of UV patches in a primary set. The state space represents the positional states of already packed patches and patches to be packed in a first subset. An action space is determined for the patches to be packed. The action space represents candidate packing actions for the patches to be packed. Each of the candidate packing actions in the action space is applied to the first patch to be packed. An updated state space is determined corresponding to each of the candidate packing actions applied to the first patch to be packed. A reward value is determined corresponding to each of the updated state spaces associated with the first patch to be packed. Each reward value corresponds to the area-average packing rate associated with each candidate packing action. The packing action corresponding to the largest reward value is determined from the candidate packing actions in the action space. The first patch to be packed is packed by adjusting its orientation and distance according to the determined packing action.

[0086] In one example, the determined packing action includes a translation action to reduce the distance between the first patch to be packed and the already packed patch, and a rotation action to adjust the orientation of the first patch to be packed. The determined packing action is applied to the first patch to be packed according to collision-constrained local optimization so that the COM of the first patch to be packed and the COM of the already packed patch are reduced to predetermined values ​​and the first patch to be packed and the already packed patch do not overlap.

[0087] In one example, the packing ratio of a first packed subset of L packed subsets is determined based on the ratio of the area of ​​the first subset to the area of ​​the first bounding box of the first packed subset. The area-average packing ratio associated with a first packed subset of L packed subsets is determined based on the ratio of (i) the sum of the areas of UV patches in the superpatch containing the first packed subset in the primary set to (ii) the sum of the areas of the subset bounding boxes of the superpatch containing the first packed subset in the primary set.

[0088] In one respect, LPNs are trained based on the Q-learning algorithm by maximizing the expected cumulative reward. The expected cumulative reward is defined as follows:

number

[0089] In one aspect, the HSN is trained. To train the HSN, a first estimated area-average packing rate is determined for a first training subset from the primary set, and a second estimated area-average packing rate is determined for a second training subset from the primary set. A first area-average packing rate for the first training subset and a second area-average packing rate for the second training subset are further determined. The HSN is updated via a margin ranking loss, and the margin ranking loss is,

number

number

[0090] In one example, the orientation of multiple superpatches is adjusted by rotating and translating them so that the spacing between them is reduced, and the first bounding box shrinks to a second bounding box. The second bounding box corresponds to a bounding size minimized according to an optimization function so that the multiple superpatches within the second bounding box do not overlap.

[0091] Then, the process proceeds to (S1099) and terminates.

[0092] Process (1000) may be appropriately adapted. The steps of Process (1000) may be changed and / or omitted. Additional steps may be added. Any appropriate order of implementation may be used.

[0093] The techniques described above may be implemented as computer software using computer-readable instructions and may be physically stored on one or more computer-readable media. For example, Figure 11 shows a computer system (1100) suitable for carrying out a particular embodiment of the disclosed subject matter.

[0094] Computer software can be coded using any suitable machine code or computer language that can follow mechanisms such as assembly, compilation, and linking to create code that contains instructions that can be executed directly or via interpretation, microcode execution, etc., by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc.

[0095] Instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, and Internet of Things devices.

[0096] The components shown in Figure 11 for the computer system (1100) are essentially illustrative and are not intended to imply any limitation on the scope of use or functionality of computer software implementing embodiments of this disclosure. Furthermore, the configuration of the components should not be construed as having any dependencies or requirements relating to any one or combination of components shown in the exemplary embodiments of the computer system (1100).

[0097] The computer system (1100) may include certain human interface input devices. Such human interface input devices may respond to input from one or more human users via, for example, haptic input (keystrokes, swipes, data glove movements, etc.), audio input (voice, applause, etc.), visual input (gestures, etc.), olfactory input (not shown), etc. Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (voices, music, ambient sounds, etc.), images (scanned images, photographic images taken from still image cameras, etc.), and video (2D video, 3D video including stereoscopic video, etc.).

[0098] The input human interface device may include one or more of the following (only one of each is shown): keyboard (1101), mouse (1102), trackpad (1103), touchscreen (1110), data glove (not shown), joystick (1105), microphone (1106), scanner (1107), and camera (1108).

[0099] The computer system (1100) may include certain human interface output devices. Such human interface output devices may stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (1110), data glove (not shown), or joystick (1105), although there may also be tactile feedback devices that do not function as input devices), audio output devices (e.g., speakers (1109), headphones (not shown)), visual output devices (e.g., screens (1110) including CRT screens, LCD screens, plasma screens, OLED screens), each with or without touchscreen input functionality, each with or without tactile feedback functionality, some of which may be capable of outputting two-dimensional visual output or more than three-dimensional output via means such as stereographic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0100] The computer system (1100) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1120) with media such as CD / DVD (1121), thumb drives (1122), removable hard drives or solid-state drives (1123), legacy magnetic media such as tapes and floppy disks (not shown), and dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0101] Those skilled in the art should also understand that the term “computer-readable medium” as used in relation to the subject matter of this disclosure does not include a transmission medium, carrier wave, or other transient signal.

[0102] The computer system (1100) may also include interfaces (1154) to one or more communication networks (1155). These networks may be, for example, wireless, wired, or optical. They may also be local, wide-area, metropolitan, vehicle and industrial, real-time, or latency-tolerant. Examples of networks include local area networks such as Ethernet® and Wi-Fi; cellular networks including GSM®, 3G, 4G, 5G, and LTE; wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; and vehicle and industrial networks including CANBus. Certain networks generally require external network interface adapters attached to specific general-purpose data ports or peripheral buses (1149) (e.g., USB ports on the computer system (1100)), while other networks are generally integrated into the core of the computer system (1100) by being attached to a system bus, as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1100) may communicate with other entities. Such communication may be one-way reception only (e.g., broadcast television), one-way transmission only (e.g., CANbus to a CANbus device), or two-way communication, such as communication with other computer systems using local or wide-area digital networks. Specific protocols and protocol stacks may be used on each of these networks and network interfaces, as described above.

[0103] The aforementioned human interface devices, human-accessible memory devices, and network interfaces may be mounted on the core (1140) of the computer system (1100).

[0104] The core (1140) may include one or more central processing units (CPUs) (1141), graphics processing units (GPUs) (1142), dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) (1143), hardware accelerators for specific tasks (1144), graphics adapters (1150), and the like. These devices may be connected via a system bus (1148) along with read-only memory (ROM) (1145), random access memory (1146), and internal mass storage devices such as hard drives and SSDs (1147) that are not accessible to the internal user. In some computer systems, the system bus (1148) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (1148) or via a peripheral bus (1149). For example, a screen (1110) may be connected to a graphics adapter (1150). The architecture of the peripheral buses includes PCI, USB, and others.

[0105] The CPU (1141), GPU (1142), FPGA (1143), and accelerator (1144) can execute certain instructions that, in combination, may constitute the aforementioned computer code. This computer code may be stored in ROM (1145) or RAM (1146). Temporary data may also be stored in RAM (1146), while persistent data may be stored, for example, in internal mass storage (1147). Fast storage and retrieval to any of the memory devices may be enabled by using cache memory that can be closely associated with one or more CPUs (1141), GPUs (1142), mass storage (1147), ROM (1145), RAM (1146), etc.

[0106] Computer-readable media may contain computer code for performing various computer implementation operations. The media and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may be of a type that is well known and available to those skilled in the computer software technology.

[0107] For example, but not limited to, a computer system having an architecture (1100), specifically a core (1140), may provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) that runs software embodied in one or more tangible computer-readable media. Such computer-readable media may be user-accessible mass storage devices as described above, as well as media associated with specific storage devices of the core (1140) of a non-transient nature, such as core internal mass storage (1147) or ROM (1145). Software implementing various embodiments of the present disclosure may be stored in such devices and executed by the core (1140). The computer-readable media may include one or more memory devices or chips according to specific needs. The software may cause the core (1140), specifically the processor (including a CPU, GPU, FPGA, etc.) therein, to execute specific processes or specific parts of specific processes described herein, including defining data structures stored in RAM (1146) and modifying such data structures according to processes defined by the software. In addition, or as an alternative, a computer system may provide functionality as a result of logic hardwired to or otherwise embodied in a circuit (e.g., an accelerator (1144)), which may, in place of or in conjunction with software, perform certain processes or certain parts of certain processes described herein. References to software may, where appropriate, encompass logic, and vice versa. References to computer-readable media may, where appropriate, encompass circuits that store software for execution (such as integrated circuits (ICs)), circuits that embody logic for execution, or both. This disclosure encompasses any appropriate combination of hardware and software.

[0108] The use of “at least one of” or “one of” in this disclosure is intended to include any one or combination of the enumerated elements. For example, references to at least one of A, B, or C, at least one of A, B, and C, at least one of A, B, and / or C, and at least one of A-C are intended to include A only, B only, C only, or any combination thereof. References to one of A or B, and one of A and B are intended to include A or B, or (A and B). The use of “one of” does not exclude any combination of the enumerated elements where applicable, such as when the elements are not mutually exclusive.

[0109] While this disclosure has described several exemplary embodiments, there are many modifications, substitutions, and various alternative equivalents that fall within the scope of this disclosure. Those skilled in the art will therefore understand that numerous systems and methods not expressly shown or described herein can be devised to embody the principles of this disclosure and thus fall within its spirit and scope.

Claims

1. A method for processing multiple UV patches of a 3D model, The steps include dividing the plurality of UV patches into a primary set of UV patches and a secondary set of UV patches based on a patch size threshold, A step of grouping the UV patches of the primary set into a plurality of super patches, wherein each of the plurality of super patches includes a different UV patch from the primary set that is packed together in a predefined shape. The steps include assembling the aforementioned multiple superpatches together into a first bounding box, The steps include adjusting the orientation of the plurality of superpatches to reduce the spacing between the plurality of superpatches, The process includes the step of filling the gaps between the UV patches in the primary set with the UV patches in the secondary set. method.

2. The step of grouping the UV patches of the primary set is: Based on a high-level group selector network (HSN) configured to identify a subset from the primary set in order to form a superpatch of the multiple superpatches, A step of selecting M subsets from the primary set, each of the M subsets comprising N UV patches, where M is a first positive integer and N is a second positive integer less than M. The steps include determining the estimated area-average packing rate associated with each of the M subsets, The steps include: rearranging the M subsets based on the estimated area-average packing ratio; A step of determining L subsets from the M subsets corresponding to L maximum estimated area average packing rates, The method according to claim 1, further comprising:

3. The step of grouping the plurality of UV patches of the primary set is: The steps include packing the UV patches from each of the L subsets together into their respective subset bounding boxes, The steps include determining the area-average packing rate associated with each of the L packed subsets, A step of determining a first superpatch of the plurality of superpatches from the L packed subsets, wherein the first superpatch corresponds to the largest area-average packing rate among the determined area-average packing rates of the L packed subsets, The method according to claim 2, further comprising:

4. The step of packing the UV patches in each of the L subsets together into the respective subset bounding boxes further includes packing the UV patches in the first subset of the L subsets into the first subset bounding box, The step of packing the UV patches from the first subset of the L subsets into the first subset bounding box is: Based on a low-level sorter network (LSN) configured to determine the packing order of the UV patches in the first subset, The steps include organizing the UV patches in the first subset into a linked graph in which the already packed patches and patches to be packed in the first subset are linked to each other, The steps include inputting the node features of the UV patches in the first subset into a graph attention network (GAT) to obtain graph features of the UV patches in the first subset, The steps include converting the graph features into corresponding Q values ​​via a multilayer perceptron (MLP) which includes an input layer, an output layer, and one or more hidden layers having stacked neurons, The further step includes determining, based on the Q value, the first of the patches to be packed that should be packed into the already packed patch, The method according to claim 3.

5. The method according to claim 4, wherein the node features of the UV patch of the first subset are determined based on a fully convolutional network (FCN) in which the UV patch of the first subset is encoded in an F-dimensional latent space, where F is a positive integer.

6. The step of packing the UV patches in the first subset of the L subsets is: Based on a low-level attitude network (LPN) configured to determine the attitude of the UV patches in the aforementioned primary set, A step of determining a state space, wherein the state space represents the positional states of the already packed patches and the patches to be packed within the first subset, A step of determining an action space for the patch to be packed, wherein the action space represents candidate packing actions for the patch to be packed, The steps include applying each of the candidate packing actions in the action space to the first of the patches to be packed, The steps include determining an updated state space corresponding to each of the candidate packing actions applied to the first of the patches to be packed, A step of determining a reward value corresponding to each of the updated state spaces associated with the first of the patches to be packed, wherein each of the reward values ​​corresponds to the area-average packing rate associated with each of the candidate packing behaviors, The steps include determining the packing behavior corresponding to the maximum reward value from the candidate packing behaviors within the aforementioned behavioral space, The steps include packing the first patch to be packed by adjusting the orientation and distance of the first patch to be packed in accordance with the determined packing action, The method according to claim 4, further comprising:

7. The determined packing action includes a translation action to reduce the distance between the first patch to be packed and the already packed patch, and a rotation action to adjust the orientation of the first patch to be packed. The determined packing action is applied to the first patch to be packed according to collision-constrained local optimization such that the center of mass (COM) of the first patch to be packed and the COM of the already packed patch are reduced to predetermined values, and the first patch to be packed and the already packed patch do not overlap. The method according to claim 6.

8. The step of determining the area-average packing rate associated with each of the L packed subsets is: A step of determining the packing ratio of the packed L subsets of the packed first subset based on the ratio of the area of ​​the first subset to the area of ​​the first bounding box of the packed first subset, (i) determining the area-average packing rate associated with the packed first subset of the packed L subsets based on the ratio of the sum of the areas of the UV patches in the superpatch including the packed first subset in the primary set and (ii) the sum of the areas of the subset bounding boxes of the superpatch including the packed first subset in the primary set; The method according to claim 4, further comprising:

9. A step of training the LPN based on the Q-learning algorithm by maximizing the expected cumulative reward, wherein the expected cumulative reward is defined as follows: [Math 1] i indicates the i-th patch of the patch to be packed, a ｉ This indicates the determined packing action from the candidate packing actions applied to the i-th patch, where r(s ｉ , a ｉ ,s ｉ＋１ ) represents the reward value for the determined packing behavior being applied to the i-th patch, and π ＬＰＮ The packing action a determined above is ｉ This indicates the probability of being selected from the candidate packing behaviors, s ｉ This represents the state space before the i-th patch is packed into the already packed patch, s ｉ＋１ Step, which shows the state space after packing the i-th patch into the already packed patch, The method according to claim 6, further comprising:

10. The process further includes the step of training the HSN, The step of training the HSN is: The steps include determining a first estimated area-average packing rate for a first training subset from the primary set and a second estimated area-average packing rate for a second training subset from the primary set, A step of determining a first area-average packing rate for the first training subset and a second area-average packing rate for the second training subset, A step of updating the HSN via margin ranking loss, wherein the margin ranking loss is [Math 2] Equivalent to, [Math 3] These are the first area-average packing rate and the first estimated area-average packing rate for the first subset, respectively. [Math 4] The steps are, respectively, the second area-average packing rate and the second estimated area-average packing rate for the second subset, where ε is the smallest positive margin, and further include, The method according to claim 2.

11. The aforementioned adjustment step is, A step of adjusting the orientation of a plurality of superpatches by rotating and translating the plurality of superpatches so that the spacing between them is reduced so that the first bounding box shrinks to a second bounding box, wherein the second bounding box corresponds to a bounding size minimized according to an optimization function so that the plurality of superpatches in the second bounding box do not overlap. The method according to claim 1, further comprising:

12. It is a device, Multiple UV patches of a 3D model are divided into a primary set of UV patches and a secondary set of UV patches based on a patch size threshold. The UV patches of the primary set are grouped into a plurality of super patches, and each of the plurality of super patches includes a different UV patch from the primary set that is packed together in a predefined shape. The aforementioned multiple superpatches are assembled together into the first bounding box. The orientation of the multiple superpatches is adjusted to reduce the spacing between the multiple superpatches. The processing circuit includes a configuration that fills the gaps between the UV patches in the primary set with the UV patches in the secondary set. Device.

13. The aforementioned processing circuit is Based on a high-level group selector network (HSN) configured to identify a subset from the primary set in order to form a superpatch of the multiple superpatches, Select M subsets from the primary set, each of the M subsets containing N UV patches, where M is a first positive integer and N is a second positive integer smaller than M. Determine the estimated area-average packing rate associated with each of the M subsets. Based on the estimated area-average packing ratio, the M subsets are rearranged. The system is configured to determine L subsets from the M subsets corresponding to L maximum estimated area-average packing rates. The apparatus according to claim 12.

14. The aforementioned processing circuit is The UV patches in each of the L subsets are packed together in their respective subset bounding boxes. Determine the area-average packing ratio associated with each of the L packed subsets. A first superpatch is determined from the L packed subsets, and the first superpatch is configured to correspond to the largest area-average packing rate among the determined area-average packing rates of the L packed subsets. The apparatus according to claim 13.

15. The L subsets include the first subset, The aforementioned processing circuit is Based on a low-level sorter network (LSN) configured to determine the packing order of the UV patches in the first subset, The UV patches in the first subset are organized into a linked graph in which the already packed patches and patches to be packed in the first subset are linked to each other. The node features of the UV patches in the first subset are input to a graph attention network (GAT) to obtain the graph features of the UV patches in the first subset. The graph features are converted into corresponding Q values ​​via a multilayer perceptron (MLP) which includes an input layer, an output layer, and one or more hidden layers having stacked neurons. Based on the Q value, the system is configured to determine which of the patches to be packed should be packed into the already packed patch. The apparatus according to claim 14.

16. The apparatus according to claim 15, wherein the node features of the UV patches of the first subset are determined based on a fully convolutional network (FCN) in which the UV patches of the first subset are encoded in an F-dimensional latent space, where F is a positive integer.

17. The aforementioned processing circuit is Based on a low-level attitude network (LPN) configured to determine the attitude of the UV patches in the aforementioned primary set, Determining a state space, wherein the state space represents the positional states of the already packed patches and the patches to be packed within the first subset, Determining the action space for the patch to be packed, wherein the action space represents candidate packing actions for the patch to be packed, Applying each of the candidate packing actions in the action space to the first of the patches to be packed, Determining an updated state space corresponding to each of the candidate packing actions applied to the first of the patches to be packed, Determining a reward value corresponding to each of the updated state spaces associated with the first of the patches to be packed, wherein each of the reward values ​​corresponds to the area-average packing rate associated with each of the candidate packing behaviors, From the candidate packing behaviors within the aforementioned behavioral space, the packing behavior corresponding to the maximum reward value is determined. The system is configured to pack the first patch to be packed by adjusting the orientation and distance of the first patch to be packed according to the determined packing action, The apparatus according to claim 15.

18. The determined packing action includes a translation action to reduce the distance between the first patch to be packed and the already packed patch, and a rotation action to adjust the orientation of the first patch to be packed. The determined packing action is applied to the first patch to be packed according to collision-constrained local optimization such that the center of mass (COM) of the first patch to be packed and the COM of the already packed patch are reduced to predetermined values, and the first patch to be packed and the already packed patch do not overlap. The apparatus according to claim 17.

19. The aforementioned processing circuit is Based on the ratio of the area of ​​the first subset to the area of ​​the first bounding box of the packed first subset, the packing ratio of the packed first subset to the packed L subsets is determined. (i) The area average packing rate associated with the packed first subset of the L packed subsets is determined based on the ratio of (i) the sum of the areas of the UV patches in the superpatch including the packed first subset in the primary set and (ii) the sum of the areas of the subset bounding boxes of the superpatch including the packed first subset in the primary set. The apparatus according to claim 15.

20. The aforementioned processing circuit is The LPN is trained based on the Q-learning algorithm by maximizing the expected cumulative reward, and the expected cumulative reward is defined as follows: [Math 5] i represents the i-th patch of the patches to be packed, a ｉ represents a determined packing action from the candidate packing actions applied to the i-th patch, r(s ｉ , a ｉ , s ｉ＋１ ) represents a reward value for applying the determined packing action to the i-th patch, π ＬＰＮ represents the probability that the determined packing action a ｉ is selected from the candidate packing actions, s ｉ represents a state space before the i-th patch is packed into the already packed patches, s ｉ＋１ is configured to represent a state space after packing the i-th patch into the already packed patches, The apparatus according to claim 17.

Images