Lossy tensor compression technique using neural network-based tensor train decomposition

Abstract

To provide a lossy tensor compression technique using neural network-based tensor-train decomposition.SOLUTION: A lossy tensor compression method performed by a computer system according to an embodiment includes: inputting, to a neural network-based tensor-train (TT) decomposition model, mode indices of a target entry to be reconstructed; and obtaining tensor-train cores from the mode indices of the target entry to be reconstructed through the neural network-based tensor-train decomposition model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The following description relates to tensor lossy compression techniques. [Background technology]

[0002] A tensor is a multidimensional array, and the cost of storing a tensor is proportional to the total number of items in the tensor, which is multiplied by the length of each dimension. As the number of dimensions of a tensor increases or the dimensions become longer, the cost of storing the tensor also increases. Therefore, techniques for compressing tensors are essential.

[0003] Tensor-Train (TT) decomposition has recently attracted attention as a technique for compressing large matrices and tensors. However, TT decomposition has limited expressive power and its accuracy is low. Deep learning models have demonstrated excellent performance due to their ability to approximate arbitrary functions, and recurrent neural networks are widely used for problems where permutations are given as input. Summary of the Invention [Problem to be solved by the invention]

[0004] A lossy tensor compression method and system is provided that applies tensor train decomposition and recurrent neural networks to compress tensors into smaller sizes while reducing errors when the tensors are restored. [Means for solving the problem]

[0005] A tensor lossy compression method performed by a computer system may include inputting mode indices of a target item to be restored into a neural network-based tensor-train (TT) decomposition model, and obtaining tensor train cores from the mode indices of the target item to be restored using the neural network-based tensor-train decomposition model.

[0006] The neural network-based tensor train decomposition model may be configured for tensor train decomposition using a recurrent neural network.

[0007] The neural network-based tensor train decomposition model may be constructed by the steps of: searching for an embedding in an embedding layer to encode each mode index; obtaining a hidden embedding for the searched embedding through an LSTM layer; generating tensor train cores from the obtained hidden embedding through a linear layer; and learning to return an approximate item value through the product of the generated tensor train cores.

[0008] The obtaining step may include searching compressed data configured with parameters and a rearrangement function of the neural network-based tensor train decomposition model.

[0009] The obtaining may include initializing parameters and a rearrangement function of the neural network-based tensor train decomposition model, rearranging and convolving the order of input tensors to generate a tensor having a dimension equal to or greater than a predetermined standard, and updating the parameters and the rearrangement function of the neural network-based tensor train decomposition model to minimize an approximation error.

[0010] The obtaining step may include performing a process of convolving the input tensor into a tensor having a dimension equal to or greater than a predetermined standard in order to reduce the compressed volume of the tensor.

[0011] The obtaining step may include rearranging the tensors before the performed convolution process.

[0012] The obtaining step may include rearranging mode indices of the input tensors so that items with similar values ​​are placed near each other by sharing mode indices of the convolved tensors.

[0013] In a computer program for executing a tensor lossy compression method executed by a computer system, the tensor lossy compression method may include inputting mode indexes of target items to be restored into a neural network-based tensor-train (TT) decomposition model, and obtaining tensor train cores from the mode indexes of the target items to be restored using the neural network-based tensor-train decomposition model.

[0014] The computer system may include a memory and a processor connected to the memory and configured to execute at least one instruction recorded in the memory, wherein the processor may input mode indexes of a target item to be restored into a neural network-based tensor-train (TT) decomposition model, and obtain tensor train cores from the mode indexes of the target item to be restored using the neural network-based tensor-train decomposition model. [Effects of the Invention]

[0015] Compared to conventional techniques, our method can improve both compression capacity and reconstruction error. We also confirmed that the compression time of the algorithm increases linearly with the number of tensor elements, demonstrating excellent scalability. Furthermore, a case study using a traffic information dataset confirmed that the rearranged results matched the geographic information in a mode that included location information. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 10 is a diagram illustrating a neural network-based tensor train decomposition operation according to one embodiment. [Figure 2] FIG. 1 is a diagram illustrating a tensor convolution operation according to an embodiment. [Figure 3] FIG. 1 illustrates a computer system in one embodiment. [Figure 4] FIG. 1 is a diagram illustrating a neural network-based tensor train decomposition method according to an embodiment. [Figure 5] 1 is an algorithm to describe the overall operation of lossy tensor compression in one embodiment. [Figure 6] 1 is an algorithm illustrating an LSTM-based compression model that generalizes tensor train decomposition, according to one embodiment. [Figure 7] 10 is an algorithm illustrating the operation of updating a reordering function in one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0018] For lossy compression algorithms, it is important that the volume of compressed data is small. Furthermore, the error when restoring the original tensor from the compressed data must be small. Compression algorithms are designed to maximize these two goals. They must also be scalable so that the compression algorithm can be executed even on large tensors. A compression algorithm can be considered scalable if the time required for compression is proportional to the number of elements in the tensor.

[0019] Traditional tensor lossy compression algorithms require assumptions about each tensor to compress it properly. For example, tensor decomposition techniques assume that the input tensor can be represented with low error due to the low-rank structure they define. Video and image compression techniques require the assumption that the values ​​stored in tensors vary smoothly along their axes. Some algorithms specialize in sparse matrices where the majority of elements are zero. However, many tensor data representing real-world data do not conform to these assumptions. Therefore, there is a need for a general algorithm that can achieve good compression performance without requiring any assumptions about tensors.

[0020] In this embodiment, a powerful lossy compression algorithm is described that exhibits small compression capacity and low recovery error without requiring any assumptions about tensors, and is capable of compressing even large tensors.

[0021] FIG. 1 is a diagram illustrating the operation of a neural network-based tensor train decomposition in one embodiment.

[0022] A computer system may provide a neural network-based tensor train decomposition. The computer system may use a neural network to generalize tensor train decomposition to increase expressive power and alleviate limitations due to the sub-rank assumption. The computer system may use a neural network to construct a neural network-based tensor train decomposition model for tensor train decomposition. Unlike conventional tensor train decomposition in which a tensor core matrix is ​​fixed by the index of each dimension, the computer system uses a recurrent neural network to obtain a matrix that depends on different mode indexes of items. The neural network-based tensor train decomposition model can approximate an input tensor using a limited number of parameters, even when the input tensor has a multidimensional structure. Furthermore, the computer system can reduce the space required for the neural network-based tensor train decomposition model by convolving the input tensor into a multidimensional tensor. This can further reduce the number of parameters required for the neural network-based tensor train decomposition model. Finally, the computer system can rearrange the modal indices of the input tensors to reveal patterns that can be exploited in a neural network-based tensor train decomposition model for improved approximation.

[0023] To understand the discussion of tensor lossy compression, let's start with a basic concept.

[0024] Matrices and Tensors: Given a real-valued matrix M of size N1 × N2, the entry in the ith row and jth column is denoted as M(i,j). The dimension of a tensor indicates the number of modes. Let X be N1 × × N d Assuming that X is a d-dimensional tensor of size (i1, ,i d )th item is X(i1, ,i d ) will be displayed.

[0025] Tensor slicing and rearrangement: mode-j index i∈[N j ],

[0026]

number

[0027]

number

[0028]

number

[0029] Frobenius norm: Frobenius norm of X

[0030]

number

[0031]

number

[0032] Next, we will explain Tensor-Train Decomposition (TTD). Tensor train decomposition is a method of dividing a given d-th order tensor X into a d-th order tensor called a tensor train core.

[0033]

number

[0034]

number

[0035] where for all k

[0036]

number

[0037]

number

[0038]

number

[0039]

number

[0040]

number

[0041]

number

[0042]

number

[0043]

number

[0044] We also provide a formal definition for the tensor lossy compression problem as follows:

[0045] [Table 1]

[0046] The compressed data D consists of a parameter set θ and a rearrangement function set π of a neural network-based tensor train decomposition model.

[0047] Figure 5 shows an algorithm for explaining the overall operation of tensor lossy compression. First, the computer system initializes a neural network-based tensor train decomposition model θ and a reordering function π. Next, the computer system reorders and convolves X to generate a higher-dimensional tensor

[0048]

number

[0049] For simplicity, we assume that the input tensor X is already properly aligned. We then provide details on how to initialize and update the reordering function π.

[0050] The computer system provides a neural network-based tensor train decomposition model for accurately approximating tensors. The computer system can accurately approximate tensor items with a small number of parameters using the neural network-based tensor train decomposition model. The computer system proposes a neural network-based tensor train decomposition model that integrates a recurrent neural network and tensor train decomposition.

[0051] Instead of directly learning the core of the tensor decomposition as a free variable, we use the (i1, ,i d For each ()-th item, we obtain the tensor train core as the output of a neural network-based tensor train decomposition model that takes as input the item mode indices. The neural network, denoted θ, is trained to approximate the items as follows:

[0052]

number

[0053] where:

[0054]

number

[0055]

number

[0056] For detailed operation of the neural network-based tensor train decomposition model, please refer to Algorithm 2 in Figure 6 and Figure 1. The neural network-based tensor train decomposition model may be composed of an embedding layer, an LSTM, and a linear layer. Each mode index i k To encode the tensor train, the neural network-based tensor train decomposition model first uses an embedding layer E k (Lines 1 and 2) k Then, the embedding e k is fed to the LSTM layer, and the hidden embedding h for 1≦k≦d (line 3) is used. k We can obtain the hidden embedding h obtained using a linear layer (lines 4 to 7). k From Tensor Train Core T k After generating the tensor train, the core T k The product of these may be returned as the approximate item value (row 8).

[0057] The computer system uses a recurrent neural network structure model to allow for the dependency between the tensor train cores and the mode indices. However, in the neural network-based tensor train decomposition model, each tensor train core Tk depends not only on the mode-k index of the target item (as in TTD) but also on the mode-j index for all j≦k. In this embodiment, the use of LSTM has been exemplified, but GRU or Scaled Dot-product Attention can also be applied.

[0058] The neural network-based tensor train decomposition model θ has the following advantages over conventional tensor train decomposition (TTD):

[0059] Context: In neural network-based tensor train decomposition models, each tensor train core depends not only on the current mode index but also on all previous mode indexes. For example, consider approximating X(2,1,2) and X(1,2,2). In the neural network-based tensor train decomposition model, the tensor train core used for the third mode is different in the two cases. However, in tensor train decomposition, the index of the third mode is the same, so the same tensor train core is used in both cases. Because neural network-based tensor train decomposition models are contextual and nonlinear, they can model tensors that cannot be easily approximated, even if tensor train decomposition uses more parameters. This increased expressiveness reduces dependence on structural assumptions about the input tensors.

[0060] Nonlinearity: Neural network-based tensor train decomposition models integrate the nonlinear operations introduced by LSTM layers, whereas tensor train decomposition does not. This contributes to improving the expressive power of neural network-based tensor train decomposition models, allowing them to better approximate tensor terms.

[0061] Simplicity: The neural network-based tensor train decomposition model shares parameters for different modes (specifically, W and b in line 6 of Algorithm 2), so the model can be generated more simply with fewer parameters than tensor train decomposition. On the other hand, tensor train decomposition requires a unique tensor train core for each mode.

[0062] Space Complexity Analysis In Theorem 1, we present the capacity of the compressed output of a neural network-based tensor train decomposition model. The hidden dimension of the LSTM and the rank of the tensor train core are denoted by h and R.

[0063] Theorem 1 (Capacity of the compressed output of a neural network-based tensor train decomposition model). The capacity of the compressed output of a neural network-based tensor train decomposition model (i.e., the number of parameters) is

[0064]

number

[0065]

number

[0066] Proof. The embedding layer is

[0067]

number

[0068]

number

[0069]

number

[0070] Optimization method for θ :Problem 1, i.e.,

[0071]

number

[0072] The computer system may provide a convolution technique for a lightweight neural network-based tensor train decomposition model. By Theorem 1,

[0073]

number

[0074] Tensor train matrix format :The convolution technique is influenced by the form of the tensor train matrix and aims to convolve the matrix as a tensor to reduce the number of parameters in the tensor train decomposition.

[0075]

number

[0076]

number

[0077]

number

[0078] Here, for each 1≦k≦d

[0079]

number

[0080]

number

[0081]

number

[0082]

number

[0083] Tensor Train Tensor Format: The tensor-train matrix format can be naturally extended to a tensor. By convolving the input tensor into a multi-dimensional tensor with shorter mode lengths, the tensor-train matrix format is extended to a tensor. Such a process is called the tensor-train tensor format. It has a size of N1××N d Given a tensor X (where

[0084]

number

[0085]

number

[0086]

number

[0087] where for all k∈{1, ,d} and l∈{1, ,d'}

[0088]

number

[0089] We choose the new dimension d' so that the convolved tensor has higher dimension than the input tensor (i.e., d'>d), but with d'=O(logN max ) while maintaining N maxdenotes the maximum mode length in the input tensor X. For real tensors, this is generally possible since the mode lengths are typically much larger than the dimensions. For example, a 4th order tensor of 256 × 256 × 256 × 256 × 256 can be convolved into an 8th order tensor where each mode has length 16. It is not always possible to construct a convolved tensor that has the same number of entries as the input tensor but meets the above criteria. In such cases, the convolved tensor may contain additional entries, whose values ​​are ignored. For real tensors, we first multiply 2 by n for all k∈{1, ,d} and l∈{1, ,d'}. k,l and modifying some of them using integers up to 5 so that the input tensor and the convolved tensor have similar numbers of items. For example, for the PEMS-SF dataset (a 3-dimensional tensor of size 963 × 144 × 440), the values ​​assigned in the form of a d × d' matrix are as follows:

[0090]

number

[0091] This results in a 10th order tensor which is 8×8×8×8×8××20×4×4×4××2.

[0092]

number

[0093] Space Complexity Analysis : We analyze the impact of convolution on the number of parameters of θ (i.e., the compressed output capacity). Simply, for all k∈[d] and l∈[d'],

[0094]

number

[0095]

number

[0096] If we treat h and R as constants, then X folded The number of parameters of the neural network-based tensor train decomposition model is as follows:

[0097]

number

[0098] where:

[0099]

number

[0100]

number

[0101]

number

[0102] The computer system may provide a reordering technique to improve the fit of the convolved tensor. Essentially, the mode indices of the input tensors are reordered before convolution so that items with similar values ​​share mode indices and are located near each other in the convolved tensor. This reordering improves the ability of the neural network-based tensor train decomposition model θ to more effectively fit the convolved tensor, since θ generates tensor train cores based on the mode indices of the target items that serve as the model's input.

[0103] In the example in Figure 2, the closer two items are to the original tensor, the more indices they tend to share in the convolved tensor. In particular, the items in the black region share only the first mode index in the convolved tensor. The adjacent items in the gray region share both the first and second mode index in the convolved tensor. In the neural network-based tensor train decomposition model θ, to approximate two items, the k-th tensor train core T in Equation (3) is used. k It is important to note that θ is the same if the first k indices are the same. Consequently, two tensor train cores are shared for items in the gray region. Therefore, the closer two items are located in the original tensor, the more inputs and tensor train cores θ is likely to share for such items. This attribute helps θ approximate items more accurately and easily by placing similar items closer to each other. In the input tensor, we reorder the mode indices and reposition them. As shown in the example in Figure 2, the gray region contains more similar items after the reordering.

[0104] The mode index rearrangement is achieved by learning the rearrangement of the π function. The computer system alternately updates the neural network-based tensor train decomposition model θ and π. Next, we explain the initialization and update procedures for π. It is important to note that mode index alignment is associated with an increase in smoothness.

[0105] Initializing a sequence : Initialize a set of reordering functions π using a surrogate loss function. Reorder the mode-k index for all k (i.e., π k Optimization) is formulated as follows:

[0106]

number

[0107] where X (k) (i) is the i-th slice of X according to the k-th mode. Note that minimizing equation (6) makes adjacent slices similar.

[0108] The problem in (6) reduces to the metric salesman problem. Assume that each node represents a slice of a tensor, and each pair of nodes forms an edge with a weight equal to the Frobenius norm of the difference between the slices. Then, we minimize (6) using the optimal solution to the TSP problem over the entire resulting graph. However, since this is computationally NP-hard, we instead find a 2-approximation solution based on the fact that the Frobenius norm satisfies the triangle inequality. Next, we remove the trunk with the largest weight from the resulting solution. We then assign each i-th node in the resulting path to π k Set to (i).

[0109] Update the order based on θ (Algorithm 3 in Figure 7)After updating the neural network-based tensor train decomposition model θ, we update the reordering ensemble π, which rearranges the order based on the updated θ and the loss function of Problem 1. This update step is explained in Algorithm 3. X π For each k-th mode, we use the mode-k index (lines 17-18) to denote the rearranged tensor in X by π.

[0110]

number

[0111]

number

[0112]

number

[0113] If you change each pair in the above process, X πIt is constructed so that similar slices tend to be located close to each other. We search for such pairs using locality-sensitive hashing (LSH) for Euclidean distance. We sample half of the indices for each mode and vectorize the corresponding slices by points in multidimensional space. We project the vectorized slices onto a random vector (lines 6-10) and divide the projected points equally to generate buckets. Then we repeatedly select two points from the same bucket. Assuming the corresponding mode indices are i1 and i2, the two pairs are

[0114]

number

[0115]

number

[0116]

number

[0117] According to this embodiment, the compression output capacity, item recovery speed, and compression speed are theoretically analyzed. Simply, the input tensor

[0118]

number

[0119] Compressed output size In Theorem 2, we present the spatial complexity of the output generated by the neural network-based tensor train decomposition model. The complexity is given by

[0120]

number

[0121]

number

[0122]

number

[0123] Theorem 2 (Capacity of the compressed output). The capacity of the compressed output D = (θ, π) generated by Algorithm 1 is

[0124]

number

[0125] Proof. This model is

[0126]

number

[0127]

number

[0128]

number

[0129] Recovery Speed Yet another important aspect of a compression algorithm is the recovery speed. Theorem 3 formulates the recovery speed for the output of a neural network-based tensor train decomposition model. The complexity is O(dR 2 ) or higher than CPD's O(dR), but only logs in the mode length.

[0130] Theorem 3 (Recovery Speed). Given the output of Algorithm 1, D = (θ, π), the input tensor items (i.e., X π Algorithm 2) takes O((d+h 2 +hR 2 )logN max )it takes time.

[0131] Proof. For each item in X, X π It takes O(d) time to obtain the mode index from all i in Eq. (4). klis between 1 and d, and l is 1~d'=O(logN max ), so we need to calculate the value between

[0132]

number

[0133] Compression Speed :Analyze the speed of the compression process.

[0134] Theorem 4 (Compression Ratio). The time complexity of Algorithm 1 with T update steps is

[0135]

number

[0136]

number

[0137] Proof sketch. To initialize all rearrangement functions of π,

[0138]

number

[0139]

number

[0140] Concatenation for actual execution time : Measure the actual execution time for compression and recomposition to check the time complexity, which actually corresponds to the number of items.

[0141]

number

[0142] Memory Requirements : The memory space complexity required for compression does not exceed the memory requirements for the mini-batch, the compressed output, and the reordering function.

[0143] Theorem 5 (Memory requirements for compression). Algorithm 1 has

[0144]

number

[0145] FIG. 3 is a diagram illustrating a computer system in one embodiment.

[0146] Computer system 300 may include at least one of an interface module 310, a memory 320, or a processor 330. In one embodiment, at least one of the components of computer system 300 may be omitted and at least one other component may be added. In one embodiment, at least any two of the components of computer system 300 may be implemented as a single integrated circuit.

[0147] The interface module 310 may provide an interface for the computer system 300. In one embodiment, the interface module 310 includes a communication module, which may communicate with an external device. The communication module may establish a communication channel between the computer system 300 and the external device and communicate with the external device via the communication channel. The communication module may include at least one of a wired communication module or a wireless communication module. The wired communication module may connect to and communicate with the external device via a wired connection. The wireless communication module may include at least one of a short-range communication module or a long-range communication module. The short-range communication module may communicate with the external device via a short-range communication method. The long-range communication module may communicate with the external device via a long-range communication method. The long-range communication module may communicate with the external device via a wireless network. In another embodiment, the interface module 310 may include at least one of an input module or an output module. The input module may input a signal used by at least one component of the computer system 300. The input module may include at least one of an input device configured for a user to directly input a signal to the computer system 300, a sensor device configured to sense the surrounding environment and generate a signal, or a camera module configured to capture video and generate video data. The output module may include at least one of a display module for visually displaying information or an audio module for outputting information as an audio signal.

[0148] The memory 320 may store various data used by at least one component of the computer system 300. For example, the memory 320 may include at least one of a volatile memory or a non-volatile memory. The data may include at least one program and input or output data associated therewith. The program may be stored in the memory 320 as software including at least one instruction.

[0149] The processor 330 may execute a program in the memory 320 to control at least one component of the computer system 300. In this way, the processor 330 may process data or perform calculations. In this case, the processor 330 may execute instructions recorded in the memory 320.

[0150] The processor 330 may input mode indexes of target items to be restored to a neural network-based tensor-train (TT) decomposition model and obtain tensor train cores from the mode indexes of the target items to be restored using the neural network-based tensor-train (TT) decomposition model. The processor 330 may perform a process of convolving a tensor into a tensor of a dimension equal to or greater than a predetermined standard to reduce the compression volume of the tensor. The processor 330 may rearrange the tensor before convolving it.

[0151] FIG. 4 is a diagram illustrating a neural network-based tensor train decomposition method according to one embodiment.

[0152] The computer system may perform neural network-based tensor train decomposition 410, tensor convolution 420, and tensor rearrangement 430 operations.

[0153] In the neural network-based tensor train decomposition 410, the computer system may use a recurrent neural network (e.g., LSTM) to enhance the expressive power of the tensor train decomposition. The computer system may construct a recurrent neural network-based tensor train decomposition model. The computer system may input the mode index (position permutation / arrangement) of the target item (element) to the neural network-based tensor train decomposition model. The computer system may output matrices constituting the tensor train decomposition using the recurrent neural network-based tensor train decomposition model. As a result, the compressed tensor can be regarded as a recurrent neural network. In this case, each matrix constituting the tensor train decomposition is determined taking into account the position of the target item in multiple dimensions, allowing for more accurate restoration of the item compared to general tensor train decomposition.

[0154] Additionally, in tensor convolution 420, the computer system may perform a process of convolving a tensor into a more multidimensional tensor to further reduce the compressed size. The number of parameters in a neural network-based tensor train decomposition model increases in proportion to the total dimensionality of the tensor. Convolving a tensor into a multidimensional tensor can further reduce the total dimensionality of the tensor. For example, convolving a two-dimensional matrix with a side length of 32 into a five-dimensional matrix with a side length of 4 reduces the total dimensionality from 64 to 20.

[0155] Finally, in tensor rearrangement 430, the computer system may rearrange the pre-convolution tensors to further improve the reconstruction accuracy of the neural network-based tensor train decomposition model. The closer a particular item is to the pre-convolution tensor, the more similar its positional ordering in the convolved tensor used as the input of the neural network-based tensor train decomposition model becomes. Therefore, the matrices output by the neural network-based tensor train decomposition model become similar, and the reconstructed values ​​become similar. Therefore, rearranging the tensors so that similar values ​​are located nearby reduces the reconstruction error of the model. Along with the parameters of the neural network-based tensor train decomposition model, the ordering information for restoring the reordered tensors to the original tensors is also considered as the result of tensor compression.

[0156] According to an embodiment, the present invention can be applied to all fields that use tensors. For example, weather data measured at air quality monitoring stations, stock market records, and video characteristic data are expressed as tensors. Since the size of such tensors increases over time, there is room for effective compression techniques. This can reduce network traffic when transmitting tensor data over the Internet, and large tensors can be stored even on devices with limited memory capacity, such as smartphones and laptops.

[0157] The parameters of an artificial neural network are primarily expressed as tensors. In the case of large language models, which have been gaining attention in recent years for their high performance, based on openAI's chatGPT, hundreds of gigabytes of memory are required to store all of the parameters. This is a capacity that would be too large for most electronic devices used by average users. If such large-scale artificial neural networks could be compressed to be stored on electronic devices with limited memory and only the necessary parts could be restored and used, it would significantly reduce the computational load on central servers and the amount of internet traffic.

[0158] Companies that can apply this technology include those that hold large tensor data such as Netflix, Meta, YouTube, and Twitch, those that produce electronic devices with small memory such as Samsung, Apple, and Huawei, and those that provide services that utilize large-scale artificial neural networks such as Google and openAI.

[0159] The above-described devices may be implemented using hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also access, record, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, a single processing device may be described. However, those skilled in the art will understand that a processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0160] Software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or may independently or collectively instruct the processing device. The software and / or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium, or device to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed and stored and executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

[0161] Methods according to embodiments may be embodied in the form of program instructions executable by various computer means and stored on a computer-readable medium. The computer-readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions stored on the medium may be specially designed for the embodiments or may be readily available to those skilled in the art of computer software. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language code, such as that generated by a compiler, but also high-level language code executed by a computer using an interpreter, for example.

[0162] Although the embodiments have been described above based on limited examples and drawings, those skilled in the art will appreciate that various modifications and variations may be made from the above description. For example, the described techniques may be performed in an order different from that described, and / or the described system, structure, device, circuit, or other element may be coupled or combined in a manner different from that described, or may be substituted or replaced by other elements or equivalents, and still achieve suitable results.

[0163] Therefore, different embodiments are within the scope of the appended claims, provided that they are equivalent to the claims. [Explanation of symbols]

[0164] 300: Computer Systems 310: Interface module 320: Memory 330: Processor

Claims

1. 1. A tensor lossy compression method implemented by a computer system, comprising: inputting mode indices of a target item to be restored into a neural network-based tensor-train (TT) decomposition model; obtaining a tensor train core from a mode index of the target item to be restored using the neural network-based tensor train decomposition model; Including, The obtaining step includes: initializing parameters and a rearrangement function of the neural network-based tensor train decomposition model, rearranging and convolving the input tensors to generate a tensor having a dimension equal to or greater than a predetermined standard, and updating the parameters and the rearrangement function of the neural network-based tensor train decomposition model to minimize an approximation error; Including, In the step of obtaining, A tensor lossy compression method, comprising: a step of convolving an input tensor into a tensor having a dimension equal to or greater than a predetermined standard in order to reduce the compressed volume of the tensor; and a step of rearranging the tensor before the convolution step.

2. The method of claim 1, wherein the neural network-based tensor train decomposition model is configured for tensor train decomposition using a recurrent neural network.

3. The neural network-based tensor train decomposition model is searching for embeddings in an embedding layer to encode each mode index; obtaining a hidden embedding for the found embedding by an LSTM layer; generating a tensor train core from the obtained hidden embedding by a linear layer; and learning to return approximate item values ​​by summing cores of the generated tensor train; The tensor lossy compression method of claim 2, wherein the tensor lossy compression method is constructed by:

4. The obtaining step includes: searching for compressed data composed of parameters and a rearrangement function of the neural network-based tensor train decomposition model; The tensor lossy compression method of claim 1 , comprising:

5. The obtaining step includes: rearranging the mode indices of the input tensors so that items with similar values ​​are placed close to each other by sharing the mode indices of the convolved tensors; The tensor lossy compression method of claim 1 , comprising:

6. 1. A computer program for performing a tensor lossy compression method executed by a computer system, comprising: The tensor lossy compression method includes: inputting mode indices of a target item to be restored into a neural network-based tensor-train (TT) decomposition model; obtaining a core of a tensor train from a mode index of the target item to be restored using the neural network-based tensor train decomposition model; Including, The obtaining step includes: initializing parameters and a rearrangement function of the neural network-based tensor train decomposition model, rearranging and convolving the input tensors to generate a tensor having a dimension equal to or greater than a predetermined standard, and updating the parameters and the rearrangement function of the neural network-based tensor train decomposition model to minimize an approximation error; Including, In the step of obtaining, A computer program that executes a process of convolving an input tensor into a tensor of a dimension equal to or greater than a predetermined standard in order to reduce the compressed volume of the tensor, and executes a process of rearranging the tensor before executing the executed convolution process.

7. 1. A computer system comprising: memory, and a processor coupled to the memory and configured to execute at least one instruction stored in the memory; The processor: The mode index of the target item to be restored is input into a neural network-based Tensor-Train (TT) decomposition model, The neural network-based tensor train decomposition model obtains a tensor train core from the mode index of the target item to be restored; The processor: Initializing parameters and a rearrangement function of the neural network-based tensor train decomposition model, rearranging and convolving the input tensors to generate a tensor having a dimension equal to or greater than a predetermined standard, and updating the parameters and the rearrangement function of the neural network-based tensor train decomposition model to minimize an approximation error; A computer system characterized in that the processor executes a process of convolving an input tensor into a tensor of a dimension equal to or greater than a predetermined standard in order to reduce the compressed volume of the tensor, and a process of rearranging the tensor before executing the executed convolution process.

Images