A parallel tensor filling method and system based on ALS

Abstract

The present invention relates to the technical field of electronic digital data processing, and discloses a parallel tensor filling method and system based on ALS. The method collects characteristic information of a data set to be filled, establishes a tensor according to preset data characteristics, and generates an initial factor matrix of the tensor; obtains the number and position of effective values of the tensor, divides the sub-tensors according to the performance of the hardware device, obtains multiple sub-tensors for parallel processing, and stores the sub-tensors in COO format; allocates a processing unit to each sub-tensor, performs a reduction operation, and assigns the updated value of each row to the currently updated factor matrix; alternately updates three factor matrices, performs multiple rounds of iterations, and after each round of iteration, performs an outer product on the updated factor matrices to obtain a filled tensor, calculates the loss between the previous and next tensors, alternately optimizes each factor matrix, and implements missing value filling. The present invention improves parallel processing efficiency and reduces overall computing delay.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to an ALS-based parallel tensor filling method and system. Background Art

[0002] A tensor is a multidimensional array structure in mathematics that can be considered a high-dimensional generalization of vectors and matrices. Tensor filling refers to the process of estimating and filling missing values using algorithms when some data is missing. This method aims to restore data integrity and availability. It is primarily used to ensure that data of varying sizes has a uniform shape in deep learning, image processing, and data preprocessing, facilitating batch processing and model training. Network traffic monitoring is a critical task in modern network management and security, encompassing anomaly detection, traffic forecasting, and attack identification. In recent years, tensor filling technology has been gradually introduced into network traffic monitoring scenarios due to its advantages in high-dimensional data recovery and feature extraction.

[0003] Due to the limitations of data collection equipment and the complexity of network environments, the traffic data obtained is often missing or incomplete. Network traffic data can be modeled as a multidimensional tensor (e.g., [time × source IP × destination IP × port × protocol]). Low-rank tensor filling methods can be used to infer missing components and thus recover traffic patterns. In practical applications, due to the limitations of data collection equipment and the complexity of network environments, the network traffic data obtained is often missing or incomplete. However, as network scale continues to grow, processing large-scale traffic tensors faces computational bottlenecks, making GPU acceleration an important means of improving computational efficiency.

[0004] Most existing methods focus on optimizing storage formats, such as using sparse matrix storage, without introducing dynamic task allocation, which may lead to unbalanced computing load and affect overall parallel efficiency. Although some methods utilize GPU acceleration, the calculation process mainly relies on global memory access, without considering that access to shared memory is 100 times faster than access to global memory. Therefore, they do not optimize GPU memory access, resulting in high access overhead. There is also much room for improvement in computing performance in this regard.

[0005] Based on the above analysis, although the existing methods have achieved certain optimization in GPU parallel computing, there are still problems such as unbalanced computing load and large global memory access overhead, which limit the performance improvement of tensor filling algorithms on large-scale data. Summary of the Invention

[0006] The purpose of the present invention is to solve the above problems and design a parallel tensor filling method and system based on ALS.

[0007] A first aspect of the present invention provides a parallel tensor filling method based on ALS, the method comprising the following steps:

[0008] Collect the feature information of the data set to be filled, build a tensor based on the preset data features, and generate the initial factor matrix of the tensor;

[0009] Obtain the number and position of effective values of the tensor, divide the sub-tensors according to the performance of the hardware device, obtain multiple sub-tensors for parallel processing, and store the sub-tensors in COO format;

[0010] Assign a processing unit to a sub-tensor and perform a reduction operation, assigning the updated value of each row to the currently updated factor matrix;

[0011] The three factor matrices are updated alternately and iterated multiple times. After each round of iteration, the outer product of the updated factor matrix is taken to obtain the filled tensor, the loss between the previous and next tensors is calculated, and the factor matrices are optimized alternately to achieve missing value filling.

[0012] Optionally, in a first implementation method of the first aspect of the present invention, three features that have a real physical relationship with each other in the data set to be filled are obtained, a corresponding three-dimensional tensor is established, missing values are filled with 0 values, and feature scaling is performed using maximum and minimum value normalization to obtain a normalized tensor.

[0013] Optionally, in a second implementation of the first aspect of the present invention, load migration of computing tasks is achieved by dynamically adjusting sub-tensor boundaries. The effective value of the sub-tensor is used as the computing load indicator, and adjacent slices of the high-load sub-tensor are extended to the low-load area. The effective value is redistributed by adjusting the slice thickness, so that the adjusted sub-tensor load satisfies:

[0014] ;

[0015] in, is the tolerance parameter, is the optimal number of valid values for each sub-tensor, The payload value of the subtensor.

[0016] Optionally, in a third implementation of the first aspect of the present invention, the entire factor matrix is traversed to find all non-zero elements. For each non-zero element, its row index, column index and corresponding value are recorded, and these indices and values are stored in three arrays respectively.

[0017] Optionally, in a fourth implementation of the first aspect of the present invention, ALS-based CP decomposition is used for tensor filling, and the factor matrix is updated alternately to continuously fit the original tensor.

[0018] Optionally, in a fifth implementation of the first aspect of the present invention, when one of the factor matrices is updated, the other two factors are placed in a shared memory, and then the required corresponding valid values in the factor matrix are extracted from the shared memory.

[0019] Optionally, in a sixth implementation of the first aspect of the present invention, the corresponding matrix after a sub-tensor is matrixed is processed by a block, and each row of the matrix is processed by a thread of this block. After all blocks are calculated, the local results of all processing units are summarized to obtain the final reduction result.

[0020] A second aspect of the present invention provides an ALS-based parallel tensor filling system, the system comprising:

[0021] The generation module is used to collect the feature information of the data set to be filled, establish a tensor based on the preset data features, and generate the initial factor matrix of the tensor;

[0022] The partitioning module is used to obtain the number and position of effective values of the tensor, divide the sub-tensor according to the performance of the hardware device, obtain multiple sub-tensors for parallel processing, and store the sub-tensors in the COO format;

[0023] The update module is used to assign a processing unit to a sub-tensor and perform a reduction operation to assign the updated value of each row to the currently updated factor matrix;

[0024] The alternating optimization module is used to alternately update the three factor matrices for multiple rounds of iterations. After each round of iteration, the outer product of the updated factor matrix is taken to obtain the filled tensor, the loss between the previous and next tensors is calculated, and each factor matrix is alternately optimized to achieve missing value filling.

[0025] The third aspect of the present invention provides an ALS-based parallel tensor filling device, which includes a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory to enable the ALS-based parallel tensor filling device to perform the various steps of the ALS-based parallel tensor filling method as described in any one of the above items.

[0026] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor, implement the various steps of the ALS-based parallel tensor filling method as described in any of the above items.

[0027] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:

[0028] 1. The present invention proposes a parallel tensor filling method based on ALS, the steps of which include: constructing a tensor and initializing a factor matrix according to data characteristics; extracting the effective value of the tensor and its position; dividing the tensor into multiple sub-tensors according to hardware performance, storing them in COO format and processing them in parallel; during the processing, placing the fixed factor matrix in shared memory to speed up access; reducing and updating the factor matrix after each sub-tensor is processed; alternatingly updating the three factor matrices, iterating multiple times until the error converges or reaches the iteration limit, and finally reconstructing the filled tensor through the outer product. This method significantly improves the efficiency of parallel computing, reduces memory access overhead, and accelerates convergence through reasonable data partitioning and shared memory optimization, and is suitable for efficient filling of large-scale sparse tensors;

[0029] 2. By partitioning the original tensor, the present invention makes the effective value of each factor matrix have a small difference, so that the effective value to be processed by the computing unit of the subsequent processing sub-tensor is not much different, achieving load balancing and avoiding the load imbalance caused by direct partitioning by rows. This helps to reduce the waiting time between different computing units, improve parallel processing efficiency, reduce overall computing latency, and further enhance the scalability and stability of the system in large-scale sparse tensor processing scenarios;

[0030] 3. The present invention shortens the access time during calculation by using shared memory to store factor matrices. Since the access speed to shared memory is much higher than that to global memory, the performance can be improved by about a hundred times. In the process of updating the factor matrix, each sub-tensor processed in parallel needs to frequently access the corresponding rows of the other two factor matrices when updating each row of the current factor matrix. After loading these two fixed factor matrices into shared memory, the number of global memory accesses can be significantly reduced, reducing the pressure on memory bandwidth, thereby speeding up the iteration speed. This optimization strategy not only improves data access efficiency, but also enhances the execution performance of the overall algorithm under the GPU parallel architecture, and is particularly suitable for scenarios with high-frequency sparse access. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0032] Figure 1 A flowchart of the ALS-based parallel tensor filling method provided in an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of an ALS update factor matrix provided by an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of evenly dividing the original tensor provided by an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of a GPU memory architecture provided by an embodiment of the present invention;

[0036] Figure 5 A schematic diagram of dividing the original tensor based on thermal conductivity provided by an embodiment of the present invention;

[0037] Figure 6 A schematic diagram of COO compression storage provided by an embodiment of the present invention;

[0038] Figure 7 A schematic diagram of a parallel acceleration mechanism based on GPU shared memory provided by an embodiment of the present invention;

[0039] Figure 8 MAE comparison of different algorithms under different data sets at different sampling rates provided by the embodiment of the present invention;

[0040] Figure 9 Comparison of RMSE of different algorithms for different data sets at different sampling rates provided by the embodiment of the present invention;

[0041] Figure 10 Memory comparison of different algorithms under different data sets at different sampling rates provided by the embodiment of the present invention;

[0042] Figure 11 The time comparison of different algorithms under different data sets at different sampling rates provided by the embodiment of the present invention;

[0043] Figure 12 A schematic diagram of the structure of an ALS-based parallel tensor filling system provided in an embodiment of the present invention;

[0044] Figure 13 A schematic structural diagram of an ALS-based parallel tensor filling device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the accompanying drawings of the embodiments of the present invention to clearly and completely describe the technical solutions of the present invention. It should be understood that the embodiments are only used to illustrate a part of the specific implementation methods of the present invention, and are not used to limit the scope of protection of the present invention. For those of ordinary skill in the art, various modifications, substitutions or equivalent improvements made without departing from the essence of the present invention should all fall within the scope of protection of the present invention.

[0046] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 The embodiment of the present invention provides a flowchart of parallel tensor filling based on ALS, which specifically includes the following steps:

[0047] Step 101: Collect feature information of the data set to be filled, establish a tensor based on the preset data features, and generate an initial factor matrix for the tensor;

[0048] Step 102: Obtain the number and position of effective values of the tensor, divide the sub-tensor according to the performance of the hardware device, obtain multiple sub-tensors for parallel processing, and store the sub-tensors in COO format;

[0049] Step 103: Assign a processing unit to each sub-tensor and perform a reduction operation to assign the updated value of each row to the currently updated factor matrix;

[0050] Step 104: Alternately update the three factor matrices, perform multiple rounds of iterations, and after each round of iteration, perform outer product on the updated factor matrices to obtain the filled tensor. Calculate the loss between the previous and next tensors, and alternately optimize each factor matrix to achieve missing value filling.

[0051] Based on the low-rank assumption of multidimensional data, the tensor completion problem usually relies on the core tool CANDECOMP / PARAFAC (CP) decomposition, which decomposes the tensor Represented as the sum of a finite number of rank-one tensors:

[0052] ;

[0053] in the formula , and Corresponding to the factor matrices under mode 1, mode 2, and mode 3 respectively, , , are the vectors of the rth rank-one components of the tensor in the three modes. In the ALS framework, the factor matrix is updated mode by mode to minimize the reconstruction error. The key significance of CP decomposition is that through the low-rank approximation (i.e. ), you can use the tensor The potential multidimensional structure of is compressed and represented as a combination of several factor matrices, thus providing a parameterized basis for subsequent missing value recovery.

[0054] Based on the above CP decomposition model, the tensor completion problem can be formalized as a constrained optimization task. Given a partial observation tensor (Observation location index set Mark), the goal is to find a complete tensor , making it The observations at are consistent and satisfy the low-rank constraint:

[0055] ;

[0056] In the formula, Represents the projection operator (retain and clear all other elements to zero). represents the CP rank, is a predefined rank parameter, and the optimization problem fundamentally uses low-rank priors to reconstruct missing elements from partial observations. CP decomposition optimizes the variables Parameterized as the joint action of factor matrices, the tensor completion problem is transformed into 、 and Update issue.

[0057] ALS solves this optimization problem iteratively by alternately fixing some variables and updating the rest:

[0058] First, update the factor matrix , as shown in the formula below, fixed and , and according to mode 1 Expand to a matrix :

[0059] ;

[0060] in express and Khatri-Rao Plot, represents pseudo-inverse;

[0061] Then fix the remaining factor matrices respectively, expand and update them according to mode 2 and mode 3 respectively and , the specific formula is as follows:

[0062] ;

[0063] ;

[0064] The above process is repeated until convergence. The advantage of ALS is that each mode is updated independently and can be easily accelerated in parallel on a GPU.

[0065] Reference Figure 2 As shown, in the update factor matrix At the xth row (x,:), the original tensor is sliced in mode -1, and the forward slice (x,:,:) of the original tensor is used. All valid values in , Perform calculations; update factor matrix At the yth row (y,:), the original tensor is sliced in mode-2, using the (:,y,:)th vertical slice of the original tensor matrix. All valid values in , Perform calculations; similarly, update the factor matrix At the zth row (z,:), the original tensor is sliced in mode-3, and the forward slice (:,:,z) of the original tensor matrix is used. All valid values in , Since the update of each factor matrix depends only on the valid values of a specific pattern slice, these update operations can be performed independently and in parallel. This feature allows us to take advantage of parallel computing to accelerate the update of factor matrices when filling tensors with ALS.

[0066] Network traffic data naturally exhibits strong sparseness and non-uniform distribution characteristics due to dynamic changes in network topology and sampling differences in monitoring equipment. Figure 3 As shown, the original tensor Moderately significant values (such as traffic peaks and abnormal events) are often concentrated in a few local sub-tensors (such as specific time periods or routing nodes), while most areas are missing or have low values. This distribution characteristic poses a severe challenge to parallel tensor filling based on ALS.

[0067] Traditional GPU parallelization methods use an equal partitioning strategy to evenly divide the tensor into sub-tensors of the same size (e.g., each sub-tensor is a horizontal slice of (1, J, K) in mode 1), which are processed independently by different GPU thread blocks. However, due to the highly uneven distribution of valid values, some sub-tensors may contain dense valid values (e.g., sub-tensors contains 452 observation points), while other sub-tensors (such as The computational load is significantly reduced (with only 238 observation points). This load imbalance leads to two key issues:

[0068] 1. Thread block idle waiting: GPU thread blocks are executed synchronously, and the next round of iteration can only be entered after all sub-tensors are processed. Figure 2 As shown, processing The thread block (Thread Block 2) completes the calculation quickly due to the light load, but needs to wait for processing The thread block (Thread Block 1) runs for a long time, resulting in a waste of hardware resources;

[0069] 2. Iteration Convergence Delay: The overall convergence speed of the ALS algorithm is limited by the execution time of the slowest thread block. When the proportion of sparse sub-tensors is high, frequent thread waiting significantly increases the time taken for a single iteration, especially when processing large-scale tensors (such as I= ), the time cost increases exponentially.

[0070] While existing methods (such as cuTC and TensorD) compress data size through sparse storage, their static partitioning strategies fail to consider the spatial locality of significant values, leading to a decoupling of computational resource utilization from data distribution. For example, in bursty traffic monitoring scenarios, traditional partitioning may split instantaneous peak regions into multiple sub-tensors, forcing a single thread block to fail to fully capture local features, further exacerbating load imbalance and convergence instability. Therefore, designing a dynamic and adaptive sub-tensor partitioning method to balance computational load across thread blocks has become a core issue in improving the parallel efficiency of ALS.

[0071] To achieve results quickly, parallel computing is undoubtedly a very effective option. With technological advancements, especially in data processing and machine learning, GPUs (graphics processing units) are becoming increasingly popular. This is due to their powerful parallel processing capabilities, allowing them to perform numerous computational tasks simultaneously, significantly increasing processing speed. Compared to traditional CPUs, GPUs are more efficient at performing complex calculations, especially when processing large amounts of data. For example, in image processing, scientific computing, and deep learning, the GPU's parallel architecture enables multiple computing units to work simultaneously, significantly reducing computation time. Therefore, using GPUs for parallel computing has become a common and effective solution for modern computing tasks.

[0072] Down Figure 4 The figure shows a rough overview of the GPU's thread and memory structure. The GPU is divided into multiple blocks, each of which contains multiple threads. A thread is the smallest execution unit of the GPU. A thread has its own local memory and register memory. Register memory is the smallest memory area in the GPU, but it is very responsive. Each block has shared memory, a memory area accessible to all threads in that block. This is also very small, but the response speed is only 4-6 clock cycles. Global memory is the largest memory area in the GPU. Generally, data uploaded to the GPU is uploaded to global memory, but its access time is 100 times slower than that of shared memory.

[0073] Although the parallel computing capabilities of GPUs provide a hardware-accelerated foundation for tensor filling, existing methods suffer from shortcomings in memory access patterns and data locality optimization, which severely limit their actual performance. This is specifically manifested in the following two aspects:

[0074] 1. Risk of memory explosion for intermediate data: The ALS iteration process requires computing the Khatri-Rao product, whose explicit storage places extremely high demands on GPU memory capacity. For example, when J=15,000, K=29,000, and R=100, this matrix occupies approximately 16GB of memory (float32 format), and a single iteration may involve multiple such intermediate matrices. For devices with limited memory (such as the RTX4090, which only has a total of 24GB), this issue forces the algorithm to reduce the rank (RR) or use low-precision calculations, which directly compromises padding accuracy.

[0075] 2. Underutilization of data locality: The fast access characteristics of GPU shared memory are not effectively utilized. For example, when processing adjacent data, threads within the same block need to repeatedly access specific rows of data in blocks B and C. However, existing methods do not cache this frequently accessed data in shared memory, resulting in redundant global memory accesses. Theoretical analysis shows that caching local rows of factor matrices in shared memory can reduce data access latency to 1% of the original latency.

[0076] While existing optimization schemes (such as GIGATENSOR) avoid intermediate matrix storage through implicit computation, their element-by-element computation fails to incorporate the GPU memory hierarchy, significantly increasing thread synchronization overhead. Furthermore, while sparse storage formats (such as CSR and COO) reduce video memory usage, they require additional parsing of index data, further increasing computational complexity. Therefore, designing a parallel acceleration mechanism that balances video memory efficiency and data locality is a key challenge in improving GPU utilization.

[0077] To address the above challenges, this study proposes the following core optimization solutions:

[0078] Thermal conductivity load balancing: Based on the spatial distribution characteristics of effective values, an adaptive sub-tensor partitioning algorithm is designed to dynamically adjust the computational load of each GPU thread block to ensure balanced thread workloads in sparse and dense areas.

[0079] Shared memory mechanism: Shared memory is used to cache frequently accessed factor matrix data, reducing global memory access latency. At the same time, an implicit element-by-element calculation mode is used to replace explicit intermediate matrix storage, effectively alleviating video memory pressure.

[0080] The following is a further description of the two methods mentioned above:

[0081] See also Figure 5 Based on the schematic diagram of the original tensor partitioning by thermal conduction, this paper proposes a thermal conduction load balancing method. Its core idea is to learn from the physical process of heat diffusion from high-density areas to low-density areas in heat conduction, and to achieve load migration of computing tasks by dynamically adjusting the sub-tensor boundaries. Specifically:

[0082] Load representation: the number of valid values of sub-tensors As a computational load indicator;

[0083] Load spreading mechanism: high load subtensors The adjacent slices are extended to the low load area, and the effective value is redistributed by adjusting the slice thickness (such as extending along the j axis in the vertical slice mode) so that the adjusted sub-tensor load meets

[0084] ;

[0085] is the tolerance parameter. The original spatial position of the valid values remains unchanged, and only the sub-tensor partitioning boundaries are changed. This ensures that the row indices of the factor matrix strictly correspond to the original tensor during ALS updates, avoiding computational logic errors.

[0086] Boundary constraints: During the adjustment process, the original spatial position of the effective value remains unchanged, and only the division boundary of the sub-tensor is changed. This ensures that the row index of the factor matrix strictly corresponds to the original tensor during the ALS update, avoiding calculation logic errors.

[0087] Through the above dynamic adjustment, the computing load of each GPU thread block tends to be balanced, maximizing the utilization of parallel computing resources, thereby reducing thread idle waiting time and improving overall iteration efficiency.

[0088] In order to make parallelism more sufficient, set as many sub-tensors as possible, but the valid values in the sub-tensors cannot be too few. If the sub-tensors all become slices of the smallest unit on that dimension, then the division will not make much sense. The size of is related to the original tensor effective value and the hardware device. When mode is 1, it means that the tensor is horizontally sliced and spread to , when mode is 2, the tensor is sliced vertically and diffused to , when mode is 3, the tensor is sliced positively, that is, ;

[0089] ;

[0090] In a for loop, the sub-tensor is partitioned sequentially, adding one slice at a time. A while loop is embedded within the for loop to calculate whether the effective value in the slice is as close to the target effective value as possible. Generally, the number of effective values is almost impossible to completely equal the target effective value because the smallest unit of expansion or contraction is a (1, J, K) sub-tensor, and the effective values within it are uncertain. Similarly, load migration can be performed on vertical and frontal slices until the load of each sub-tensor is balanced within a certain range.

[0091] Non-zero values are partitioned based on a heat load balancing method, resulting in one or more batches of sub-tensors. These sub-tensors update their corresponding rows and can be processed in parallel. Each GPU block corresponds to a sub-tensor, and multiple blocks can process simultaneously. Of course, a mask must be added during calculations to distinguish valid values from zero values.

[0092] See also Figure 6 For sparse matrices, COO compression storage can be used first. The COO format is a common sparse tensor or sparse matrix storage method, which is mainly used to save the position of non-zero elements and their corresponding values. In the COO format, data is stored in three one-to-one arrays: row index array (row indices), column index array (column indices), and value array (values). For multi-dimensional tensors, it will also be expanded to multiple coordinate arrays. For example, a three-dimensional tensor will use i_indices, j_indices, and k_indices to represent the indexes in the three dimensions respectively. Specifically, Figure 5 As shown, the subtensors are first matrixed and expanded according to mode-1, mode-2, and mode-3, respectively. The position of each nonzero element is determined by the combination of its coordinates in each dimension, and its corresponding value is stored in the values array. Therefore, the information of the nth nonzero element can be fully described by row[n], col[n], and values[n] (or multiple coordinate arrays for higher-dimensional tensors). This structure is concise and intuitive, making it easy to traverse and reconstruct the original data.

[0093] According to the definition of Kronecker product, The dimension of .when and When the dimension is large, it may reach hundreds of thousands or even millions, resulting in huge storage overhead. Such a large number of intermediate results are difficult to store and compute directly on the GPU. GigaTensor can be used to avoid the problem of data expansion. The GigaTensor paper mentions that updating the factor matrix A can be divided into three steps:

[0094] ;

[0095] ;

[0096] ;

[0097] In order to solve the problem of intermediate data explosion caused by explicit matrix calculation in Step 1, GigaTensor is no longer explicitly constructed. , just construct , , , and then perform element-by-element calculations: use a for loop to Divide 、 、 The calculation is carried out in three steps, which greatly reduces the memory space overhead;

[0098] ;

[0099] ;

[0100] ;

[0101] After dividing the effective value according to the heat conduction load balancing method, a group of sub-tensors are obtained. It is known that these sub-tensors update the corresponding rows respectively, so these sub-tensors can be processed in parallel, that is, one block in the GPU processes one sub-tensor, and multiple blocks process in parallel.

[0102] like Figure 7 As shown, the main thing that needs to be optimized is the calculation of M1. According to the formula, updating the matrix M1 requires three for loops. If r is selected in the first loop, it is updated column by column. It can be found that no matter how much r is (as long as it is within the range), it will not affect the calculation of the second and third loops. Therefore, the first loop can be opened to allow one thread in the block to process an r value, and all threads can also process in parallel, thus further utilizing the parallel characteristics of the GPU to reduce loop nesting. In addition, the factor matrices B and C need to be used for calculations multiple times in the innermost loop, so the factor matrices B and C can be placed in the shared memory of the block. Of course, the result of each thread processing a column needs to be stored in an array, so this array can also be placed in shared memory for easy access by each thread.

[0103] The key lies in the writing of the kernel function. The specific idea is: first initialize bx and tx, which are the current block number and the current thread program number respectively; then put the factor matrices B, C and the array storing the current block results from the global memory into the shared memory, and the tx-th thread processes the tx-th column. Because the number of columns of the matrix is fixed to R, the number of threads required cannot exceed R. In the first-level for loop, all the elements of the rows of the column are processed in turn; in the second-level for loop, the accumulation in the corresponding formula is performed, which requires the specific elements of the i-th row of the factor matrices B and C (the i-th row of B). Column, row i of C The calculation is divided into three steps 、 、 To calculate, It will and The multiplication results are accumulated and added is a regularization parameter to prevent overfitting; each thread will get a , put it into a specific position of the array. Because each thread processes a different column, completing the second-level for loop only processes a specific row, so the threads will not conflict.

[0104] It should be explained that the above filling needs to fill the missing data in the dataset. The specific process is:

[0105] 1) Repeated iteration:

[0106] After each update of the three factor matrices, the outer product of the three factor matrices is performed to calculate the padded tensor, and the relative root mean square error (RMSE) between the padded tensor and the original tensor is calculated:

[0107] ;

[0108] And the relative mean absolute error (MAE):

[0109] ;

[0110] In the formula, , , and . Represents the total number of elements, Represents the first , , data, Represents the first , , data. is the missing value index set, is the total number of missing values. Repeat this process. There are usually two stopping conditions: one is that the difference in recovery loss (such as mean square error or other measurement indicators) between two consecutive iterations is less than a given threshold. In the embodiment of the present invention, it is set to ; The second is reaching the maximum number of iterations. When any of the stopping conditions is met, the iteration terminates.

[0111] 2) Obtain the converged factor matrix

[0112] When the above iterative process reaches convergence, the final factor matrices A, B, and C are obtained.

[0113] 3) Recover the tensor using the factor matrix

[0114] According to the formula , substituting the converged factor matrices A, B, and C into the original tensor to calculate the values of the missing items. Here, i, j, and k traverse the entire index range of the tensor. For the originally missing data locations, the restored values are calculated, thus achieving complete recovery of the entire original tensor data and obtaining the complete, padded tensor data for subsequent analysis and application.

[0115] like Figure 8-11 As shown, the experimental hardware is an NVIDIA GeForce RTX 4090 GPU. In a specific implementation, six public real-world datasets are used to verify the effectiveness of the algorithms involved in the embodiments of the present invention. These datasets include the Chengdu traffic dataset (size: 2016*128*64), the Beijing traffic dataset (size: 1530*128*128), the New York taxi dataset (NYC) (size: 30*30*1464), the PeMS traffic dataset (size: 228*288*44), the ECW dataset 08 (size: 797*720*16), and the dataset 09 (size: 1022*720*16). The six datasets are unevenly sampled to model the experimental conditions of data imbalance, with sampling rates ranging from 10% to 90%.

[0116] The rapid and efficient data recovery of GIGA_ALS was evaluated based on six aspects: the accuracy convergence of GIGA_ALS at different sampling rates using a balanced data partitioning method, and the impact of GIGA_ALS's high-dimensional structure on accuracy. GIGA_ALS was also compared with TensorD_ALS (which uses TensorFlow for CP decomposition), cuTC_sgd, cuTC_ALS, and pyCP_APR (Alternating Poisson Regression), all of which leverage the parallel nature of GPUs for CP decomposition.

[0117] To verify the effectiveness of the load balancing strategy, the time consumption and speedup ratio before and after using the strategy (Time1) are compared.

[0118] ;

[0119] Without thermal load balancing, a block processes a sub-tensor of a minimum unit slice, taking Time 1. With thermal load balancing, a block processes a sub-tensor (composed of multiple minimum unit slices), taking Time 2. In both cases, shared memory is not used to evaluate the contribution of thermal load balancing to model performance.

[0120] The speedup ratios of the ECW_08 dataset using thermal conduction load balancing at different sampling rates are 2.002, 2.553, 2.882, 3.033, 3.136, 3.457, 3.671, 3.661, and 3.865 respectively; the speedup ratios of the ECW_09 dataset using thermal conduction load balancing at different sampling rates are 2.487, 2.938, 3.404, 3.687, 3.815, 3.843, 4.146, 4.146, and 3.290 respectively; the speedup ratios of the CDTaxi dataset using thermal conduction load balancing at different sampling rates are 1.55, 1.82, 2.05, 2.18, 2.39, 1.69, 1.92, 1.51, and 1.98 respectively; The speedup ratios of NYC dataset using thermal conduction load balancing at different sampling rates are 1.428, 1.136, 1.384, 1.375, 1.382, 1.356, 1.281, 1.333, and 1.315 respectively; the speedup ratios of PeMS dataset using thermal conduction load balancing at different sampling rates are 1.182, 1.202, 1.352, 1.545, 1.545, 1.630, 1.520, 1.486, and 1.576 respectively; the speedup ratios of BJTaxi dataset using thermal conduction load balancing at different sampling rates are 1.389, 1.724, 1.753, 1.718, 1.511, 1.692, 1.829, 2.412, and 2.139 respectively.

[0121] Key conclusions: The load balancing strategy offers significant advantages in highly sparse scenarios: It achieves speedups of up to 3.865 and 4.146, demonstrating effective mitigation of load imbalance caused by "long strips." Verification of generalizability: For dense datasets such as NYC and PeMS, the strategy still improves efficiency (speedups of 1.315-4.052), but the magnitude of this improvement is affected by data distribution. For example, NYC achieves a speedup of 1.281 at a sampling rate of 0.7, while PeMS achieves 4.052 at 0.9, demonstrating that the strategy is sensitive to data locality. Furthermore, the load balancing threshold impacts computational resource utilization: Computational resource utilization is optimal when the effective value of a subtensor approaches a set threshold (ε). For example, CDTaxi experiences a sudden drop in speedup (1.69 → 1.125) at a sampling rate of 0.5, reflecting the need for dynamic adjustment of thresholds based on the data (the thresholds for each dimension of the same dataset were fixed in the experiments).

[0122] The speedup ratios of the shared memory acceleration mechanism for the dataset ECW_08 at different sampling rates are 1.375, 1.255, 1.472, 1.485, 1.566, 1.677, 1.465, 1.580, and 1.528 respectively; the speedup ratios of the shared memory acceleration mechanism for the dataset ECW_09 at different sampling rates are 1.047, 1.262, 1.409, 1.432, 1.380, 1.522, 1.603, 1.517, and 1.445 respectively; the speedup ratios of the shared memory acceleration mechanism for the dataset CDTaxi at different sampling rates are 1.101, 1.250, 1.370, 1.339, 1.192, 1.125, 1.176, 1.415, and 1.4 27; the speedup ratios of the shared memory acceleration mechanism for the NYC dataset at different sampling rates are 1.838, 1.861, 1.749, 1.774, 1.764, 1.826, 2.058, 2.064, and 1.987 respectively; the speedup ratios of the shared memory acceleration mechanism for the PeMS dataset at different sampling rates are 1.713, 2.067, 1.860, 1.961, 2.195, 3.022, 3.747, 4.070, and 4.052 respectively; the speedup ratios of the shared memory acceleration mechanism for the BJTaxi dataset at different sampling rates are 1.090, 1.078, 1.143, 1.287, 1.511, 1.277, 1.280, 1.088, and 1.225 respectively;

[0123] Key conclusions: Significantly reduced memory bandwidth pressure: The shared memory mechanism achieves a speedup of 4.052 for the PeMS dataset at a sampling rate of 0.9, indicating a significant improvement in memory access efficiency for high-frequency data. Limited benefits for sparse data: The speedup of BJTaxi is only 1.090 at a sampling rate of 0.1, due to insufficient shared memory utilization under sparse data, requiring coordinated optimization in conjunction with a load balancing strategy. Memory-computation balance: The speedup of ECW_08 and NYC increases with increasing sampling rate (1.375→1.528, 1.838→1.987), indicating that shared memory is more advantageous for intensive computing scenarios.

[0124] Depend on Figure 7-8 As can be seen, in terms of MAE and RMSE, GIGA_ALS achieves the lowest error (MAE reduction of 10%-15%) on datasets such as ECW_08 and PeMS, thanks to its heat conduction strategy that reduces local overfitting caused by load imbalance. Robustness verification demonstrates that in CDTaxi and NYC, the errors of the algorithms are similar, but GIGA_ALS converges the fastest (reducing the number of iterations by 20%), thanks to shared memory that accelerates matrix updates.

[0125] When the loss is basically the same, compare the memory and time ( Figure 9-10During calculations, TensorD replaces missing values with 0, rather than extracting valid values and their locations. Zeros also occupy memory, so TensorD's GPU memory usage does not change with changes in sampling rate. Because cuTC preprocesses the data and uses sparse matrix storage, its memory usage is relatively low. SGD processes only one or a few samples at a time, rather than the entire dataset. This means that minimal GPU memory is required to calculate gradients and update model parameters. Efficiency and resource comparison results: Memory usage: Taking the CDTaxi dataset as an example, GIGA_ALS's memory usage increased from 212MB to 308MB, an increase of 96MB, far less than other algorithms. This steady growth demonstrates that the algorithm is more adaptable to sampling rate changes and can maintain low memory usage across different sampling rates. Compared to TensorD, the average memory usage was reduced by 80.45%, and the maximum memory usage was reduced by 84.06%. Time performance: Across four datasets with different sampling rates, this algorithm achieved a time speedup of up to 90% compared to the other three baseline algorithms. While ensuring accuracy, the algorithm in this paper significantly improves computational efficiency and is suitable for large-scale network traffic monitoring scenarios with high real-time requirements and uneven data distribution.

[0126] See also Figure 12 , a schematic diagram of the structure of an ALS-based parallel tensor filling system provided in an embodiment of the present invention, the system includes:

[0127] The generation module is used to collect the feature information of the data set to be filled, establish a tensor based on the preset data features, and generate the initial factor matrix of the tensor;

[0128] The partitioning module is used to obtain the number and position of effective values of the tensor, divide the sub-tensor according to the performance of the hardware device, obtain multiple sub-tensors for parallel processing, and store the sub-tensors in the COO format;

[0129] The update module is used to assign a processing unit to a sub-tensor and perform a reduction operation to assign the updated value of each row to the currently updated factor matrix;

[0130] The alternating optimization module is used to alternately update the three factor matrices for multiple rounds of iterations. After each round of iteration, the outer product of the updated factor matrix is taken to obtain the filled tensor, the loss between the previous and next tensors is calculated, and each factor matrix is alternately optimized to achieve missing value filling.

[0131] Figure 13Figure 6 is a schematic diagram of the structure of an ALS-based parallel tensor filling device provided in an embodiment of the present invention. This ALS-based parallel tensor filling device 600 may vary significantly depending on configuration or performance. It may include one or more central processing units (CPUs) 610 (e.g., one or more processors), memory 620, and one or more storage media 630 (e.g., one or more mass storage devices) storing application programs 633 or data 632. The memory 620 and storage medium 630 may be either ephemeral or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations within the ALS-based parallel tensor filling device 600. Furthermore, the processor 610 may be configured to communicate with the storage medium 630, executing the series of instruction operations stored in the storage medium 630 on the ALS-based parallel tensor filling device 600 to implement the method provided in the above embodiment.

[0132] The ALS-based parallel tensor filling device 600 may further include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input and output interfaces 660, and / or one or more operating devices 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that Figure 13 The ALS-based parallel tensor filling device structure shown does not constitute a limitation on the computer device provided by the present invention, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0133] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions. When the instructions are executed on a computer, the computer executes the various steps of the ALS-based parallel tensor filling method provided in the above embodiments.

[0134] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0136] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A parallel tensor filling method based on ALS, characterized in that: The method comprises the following steps: Collect the feature information of the data set to be filled, build a tensor based on the preset data features, and generate the initial factor matrix of the tensor; Obtain the number and position of effective values of the tensor, divide the sub-tensors according to the performance of the hardware device, obtain multiple sub-tensors for parallel processing, and store the sub-tensors in COO format; Assign a processing unit to a sub-tensor and perform a reduction operation, assigning the updated value of each row to the currently updated factor matrix; The three factor matrices are updated alternately and iterated multiple times. After each iteration, the outer product of the updated factor matrix is taken to obtain the filled tensor. The loss between the previous and next tensors is calculated. The factor matrices are optimized alternately to achieve missing value filling. The load migration of computing tasks is achieved by dynamically adjusting the sub-tensor boundaries. The effective value of the sub-tensor is used as the computing load indicator. The adjacent slices of the high-load sub-tensor are extended to the low-load area. The effective value is redistributed by adjusting the slice thickness so that the adjusted sub-tensor load meets the following requirements: ； in, is the tolerance parameter, is the optimal number of valid values for each sub-tensor, The payload value of the subtensor.

2. The parallel tensor filling method based on ALS according to claim 1, characterized in that: Obtain three features that have a real physical relationship with each other in the data set to be filled, establish the corresponding three-dimensional tensor, fill the missing values with 0, use maximum and minimum value normalization to perform feature scaling, and obtain the normalized tensor.

3. The parallel tensor filling method based on ALS according to claim 1, characterized in that: Traverse the entire factor matrix, find all non-zero elements, and for each non-zero element, record its row index, column index, and corresponding value, and store these indices and values in three arrays respectively.

4. The parallel tensor filling method based on ALS according to claim 1, characterized in that: Use ALS-based CP decomposition for tensor filling, alternately update the factor matrix, and continuously fit the original tensor.

5. The ALS-based parallel tensor filling method according to claim 4, wherein: When updating one of the factor matrices, the other two factors are placed in the shared memory, and then the corresponding valid values required in the factor matrix are extracted from the shared memory.

6. The ALS-based parallel tensor filling method according to claim 1, wherein: The corresponding matrix after a sub-tensor is matrixed is processed by a block, and each row of the matrix is processed by the thread of this block. After all blocks are calculated, the local results of all processing units are summarized to obtain the final reduction result.

7. A parallel tensor filling system based on ALS, characterized in that: The system includes: The generation module is used to collect the feature information of the data set to be filled, establish a tensor based on the preset data features, and generate the initial factor matrix of the tensor; The partitioning module is used to obtain the number and position of effective values of the tensor, divide the sub-tensors according to the performance of the hardware device, obtain multiple sub-tensors for parallel processing, and store the sub-tensors in the COO format: the load migration of computing tasks is achieved by dynamically adjusting the sub-tensor boundaries. The effective value of the sub-tensor is used as the computing load indicator, and the adjacent slices of the high-load sub-tensor are extended to the low-load area. The effective value is redistributed by adjusting the slice thickness so that the adjusted sub-tensor load meets the following requirements: ； in, is the tolerance parameter, is the optimal number of valid values for each sub-tensor, is the effective load value of the sub-tensor; The update module is used to assign a processing unit to a sub-tensor and perform a reduction operation to assign the updated value of each row to the currently updated factor matrix; The alternating optimization module is used to alternately update the three factor matrices for multiple rounds of iterations. After each round of iteration, the outer product of the updated factor matrix is taken to obtain the filled tensor, the loss between the previous and next tensors is calculated, and each factor matrix is alternately optimized to achieve missing value filling.

8. A parallel tensor filling device based on ALS, characterized in that: The ALS-based parallel tensor filling device includes a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the ALS-based parallel tensor filling device performs the various steps of the ALS-based parallel tensor filling method as described in any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the various steps of the ALS-based parallel tensor filling method as described in any one of claims 1 to 6 are implemented.

Images