Task execution method and program product

Abstract

This application provides a task execution method and program product, which can be applied to the field of computer technology. The task execution method includes: acquiring multiple target data to be processed by a processor; and determining multiple vector widths selectable by the processor for processing the multiple target data, wherein the vector width represents the amount of data that the processor can process in parallel at one time, and the multiple vector widths are determined based on the physical bit width of the hardware unit used for parallel processing in the processor and the data type of the target data; determining at least one target vector width from the multiple vector widths in descending order, wherein the total amount of data that the processor can process corresponding to the at least one target vector width is equal to the total amount of data of the multiple target data; dividing the multiple target data into at least one data group corresponding to the at least one target vector width; and executing at least one batch of processing tasks on each of the at least one data group.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, more particularly, to a task execution method and a program product. BACKGROUND

[0002] For the Graphics Processing Unit (GPU) computing framework and deep learning acceleration library, the related method usually adopts a unified vectorization processing strategy for the data to be processed, or adopts a data padding or data truncation method to process data that is not suitable for the unified vectorization processing strategy. However, the related method at least has the following problems: the padding method will introduce redundant calculation and destroy the integrity of the original data, and the truncation method will cause part of the data to be unable to be processed. SUMMARY

[0003] In view of the above problems, the present application provides a task execution method and a program product.

[0004] According to a first aspect of the present application, a task execution method is provided, comprising: obtaining a plurality of target data to be processed by a processor; and determining a plurality of vector widths selectable for the processor to process the plurality of target data, wherein the vector width represents the amount of data that the processor can process in parallel at a time, and the plurality of vector widths are determined according to the physical bit width of a hardware unit for parallel processing in the processor and the data type of the target data; determining at least one target vector width from the plurality of vector widths in descending order of the plurality of vector widths, wherein the total amount of data that the processor can process corresponding to the at least one target vector width is equal to the total data amount of the plurality of target data; dividing the plurality of target data into at least one data group corresponding to the at least one target vector width; and respectively executing at least one batch of processing tasks on the at least one data group, wherein one batch of processing tasks is used to process one data group.

[0005] A second aspect of the present application provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0006] A third aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the steps of the above method.

[0007] A fourth aspect of the present application further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the above method. BRIEF DESCRIPTION OF DRAWINGS

[0008] The above and other objects, features and advantages of the present application will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0009] Figure 1 An application scenario diagram of a task execution method and program product according to an embodiment of the present application is shown;

[0010] Figure 2 A flowchart of a task execution method according to an embodiment of the present application is shown;

[0011] Figure 3 A flowchart of a task execution method for non-continuous data according to an embodiment of the present application is shown;

[0012] Figure 4 A flowchart of a task execution method according to another embodiment of the present application is shown;

[0013] Figure 5 A structural block diagram of a task execution apparatus according to an embodiment of the present application is shown;

[0014] Figure 6 A block diagram of an electronic device suitable for implementing a task execution method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0015] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely exemplary of the present application, and is not intended to limit the scope of the present application. In the following detailed description of the embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0016] The terms used herein are merely used to describe specific embodiments, and are not intended to limit the present application. The terms "include" and "have" and the like used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0017] All terms used herein, including technical and scientific terms, have the same meanings as those generally understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the present description, and should not be interpreted in an idealized or overly formal manner.

[0018] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0019] In modern deep learning and scientific computing, GPU-accelerated tensor operations are a core foundation supporting large-scale model training and inference tasks. With the exponential growth of model parameters and the increasing complexity of data dimensions, higher demands are placed on the computational efficiency of tensor operations. Especially in typical application scenarios such as computer vision and natural language processing, element-level operations, as the most basic computational mode, directly impact overall computational performance. Related GPU vectorization methods face two main problems when handling tensor operations in real-world scenarios: First, when the total amount of data is not an integer multiple of the vector width, traditional methods cannot effectively handle the remaining data, leading to wasted computational resources and performance degradation. Here, the remaining data refers to data in the total amount of data that is not an integer multiple of the vector width. Second, when facing non-contiguous memory access patterns such as step access and interval access, related vectorization techniques are ineffective, significantly reducing GPU memory bandwidth utilization.

[0020] Specifically, on the one hand, related methods usually use data padding or truncation to process data that does not meet the requirement of being an integer multiple of the vectorization width. However, data padding introduces redundant calculations and destroys the integrity of the original data, while data truncation results in some data not being processed.

[0021] On the other hand, for non-contiguous memory access, two common approaches are used: one is to completely abandon vectorization and degenerate into element-wise serial processing; the other is to rearrange the data before performing vectorization computation. However, completely abandoning vectorization cannot fully utilize the GPU's Single Instruction Multiple Data (SIMD) computing units, resulting in wasted hardware resources. Furthermore, data rearrangement itself incurs additional overhead, and in some scenarios, the cost of rearrangement may outweigh the benefits of vectorization. Moreover, fixed processing strategies cannot be dynamically adjusted and optimized based on actual data characteristics, making it difficult to maintain optimal performance across different application scenarios.

[0022] In view of this, embodiments of this application provide a task execution method, comprising: acquiring multiple target data to be processed by a processor, and determining multiple vector widths selectable by the processor for processing the multiple target data, wherein the vector width represents the amount of data that the processor can process in parallel at one time, and the multiple vector widths are determined according to the physical bit width of the hardware unit used for parallel processing in the processor and the data type of the target data; determining at least one target vector width from the multiple vector widths in descending order, wherein the total amount of data that the processor can process corresponding to the at least one target vector width is equal to the total amount of data of the multiple target data; dividing the multiple target data into at least one data group corresponding to the at least one target vector width; and executing at least one batch of processing tasks on the at least one data group respectively, wherein one batch of processing tasks is used to process one data group.

[0023] Figure 1 The diagram illustrates an application scenario of the task execution method and program product according to embodiments of this application.

[0024] like Figure 1 As shown, the application scenario 100 according to this embodiment may include multiple target data and multiple vector widths. For example, the multiple target data may include target data 1, target data 2... target data M, and the multiple vector widths may include vector width 1, vector width 2... vector width K. The multiple vector widths may be different, for example, the multiple vector widths may include: vector width of 8, vector width of 4, vector width of 2, etc.

[0025] For example, the process involves acquiring multiple target data sets to be processed by the processor, and determining multiple vector widths that the processor can choose to process the multiple target data sets. Each vector width represents the amount of data the processor can process in parallel at a time. The multiple vector widths are determined based on the physical bit width of the hardware units used for parallel processing in the processor and the data type of the target data. At least one target vector width is determined from the multiple vector widths in descending order of size. The total amount of data that the processor can process corresponding to the at least one target vector width is equal to the total amount of data in the multiple target data sets. The multiple target data sets are divided into at least one data group corresponding to the at least one target vector width. At least one batch of processing tasks is executed on each of the at least one data group, where each batch of processing tasks is used to process one data group. It should be noted that... Figure 1 The target vector width and the amount of target data are for illustrative purposes only.

[0026] The following will be based on Figure 1 The described scene, through Figures 2-4 The task execution method of the embodiments of this application will be described in detail.

[0027] Figure 2 A flowchart of a task execution method according to an embodiment of this application is shown.

[0028] like Figure 2 As shown, the task execution method of this embodiment includes operations S210 to S240.

[0029] In operation S210, multiple target data to be processed by the processor are acquired, and multiple vector widths that the processor can select to process the multiple target data are determined. Here, the vector width represents the amount of data that the processor can process in parallel at one time, and the multiple vector widths are determined based on the physical bit width of the hardware unit in the processor used for parallel processing and the data type of the target data.

[0030] For example, the embodiments of this application can be used in scenarios that require processing batch data, such as scenarios that process multiple target data. Regarding processor hardware performance, taking a graphics processing unit (GPU) as an example, a GPU can be used to process batch data in parallel. Batch processing maximizes the utilization of the SIMD unit and fully utilizes the GPU's hardware performance. However, in scenarios where only one data point is processed, the SIMD unit struggles to perform its function. Regarding vector width, since the amount of data a processor can process in parallel at one time has a fixed range (e.g., 4 or 8 data points), it cannot process data outside this range. Vector width determines the amount of data to be processed in each batch of parallel processing for multiple target data points. However, in the case of only one data point, the computational overhead of a single data point is extremely low. Matching the single data point with the vector width incurs an instruction overhead far exceeding the data processing itself, resulting in redundant overhead.

[0031] For example, the processor may include a graphics processing unit (GPU). The target data may correspond to at least one data type, which may include single-precision floating-point numbers, half-precision floating-point numbers, etc. The hardware unit for parallel processing may include a SIMD unit, which can be used for parallel data processing, such as vectorization processing. Vectorization processing may include: packaging multiple target data into a continuous, regular vector, and completing the calculation or read / write operation in one go. For example, the amount of data that the processor can process in parallel at one time has multiple selectable ranges, such as processing 8 data points or 4 data points in parallel at one time, and correspondingly, multiple vector widths can be determined, such as a vector width of 8, a vector width of 4, etc.

[0032] According to embodiments of this application, the physical bit width of the hardware unit for parallel processing and the data type of the target data determine the vector width, eliminating the need to rewrite code for different hardware and improving hardware adaptability.

[0033] In operation S220, at least one target vector width is determined from the multiple vector widths in descending order, wherein the total amount of data that the processor can process corresponding to the at least one target vector width is equal to the total amount of data of the multiple target data.

[0034] For example, following the order of multiple vector widths from largest to smallest, first match the total amount of target data with the multiple vector widths sequentially to determine the target vector width corresponding to the first batch; then calculate the current remaining data amount (i.e., the total data amount minus the data amount of the matched batches), and continue to match the current remaining data amount with the remaining vector widths in descending order to determine the target vector width of the corresponding batch, and update the remaining data amount; repeat the operation of calculating the current remaining data amount and matching the current remaining data amount with the remaining vector widths to determine the target vector width of the corresponding batch until the updated remaining data amount is 0, thus completing the determination of the target vector width for all batches.

[0035] For example, if the total data volume of multiple target data sets is 10, and the available vector widths are 8, 4, 2, and 1, we can first match the maximum vector width of 8. Since 10 ≥ 8, the target vector width for the first batch is determined to be 8. The current remaining data volume is: total data volume 10 - target vector width of the first batch 8, which is 2. Next, we match the current remaining data volume 2 with the next vector width 4. Since 2 is less than 4, the current remaining data volume 2 does not match the vector width 4, so we continue with a downgraded matching, that is, matching the current remaining data volume 2 with the vector width 2. Since the current remaining data volume 2 is equal to the vector width 2, the target vector width for the second batch is determined to be 2. We update the current remaining data volume, which is: current remaining data volume 2 - target vector width of the second batch 2 = 0. Since the updated current remaining data volume is 0, we can stop matching, resulting in target vector widths of 8 and 2.

[0036] For example, the total data size of multiple target datasets might be an integer multiple of the target vector width of the first batch, such as a total data size of 16 and a target vector width of 8 for the first batch. In this case, using the current remaining data size, the multiple target datasets can be directly divided into at least one data group, with each data group containing data equal to the target vector width of the first batch, and the number of data groups being the ratio of the total data size of the multiple target datasets to the target vector width of the first batch. For example, the multiple target datasets can be split into two data groups, each containing 8 data groups. This eliminates the need for additional processing of the current remaining data size, ensuring that each load-store operation meets GPU memory alignment requirements and maximizing memory bus utilization. For example, loop unrolling optimization instructions can be used to eliminate loop control overhead, allowing vectorized computation to be completed within the shortest instruction pipeline. For example, register arrays can be used for temporary storage of intermediate results, avoiding repeated global memory accesses.

[0037] According to embodiments of this application, by determining multiple target vector widths from multiple vector widths in descending order, even when the total amount of data is not an integer multiple of the vector width, a smaller vector width can be tried through degradation exploration, so that the most efficient vector width can be selected under any remaining data size. This ensures the optimization of memory access patterns and improves the utilization of hardware units used for parallel processing.

[0038] In operation S230, multiple target data are divided into at least one data group corresponding to at least one target vector width.

[0039] For example, a target vector width can correspond to a data group, and the data size of the data group is the same as the target vector width. For example, multiple target data can be divided into two data groups. The first data group corresponds to a target vector width of 8, so the data size of this data group is 8. The second data group corresponds to a target vector width of 2, so the data size of this data group is 2.

[0040] In operation S240, at least one batch of processing tasks is performed on at least one data group, wherein a batch of processing tasks is used to process one data group.

[0041] For example, when the target vector width corresponding to the data group is a preset value (e.g., 1), the processing task may include using a low-overhead element-by-element processing path for the data group, such as scalar processing. For instance, when the target vector width is 1, the scalar computation unit in the processor may be activated, and another instruction may be generated to process the data group with a target vector width of 1 one by one in response to the instruction.

[0042] For example, when the target vector width is greater than a preset value (such as a target vector width of 2 or 8), the processing task may include performing vectorization processing on the data in the data group corresponding to the target vector width. For instance, a vector computation unit in the processor that processes multiple data simultaneously can be enabled, and an instruction can be generated to process all data in the data group simultaneously in response to that instruction. Thus, a progressive degradation path from complete vectorization to element-wise processing can be constructed. Through this collaborative design of main path vectorization and boundary path refinement, the contradiction between vectorization efficiency and data regularity in GPU element-level operation scenarios is resolved. While ensuring memory access efficiency and maintaining computational correctness, high-performance coverage of the entire range from complete data blocks to the tail number is achieved, eliminating the tail performance bottleneck in related vectorization processing methods and providing a reliable optimization foundation for GPU computing under irregular data scales.

[0043] According to embodiments of this application, by determining multiple vector widths that the processor can select to process multiple target data, and determining multiple target vector widths from multiple vector widths in descending order of the multiple vector widths, the most efficient vectorization level can be selected under any remaining data size. This ensures that all data is processed efficiently even when the total amount of data is not an integer multiple of the vector width, thereby maximizing the utilization of SIMD units and reducing memory access overhead.

[0044] According to an embodiment of this application, determining at least one target vector width from multiple vector widths in descending order includes performing at least one width matching operation. The i-th width matching operation in the at least one width matching operation includes: determining the amount of data to be matched in the i-th time based on the total amount of multiple target data and the results of the previous i-1 width matching operations, where i is a positive integer; and determining the target vector width that is closest to the amount of data to be matched in the i-th time from the multiple vector widths in descending order as the result of the i-th width matching operation.

[0045] For example, the sum of the width values ​​of all target vectors corresponding to the results of the first i-1 width matching operations can be determined, and the amount of data to be matched in the i-th operation can be determined based on the total amount of data and the sum of the width values.

[0046] For example, the target vector width that is closest to the amount of data to be matched in the i-th time can be: the target vector width whose vector width value is less than the amount of data to be matched in the i-th time, and whose difference between the vector width value and the amount of data to be matched in the i-th time is the smallest.

[0047] For example, multiple vector widths include 8, 4, 2, and 1. The first width matching operation is as follows: the total data volume of 10 is matched with the maximum vector width of 8. Since the maximum vector width of 8 is less than the total data volume of 10, and the difference between it and the total data volume of 10 is the smallest among the multiple vector widths, the maximum vector width of 8 is chosen as the result of the first width matching operation. The second matching operation requires matching the difference between the total data volume of 10 and the maximum vector width of 8, which is 2.

[0048] According to the embodiments of this application, based on the data volume of multiple target data and the results of the first i-1 width matching operations, the data volume to be matched in the i-th time is determined. Then, according to the order of multiple vector widths from largest to smallest, the target vector width that is closest to the data volume to be matched in the i-th time is selected. By comprehensively analyzing multi-dimensional data such as the total data volume, the data volume to be matched in the i-th time, and the vector width, the optimal vectorization width can be accurately determined, thereby determining the optimal task processing strategy.

[0049] According to an embodiment of this application, determining the amount of data to be matched for the i-th time based on the total amount of data of multiple target data and the results of the first i-1 width matching operations includes: determining the total amount of data corresponding to the target vector width of the results of the first i-1 width matching operations; and taking the difference between the amount of data of multiple target data and the total amount of data of the target vector width as the amount of data to be matched for the i-th time.

[0050] For example, when i=1, the amount of data to be matched is the total amount of data from multiple target data; when i=2, the amount of data to be matched is the difference between the total amount of data and the target vector width corresponding to the result of the first width matching operation; when i=3, the amount of data to be matched is the difference between the total amount of data and the target vector width corresponding to the result of the first two width matching operations.

[0051] According to the embodiments of this application, by determining the amount of data to be matched in the i-th time based on the data volume of multiple target data and the results of the first i-1 width matching operations, the remaining data beyond the integer multiple of the vectorized width can be accurately identified, thereby effectively processing data elements that do not meet the complete vectorization conditions and ensuring that computational correctness and high performance can be maintained under any data scale.

[0052] According to an embodiment of this application, the result of the width matching operation for the i-th operation is determined by selecting the target vector width that is closest to the amount of data to be matched in the i-th operation, in descending order of vector width. This includes: starting from the target vector width corresponding to the result of the (i-1)-th width matching operation, selecting the target vector width that is closest to the amount of data to be matched in the i-th operation, in descending order of vector width.

[0053] For example: For the first width matching operation: starting from the largest vector width among multiple vector widths, match the total amount of data with the largest vector width 8. The target vector width corresponding to the result of the first width matching operation includes the largest vector width.

[0054] For the second width matching operation: for example, if the amount of data to be matched in the second time is 6, start matching from the maximum vector width. Since the amount of data to be matched in the second time is 6, which is less than the maximum vector width of 8, continue to downgrade the matching, that is, match the amount of data to be matched in the second time is 6 with the vector width of 4, and determine the vector width of 4 as the target vector width that is closest to the amount of data to be matched in the second time.

[0055] For the third width matching operation: for example, if the amount of data to be matched in the third operation is 2, the matching starts from the vector width of 4. For example, if the vector width of 2 is determined to be the target vector width that is closest to the amount of data to be matched in the third operation, then in the fourth width matching operation, the matching starts from the vector width of 2.

[0056] According to the embodiments of this application, starting from the target vector width corresponding to the result of the (i-1)th width matching operation, the target vector width closest to the amount of data to be matched in the i-th operation is determined as the result of the i-th width matching operation in descending order of vector width. Each time, matching only starts from the target vector width corresponding to the result of the (i-1)th width matching operation, without having to repeatedly match from the maximum vector width, which can reduce the number of matching operations and improve matching efficiency.

[0057] For example, the remaining data that is too large to fit a full vector width can be temporarily stored in a register to achieve register-level data temporary storage, thereby minimizing the performance overhead caused by mantissa processing.

[0058] According to embodiments of this application, determining multiple vector widths that a processor can select for processing multiple target data includes: determining an upper limit on the amount of data that the processor can process in parallel at one time based on the physical bit width of the hardware unit used for parallel processing in the processor; determining the bit width of a single target data based on the data type of the target data; determining a maximum vector width based on the ratio of the upper limit of the data amount to the bit width of the target data; and performing a binary splitting operation on the maximum vector width in sequence to obtain multiple vector widths.

[0059] For example, the maximum vector width characterizes the upper limit of the maximum parallel capability under the constraints of the physical bit width of the hardware unit used for parallel processing in the processor and the data type of the target data. For instance, the physical bit width refers to the total data bit width of the processor's SIMD computing units, such as 128 bits, 256 bits, 512 bits, etc., which characterizes the upper limit of the processor's parallel capability at the hardware level, determining the maximum number of bits of data that can be processed simultaneously. The data type of the target data characterizes the number of bits occupied by a single data item, determining how much hardware bit width each data item requires. Data types could be, for example, single-precision floating-point numbers, half-precision numbers, etc.

[0060] For example, the maximum vector width can be the ratio of the upper limit of the data volume to the bit width of the target data (which can be rounded down). For example, if the physical bit width of the processor's parallel hardware unit is 256 bits and the data type of the target data is a single-precision floating-point number, which occupies 32 bits of binary data, the maximum vector width can be 256 bits / 32 bits, which is 8.

[0061] For example, after determining the maximum vector width to be 8 based on the physical bit width and the data type of the target data, the maximum vector width can be split into two parts to obtain a vector width of 4. Then, the vector width of 4 can be split into two parts to obtain a vector width of 2.

[0062] According to an embodiment of this application, the maximum vector width among multiple vector widths is determined by the physical bit width of the hardware unit used for parallel processing and the data type of the target data. Then, the maximum vector width is sequentially split into multiple vector widths by performing a binary splitting operation, which can automatically adapt to different data types and different vector widths.

[0063] For example, vectorized templates can be integrated into the system. These templates can automatically generate multiple vector widths based on data type and physical bit width, eliminating the need for rewriting code and reducing workload. During computation, the target data can automatically switch between different data types. For instance, if the target data is of half-precision type, which offers fast data transfer and is suitable for large-scale data storage, vectorized templates can automatically convert the data type from half-precision to single-precision floating-point numbers for computation, thereby improving computational accuracy. This enables mixed-precision conversion from half-precision to single-precision floating-point numbers. It adapts to both high-speed, large-scale data scenarios and high-precision computational scenarios, thus broadening its applicability.

[0064] For example, multiple target data may be continuous data or non-continuous data, and non-continuous data may further include partially continuous data and completely non-continuous data.

[0065] For example, the target data can be multidimensional data (such as matrices, tensors, etc.), which correspond to multiple access dimensions. When multiple target data maintain a continuous address layout across all access dimensions, the multiple target data can be determined to be continuous data, and the multiple target data can be directly vectorized.

[0066] For example, if multiple target data are non-contiguous, but if the multiple target data only maintain a continuous memory address layout in a few access dimensions, that is, the multiple target data are partially continuous, the multiple target data can also be vectorized.

[0067] For example, if the memory addresses of multiple target data are completely random and irregular, such as not maintaining a continuous memory address layout across all access dimensions, then the multiple target data are determined to be completely non-continuous data.

[0068] According to an embodiment of this application, before performing at least one batch of processing tasks on at least one data group, the method further includes: for each data group, if the actual input offset and the expected input offset of the target data are not the same, determining that the first memory address where the target data in the data group is written into memory is not contiguous, wherein the actual input offset represents the real memory address where the target data is written into memory, and the expected input offset represents the expected memory address where multiple target data are to be written into memory in a contiguous writing manner; if the first memory address is not contiguous, transferring the target data in the data group to the processor's register according to the respective data index of the target data in the data group.

[0069] For example, in the current access dimension, it can be determined whether the first memory address where the target data is written into memory is continuous within the data group under the current access dimension, based on whether the actual input offset and the expected input offset of the target data are the same. When there are individual access dimensions where the actual input offset and the expected input offset of the target data are the same, it can be determined as partially continuous data.

[0070] For example, when the actual input offset and the expected input offset of the target data are different across all access dimensions, the data can be determined to be completely discontinuous. The actual input offset can characterize the real storage address of the target data; for example, it represents the actual memory access address when the target data is loaded by the processor. The expected input offset is a theoretical reference value, representing the expected memory address to be written into memory in a contiguous write manner; for example, it can represent the memory address assuming all target data is stored compactly and sequentially in contiguous memory blocks.

[0071] For example, when the target data within a data group is completely non-contiguous, the target data within the data group can be reassembled into a contiguous buffer. The contiguous buffer may include processor registers, and the target data within the data group can be transferred to the processor registers according to their respective data indices.

[0072] For example, transferring target data within a data group to the processor's registers according to the data index can include: arranging the target data within the data group closely in a set of registers according to the data index, so that the target data within the data group is arranged continuously, thereby enabling simultaneous processing of the target data within the data group, thus significantly reducing the number of global memory accesses, and even with discontinuous data distribution, the processor's computing units can maintain a utilization rate close to 100%.

[0073] According to embodiments of this application, by transferring the target data within a data group to the processor's registers, non-contiguous memory access patterns can be merged into contiguous register operations, thereby enabling efficient vectorized computation. By transferring the target data within a data group to the processor's registers, vectorized step-by-step access operations are transformed into well-organized computations that can be processed in batches. This allows the GPU's SIMD computing capabilities to be fully utilized even under complex memory layouts, achieving near-peak performance in step-by-step access scenarios.

[0074] According to embodiments of this application, the consistency between the actual input offset and the expected input offset can accurately determine whether the first memory address is contiguous. Non-contiguous access patterns for non-contiguous data lead to multiple, fragmented memory transactions, making it impossible to utilize the hardware's merge access characteristics, resulting in extremely low memory bandwidth utilization. However, by transferring the target data within a data group to the processor's registers according to their respective data indices when the first memory address is not contiguous, the register array can be used as a temporary buffer to collect data elements scattered throughout the video memory into a contiguous register space, thereby improving data processing efficiency. This approach can then be adapted to non-contiguous access scenarios such as tensor transpose and slicing operations in deep learning models.

[0075] According to an embodiment of this application, the task execution method further includes: determining the actual input offset for each target data based on the data index of the target data and the input step size corresponding to the target data, wherein the input step size represents the difference between the actual memory addresses of two adjacent target data being written into memory.

[0076] For example, the actual input offset can be determined as shown in Equation (1).

[0077] (1)

[0078] in, Where I is the actual input offset, and 'I' is the data index. Enter the step size.

[0079] For example, the target data within the data group includes data A, data B, ..., data F, where data A has a data index of 0, data B has a data index of 1, ..., data F has a data index of 5. If the difference between the actual memory addresses of data A and data B is 8 bytes, then the actual input offset of data B is 8 bytes.

[0080] According to embodiments of this application, the desired input offset can be determined for each target data based on the data index of the target data and the data length corresponding to the target data.

[0081] For example, the desired input offset can be determined as shown in equation (2).

[0082] (2)

[0083] in, This represents the expected input offset. 'I' has the same meaning as above, representing the data index. This represents the data length.

[0084] For example, the target data within a data group includes data A, data B, ..., data F, where data A has a data index of 0, data B has a data index of 1, ..., data F has a data index of 5. When data A, data B, ..., data F are written to memory in a sequential manner, the difference between the memory addresses of two adjacent data items should be the data length. For example, if the data length is 4 bytes, then the difference between the memory addresses of two adjacent data items should be 4 bytes.

[0085] According to embodiments of this application, by using data index, input step size, and data length, it is possible to accurately determine whether the first memory address is contiguous, thereby supporting step access modes of arbitrary complexity.

[0086] According to an embodiment of this application, when the actual input offset and the expected input offset of each target data are the same, it can be determined that the first memory addresses where the target data in the data group are written into memory are consecutive.

[0087] For example, when the actual input offset and the expected input offset are the same, it means that all target data within the data group is written to memory in a continuous write manner, and therefore the first memory addresses are contiguous. Conversely, when the actual input offset and the expected input offset are different, it means that the first memory addresses are not contiguous. For instance, for data B with data index 1, the difference between the actual memory addresses of data A and data B is 8 bytes, and the actual input offset of data B is 8 bytes. However, the difference between the memory addresses of two adjacent data should be 4 bytes, indicating that there is other data between data A and data B, meaning that the first memory addresses of data A and data B are not contiguous.

[0088] According to the embodiments of this application, after the target data in the data group is transferred to the processor's register, the continuously arranged register data can be immediately processed at full speed using vectorization. Full speed vectorization can include: immediately handing the register data over to the SIMD computing unit for parallel processing to obtain the processing result without wasting any time in between, thereby maximizing computing efficiency. Thus, a seamless connection between data reorganization and data computation can be achieved, improving the processor's computing efficiency.

[0089] According to an embodiment of this application, the task execution method further includes: for the processing results obtained from each target data in the data group in the processing register, determining the second memory address of each processing result according to the output step size corresponding to the processing result and the data index of the target data corresponding to the processing result, wherein the output step size represents the difference between the corresponding memory addresses in memory where the processing results of two adjacent target data are written; for each target data in the data group, writing the processing result to the second memory address in memory according to the order of the data index.

[0090] For example, after obtaining the processing result, it is necessary to accurately write the processing result back to the corresponding memory address. The memory address to be written back to, i.e., the second memory address, can be accurately determined based on the output step size and the data index of the target data corresponding to the processing result. Here, one target data can correspond to one processing result, and the output step size can define how many bytes should be spaced between the calculation results of two adjacent target data in the output memory.

[0091] For example, based on the output step size corresponding to the processing result and the data index of the target data corresponding to the processing result, the target memory offset is determined. Then, based on the target memory offset and the memory start address, the second memory address is determined. For example, the target memory offset represents the number of bytes of the second memory address relative to the memory start address. For instance, an offset of 8 bytes means writing back from the start address plus 8 bytes.

[0092] The target memory offset can be determined by formula (3).

[0093] (3)

[0094] in, The target memory offset. The data index for the target data corresponding to the processing result. This is for outputting the step size.

[0095] According to an embodiment of this application, the target memory offset is determined by the data index and output step size of the target data corresponding to the processing result, and then the second memory address is determined. This can reconstruct the non-contiguous layout of the target memory and ensure that each calculation result can be accurately written back to the corresponding memory location.

[0096] According to embodiments of this application, data is collected from non-contiguous memory addresses into a contiguous local buffer during the collection phase; vectorized computation is performed on the data in the local buffer during the data computation phase; and the computation results are written back to non-contiguous memory addresses during the result distribution phase. Through the coordinated design of these three phases—data collection, data computation, and result distribution—the contradiction between vectorization efficiency and memory regularity in GPU computing scenarios can be resolved. Compared to related methods, this approach maintains algorithm versatility while achieving high-performance coverage across all scenarios, from contiguous to non-contiguous memory. It provides a reliable underlying optimization foundation for complex tensor operations, making it suitable for handling non-contiguous access scenarios such as tensor transpose and slicing operations in deep learning models.

[0097] According to embodiments of this application, processing at least one data group comprising multiple target data includes: determining the amount of valid data based on the processor's data processing performance; and processing the target data within the amount of valid data.

[0098] For example, data processing performance includes at least one of the following: current power consumption limit, processor core temperature, available video memory capacity, current computing load, etc.

[0099] For example, the amount of effective data can represent the boundary of the amount of data that the current processor's hardware can stably and efficiently process. Current power consumption limits, processor core temperature, available video memory capacity, and current computing load data processing performance can reflect the upper limit of the hardware's capabilities. If processing is forced beyond this limit, it will lead to crashes, overheating, or a sharp drop in efficiency. Therefore, the amount of data that the processor needs to process can be dynamically adjusted based on the amount of effective data, for example, processing only the target data within the amount of effective data.

[0100] For example, the target data within the effective data volume can be determined by the data index of the target data. For instance, if the effective data volume is 10, only the target data with data indices 0 to 9 can be processed.

[0101] According to the embodiments of this application, by processing only the target data within the amount of valid data, the data can be processed within the upper limit of the hardware's ability to process stably and efficiently, without processing redundant data that exceeds the hardware's capabilities. This avoids redundant calculations and ensures stable hardware operation, preventing a sudden drop in performance.

[0102] Figure 3 A flowchart illustrating task execution for non-continuous data according to an embodiment of this application is shown. Figure 3 As shown, the task execution for non-continuous data in this embodiment includes operations S310 to S340.

[0103] During the S310 operation, in the data collection phase, data is read from non-contiguous addresses into the buffer.

[0104] For example, during the data collection phase, data can be intelligently reassembled from non-contiguous address spaces into a temporary reassembly buffer. For instance, a register array can be used as a temporary reassembly buffer to collect data scattered throughout video memory into a contiguous register space.

[0105] When operating S320, determine whether valid data has been stored. If it is determined that no valid data has been stored, end the current processing flow.

[0106] For example, before processing a piece of data, the system can perform a validity check to determine whether valid data has been stored. If no valid data is found within the target data block, the system can immediately abandon all subsequent operations. By employing this lightweight valid element counting mechanism, useless padding values ​​or invalid data outside the boundaries can be avoided, thus preventing the waste of computing resources caused by processing invalid data.

[0107] When operating S330, if valid data is confirmed to be stored, the continuously arranged register data is vectorized at full speed during the data calculation phase.

[0108] For example, during the data computation phase, the recombined data can be processed in batches using SIMD. Batch SIMD processing includes precision conversion operations (cast operations), which can perform 128-bit operations at once.

[0109] During operation S340, in the result distribution stage, the calculation results are written to non-contiguous addresses.

[0110] For example, during the result distribution stage, the calculation results can be accurately written back to the target non-contiguous address based on the data index and output step size.

[0111] According to the embodiments of this application, a three-stage pipeline architecture of data collection, data computation, and result distribution is set up. This architecture adopts a lightweight effective element counting mechanism to count the amount of effective data that can be processed in real time during the data collection process. In addition, through the built-in step-aware address calculator, the memory access mode is determined according to the data index, step size, etc., which can automatically adapt to various step access modes, such as automatic adaptation to complex scenarios such as fixed step size and block access. Furthermore, by maintaining a unified register resource management strategy, the system ensures that the data collected in the registers is always kept in the registers before all computations are completed and finally distributed and written back. This ensures that the data is always kept at the most efficient storage level throughout the entire process of collection, computation, and distribution. The above design reduces the overhead of non-contiguous memory access to less than 35% of related methods, almost reaching the performance level of contiguous memory access.

[0112] Figure 4 A flowchart of a task execution method according to another embodiment of this application is shown. Figure 4 As shown, the task execution method of this embodiment includes operations S410 to S480.

[0113] When operating S410, the optimal execution path is automatically selected based on data characteristics.

[0114] For example, the optimal execution path may include path 1 and path 2. The operations of path 1 may include operations S420 to S460, and the operations of path 2 may include operations S470 to S480.

[0115] For example, two kernel versions can be generated during compilation: one is a secure version with boundary checks, which corresponds to path 1 and is used to prioritize the correctness of calculations; the other is a high-performance version without boundary checks, which corresponds to path 2 and is used to eliminate the overhead of conditional judgments, suitable for scenarios with regular data.

[0116] For example, by using template metaprogramming techniques with preset call entry points, automatic strategy dispatch can be achieved, automatically selecting the corresponding execution strategy at compile time without the need for manual strategy switching. This allows for the construction of a general-purpose element-level GPU kernel function startup system. Through deep integration of kernel version building during compilation and adaptive path selection at runtime, a multi-layered optimized computing framework can be built, capable of automatically selecting the optimal execution path based on data characteristics, maximizing performance while maintaining interface consistency.

[0117] When operating the S420, the system determines whether the memory layout is continuous during runtime.

[0118] For example, the relationship between data strides and data logical dimensions can be dynamically detected at runtime to accurately determine memory continuity. The data logical dimension represents the logical organization and dimensional division of multidimensional data. For instance, when the data stride is the product of the logical dimensions of the lower-dimensional data (i.e., the stride of each dimension is exactly equal to the total number of bytes of all lower-dimensional elements), it can be determined as a contiguous memory layout; otherwise, it can be determined as a non-contiguous access pattern.

[0119] When operating S430, if the memory layout is contiguous, a first predefined function is called to perform contiguous memory processing.

[0120] For example, a contiguous memory layout can include: data is stored consecutively in physical memory, with no gaps between adjacent data addresses, i.e., no extra bytes. The first predefined function can be used to: process data in batches according to the maximum vector width; if the total amount of data is not an integer multiple of the vector width, then through a fallback approach, match the remaining data with a smaller vector width.

[0121] When operating S440, adaptive mantissa processing is performed to automatically process the remaining data.

[0122] According to embodiments of this application, for cases where the total amount of data cannot be divided evenly by the vector width, an intelligent mantissa processing mechanism is used to ensure that all data is processed correctly while maintaining the advantages of vectorized computation, achieving the highest vectorization efficiency. For example, when the remaining data is 1, scalar processing can be initiated to ensure that all data can be processed.

[0123] In operation S450, a second predefined function is called to perform non-contiguous memory processing.

[0124] For example, the second predefined function can be used to reassemble the valid data and then partially vectorize the data when the data is stored in memory in a scattered manner. For example, the actual memory address of each valid data can be calculated according to the data step size, the scattered valid data can be read into the register in batches, reassembled into a continuous vector, and then vectorization calculation can be performed on the reassembled data.

[0125] When operating the S460, stepping is performed during partial vectorization operations. Stepping includes, for example, locating the actual memory address of each valid data point by using the data index and the input step size when processing non-contiguous memory data, and accurately locating the memory address to which the calculation result is written back based on the output step size according to the data index.

[0126] According to embodiments of this application, for non-contiguous memory access patterns, effective partial vectorization processing can fully utilize the GPU's SIMD computing power even under complex memory layouts, achieving vectorized loading or storage efficiency below theoretical peak performance. Through the synergy of intelligent mantissa processing and partial vectorization processing, the computational efficiency of element-level operations in various practical application scenarios can be significantly improved.

[0127] According to embodiments of this application, the most suitable vectorization width and processing path can be dynamically selected by using multi-dimensional parameters such as total data volume, memory alignment status, and step size mode. By employing a hierarchical decision tree mechanism, the memory continuity characteristics are first determined, and then the matching degree between the data size and the vectorization width is analyzed to determine the optimal computation strategy.

[0128] In operation S470, a third predefined function is called to perform fully vectorized processing with no-boundary checks.

[0129] When operating S480, assuming the data is aligned and complete, perform a simple vectorization operation.

[0130] For example, data alignment and integrity can include situations where multiple data sets are stored contiguously, and the size of these data sets is an integer multiple of a predetermined vector width. Fully vectorized processing without boundary checks can include simple vectorization operations, which can directly process multiple data sets in batches according to a predetermined vector width without performing partial vectorization, data reassembly, or other operations, thereby maximizing GPU parallel efficiency.

[0131] According to embodiments of this application, by acquiring multiple target data to be processed by the processor and determining multiple vector widths selectable by the processor for processing multiple target data, and determining multiple vector widths based on the physical bit width of the hardware unit used for parallel processing in the processor and the data type of the target data, and by transferring the target data in the data group to the processor's register according to the data index of each target data in the data group when the first memory address is not contiguous, it can automatically adapt to different data scales, memory layouts and hardware architectures, and maintain stable performance under various boundary conditions; by using template metaprogramming technology to achieve automatic policy dispatch through preset call entry points, development efficiency can be improved; and by encapsulating complex optimization details through a unified programming interface, near-optimal performance can be obtained without manual parameter tuning.

[0132] According to embodiments of this application, efficient computation can be achieved through a dual innovative mechanism of adaptive mantissa processing and partial vectorization reorganization. Specifically, on the one hand, by introducing an adaptive mantissa processing mechanism, data boundaries are intelligently identified, and data elements that do not meet the conditions for full vectorization are specifically processed. Furthermore, by automatically switching between full vectorization and element-wise processing, all data can be processed efficiently, ensuring computational correctness. On the other hand, through a partial vectorization reorganization mechanism, data collection and batch computation are performed for non-contiguous memory access patterns, adapting to complex memory layouts while maintaining the advantages of vectorization. This dual innovative mechanism of adaptive mantissa processing and partial vectorization reorganization allows for deep optimization of irregular data scales and non-contiguous access patterns, significantly improving the execution efficiency of various element-level operations. The collaborative work of the adaptive mantissa processing mechanism and the partial vectorization reorganization mechanism constitutes a complete adaptive vectorized computation system. By analyzing the scale characteristics and memory layout of the input data, the optimal execution path is determined, adapting to various task execution requirements.

[0133] For example, the task execution method of the embodiments of this application can be applied to a variety of fields, such as not only artificial intelligence, but also other fields that require processing multidimensional data and complex data formats, such as scientific computing, image processing, and signal processing. By using a similar approach to the task execution method of the embodiments of this application, the efficiency of task execution and computation in these fields can be extended and optimized.

[0134] Based on the above task execution method, this application also provides a task execution apparatus. The following will be combined with... Figure 5 The device is described in detail.

[0135] Figure 5 A structural block diagram of a task execution apparatus according to an embodiment of this application is shown.

[0136] like Figure 5 As shown, the task execution device 500 of this embodiment includes an acquisition module 510, a vector width determination module 520, a partitioning module 530, and a task execution module 540.

[0137] The acquisition module 510 is used to acquire multiple target data to be processed by the processor, and to determine multiple vector widths that the processor can select for processing the multiple target data. The vector width represents the amount of data that the processor can process in parallel at one time, and the multiple vector widths are determined based on the physical bit width of the hardware unit in the processor used for parallel processing and the data type of the target data. In one embodiment, the acquisition module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0138] The vector width determination module 520 is used to determine at least one target vector width from multiple vector widths in descending order, wherein the total amount of data that the processor can process corresponding to the at least one target vector width is equal to the total amount of data of the multiple target data. In one embodiment, the vector width determination module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0139] The partitioning module 530 is used to partition multiple target data into at least one data group corresponding to at least one target vector width. In one embodiment, the partitioning module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0140] The task execution module 540 is used to execute at least one batch of processing tasks on at least one data group, wherein one batch of processing tasks is used to process one data group. In one embodiment, the task execution module 540 may be used to execute the operation S240 described above, which will not be repeated here.

[0141] According to an embodiment of this application, the vector width determination module 520 includes a first determination submodule and a second determination submodule. The first determination submodule determines the amount of data to be matched in the i-th time based on the total amount of multiple target data and the results of the first i-1 width matching operations, where i is a positive integer; the second determination submodule is used to determine the target vector width that is closest to the amount of data to be matched in the i-th time among the multiple vector widths in descending order as the result of the width matching operation.

[0142] According to an embodiment of this application, the task execution device 500 further includes a continuity determination module, used to determine, for each data group, that the first memory address where the target data is written into memory is not contiguous when the actual input offset and the expected input offset of the target data are not the same, wherein the actual input offset represents the real memory address where the target data is written into memory, and the expected input offset represents the expected memory address where multiple target data are to be written into memory in a continuous writing manner; it also includes a transfer module, used to transfer the target data in the data group to the processor's register according to the data index of the target data in the data group when the first memory address is not contiguous.

[0143] According to an embodiment of this application, the task execution device 500 further includes a first offset determination module, which is used to determine the actual input offset for each target data according to the data index of the target data and the input step size corresponding to the target data, wherein the input step size represents the difference between the actual memory addresses of two adjacent target data being written into memory.

[0144] According to an embodiment of this application, the task execution device 500 further includes a first offset determination module, which is used to determine the expected input offset for each target data based on the data index of the target data and the data length corresponding to the target data.

[0145] According to an embodiment of this application, the task execution device 500 further includes a memory address determination module, used to determine the second memory address of each processing result for the processing results obtained from each target data in the data group in the processing register, based on the output step size corresponding to the processing result and the data index of the target data corresponding to the processing result, wherein the output step size represents the difference between the memory addresses corresponding to the processing results of two adjacent target data being written into memory; and also includes a writing module, used to write the processing results into the second memory address in memory according to the order of the data index for each target data in the data group.

[0146] According to an embodiment of this application, the task execution module 540 includes a third determining submodule for determining the amount of valid data based on the data processing performance of the processor; and a processing submodule for processing target data within the amount of valid data.

[0147] According to an embodiment of this application, the first determining submodule includes: a first determining unit, configured to determine the total data volume of the target vector width corresponding to the result of the first i-1 width matching operations; and a second determining unit, configured to use the difference between the data volume of multiple target data and the total data volume of the target vector width as the data volume to be matched in the i-th operation.

[0148] According to an embodiment of this application, the second determining submodule includes a third determining unit, which is used to determine the target vector width that is closest to the amount of data to be matched in the i-th width matching operation, starting from the target vector width corresponding to the result of the (i-1)th width matching operation, in descending order of vector width.

[0149] According to an embodiment of this application, the acquisition module 510 includes: an upper limit determination submodule, used to determine the upper limit of the amount of data that the processor can process in parallel at one time, represented by the maximum vector width, based on the physical bit width; a bit width determination submodule, used to determine the bit width of a single target data based on the data type; a vector width determination submodule, used to determine the maximum vector width based on the ratio of the upper limit of the data amount to the bit width of the target data; and a splitting submodule, used to perform a binary splitting operation on the maximum vector width in sequence to obtain multiple vector widths.

[0150] According to embodiments of this application, any multiple modules among the acquisition module 510, vector width determination module 520, partitioning module 530, and task execution module 540 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the acquisition module 510, vector width determination module 520, partitioning module 530, and task execution module 540 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 510, vector width determination module 520, partitioning module 530, and task execution module 540 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0151] Figure 6 A block diagram of an electronic device suitable for implementing a task execution method according to an embodiment of this application is shown.

[0152] like Figure 6 As shown, an electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0153] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 602 and / or RAM 603. It should be noted that programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in one or more memories.

[0154] According to embodiments of this application, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0155] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0156] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0157] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0158] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0159] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0160] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0161] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0163] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0164] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A task execution method, characterized in that, The method includes: Acquire multiple target data to be processed by the processor; The upper limit of the amount of data that the processor can process in parallel at one time is determined based on the physical bit width of the hardware unit used for parallel processing in the processor. The bit width of a single target data item is determined based on the data type of the target data. The maximum vector width is determined based on the ratio of the upper limit of the data volume to the bit width of the target data; The maximum vector width is sequentially split into two vector widths to obtain multiple vector widths that the processor can select for processing the multiple target data. The vector width represents the amount of data that the processor can process in parallel at one time. The multiple vector widths are determined based on the physical bit width of the hardware unit in the processor used for parallel processing and the data type of the target data. According to the order of the multiple vector widths from largest to smallest, the total data volume of the multiple target data is matched with the multiple vector widths in sequence to determine the target vector width corresponding to the first batch; Calculate the current remaining data volume, continue to match the current remaining data volume with the remaining vector width in descending order of vector width, determine the target vector width for the corresponding batch, and update the remaining data volume; Repeat the operation of calculating the current remaining data volume, matching the current remaining data volume with the remaining vector width, and determining the target vector width of the corresponding batch until the updated remaining data volume is zero, thus completing the determination of the target vector width of all batches. Among them, the total amount of data that the processor can process corresponding to at least one target vector width is equal to the total amount of data of the multiple target data. The plurality of target data are divided into at least one data group corresponding to the width of the at least one target vector; At least one batch of processing tasks is performed on each of the at least one data group, wherein a batch of processing tasks is used to process one of the data groups; Before performing at least one batch of processing tasks on each of the at least one data group, the method further includes: For each of the data groups, if the actual input offset and the expected input offset of the target data are not the same, it is determined that the first memory address where the target data is written into memory within the data group is not contiguous. Here, the actual input offset represents the real memory address where the target data is written into memory, and the expected input offset represents the expected memory address where multiple target data are to be written into memory in a contiguous writing manner. If the first memory address is not contiguous, the target data in the data group is transferred to the processor's register according to the respective data index of the target data in the data group.

2. The method according to claim 1, characterized in that, Determining at least one target vector width from the plurality of vector widths in descending order includes performing at least one width matching operation, wherein the i-th width matching operation in the at least one width matching operation includes: Based on the total data volume and previous data of the multiple target data The result of the width matching operation determines the amount of data to be matched in the i-th operation, where i is a positive integer; According to the order of the multiple vector widths from largest to smallest, the target vector width that is closest to the amount of data to be matched in the i-th time is determined as the result of the width matching operation in the i-th time.

3. The method according to claim 1, characterized in that, The method further includes: For each of the target data, the actual input offset is determined based on the data index of the target data and the input step size corresponding to the target data, wherein the input step size represents the difference between the actual memory addresses of two adjacent target data that are written into memory; For each of the target data, the expected input offset is determined based on the data index of the target data and the data length corresponding to the target data.

4. The method according to claim 1, characterized in that, The method further includes: For the processing results obtained from processing each target data in the data group in the register, the second memory address of each processing result is determined according to the output step size corresponding to the processing result and the data index of the target data corresponding to the processing result. The output step size represents the difference between the memory addresses corresponding to the processing results of two adjacent target data that are written into memory. For each target data within the data group, the processing result is written to the second memory address in memory according to the order of the data index.

5. The method according to claim 1, characterized in that, Processing the plurality of target data comprising the at least one data group includes: The amount of valid data is determined based on the data processing performance of the processor. The target data within the amount of valid data to be processed.

6. The method according to claim 2, characterized in that, The total data volume based on the multiple target data and the previous The result of the width matching operation, determining the amount of data to be matched in the i-th time includes: determining the previous... The total amount of data corresponding to the target vector width as a result of each width matching operation; The difference between the total amount of data of the multiple target data and the total amount of data of the target vector width is taken as the amount of data to be matched in the i-th time.

7. The method according to claim 2, characterized in that, The step of determining the target vector width that is closest to the amount of data to be matched in the i-th time among the multiple vector widths in descending order as the result of the width matching operation includes: From the Starting with the target vector width corresponding to the result of the first width matching operation, the target vector width that is closest to the amount of data to be matched in the i-th operation is determined as the result of the i-th width matching operation, in descending order of vector width.

8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the steps of the task execution method according to any one of claims 1 to 7.

Images