Method and device for optimizing data movement tasks

Abstract

Embodiments of the present disclosure provide a method and device for optimizing data movement tasks associated with machine learning models. The method may include identifying a data movement task in an executable program of a device, identifying one or more operations associated with the data movement task, determining the hardware capabilities of the device and the requirements for optimizing the data movement task, and in response to a determination that the hardware capabilities of the device meet the requirements, mapping the identified data movement task to data writing of the identified one or more operations associated with the data movement task.

Description

Technical Field

[0001] The present disclosure relates generally to the field of computing optimization, and more particularly to methods and devices for optimizing data movement tasks. Background Art

[0002] Due to the increasing speed gap between the CPU and the memory outside the CPU chip, reducing the computation time of data movement operations that are widely present in computer software tasks has become a focus of attention. Operations such as copying and moving can be very time-consuming. The acceleration of these operations can greatly improve computing performance and can accelerate model learning in machine learning on heterogeneous devices such as FPGAs and NPUs. In an example where a computation operation is followed by a data movement operation, the optimization of the data movement operation provided by traditional solutions is separate from the optimization of the computation operation, and the performance improvement produced by this optimization is limited. Traditional solutions, such as using manual assembly code and multi-threaded data copying, do not fully address this problem. This requires more efficient optimization of data movement operations. Summary of the invention

[0003] An embodiment of the present disclosure provides a method for optimizing a data movement task associated with a machine learning model. The method may include identifying a data movement task in an executable program of a device, identifying one or more operations associated with the data movement task, determining the hardware capabilities of the device and the need to accelerate the data movement task, and in response to a determination that the hardware capabilities of the device meet the needs, mapping the identified data movement task to data writing of the identified one or more operations associated with the data movement task.

[0004] The embodiment of the present disclosure also provides a device for optimizing data movement tasks associated with machine learning models. The device may include a memory storing a set of instruction sets, and one or more processors configured to execute the instruction sets, so that the device performs: identifying a data movement task in an executable program of the device, identifying one or more operations associated with the data movement task, determining the hardware capabilities of the device and the need to accelerate the data movement task, and, in response to the hardware capabilities of the device meeting the requirements, mapping the identified data movement task to the data writing of the identified one or more operations associated with the data movement task.

[0005] An embodiment of the present disclosure also provides a non-volatile computer-readable medium storing a set of instructions executable by at least one processor of a computer to enable the computer to perform a method for optimizing a data movement task related to a machine learning model, the method may include identifying a data movement task in an executable program of a device; identifying one or more operations associated with the data movement task; determining the hardware capabilities of the device and the requirements for optimizing the data movement task; and, in response to a determination that the hardware capabilities of the device meet the requirements, mapping the identified data movement task to data writes of the identified one or more operations associated with the data movement task. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The description of the specific embodiments and exemplary embodiments of the present disclosure is used to explain the present disclosure and is not intended to constitute an undue limitation to the present disclosure. In the drawings:

[0007] Figure 1 A schematic diagram showing an exemplary deep learning acceleration system consistent with embodiments of the present disclosure.

[0008] Figure 2 A block diagram illustrating example components of a system including an apparatus for optimizing data movement tasks consistent with an embodiment of the present invention.

[0009] FIG. 3A shows a schematic diagram of an exemplary computational model before optimization consistent with an embodiment of the present disclosure.

[0010] FIG. 3B is a schematic diagram of an exemplary optimized computational model consistent with an embodiment of the present disclosure.

[0011] Figure 4 A flow chart illustrating an exemplary method of optimizing data movement tasks consistent with embodiments of the present disclosure.

[0012] Figure 5 A flow chart illustrating an exemplary method of optimizing data movement tasks consistent with embodiments of the present disclosure.

[0013] Fig. 6A A schematic diagram illustrating an exemplary data movement optimization of a merge computing process consistent with an embodiment of the present disclosure.

[0014] Figure 6B A schematic diagram showing data movement optimization of an interleaved computing process consistent with an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] In order to facilitate understanding of the solutions in the present disclosure, the technical solutions in certain embodiments of the present disclosure will be described with reference to the attached drawings. It should be noted that the described embodiments are only a part of the embodiments of the present disclosure, rather than all embodiments. In accordance with the present disclosure, other embodiments can be obtained without departing from the principles of the present disclosure. Such embodiments should also belong to the protection scope of the present disclosure.

[0016] In order to improve computing performance, some current solutions focus on improving the process of data movement tasks, such as using manual assembly code and multi-threaded data copying. The present disclosure incorporates the data movement task into an operation (such as a computing operation) performed before the data movement task, and redirects the output operation object of the data movement task to an offset generated by a function, thereby eliminating the data movement task, thereby solving this problem.

[0017] In one example, the optimization of data movement tasks can be used in a machine learning system for a point-of-sale (POS) machine that uses a field programmable gate array (FPGAs) to accelerate model learning. It is worth noting that embodiments of the present disclosure can be used in many systems, including autonomous driving systems, speech recognition systems, and identity recognition systems that benefit from computational acceleration. The optimization method can be applied to heterogeneous devices such as FPGAs and neural processing units (NPUs). It is worth noting that this optimization can be used for any type of compiler and computing framework. It is also worth noting that embodiments can be used for CPUs and any type of accelerator (e.g., FPGAs, NPUs, GPUs, GRUs, and ASICs).

[0018] Figure 1 1 shows a block diagram of an exemplary deep learning acceleration system 100 according to an embodiment of the present invention. The deep learning accelerator system 100 may include a neural network processing unit (NPU) 102, an NPU memory 104, a host CPU 108, a host memory 110 associated with the host CPU 108, and a disk 112. The acceleration system 100 may perform data movement optimization. It is worth noting that although Figure 1 An accelerator system using an NPU is shown, but any type of accelerator may be used.

[0019] like Figure 1 As shown, NPU 102 can be connected to host CPU 108 via a peripheral interface (not shown). As described herein, a neural network processing unit (e.g., NPU 102) can be a computing device for accelerating neural network computing tasks. In some embodiments, NPU 102 may be configured to function as a coprocessor for host CPU 108.

[0020] In some embodiments, NPU 102 may include a compiler (not shown). A compiler may be a program or computer software that converts computer code written in a programming language into NPU instructions to create an executable program. In processing applications, a compiler may perform various operations such as preprocessing, lexical analysis, parsing, semantic analysis, converting an input program into an intermediate representation, code optimization, code generation, or a combination thereof.

[0021] In some embodiments, the compiler may be located on a host unit (e.g., Figure 1 The host CPU 108 or host memory 110 in the NPU 102 is configured to push one or more commands to the NPU 102. Based on these commands, the task manager (not shown) of the NPU 102 can assign any number of tasks to one or more processing elements of the NPU 102. Some of these commands may instruct the DMA unit to load instructions and data from the host memory to the global memory. The loaded instructions can then be distributed to each processing element assigned a corresponding task, and one or more processing elements can process these instructions.

[0022] It is worth noting that the first few instructions received by the processing element may be instructions for the processing element to load / store data from the global memory to the local memory of one or more of the processing elements (e.g., the memory of a processing element or the local memory of each active processing element). Then, each processing element can start the instruction pipeline, which includes fetching instructions from the local memory (e.g., through a fetch unit), decoding the instructions (e.g., through an instruction decoder), generating a local memory address (e.g., corresponding to an operand), reading source data, executing or loading / storing operations, and then writing back the results.

[0023] The host CPU 108 may be associated with a host memory 110 and a disk 112. In some embodiments, the host memory 110 may be an integral memory associated with the host CPU 108 or an external memory. The host memory 110 may be a local memory or a global memory. In some embodiments, the disk 112 may include an external memory configured to provide additional memory to the host CPU 108.

[0024] Figure 2 1 is a block diagram showing example components of a system consistent with an embodiment of the present invention, the system including an apparatus for optimizing data movement tasks. The apparatus 200 for optimizing data movement tasks can be implemented within the system. In some embodiments, the system can be Figure 1 In some embodiments, the device 200 may be a neural network acceleration system 100. Figure 1A compiler on the host CPU 108 or host memory 110 is shown. It is worth noting that while the compiler is used to perform data movement optimizations, the optimized data movement tasks can also be implemented in any framework that uses optimizations (e.g., TensorFlow for deep learning). This framework can be an abstraction in which software that provides general functionality can be selectively changed through additional user-written code to provide application-specific software. The device 200 includes a data move task finder 210, a task analyzer 220, a storage allocator 230, an address redirector 240, and a data move task deleter 250.

[0025] The data movement task finder 210 can identify the device (such as Figure 1 The deep learning acceleration system 100 shown in the figure can be used to perform a data movement task (DMT) in an executable program. The data movement task can include an operation of moving or copying data. The data movement task finder 210 can also identify one or more operations associated with the data movement task. The associated operation includes an operation previously performed by the data movement task. The operation can be a calculation operation, such as multiplication and addition. In the apparatus 200, Figure 1 In the example of a compiler on the host CPU 108 or host memory 110 shown, the input of the compiler can be a model definition and a model data file. The module definition describes the main inputs of the model, the intermediate calculations, and the outputs of the model. The model data file may include weights, biases, parameters, and coefficients. The compiler may first analyze the structure and operation of the model, identify and optimize the parts that can be optimized, convert the model definition into an intermediate representation (IR), and generate binary instructions according to the instruction set architecture (ISA) of the device. The input and output of the model may be tensors (such as the input tensors and output tensors shown in Figures 3A and 3B). Figure 2 The data movement task finder 210 shown can identify data movement type operations, such as join operations (such as merging arrays as shown in FIG. 3A ) and split operations when analyzing the operations of the model. In the example of merging array operations, hardware may be required to connect two three-dimensional input data along one dimension. For example, data matrix

[10]

[20]

[30] and data matrix

[10]

[40]

[30] are connected along the second dimension to generate the result of

[10]

[60]

[30] . In the example of split operations, multidimensional data can be split along one dimension to generate multiple output data. For example, data matrix [2][3][4] can be split along the third dimension to generate four outputs of [2][3]. The above example operations can be identified as data movement types.

[0026] Consistent with the embodiments of the present disclosure, a schematic diagram of an exemplary computing model before optimization is shown in FIG3A; consistent with the embodiments of the present disclosure, a schematic diagram of an exemplary computing model after optimization is shown in FIG3B. As shown in FIG3A, blocks of different sizes represent examples of input tensors and output tensors. It is worth noting that, although the optimization can be used for deep learning, calculations using any data blocks can be optimized using the data movement task. In deep learning, the input refers to the model structure and definition. In one example, the data block used as input can be a matrix, a vector, an integer, or a floating point number. The data block can be defined in granularity. For example, when using NPU and FPGA, the data block used as input can have a larger granularity, while when using CPU and GPU, the data block used as input can represent an integer. The exemplary computing model shown in FIG3A can involve three operations, including a matrix multiplication (MatMul) operation, an addition (Add) operation, and a merge array (Concat) operation. In the example given, four input tensors are calculated to produce an output tensor. The compiler can confirm that the merge array operation connects two data operation objects along the second axis.

[0027] Now back to Figure 2 , the task analyzer 220 can determine whether the hardware capabilities of the device meet the requirements related to the data movement task. The task analyzer 220 can also determine the specifications of the output operation object (the output tensor shown in Figure 3A) of the identified operation related to the data movement task. Not all data movement type operations can be optimized because the patterns of some operations cannot be extracted or the hardware cannot optimize these operations. These requirements are met when the minimum specification of the data configured to be written by the hardware is equal to or less than the minimum specification of the output operation object. For example, an accelerator such as an NPU may be limited in hardware capabilities because the ISA of the NPU may not support such optimization. For example, although the minimum specification that the NPU is configured to write may be 10 integers, when the data movement task is merged into the identified operation, the optimization may require writing one integer at a time. Since the minimum specification is 10 integers, the NPU cannot write an integer less than 10, or skip an integer to map the offset for optimization. The accelerator may be limited by hardware capabilities, resulting in the accelerator being unable to write the operation result in a certain way required for optimization. If the hardware is able to optimize the operation, the data movement operation can be eliminated. The data movement task may incorporate a computing operation that is performed prior to the data movement operation.

[0028] When the device is confirmed (e.g. Figure 1The hardware of the deep learning acceleration system 100 shown in the figure meets the requirements for optimizing the data movement task, and the address redirector 240 can map the identified data movement task to the identified operation associated with the data movement task. The address redirector 240 can generate a function to map the input offset and the output offset. The offset can be an integer representing the distance between the base address and the given operand. When the input offset of the data movement task is given, the function can infer the output offset of the data movement task. When writing data in a calculation operation performed immediately before the data movement task, the output offset generator can be used to write the result of the calculation operation to the output offset. The output offset generator can be adjusted by this function. The address redirector 240 can adjust the output offset generator of the calculation operation by using the function of inferring the output offset of the data movement task, redirecting the result of the calculation operation to the output offset of the data movement task, thereby merging the data movement task into the calculation operation.

[0029] The exemplary computing models before and after optimization are shown in FIG3A and FIG3B , wherein the optimized computing model in FIG3B is executed without performing the data movement task. Figure 2 , the address redirector 240 in the exemplary compiler can mark in the matrix multiplication (MatMul) operation and the addition (Add) operation to indicate that the result is written at a distance-based interval, and adjust the instructions for the matrix multiplication operation and the addition operation based on the offset change of the result. The result of the matrix multiplication operation is a temporary tensor of 2*4 size (such as a 2*4 block in FIG. 3A), and the result of the addition operation is a temporary tensor of 2*1 size (such as a 2*1 block in FIG. 3A). In the matrix multiplication operation, every time four data units are written to the memory of the device, the length of one data unit is skipped, and the skipped space is reserved for the result of the addition operation. In the addition operation, every time a data unit is written, the length of 4 data units is skipped, and the skipped space is reserved for the result of the matrix multiplication operation. The compiler can then generate instructions to merge the calculation operation and the data movement operation. As shown in FIG. 3B, the optimized calculation model can perform two calculation operations without performing the data movement operation. Therefore, when the compiler combines the calculation operation with the data movement task and optimizes the data movement operation, the calculation performance can be improved.

[0030] Replay Figure 2 The storage allocator 230 is provided before an operation (such as a computing operation), and the storage allocator 230 can allocate storage space that is sufficient for the storage operation before the operation is performed. The storage allocator 230 can work in conjunction with the address redirector 240 to optimize the data movement task.

[0031] The data movement task deleter 250 can eliminate the mapped data movement tasks. When the data movement task is merged into the operation before the data movement task, the execution of the data movement task is replaced by redirecting the result and writing it to the output offset generated by the function, which is a function based on the specification of the output operation object of the operation. As shown in Figure 3B, the results of the matrix multiplication operation and the addition operation are connected without performing a connection operation. Accordingly, the data movement task can be eliminated. Therefore, the resources dedicated to the data movement task before optimization can be released after optimization. The overall computing power of the accelerator is improved.

[0032] Figure 4 Flowchart showing an exemplary method 400 for optimizing data movement tasks consistent with an embodiment of the present disclosure. The method 400 may include the following steps. The method 400 may be performed by a processing device (e.g., Figure 1 The compiler is executed on the host CPU 108 or NPU 102).

[0033] In step 401, a data movement task is identified in an executable program of a device such as an accelerator. A data movement task may be an operation of moving or copying data. For example, when a compiler can analyze the behavior of an operation, the compiler can determine whether the operation is a data movement task. In the example of a join operation (as shown in FIG. 3A ) and a split operation, the compiler can determine how to join input operation objects along a dimension to generate an output operation object of the join operation, and how to split the input operation realization along a dimension to generate an output operation object of the split operation. The compiler can identify that join operations and split operations are data movement tasks. The number of input operation objects of a data movement task may be different. The number of output operation objects of a data movement task may also be different.

[0034] In step 402, one or more operations associated with the data movement task are identified. The one or more operations associated with the data movement task include operations performed before the data movement task. The operations can be computational operations, such as multiplication and addition. The compiler can identify the operations immediately preceding the data movement task in order to merge the data movement task and the operations.

[0035] In step 403, the hardware capabilities of the device and the requirements for optimizing the data movement task are determined. If the hardware capabilities meet the requirements, the method 400 proceeds to step 404. If the hardware capabilities do not meet the requirements, the method 400 ends. Step 403 may include determining the specifications of the output operation object (e.g., the output tensor shown in Figure 3A) of the identified operation associated with the data movement task. The compiler may determine whether the operation can be optimized based on the definition of the operation and the hardware capabilities. The requirements can be met when the minimum specification of the written data configured by the hardware is equal to or less than the minimum specification of the output operation object. For example, an accelerator such as an NPU may be limited in hardware capabilities because the ISA of the NPU may not support optimization. For example, the minimum specification that the NPU is configured to write may be 10 integers, and the optimization may require writing an integer less than 10 and skipping an integer when the data movement task is merged into the identified operation. Since the minimum specification is 10 integers, the NPU cannot write an integer or skip an integer to map an offset for optimization.

[0036] In step 404, if the hardware capability of the device meets the requirements for optimizing the data movement task, the identified data movement task is mapped to data writing of one or more operations associated with the data movement task. For example, the operations may include calculation operations such as matrix multiplication and addition. The identified data movement task is mapped to data writing of one or more operations associated with the identified data movement task. Figure 5 Step 404 may also include step 501 and step 502, such as Figure 5 shown. Figure 5 A flow chart showing an exemplary method 500 for optimizing data movement tasks consistent with an embodiment of the present disclosure is shown. The method 500 may include the following steps.

[0037] In step 501, a function is generated that maps the output offset of the identified data move task to the input offset of the identified data move task. The offset can be an integer representing the distance between the base address and a given operand. The base address can be used as a reference point for other addresses. The base address can represent the starting address of a storage space. The output offset generator can be used to write the result of the identified operation associated with the data move task to a certain offset. The function can map the offset address using the following method. The function can modify the page table of the operating system. The page table is a data structure mapping between the virtual address used in the program and the physical address used in the hardware. This function can construct and maintain a lookup table for the offset mapping. The function can call a designed formula. The function can also directly modify the way the offset changes, for example, from increasing by 1 increment each time to increasing by 2, 2, or 5 increments regularly. The increment can be determined based on the specification of the output operation object of the operation associated with the data move task. Any of the above methods can be used to convert the old address to a new address (for example, address 100 to address 300) to write the operation result. The physical address of the written result is obtained by adding the base address to the offset.

[0038] In step 502, a function is used to redirect the output offset of the identified data move task according to the specification of the output operation object of the identified operation associated with the identified data move task. The function can be used to adjust the output offset generator so that the result of the identified operation is redirected and written to the output offset generated by the function. The output offset can be adjusted to obtain an actual address to write the result of the data move task.

[0039] Please refer again Figure 4 , in step 405, the mapped data movement tasks are eliminated. After the identified data movement tasks are mapped, the data movement tasks are merged into the associated operations. As shown in FIG. 3A and FIG. 3B , after optimization, the merge operation of the two temporary tensors in FIG. 3A is eliminated. In FIG. 3B , the results of the matrix multiplication operation and the addition operation are merged without performing the join operation. Therefore, the mapped data movement tasks can be deleted, thereby optimizing the calculation.

[0040] In step 406, it is determined whether there are more identified data movement tasks to be processed. If there is another identified data movement task to be processed, method 400 continues to step 403. If there are no more identified data movement tasks to be processed, method 400 ends.

[0041] Fig. 6A FIG. 1 is a schematic diagram showing an exemplary data movement optimization of a merge computing process consistent with an embodiment of the present disclosure. Fig. 6AAs shown, calculation process 1 and calculation process 2 are merged in sequence through a merge operation. After optimization, the results of calculation process 1 and calculation process 2 are merged in sequence without a merge operation. Thus, the data movement task associated with the calculation process is eliminated using the optimization technique consistent with the embodiment.

[0042] Figure 6B FIG. 1 is a schematic diagram showing data movement optimization of an interleaved computing process consistent with an embodiment of the present disclosure. Figure 6B As shown, the alternating computing process 1 and computing process 2 are interleaved by interleaving operations. After optimization, the results of computing process 1 and computing process 2 are interleaved without interleaving operations. The data movement task is eliminated.

[0043] It should be noted that the computing process 1 and computing process 2 shown in FIG.6A and FIG.6B can run on different hardware (for example, on two independent GPUs, or computing process 1 on the CPU and computing process 2 on the GPU), provided that the CPU and GPU can access shared memory, and both the CPU and GPU can write the results to the offset address reserved based on the result size of computing process 1 and computing process 2. Computing process 1 and computing process 2 can also be executed by an accelerator (such as a GPU). When computing process 1 and computing process 2 are executed on a CPU or a GPU, the CPU and the GPU can optimize operations according to the embodiments of the present disclosure. When any process runs on FPGAs or NPUs, it may be necessary to determine whether the hardware of the FPGAs or NPUs can perform such optimizations, because the ISA of the hardware may not support such optimizations.

[0044] In some embodiments, a non-volatile computer-readable storage medium including instructions is also provided, and these instructions can be used by a device (eg, Figure 1 Common forms of non-volatile storage include, for example, floppy disks, diskettes, hard disks, solid-state drives, tapes, or any other magnetic data storage medium, CD-ROMs, any other optical data storage medium, any punched physical medium, RAM, PRAM, and EPROM, Flash EPROM or other flash memory, NVRAM, cache, registers, any other memory chip or cartridge memory, and network versions thereof. A device may include one or more processors (such as a CPU and a processing accelerator), input / output interfaces, network interfaces, and / or memory.

[0045] It should be noted that relational terms (e.g., "first" and "second") herein are used only to distinguish an entity or operation from another entity or operation, and do not require or imply any actual relationship or order between these entities or operations. In addition, the words "comprise," "have," "include," and "includes," and other similar forms are equivalent in meaning and are open ended, in that one or more items following any of these words are not meant to be an exhaustive list of such items, nor are they meant to be limited to the listed items.

[0046] Here, unless otherwise specified, the word "or" includes all possible combinations unless they are not feasible. For example, if it is expressed that a component may include A or B, then unless otherwise specified or not feasible, the component may include A, or B, or A and B. As a second example, if it is expressed that a component may include A, B, or C, then unless otherwise specified or not feasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A, B, and C.

[0047] It should be understood that the above embodiments can be implemented by hardware, or software (program code) or a combination of hardware and software. If implemented by software, it can be stored in the above-mentioned computer-readable medium. When executed by a processor, the software can execute the disclosed method. The computing unit and other functional units described in the present invention can be implemented by hardware, software or a combination of hardware and software. It will also be understood by those of ordinary skill in the art that a plurality of the above-mentioned modules / units can be combined into one module / unit, and each of the above-mentioned modules / units can be further divided into a plurality of sub-modules / sub-units.

[0048] In the foregoing description, embodiments have been described with reference to many specific details, which may vary according to different implementations. Certain adjustments and modifications may be made to the described embodiments. Other embodiments will be apparent to those skilled in the art from the description and practice of the invention disclosed herein. This description and examples are to be considered as exemplary only, and the true scope and nature of the invention are indicated by the claims. The sequence of steps shown in the accompanying drawings is for illustrative purposes only and is not limited to any particular sequence of steps. Therefore, it will be appreciated by those skilled in the art that the steps may be performed in different orders when implementing the same method.

[0049] In the drawings and specifications, exemplary embodiments have been disclosed. However, many variations and modifications may be made to these embodiments. Therefore, although specific terms are used, they are used only in a generic and descriptive sense and not for the purpose of limitation.

Claims

1. A method for optimizing data movement tasks associated with a machine learning model, comprising: identifying data movement tasks in executable programs of the device; identifying one or more operations associated with the data movement task; determining a specification of an output operation object of the identified one or more operations associated with the data movement task; When the minimum specification of the device configured to be written is equal to or less than the minimum specification of the output operation object, the hardware capability meets the requirement; In response to a determination that the hardware capability of the device meets the requirement, the identified data movement task is mapped to data writing of the identified one or more operations associated with the data movement task.

2. The method of claim 1 , wherein mapping the identified data movement task to write data of an operation associated with the data movement task comprises: generating a function that maps the output offset of the identified data movement task to a given input offset of the identified data movement task; as well as An output offset of the identified data movement task is redirected using a function based on the specification of the output operation object.

3. The method according to claim 2, wherein the function maps the offset in one of the following ways: modifying the page table of the operating system, constructing and maintaining a lookup table, calling a formula, or modifying the way the offset changes.

4. The method according to claim 1, further comprising: Prior to performing the identified one or more operations associated with the data movement task, a storage allocator is provided.

5. The method according to claim 1, further comprising: The mapped data movement tasks are eliminated.

6. An apparatus for optimizing data movement tasks associated with a machine learning model, the apparatus comprising: A memory that stores a set of instructions; and One or more processors configured to execute a set of instructions to cause the device to: identifying data movement tasks in executable programs of the device; identifying one or more operations associated with the data movement task; determining a specification of an output operation object of the identified one or more operations associated with the data movement task; When the minimum specification of the device configured to be written is equal to or less than the minimum specification of the output operation object, the hardware capability meets the requirement; as well as In response to a determination that the hardware capability of the device meets the requirement, the identified data movement task is mapped to data writing of the identified one or more operations associated with the data movement task.

7. The apparatus of claim 6, wherein mapping the identified data movement task to write data of an operation associated with the data movement task comprises: generating a function that maps the output offset of the identified data movement task to a given input offset of the identified data movement task; as well as The output offset of the identified data movement task is redirected using a function based on the specification of the output operation object.

8. The apparatus according to claim 7, wherein the function maps the offset in one of the following ways: modifying a page table of an operating system, constructing and maintaining a lookup table, calling a formula, or modifying a way in which the offset changes.

9. The apparatus of claim 6, wherein the one or more processors are configured to execute the set of instructions to cause the device to further perform: Prior to performing the identified one or more operations associated with the data movement task, a storage allocator is provided.

10. A non-transitory computer-readable medium storing a set of instructions executable by at least one processor of a computer to cause the computer to perform a method for optimizing a data movement task associated with a machine learning model, the method comprising: identifying a data movement task in an executable program of the device; identifying one or more operations associated with the data movement task; determining a specification of an output operation object of the identified one or more operations associated with the data movement task; When the minimum specification of the device configured to be written is equal to or less than the minimum specification of the output operation object, the hardware capability meets the requirement; as well as In response to a determination that the hardware capability of the device meets the requirement, the identified data movement task is mapped to data writing of the identified one or more operations associated with the data movement task.

11. The computer-readable medium of claim 10, wherein mapping the identified data movement task to write data of an operation associated with the data movement task comprises: generating a function that maps the output offset of the identified data movement task to a given input offset of the identified data movement task; as well as The output offset of the identified data movement task is redirected using a function based on the specification of the output operation object.

12. The computer-readable medium of claim 11, wherein the function maps the offset by one of the following means: modifying a page table of an operating system, constructing and maintaining a query table, calling a formula, or modifying an offset change manner.

13. The computer-readable medium of claim 10, wherein a set of instructions is executable by at least one processor of a computing device to cause the computing device to further perform: Prior to performing the identified one or more operations associated with the data movement task, a storage allocator is provided.

14. The computer-readable medium of claim 10, wherein the set of instructions is executable by at least one processor of the computing device to cause the computing device to further perform: The mapped data movement tasks are eliminated.

Images