Memory management method and device, hardware accelerator and storage medium

Abstract

The invention provides a memory management method and device, a hardware accelerator and a storage medium. The memory management method comprises the following steps: reading a first memory use instruction in a target operator; when it is determined that the barrier object is used by the first memory use instruction, associating a first memory used by the first memory use instruction with the barrier object; reading a pass-waiting notification barrier instruction in the target operator; when it is determined that the waiting for notification barrier instruction is associated with the barrier object and the first memory is also used, a life cycle end time of the first memory is set as a time at which execution of the waiting for notification barrier instruction is completed. Therefore, the end time of the life cycle of the first memory can be prolonged to the end of the execution of the waiting notification barrier instruction, and the first memory is not allocated to other memory use instructions during the period, so that a memory trampling event which is easy to occur in memory management in a traditional method is avoided, and the operator performance and the execution efficiency can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of memory management technology, and in particular to a memory management method, device, hardware accelerator, storage medium and computer program. Background Art

[0002] In asynchronous computing scenarios, operator developers develop operators based on the board and card, and these operators also apply for various types of memory for data storage and computation. These operators include various types of instructions, including but not limited to input / output instructions and computation instructions, and support asynchronous execution of these various types of instructions. For example, in related technologies, synchronization control mechanisms, such as barrier objects, can be used to control the execution of these various instructions in a parallel pipeline.

[0003] However, in the process of implementing this application, the inventors discovered that in traditional memory management schemes, there is a risk that the memory used by the previous instruction will continue to be used by the next instruction without being released, thereby causing a memory stampede event. In order to prevent the above-mentioned memory stampede event from occurring, it is usually necessary to adjust the instructions that were originally expected to run in a pipeline manner to run in a serial manner; although this can avoid the occurrence of memory stampede events, it will also affect the performance of the operator and reduce the execution efficiency of the operator. Summary of the Invention

[0004] The present application provides a memory management method, device, hardware accelerator, storage medium and computer program to address the defects of traditional memory management methods that are prone to memory stampede events and affect operator performance and execution efficiency.

[0005] This application provides a memory management method, including: Read the first memory usage instruction in the target operator; When determining that the first memory usage instruction uses a barrier object, associating a first memory used by the first memory usage instruction with the barrier object; Reading a wait-through notification barrier instruction in the target operator; When it is determined that the wait-for-notification barrier instruction is associated with the barrier object and also uses the first memory, the lifecycle end time of the first memory is set to the time when the wait-for-notification barrier instruction is completed.

[0006] According to the memory management method provided in this application, the reading of the first memory usage instruction in the target operator includes: Reading a first parameter list of the first memory usage instruction, the first parameter list including the first memory; The determining that the first memory usage instruction uses a barrier object includes: Determine that the first parameter list also includes the barrier object.

[0007] According to the memory management method provided by the present application, the reading of the wait-for-pass notification barrier instruction includes: Reading a second parameter list of the wait-for-pass notification barrier instruction; The determining that the wait is associated with the barrier object through a notify barrier instruction and also uses the first memory includes: Determine whether the second parameter list includes the barrier object and the first memory.

[0008] The memory management method provided by this application also includes: Reading a second memory usage instruction in the target operator, where the execution order of the second memory usage instruction in the target operator is between the first memory usage instruction and the wait-for-pass notification barrier instruction; When it is determined that the memory indicated by the second memory usage instruction is of the same type as the first memory, a second memory is allocated to the second memory usage instruction, and the memory addresses of the second memory and the first memory do not overlap.

[0009] According to the memory management method provided in the present application, the first memory and the second memory are both group shared memories.

[0010] The memory management method provided by this application also includes: After the wait-for-notification barrier instruction is executed, the first memory is released.

[0011] The present application also provides a memory management device, comprising: A first reading module, configured to read a first memory usage instruction in a target operator; an associating module, configured to, when determining that the first memory use instruction uses a barrier object, associate the first memory used by the first memory use instruction with the barrier object; A second reading module is used to read the wait-for-pass notification barrier instruction in the target operator; A setting module is used to set the life cycle end time of the first memory to the time when the wait-for-notification barrier instruction is completed when it is determined that the wait-for-notification barrier instruction is associated with the barrier object and also uses the first memory.

[0012] The present application also provides a hardware accelerator, comprising: implementing any of the memory management methods described above when the hardware accelerator executes a target operator.

[0013] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the memory management method described in any one of the above items is implemented.

[0014] The present application also provides a computer program, which, when executed by a processor, implements any of the above-mentioned memory management methods.

[0015] The memory management method, device, hardware counter, storage medium and computer program product provided in the present application read a first memory usage instruction in a target operator; when it is determined that the first memory usage instruction uses a barrier object, the first memory used by the first memory usage instruction is associated with the barrier object; read a wait-for-pass notification barrier instruction in the target operator; when it is determined that the above-mentioned wait-for-pass notification barrier instruction is associated with the barrier object and also uses the first memory, the life cycle end time of the first memory is set to the time when the wait-for-pass notification barrier instruction is completed. Therefore, in the process of memory management, the life cycle of the first memory is clearly extended to the time waiting for the execution of the notification barrier instruction to be completed, ensuring that the first memory use instruction can independently use the above-mentioned first memory during the life cycle of the first memory. Even if there are other memory use instructions, such as input and output (I / O) instructions or calculation instructions that request the same type of memory, the memory management device will no longer allocate memory with the same address, and can effectively avoid memory stampede incidents. At this time, the above-mentioned input and output (I / O) instructions or calculation instructions can be pipelined and run in parallel with the first memory use instruction, thereby effectively improving the performance of the operator and the execution efficiency of the operator, and effectively improving the resource utilization of the board. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in this application or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 This is a schematic diagram of an operator of a memory management solution in the related art; Figure 2 This is a flowchart of a memory management method provided by an embodiment of the present application; Figure 3 This is a schematic diagram of an operator of a memory management solution provided in an embodiment of the present application; Figure 4It is a serial execution logic diagram in the related art; Figure 5 This is a schematic diagram of the parallel execution logic provided by the embodiment of the present application; Figure 6 It is a structural diagram of the memory management device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0019] Figure 1 This is a diagram of an operator of a memory management solution in the related art, which shows the program code included in the traditional operator, that is, including Figure 1 The memory usage instructions shown in the related art provide a technical solution for memory management based on the above memory usage instructions. Specifically, the memory management technical solution may include: (1) Data loading instruction Load (GSM); The function of the data load instruction is to load data from external storage into group shared memory (GSM), where group shared memory is a type of memory. The data record instruction is a type of IO instruction. In a memory management scheme, a memory address may be allocated to the data record instruction, and the memory allocated thereto may be group shared memory (GSM) to store the data loaded according to the data load instruction. (2) Register load instruction LoadGSM2TLR(TLR, GSM, barrier); This register load instruction indicates that the data stored in the group shared memory GSM will be loaded into the timer load register (Timer Load Register, TLR for short). At the same time, since the parameter list of the register record instruction includes barrier, it means that the register load instruction starts the synchronization control mechanism, that is, a barrier object (barrier) needs to be assigned to it. The barrier mechanism is mainly to coordinate the execution order of multiple parallel execution units, such as the execution order of instruction streams, and for a barrier object, it can include two stages: passing the notification barrier (Pass barrier) and waiting to pass the notification barrier (Sync barrier). When executing this register load instruction, it is equivalent to the above-mentioned Passbarrier stage, indicating that the first memory usage instruction uses a barrier object (barrier).

[0020] In the related art, there is no limit on the release time of the memory used by the register loading instruction LoadGSM2TLR (TLR, GSM, barrier). Generally, after this instruction is executed, the first memory used by it, namely GSM, will be released. At this time, for a subsequently inserted asynchronous instruction using GSM, there is a probability that the original data in GSM will be destroyed before the current register loading instruction has finished using GSM.

[0021] (3) Asynchronous computing instruction instructionAsync(GSM); This asynchronous computing instruction asynchronously executes the computing task after the above-mentioned register loading instruction LoadGSM2TLR (TLR, GSM, barrier), and uses the same type of memory, namely the group shared memory GSM. Therefore, the memory management device can allocate memory for it. Normally, since the memory used by the previous register loading instruction does not have a clear specific usage time, it is released according to the principle after the instruction is executed. At this time, when allocating memory for the asynchronous computing instruction instructionAsync (GSM), since the required memory type is the same, the first memory with the same memory address will be allocated to it. However, if the specific execution completion time of the previous register loading instruction is uncertain at this time, if the previous register loading instruction LoadGSM2TLR has not finished using GSM, a memory stampede may occur, destroying the original data in GSM.

[0022] (4) Waiting for the notification barrier instruction waitloadInst(barrier); As mentioned above, the synchronization control mechanism is started through the register load instruction, and a barrier object (barrier) is allocated. This instruction is equivalent to forcing the synchronization point of related instructions in the Sync barrier stage. That is, after executing this instruction, the above register load instruction is completed, and subsequent operations can be executed and enter the next stage. This wait is only used for the synchronization of instruction execution by notifying the barrier instruction, and does not consider the synchronization of memory usage. For the asynchronous calculation instruction instructionAsync(GSM) in (3) above, since it will cause a memory stampede event, the above calculation instruction can be completed, that is, executed in the subsequent (5), thereby avoiding the occurrence of a memory stampede event.

[0023] (5) Asynchronous computing instruction instructionAsync(GSM); To avoid memory stampede, developers are forced to postpone the computational tasks that should be executed asynchronously in (3) until after the Sync Barrier stage when developing operators. This prevents them from forming a pipelined operation with the data loading instructions in (2).

[0024] (6) Matrix multiplication and accumulation calculation instruction MMA (TLR); The function of this calculation instruction is to perform matrix multiplication and accumulation, which uses the data loaded into the timer load register (TLR) in the register load instruction in (2) above.

[0025] Although barrier objects are used in the aforementioned related technologies, their primary function is to synchronize instructions. They do not limit the lifecycle of the memory used by the instructions, particularly the release time of the memory used by the instructions. Therefore, if various instructions, such as input / output (I / O) instructions and computational instructions, are run asynchronously, there is a certain probability of a memory stampede. Because memory stampedes in related technologies are prone to occur, instructions that could otherwise be executed in parallel need to be executed serially. This, however, reduces operator performance and execution efficiency. The present application provides a corresponding solution.

[0026] Figure 2 A flowchart of the memory management method provided in the embodiment of the present application is shown in FIG. Figure 2 As shown, the memory management method provided in the embodiment of the present application includes: Step 101: Read the first memory usage instruction in the target operator; The target operator of this step exists in the form of program code and can include multiple types of instructions, such as input and output (I / O) instructions, calculation instructions, etc., and the above-mentioned input and output (I / O) instructions and calculation instructions will use memory during the operation of the target operator. Therefore, the above-mentioned instructions in the above-mentioned target operator can also be called first memory use instructions. In the embodiment of the present application, the first memory use instruction can be, for example, an input and output (I / O) instruction, such as the above-mentioned Figure 1 The register loading instruction LoadGSM2TLR (TLR, GSM, barrier) instruction shown in the figure may include the instruction name LoadGSM2TLR and the first parameter list (TLR, GSM, barrier). The instruction is used to load the data stored in a group shared memory GSM into the timer load register TLR.

[0027] Step 102: When it is determined that the first memory usage instruction uses a barrier object, associate the first memory used by the first memory usage instruction with the barrier object; In the embodiment of the present application, the barrier object (Barrier), as a synchronization control mechanism, not only controls the execution order of instructions, but also controls the declaration cycle of the first memory used by the first memory usage instruction. Specifically, when determining that the first memory usage instruction uses the barrier object (Barrier) in this step, that is, in the notification passing the barrier (PassBarrier) stage, the first memory is associated with the barrier object, and then the release time of the first memory is clarified, thereby determining the life cycle of the first memory. Specifically, it can be in the subsequent waiting notification passing the barrier (Sync Barrier) stage, determining the release time of the above-mentioned first memory as the time to wait for the notification passing the barrier instruction to complete, thereby clarifying the start time of the life cycle of the first memory. In this case, if the same type of memory needs to be allocated for other memory usage instructions, memory with the same memory address will no longer be allocated within the life cycle of the first memory, thereby avoiding the occurrence of memory stampede events.

[0028] For the target operator, it can have multiple barrier objects, each of which constrains the execution order of some instructions. In this embodiment of the present application, the first memory used by the first memory-using instruction can be determined by reading the first parameter list of the first memory-using instruction. That is, the first parameter list includes the first memory and the barrier object. When it is determined that the barrier object is included in the first parameter list, it is determined that the first memory-using instruction uses the barrier object.

[0029] Step 103: Read the wait-through notification barrier instruction in the target operator; In this step, during the barrier object's wait-for-notification-barrier (sync barrier) phase, a wait-for-notification-barrier instruction is used to indicate the synchronization point of the first memory usage instruction associated with it. In the target operator, the wait-for-notification-barrier instruction can typically be a waitloadInst(barrier, GSM) instruction. In related technologies, the aforementioned wait-for-notification-barrier instruction is only associated with the barrier object to control the execution order of the first memory usage instruction. However, in the embodiment of the present application, the wait-for-notification-barrier instruction can be associated with the barrier object and also with the first memory, thereby constraining the lifecycle of the first memory.

[0030] Step 104 : When it is determined that the Wait Through Notification Barrier instruction is associated with the barrier object and also uses the first memory, set the lifecycle end time of the first memory to the time when the Wait Through Notification Barrier instruction is completed.

[0031] In the embodiment of the present application, specifically, the associated barrier object and the used first memory may be recorded in the second parameter list of the waiting-for-notification-barrier instruction.

[0032] Therefore, in this step, it can be determined that the second parameter list includes the barrier object and the above-mentioned first memory, so as to determine the waiting time to be associated with the barrier object through the notification barrier instruction, and also use the first memory; and in this case, the release time of the first memory, that is, the end time of the life cycle is set to the time of waiting for the completion of the execution of the notification barrier instruction, so that during the time of waiting for the completion of the execution of the notification barrier instruction, since it is clear that the first memory will not be released and will always be in a state of being used, the memory management device will no longer allocate it to other instructions for use.

[0033] The memory management method provided in the embodiment of the present application can set the end time of the life cycle of the first memory to the time for the execution of the instruction waiting to be executed through the notification barrier, so that the memory management device will no longer allocate the first memory to other instructions during the process of the first memory use instruction using the first memory. For other memory use instructions that need to use the same type of memory, other memory other than the first memory will be allocated to them, that is, memory with other addresses, so that other memory use instructions will not have data conflicts with the first memory use instruction, thereby effectively avoiding the occurrence of memory stampede events. In this case, other memory use instructions and the first memory use instruction can be executed in parallel, thereby realizing a pipeline instruction execution method, thereby improving the execution efficiency of the operator and improving the performance of the operator.

[0034] In the above embodiment of the present application, two memory usage instructions using the same type of memory can be executed in parallel without causing a memory stampede. That is, the target operator can also include a second memory usage instruction. In this case, the method for performing memory management for the target operator can further include: A second memory usage instruction in the target operator is read. The execution order of the second memory usage instruction in the target operator is between the first memory usage instruction and the wait-for-notification-barrier instruction, that is, the second memory usage instruction is executed after the first memory usage instruction and before the wait-for-notification-barrier instruction.

[0035] At this time, in an embodiment of the present application, when it is determined that the second memory used by the second memory use instruction is of the same type as the first memory, the second memory is allocated to the second memory use instruction, and the memory addresses of the second memory and the first memory do not overlap. Since the same type of memory has the same allocated address range, compared to the related art, the first memory is still allocated to the second memory use instruction. In the embodiment of the present application, the life cycle of the first memory is explicitly extended to the moment when the execution of the notification barrier instruction is completed, that is, the first memory is released at that moment. Therefore, the first memory will no longer be allocated to the second memory use instruction, but the second memory will be allocated, thereby effectively avoiding the occurrence of memory stampede incidents.

[0036] In the embodiment of the present invention, the first memory usage instruction and the second memory usage instruction include but are not limited to any one of an input / output (I / O) instruction and a calculation instruction.

[0037] In the embodiments of the present application, the first memory and the second memory may be various types of memory, such as global memory, group shared memory, etc. Group shared memory (GSM) may be a memory area shared between a group of processes or threads in multi-process / multi-threaded programming. This type of memory allows members of the same group to directly read and write to the same physical memory, thereby achieving efficient data exchange. This type of memory is prone to the aforementioned memory stampede incident, so the corresponding problem can be solved by the technical solution provided in the embodiments of the present application.

[0038] Correspondingly, in the technical solution provided in the embodiment of the present application, it is also possible to further release the above-mentioned first memory after waiting for the notification barrier instruction to be executed. At this time, the above-mentioned first memory will no longer be used, and no memory stampede event will occur, and the above-mentioned first memory can also be applied to subsequent other memory usage instructions.

[0039] Figure 3This is a schematic diagram of an operator of a memory management solution in an embodiment of the present application, which shows the program code included in the operator in the embodiment of the present application, namely, including Figure 3 The memory usage instructions shown are used, and memory management can be performed based on the above memory usage instructions. Specifically, the memory management solution may include: (1) Data loading instruction Load (GSM); The function of the data load instruction is to load data from external storage into the group shared memory GSM. The data recording instruction is a type of IO instruction. In the memory management scheme, a memory address can be allocated to the data load instruction. The memory allocated to it can be the group shared memory GSM, which can also be called the first memory, to store the data loaded according to the data load instruction; (2) Register load instruction LoadGSM2TLR(TLR, GSM, barrier); The register load instruction indicates that the data stored in the first memory, that is, the group shared memory GSM, will be loaded into the timer load register (Timer Load Register, abbreviated as: TLR). At the same time, since the parameter list of the register record instruction includes barrier, it means that the register load instruction starts the synchronization control mechanism, that is, a barrier object (barrier) needs to be assigned to it. In the embodiment of the present application, the barrier object can coordinate the execution order of multiple parallel execution units, such as the execution order of instruction streams, and for a barrier object, it can include two stages: passing the notification barrier (Pass barrier) and waiting for passing the notification barrier (Sync barrier). When this instruction is executed, it is equivalent to the above-mentioned Pass barrier stage, indicating that the first memory usage instruction uses a barrier object (barrier).

[0040] In the technical solution of the embodiment of the present application, the release time of the first memory used by the register loading instruction LoadGSM2TLR (TLR, GSM, barrier) is also limited, that is, it can be extended to the waiting for passing the notification barrier (Sync barrier) stage. After the waiting for passing the notification barrier instruction is executed, the above-mentioned first memory is released. For details, please refer to the subsequent description of the waiting for passing the notification barrier instruction waitloadInst (barrier) in (4).

[0041] After the memory management device learns that the register load instruction uses a barrier object and the specific release time of the first memory used by the register load instruction, it will not allocate memory with the same memory address as the first memory to other memory use instructions before the barrier object is released, thereby avoiding a memory stampede. (3) Asynchronous computing instruction instructionAsync(GSM); This asynchronous computing instruction asynchronously executes the computing task after the register load instruction LoadGSM2TLR(TLR, GSM, barrier) in (2) above, and uses the same type of memory, namely, the group shared memory GSM. Therefore, the memory management device can allocate memory for it. In the embodiment of the present application, since it is clear that the life cycle of the first memory used by the previous register load instruction will continue until the time when the notification barrier instruction waitloadInst(barrier) in (4) is executed, although the memory used by the two is of the same type, the same memory cannot be allocated to them. According to the technical solution provided in the embodiment of the present application, a second memory will be allocated for it, and the second memory has a different memory address from the first memory. At this time, the memory stampede incident will no longer occur.

[0042] (4) Waiting for the notification barrier instruction waitloadInst(barrier, GSM); As described above, the register load instruction is used to start the synchronization control mechanism and a barrier object is allocated. This instruction is equivalent to forcing the synchronization point of the relevant instructions in the Sync barrier stage, that is, after executing this instruction, the above register load instruction is completed, and subsequent operations can be executed and enter the next stage. In addition, the wait-through notification barrier instruction can also be used to indicate the release time of the first memory used by the register load instruction in (2), that is, the above first memory is released after the execution of this wait-through notification barrier instruction is completed. Specifically, the above first memory can be recorded in the parameter list of the wait-through notification barrier instruction, which is equivalent to the wait-through notification barrier instruction using the above first memory. In this way, the end time of the life cycle of the first memory is indicated. Through this implementation method, the occurrence of memory stampede events can be effectively avoided.

[0043] (5) Matrix multiplication and accumulation calculation instruction MMA (TLR); The function of this calculation instruction is to perform matrix multiplication and accumulation, which uses the data loaded into the timer load register (TLR) in the register load instruction in (2) above.

[0044] It can be seen that compared with traditional memory management schemes, there is a risk that the first memory used by the previous memory usage instruction will be used by the next memory usage instruction without being released, thereby causing a memory stampede event. In order to prevent the above-mentioned memory stampede event from occurring, it is usually necessary to adjust the instructions that were originally expected to run in a pipeline manner to run in a serial manner; in the technical solution provided in the embodiment of the present application, since the life cycle of the first memory used by the previous memory usage instruction is clarified, the occurrence of a memory stampede event can be effectively avoided, so that the two memory usage instructions can run in parallel in a pipeline manner, thereby improving the performance of the operator and the execution efficiency of the operator.

[0045] In the memory management method provided in the embodiment of the present application, the first memory is released only after the execution of the notification barrier instruction is completed. Therefore, in the embodiment of the present application, the time when the first memory starts to be enabled can be marked by the notification barrier (Pass Barrier), and the release timing of the first memory can be further controlled by waiting for the notification barrier (Sync Barrier), so that the life cycle of the first memory is extended from the Pass Barrier to the Sync Barrier, forming a memory safety use window, that is, within the life cycle of the above-mentioned first memory, it is no longer allowed to be allocated to other memory use instructions.

[0046] For traditional instruction serial execution logic, Figure 4 As shown, after a certain IO instruction 1 uses memory A, since the release cycle of memory A cannot be determined, when the subsequent calculation instruction 1 needs to allocate memory, it is still possible to allocate memory A for it, and a memory trampling event may occur at this time. The example method of this application can be as follows Figure 5 As shown, it can establish a memory safety usage window, that is, a barrier object (Barrier) can be used through the IO instruction 1, that is, in the PassBarrier stage, memory A is associated with the barrier object, and the memory management device will also mark memory A as being in use and extend its life cycle until the subsequent wait is completed by the notification instruction. Therefore, the first memory can no longer be allocated to other memory usage instructions at this time; at this time, when an asynchronous instruction is inserted, that is, the calculation instruction 1, memory A will not be allocated for it, but will be replaced by allocating memory B for it. At this time, no memory trampling incident will occur. Memory A will not be released until the wait is completed by the notification instruction.

[0047] The memory management method provided in the embodiment of the present application can mark the starting point of memory usage when the notification of the barrier object passes the barrier stage, and determine the end point of memory usage in the waiting stage for the notification to pass the barrier, thereby forming a memory safety usage window between the starting point and the end point. Within the above-mentioned memory safety usage window, the memory management device no longer allocates memory with the same memory address to other memory usage instructions that use the same type of memory. This mechanism can effectively break the limitation of traditional solutions that memory stampede incidents are prone to occur when memory usage instructions are executed in parallel, so that IO operation instructions and computing operation instructions can be truly executed in parallel; the utilization rate of the computing unit is significantly improved through instruction-level parallelism, which is suitable for scenarios such as AI training and real-time data processing that require high throughput.

[0048] The memory management device provided in the present application is described below. The memory management device described below and the memory management method described above can be referenced to each other.

[0049] Figure 6 A schematic diagram of the structure of the memory management device provided in the embodiment of the present application is shown as follows: Figure 6 As shown, the memory management device provided in the embodiment of the present application includes: A first reading module 601 is configured to read a first memory usage instruction in a target operator; An associating module 602 is configured to, when determining that the first memory use instruction uses a barrier object, associate the first memory used by the first memory use instruction with the barrier object; The second reading module 603 is used to read the wait-for-pass notification barrier instruction in the target operator; The setting module 604 is configured to set the lifecycle end time of the first memory to the time when the wait-for-notification barrier instruction is completed when it is determined that the wait-for-notification barrier instruction is associated with the barrier object and also uses the first memory.

[0050] The memory management device provided in an embodiment of the present application reads a first memory usage instruction within a target operator; upon determining that the first memory usage instruction uses a barrier object, associates the first memory used by the first memory usage instruction with the barrier object; reads a wait-for-pass notification barrier instruction within the target operator; and upon determining that the wait-for-pass notification barrier instruction is associated with the barrier object and also uses the first memory, sets the lifecycle end time of the first memory to the time when the wait-for-pass notification barrier instruction completes execution. This ensures that, during the memory management process, the first memory usage instruction can independently use the first memory within its lifecycle. Even if other memory usage instructions, such as input / output (I / O) instructions or computation instructions, request the same type of instruction, the memory management module will not re-allocate the same memory, effectively avoiding memory stampede events. In this case, the I / O instructions or computation instructions can be pipelined and executed asynchronously and in parallel with the first memory usage instruction, thereby improving operator performance, operator execution efficiency, and board resource utilization.

[0051] An embodiment of the present application also provides a hardware accelerator, comprising: when the hardware accelerator executes a target operator program, implementing a memory management method as described in any one of the above, the method comprising: reading a first memory usage instruction in the target operator; when it is determined that the first memory usage instruction uses a barrier object, associating a first memory used by the first memory usage instruction with the barrier object; reading a wait-for-pass notification barrier instruction in the target operator; when it is determined that the wait-for-pass notification barrier instruction is associated with the barrier object and also uses the first memory, setting the lifecycle end time of the first memory to the time when the wait-for-pass notification barrier instruction is completed.

[0052] In an embodiment of the present application, the hardware accelerator supports asynchronous execution instructions, and the asynchronous execution instructions include: IO instructions and calculation instructions.

[0053] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the memory management method provided by the above-mentioned methods, the method comprising: reading a first memory usage instruction in a target operator; when determining that the first memory usage instruction uses a barrier object, associating the first memory used by the first memory usage instruction with the barrier object; reading a wait-for-pass notification barrier instruction in the target operator; when determining that the wait-for-pass notification barrier instruction is associated with the barrier object and also uses the first memory, setting the life cycle end time of the first memory to the time when the wait-for-pass notification barrier instruction is executed.

[0054] On the other hand, the present application also provides a computer program, which, when executed by a processor, is implemented to execute the memory management method provided by the above-mentioned methods, the method including: reading a first memory usage instruction in a target operator; when determining that the first memory usage instruction uses a barrier object, associating the first memory used by the first memory usage instruction with the barrier object; reading a wait-for-pass notification barrier instruction in the target operator; when determining that the wait-for-pass notification barrier instruction is associated with the barrier object and also uses the first memory, setting the life cycle end time of the first memory to the time when the wait-for-pass notification barrier instruction is executed.

[0055] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0056] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A memory management method, characterized in that: include: Read the first memory usage instruction in the target operator; When determining that the first memory usage instruction uses a barrier object, associating a first memory used by the first memory usage instruction with the barrier object; Reading a wait-through notification barrier instruction in the target operator; When it is determined that the wait-for-notification barrier instruction is associated with the barrier object and also uses the first memory, the lifecycle end time of the first memory is set to the time when the wait-for-notification barrier instruction is completed.

2. The memory management method according to claim 1, wherein: The reading the first memory usage instruction in the target operator includes: Reading a first parameter list of the first memory usage instruction, the first parameter list including the first memory; The determining that the first memory usage instruction uses a barrier object includes: Determine that the first parameter list also includes the barrier object.

3. The memory management method according to claim 2, wherein: The reading of the wait-through notification barrier instruction includes: Reading a second parameter list of the wait-for-pass notification barrier instruction; The determining that the wait is associated with the barrier object through a notify barrier instruction and also uses the first memory includes: Determine whether the second parameter list includes the barrier object and the first memory.

4. The memory management method according to any one of claims 1 to 3, characterized in that: Also includes: Reading a second memory usage instruction in the target operator, where the execution order of the second memory usage instruction in the target operator is between the first memory usage instruction and the wait-for-pass notification barrier instruction; When it is determined that the memory indicated by the second memory usage instruction is of the same type as the first memory, a second memory is allocated to the second memory usage instruction, and the memory addresses of the second memory and the first memory do not overlap.

5. The memory management method according to claim 4, characterized in that: The first memory and the second memory are both group shared memories.

6. The memory management method according to any one of claims 1 to 3, characterized in that: Also includes: After the wait-for-notification barrier instruction is executed, the first memory is released.

7. A memory management device, characterized in that: include: A first reading module, configured to read a first memory usage instruction in a target operator; an associating module, configured to, when determining that the first memory use instruction uses a barrier object, associate the first memory used by the first memory use instruction with the barrier object; A second reading module is used to read the wait-for-pass notification barrier instruction in the target operator; A setting module is used to set the life cycle end time of the first memory to the time when the wait-for-notification barrier instruction is completed when it is determined that the wait-for-notification barrier instruction is associated with the barrier object and also uses the first memory.

8. A hardware accelerator, characterized in that: include: The hardware accelerator implements the memory management method according to any one of claims 1 to 6 when executing the target operator.

9. A non-transitory readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the memory management method according to any one of claims 1 to 6 is implemented.

10. A computer program, wherein when executed by a processor, the computer program implements the memory management method according to any one of claims 1 to 6.

Images