Memory management circuit, method, and electronic device

Abstract

A memory management method, applicable to a memory management circuit, includes: moving a data unit from a repatriation page of a first memory device to a reallocation page of a second memory device to update storage space provided for an ongoing task, and generating a status table to record the status of the data unit during the movement of the data unit; maintaining a mapping table of the ongoing task during the movement of the data unit, and updating the mapping table according to the movement result of the data unit; and performing an address mapping operation during the movement of the data unit according to the access command, the maintained mapping table, and the status table.

Description

Technical Field

[0001] This invention relates to memory management, and more specifically, to memory management circuits, electronic devices (e.g., electronic devices including memory management circuits), and memory management methods. Prior Technology

[0002] In data computing systems, low-latency storage devices implemented using software-managed Static Random Access Memory (SRAM) can be placed near the computing engine, resulting in better performance in terms of power efficiency, performance, and circuit area compared to cache memory devices. However, low-latency storage devices also have some drawbacks. For example, when multiple tasks are scheduled and start or end at different times, the overall optimal allocation of storage resources for these tasks may change over time, but the storage allocation of low-latency storage devices is difficult to change dynamically (e.g., it is not possible to arbitrarily remove occupied storage space from tasks or allocate free storage space to tasks during task execution). Furthermore, low-latency storage devices are not good at data isolation, thus requiring additional costs to encrypt data in different contexts using different keys.

[0003] Therefore, a new mechanism is needed to manage storage resources for AI computing tasks so that low-latency storage devices can be utilized more efficiently. Summary of the Invention

[0004] This invention provides memory management circuits, methods, and electronic devices to ensure that memory resources can be used effectively.

[0005] In an optional embodiment, the present invention provides a memory management circuit, comprising: an allocation control circuit configured to move a data unit from a withdrawal page of a first memory device to a reallocation page of a second memory device to update the storage space provided for an ongoing task, and to generate a status table to record the status of the data unit during the movement of the data unit; a mapping table control circuit configured to maintain a mapping table of the ongoing task during the movement of the data unit, and to update the mapping table according to the movement result of the data unit, wherein the mapping table records the mapping relationship between the address used in the access command of the ongoing task and the address of the first memory device or the second memory device; and address mapping logic coupled to the allocation control circuit and the mapping table control circuit, configured to perform an address mapping operation according to the access command, the mapping table maintained by the mapping table control circuit, and the status table during the movement of the data unit.

[0006] In another alternative embodiment, the present invention provides an electronic device comprising: a computing circuit configured to perform an ongoing task; a first memory device configured to selectively provide a first storage space to the computing circuit; a second memory device configured to selectively provide a second storage space to the computing circuit; and a memory management circuit coupled to the computing circuit, the first memory device, and the second memory device, including a controller and address mapping logic, wherein the controller includes: allocation control circuitry configured to move data units from a withdrawn page of the first memory device to a reallocated page of the second memory device to update the storage space provided for the ongoing task, and to generate a status table to record the data. The state of the unit during the movement of the data unit; and the mapping table control circuit, configured to maintain the mapping table of the ongoing task during the movement of the data unit, and update the mapping table according to the movement result of the data unit, wherein the mapping table records the mapping relationship between the address used in the access command of the ongoing task and the address of the first memory device or the second memory device; wherein the address mapping logic couples the allocation control circuit and the mapping table control circuit, and is configured to perform address mapping operation according to the access command, the mapping table maintained by the mapping table control circuit and the state table during the movement of the data unit.

[0007] In another optional embodiment, the present invention provides a memory management method applicable to a memory management circuit, comprising: moving a data unit from a withdrawal page of a first memory device to a reallocation page of a second memory device to update the storage space provided for an ongoing task, and generating a status table to record the status of the data unit during the movement of the data unit; maintaining a mapping table of the ongoing task during the movement of the data unit, and updating the mapping table according to the movement result of the data unit, wherein the mapping table records the mapping relationship between the address used in the access command of the ongoing task and the address of the first memory device or the second memory device; and performing an address mapping operation during the movement of the data unit according to the access command, the maintained mapping table and the status table. Simple Explanation of the Diagram

[0008] Figure 1 shows an electronic device 10 according to an embodiment of the present invention. Figure 2 illustrates the first exemplary challenge encountered by TCM 121. Figure 3 illustrates the second exemplary challenge encountered by TCM 121. Figure 4 illustrates the third exemplary challenge encountered by TCM 121. Figure 5 illustrates the retraction operation of TCM 121 according to an embodiment of the present invention. Figure 6 illustrates the reallocation operation of TCM 121 according to an embodiment of the present invention. Figure 7 shows a diagram of data isolation of TCM 121 according to an embodiment of the present invention. Figure 8 illustrates the workflow of the memory management method according to an embodiment of the present invention. Implementation

[0009] Certain terms are used in the following descriptions and requests that refer to specific components. Those skilled in the art will understand that electronics manufacturers may use different names to refer to the same component. This document is not intended to distinguish between components with different names but identical functions. In the following descriptions and requests, the terms “comprising” and “including” are used in an open-ended manner and should therefore be interpreted as “including, but not limited to…”. Furthermore, the term “coupled” is intended to indicate an indirect or direct electrical connection. Therefore, if one device is coupled to another device, the connection can be a direct electrical connection or an indirect electrical connection via other devices and connections.

[0010] Figure 1 illustrates an electronic device 10 (e.g., an artificial intelligence (AI) related system) according to an embodiment of the present invention. The electronic device 10 includes one or more computing circuits (or computing engines), such as accelerators 111, 112, and 113 (e.g., AI acceleration circuitry), a first memory device, such as tightly coupled memory (TCM) 121, a second memory device, such as dynamic random access memory (DRAM) 122, and memory management circuitry 100 coupled to accelerators 111, 112, and 113, TCM 121, and DRAM 122. TCM 121 may be implemented using software-managed static random access memory (SRAM) placed near accelerators 111, 112, and 113; it may also be referred to as temporary memory or a software-managed cache, but the invention is not limited thereto. More specifically, compared to a typical cache or DRAM 122, TCM 121 can achieve lower power consumption and better performance (e.g., faster access speed). In this embodiment, accelerators 111, 112, and 113 are configured to perform one or more tasks. When accelerators 111, 112, and 113 perform one or more tasks, storage resources / storage space (e.g., storage pages, typically 4 KB, 8 KB, 16 KB, 32 KB, or 64 KB) of TCM 121 and DRAM 122 are provided to the accelerators. More specifically, memory management circuitry 100 is configured to manage the allocation of TCM storage resources of TCM 121 and DRAM storage resources of DRAM 122 during the execution of one or more tasks.

[0011] In some embodiments, memory management circuitry 100 includes a controller 130 and address mapping logic 140 (e.g., address mapping logic circuitry), wherein address mapping logic 140 is coupled to controller 130. Controller 130 is configured to perform memory allocation of TCM 121 and DRAM 122 to update storage space provided for computing circuitry (e.g., an ongoing task of accelerator 111). Controller 130 includes allocation control circuitry (e.g., synchronization management circuitry 131) and mapping table control circuitry (e.g., proxy mapping table control circuitry 132). Synchronization management circuitry 131 is configured to move data cells from withdrawn pages in a first memory device to re-assigned pages in a second memory device to update storage space provided for an ongoing task and to generate a state table to record the state of the data cell during the data cell's movement. The state table is used to maintain data consistency between withdrawn pages in the first memory device and re-assigned pages in the second memory device. The proxy mapping table control circuit 132 is configured to maintain the mapping table of the ongoing task during the movement of the data unit and update the mapping table according to the movement result of the data unit, wherein the mapping table records the mapping relationship between the address used in the access command of the ongoing task and the address of the first memory device and / or the second memory device. In one embodiment, the synchronization management circuit 131 is configured to move the data unit from the withdrawal page (e.g., TCM1) of TCM 121 to the reallocation page (e.g., DRAM0) of DRAM 122 to update the storage space provided for the ongoing task. The synchronization management circuit 131 also generates a status table 141S (possibly stored in the mapping table cache 141 of the address mapping logic 140), wherein the status table 141S records the status of the data unit moved during the movement of the data unit from the withdrawal page of TCM 121 to the reallocation page of DRAM 122. In another example, the synchronization management circuit 131 is configured to move data units from a withdrawn page (e.g., DRAM0) of DRAM 122 to a reallocation page (e.g., TCM0) of TCM 121 to update the storage space provided for the ongoing task. The synchronization management circuit 131 also generates a status table 141S' (possibly stored in the mapping table cache 141 of the address mapping logic 140), wherein the status table 141S' records the status of the data units moved during the process of moving data units from the withdrawn page of DRAM 122 to the reallocation page of TCM 121.In one embodiment, the proxy mapping table control circuit 132 is configured to maintain a mapping table 141M (possibly stored in the mapping table cache 141 of the address mapping logic 140) during the movement of data units, and to update the mapping table 141M according to the result of the data unit movement. The mapping table 141M records the mapping relationship between the address used in the access command (sent by the computing engine) and the address of DRAM 122 and / or TCM 121. The mapping table 141M is updated according to the movement result of replacing a withdrawn page of TCM 121 with a reallocated page of DRAM 122, and / or according to the movement result of replacing a withdrawn page of DRAM 122 with a reallocated page of TCM 121. The address mapping logic 140 is configured to perform address mapping operations related to TCM 121 and DRAM 122. In one embodiment, the address mapping logic 140 is configured to perform address mapping operations based on the access command (sent by the computing engine), the maintained mapping table 141M, and the status table during the data unit movement process. Address mapping logic 140 may include a mapping table cache 141, which further includes a mapping table 141M, a status table 141S, and a status table 141S'. In this case, address mapping logic 140 is configured to perform address mapping operations based on the access command of the ongoing task, the maintained mapping table 141M, and the status table (status table 141S or status table 141S') during data unit movement. In some embodiments, address mapping logic further includes a collision buffer 142. Synchronization management circuitry 131 determines whether to retain a specific access command in collision buffer 142 based on the status of a specific data unit corresponding to a specific access command in the data unit. For example, when the status of a specific data unit corresponding to a specific access command indicates that the specific data unit has been copied from the TCM or DRAM but not written to the reallocation page in the DRAM or TCM, the specific access command in the access command is retained in collision buffer 142. It should be noted that the architecture and layout shown in FIG1 are for illustrative purposes only and are not intended to limit the invention. For example, one or more circuit blocks may be further added to the memory management circuitry 100. Furthermore, the collision buffer 142 need not be implemented in the address mapping logic 140. For example, the collision buffer 142 can be implemented in the controller 130 or other parts of the memory management circuitry 100. Furthermore, the mapping table 141M, the state table 141S, and the state table 141S' need not be stored in the mapping table cache 141. For example, the mapping table 141M, the state table 141S, and / or the state table 141S' can be stored in other parts of the controller or the memory management circuitry 100.

[0012] Figure 2 illustrates the first exemplary challenge encountered by TCM 121. As shown in Figure 2, scheduler 20 can sequentially assign tasks (e.g., T0, T1, T2, and T3) to the corresponding computing circuits. For example, tasks T0 and T1 can be assigned to computing engine 110, which can be any of the accelerators 111, 112, and 113 shown in Figure 1. In this embodiment, tasks T0, T2, and T3 are normal tasks, while task T1 is a high-priority task (its priority is higher than tasks T0, T2, and T3). According to the scheduler's arrangement, task T0 is the first task assigned to computing engine 110. More specifically, at the moment computing engine 110 receives task T0, some or all of the TCM 121's storage resources (e.g., storage pages) may be available and therefore assigned to task T0. When scheduler 20 assigns task T1 to computing engine 110, some or all of the storage resources of TCM 121 may already be occupied by task T0, and computing engine 110 cannot access TCM 121 to execute task T1 (which has a higher priority). Therefore, the present invention proposes a memory management circuit 100 to reclaim the storage resources of TCM 121 used by T0 (e.g., storage page TCM1 in FIG. 5), and to reallocate the storage resources of TCM 121 (e.g., storage page TCM1 in FIG. 5) to task T1 without stopping the execution of tasks T0 and T1.

[0013] Figure 3 illustrates the second exemplary challenge encountered by TCM 121. In this embodiment, task T2 is assigned to a first computing engine (labeled "Engine-1" in Figure 3 for simplicity), and task T3 is assigned to a second computing engine (labeled "Engine-2" in Figure 3 for simplicity), where the first and second computing engines can be two of accelerators 111, 112, and 113, respectively. As shown in Figure 3, when the second computing engine receives task T3, task T2, executed by the first computing engine, is already in progress and occupies the resources of TCM 121. The second computing engine can choose to use alternative storage space, such as DRAM 122, to execute task T3. After task T2 is completed, TCM 121 becomes idle, but task T3 is already in progress in the alternative storage space DRAM 122. Therefore, unless the memory allocation for task T3 is reconfigured, TCM 121 will not be utilized. Based on this situation, the present invention proposes a memory management circuit 100 to reallocate the storage resources of TCM 121 (e.g., storage page TCM1 in FIG. 5) to the task T3 executed by the second computing engine when the task T2 executed by the first computing engine is completed or not completed. More specifically, the present invention proposes to transfer the data related to task T3 from DRAM 122 to TCM 121 during the reallocation without stopping the execution of task T3.

[0014] Figure 4 illustrates the third exemplary challenge encountered by TCM 121. As shown in Figure 4, task T4 belongs to a first context (labeled "Context-1" in Figure 4 for simplicity), and task T5 belongs to a second context (labeled "Context-2" in Figure 4 for simplicity). Since tasks T4 and T5 belong to different contexts (e.g., assigned by different users), the data associated with task T4 and the data associated with task T5 need to be isolated. In this embodiment, both tasks T4 and T5 are executed by the same computing engine (labeled "Engine-0" in Figure 4 for simplicity), requiring a period of silence to completely clear the SRAM contents (i.e., the data associated with task T4) before the storage resources (e.g., storage pages) previously storing the data associated with task T4 can be reallocated to task T5. In practice, the silence time for clearing all data associated with task T4 can be very long. Therefore, the present invention proposes a memory management circuit 100 to prevent TCM 121 from spending a long period of silence to clear all data of task T4 and to apply a reallocation scheme to reduce the waiting time of task T5. Furthermore, this reallocation mechanism allows task T5 to perform TCM 121 cleanup simultaneously when it starts executing on DRAM 122. Once TCM 121 is cleaned up, the TCM resources can be reallocated to T5 without stopping its execution.

[0015] Figure 5 illustrates the repossession operation of TCM 121 according to an embodiment of the present invention. In this embodiment, TCM 121 may include multiple TCM pages such as TCM0, TCM1, TCM2, TCM3, TCM4, and TCM5, and DRAM 122 may include multiple DRAM pages such as DRAM0, DRAM1, DRAM2, DRAM3, DRAM4, and DRAM5. When computing engine 50 (which may be any one of accelerators 111, 112, and 113) requests storage space to perform a task (e.g., task T0 in Figure 2), memory management circuitry 100 allocates a portion of TCM 121 (e.g., TCM pages TCM1 and TCM4) and a portion of DRAM 122 (e.g., DRAM pages DRAM1, DRAM2, DRAM3, and DRAM4) as storage space to serve the task (in other embodiments, a task may occupy only one type of memory resource, such as only TCM 121 or only DRAM 122). This task becomes an ongoing task (e.g., a task being processed) when computing engine 50 begins executing the task. Depending on the memory allocation, when the computing engine 50 sends an access command for an ongoing task to the memory management circuit 100, the access command may carry addresses in address regions ADDR0, ADDR1, ADDR2, ADDR3, ADDR4, or ADDR5. The memory management circuit 100 (e.g., its address mapping logic 140) can perform an address mapping operation according to the mapping table 141M to point the access addresses in ADDR0, ADDR1, ADDR2, ADDR3, ADDR4, or ADDR5 to the corresponding storage spaces in the TCM. In one embodiment, the mapping table 141M points multiple addresses of ADDR0 to multiple corresponding addresses of TCM page TCM4, multiple addresses of ADDR1 to multiple corresponding addresses of TCM page TCM1, multiple addresses of ADDR2 to multiple corresponding addresses of DRAM page DRAM1, multiple addresses of ADDR3 to multiple corresponding addresses of DRAM page DRAM2, multiple addresses of ADDR4 to multiple corresponding addresses of DRAM page DRAM3, and multiple addresses of ADDR5 to multiple corresponding addresses of DRAM page DRAM4.

[0016] When a new task with higher priority (e.g., task T1 in Figure 2) needs to be executed but TCM 121 is already fully occupied, memory management circuitry 100 (e.g., synchronization management circuitry 131) may receive a TCM management command that may instruct memory management circuitry 100 to select one or more specific TCM pages of TCM 121 occupied by the ongoing task to be reassigned to the new task. For better illustration, assume that memory management circuitry 100 (e.g., synchronization management circuitry 131) selects to reassign TCM page TCM1 to the new task, and more specifically, assume that memory management circuitry 100 (e.g., synchronization management circuitry 131) replaces TCM page TCM1 with an available DRAM page (e.g., DRAM page DRAM0) and configures that available DRAM page to serve the ongoing task being executed by computing engine 50. The TCM page TCM1 to be reassigned to the new task may be a withdrawn page of the ongoing task, while the DRAM page DRAM0 may be a reassigned page of the ongoing task. More specifically, after the synchronization management circuit 131 selects TCM page TCM1 as the withdrawal page, the synchronization management circuit 131 may divide the data related to the ongoing task stored in TCM page TCM1 into N data units. These N data units are initially stored in N TCM subpages LINE-0, LINE-1, LINE-2, LINE-3, LINE-4, LINE-5, ..., and LINE-N, respectively. These N data units will be moved to the corresponding alternative storage space, such as DRAM subpages T2D-0, T2D-1, T2D-2, T2D-3, T2D-4, T2D-5, ..., and T2D-N of DRAM page DRAM0. As shown in Figure 5, the synchronization management circuit 131 may generate a state table 141S, which records the states S-0, S-1, S-2, S-3, S-4, S-5, ..., and SN of N data units (stored in N TCM sub-pages LINE-0, LINE-1, LINE-2, LINE-3, LINE-4, LINE-5, ..., and LINE-N, respectively). These data units will be moved from TCM page TCM1 to DRAM page DRAM0. Specifically, when a page (e.g., TCM page TCM1) is selected as a revoked page, the synchronization management circuit 131 may mark the states of all N data units stored on the revoked page (more specifically, stored in N TCM sub-pages LINE-0, LINE-1, LINE-2, LINE-3, LINE-4, LINE-5, ..., and LINE-N) as a first state, such as "incomplete". The page is selected as a revoked page when it is selected to be revoked from the storage space provided for an ongoing task.The synchronization management circuit 131 may copy one or more data units with an "incomplete" state. When a specific data unit is copied from a withdrawn page but not written to a reallocated page, the synchronization management circuit 131 may change the state of a specific data unit from a first state (e.g., "incomplete") to a second state (e.g., "copying in progress"). For example, when a data unit stored in TCM subpage LINE-0 is copied from LINE-0 but not written to a DRAM subpage, such as T2D-0, the synchronization management circuit 131 may change state S-0 from "incomplete" to "copying in progress." When a data unit stored in TCM subpage LINE-1 is copied from LINE-1 but not written to a DRAM subpage, such as T2D-1, the synchronization management circuit 131 may change state S-1 from "incomplete" to "copying in progress." Similarly, when a data unit stored in the TCM subpage LINE-N is copied from LINE-N but not written to the DRAM subpage, such as T2D-N, the synchronization management circuit 131 may change the state SN from "incomplete" to "copying". In this embodiment, the N data units stored in the retraction page may be copied at different times (e.g., N data units copied one by one), so the states S-0, S-1, S-2, S-3, S-4, S-5, ... and SN may be changed from "incomplete" to "copying" at different times. Furthermore, the synchronization management circuit 131 may change the state of a specific data unit from a second state (e.g., "copying" state) to a third state (e.g., "complete") when the copied data unit is written to a redistributed page, such as the DRAM page DRAM0. For example, when a copied data unit corresponding to a data unit stored in TCM subpage LINE-0 is written to DRAM subpage T2D-0, the synchronization management circuit 131 may change state S-0 from "copying" to "complete". When a copied data unit corresponding to a data unit stored in TCM subpage LINE-1 is written to DRAM subpage T2D-1, the synchronization management circuit 131 may change state S-1 from "copying" to "complete". Similarly, when a copied data unit corresponding to a data unit stored in TCM subpage LINE-N is written to DRAM subpage T2D-N, the synchronization management circuit 131 may change state SN from "copying" to "complete". In this embodiment, copied data units corresponding to N data units may be written to DRAM page DRAM0 at different times (e.g., written one by one), therefore states S-0, S-1, S-2, S-3, S-4, S-5, ... and SN may be changed from "copying" to "complete" at different times.After all N data units have their states S-0, S-1, S-2, S-3, S-4, S-5, ... and SN marked as "complete", the proxy mapping table generator 132 may update the mapping table based on the move results (or reallocation results) (indicating the updated allocation storage space formed by TCM page TCM4 and DRAM pages DRAM0, DRAM1, DRAM2, DRAM3, and DRAM4).

[0017] In this embodiment, during the movement of a data unit from TCM page TCM1 to DRAM page DRAM0, when a specific access command accesses an unrevoked memory space (e.g., any memory space in TCM page TCM4 and DRAM pages DRAM1, DRAM2, DRAM3, and DRAM4), the memory management circuit 100 (e.g., address mapping logic 140) can perform an address mapping operation according to the mapping table 141M (unupdated version) to point the access address of the specific access command to the corresponding memory space. When a specific access command accesses a memory space that will be revoked (e.g., a specific TCM subpage in TCM page TCM1), the memory management circuit 100 (e.g., address mapping logic 140) can perform an address mapping operation according to the mapping table 141M (unupdated version) and the status table 141S. When the memory management circuit 100 receives a specific access command from the computing circuit 50, the state of the specific data unit is in the first state (e.g., "incomplete"), which means that the specific data unit stored in the specific TCM subpage has not yet been copied. The address mapping logic 140 addresses the specific data unit in the withdrawn page according to the mapping table and causes the specific access command to access the specific data unit in the withdrawn page (e.g., the specific TCM subpage of TCM page TCM1). When the memory management circuit 100 receives a specific access command from the computing circuit 50, the state of the specific data unit is in the second state (e.g., "copying"), which means that the specific data unit stored in the specific TCM subpage has been copied from the withdrawn page but has not yet been moved / written to the reallocated page (e.g., the DRAM subpage corresponding to the specific TCM subpage). This may be referred to as a collision event. The synchronization management circuit 131 can keep the specific access command corresponding to the specific data unit in the collision buffer 142. The address mapping logic 140 can prevent the access command from being executed, so as to temporarily prevent (at least until the state of the specific data unit becomes "complete") the execution of the specific access command. When the status table indicates that the status of a specific data unit changes from the second state to the third state (i.e., from "copying" to "complete"), the synchronization management circuit 131 can notify the address mapping logic 140, which addresses the specific data unit in the reallocated page according to the maintained mapping table 141M and the status table (e.g., status table 141S), and makes an access command (obtained from the collision buffer 142) access the specific data unit in the reallocated page (e.g., the DRAM subpage corresponding to a specific TCM subpage).When the memory management circuit 100 receives a specific access command from the computing circuit 50, the specific data unit is in a third state (e.g., "completed"), which means that the specific data unit stored in the specific TCM subpage (or the copy data unit corresponding to the specific data unit) has been moved to a reassigned page (e.g., the DRAM subpage corresponding to the specific TCM subpage). The address mapping logic 140 addresses the specific data unit in the reassigned page according to the mapping table 141M (unupdated version) and the status table (e.g., status table 141S), and causes the specific access command to access the copy data unit in the reassigned page (e.g., the DRAM subpage of DRAM page DRAM0). Under the above control scheme, during the reassignment process of replacing TCM page TCM1 with DRAM page DRAM0, the computing engine 50 will not be aware of the reassignment (e.g., the execution of the ongoing task will not be interrupted by the reassignment). When the TCM 121 storage resource (e.g., storage page TCM1 in Figure 5) allocated to T0 is reclaimed, the TCM 121 storage resource (e.g., storage page TCM1 in Figure 5) can be reassigned to task T1, which has a higher priority than T0. This ensures that task T1, with a higher priority than T0, can utilize the TCM 121 storage resource more quickly.

[0018] Figure 6 illustrates the reallocation operation of TCM 121 according to an embodiment of the present invention. In this embodiment, when the computing engine 50 requests storage space to execute a task (e.g., a new task), since TCM 121 is completely occupied by a previous task, TCM pages TCM0, TCM1, TCM2, TCM3, TCM4, and TCM5 are therefore classified into the occupancy pool 610, and the memory management circuit 100 allocates DRAM 122 (e.g., DRAM pages DRAM0, DRAM1, DRAM2, DRAM3, DRAM4, and DRAM5) as storage space for the task (i.e., the new task). When the computing engine 50 begins executing a task (e.g., a new task), the task may be an example of the aforementioned ongoing task (e.g., a task currently being processed). Depending on the memory allocation, when the computing engine 50 sends an access command for an ongoing task to the memory management circuit 100, the access command may carry an address in the address region ADDR0, ADDR1, ADDR2, ADDR3, ADDR4, or ADDR5. The memory management circuit 100 (e.g., its address mapping logic 140) can perform an address mapping operation according to the mapping table 141M to point the address in ADDR0, ADDR1, ADDR2, ADDR3, ADDR4, or ADDR5 to the corresponding storage space in the DRAM. In one embodiment, mapping table 141M points multiple addresses of ADDR0 to multiple corresponding addresses of DRAM page DRAM0, multiple addresses of ADDR1 to multiple corresponding addresses of DRAM page DRAM1, multiple addresses of ADDR2 to multiple corresponding addresses of DRAM page DRAM2, multiple addresses of ADDR3 to multiple corresponding addresses of DRAM page DRAM3, multiple addresses of ADDR4 to multiple corresponding addresses of DRAM page DRAM4, and multiple addresses of ADDR5 to multiple corresponding addresses of DRAM page DRAM5.

[0019] In this embodiment, when a previous task occupying a page of TCM 121 is completed or not yet fully completed (but the data stored in the occupied page has been used by a previous task), the synchronization management circuit 131 can clear the previous data stored in the occupied page to convert the occupied page into a blank page (e.g., by clearing the previous data stored in the occupied page by filling the occupied page with zero or a default value), and move the blank page to the reallocation pool 620 to allow the blank page to be selected as a reallocation page. For example, after a previous task occupying TCM 121 is completed, the synchronization management circuit 131 can begin clearing the previous data stored in each TCM page TCM0, TCM1, TCM2, TCM3, TCM4, and TCM5, thereby moving each TCM page TCM0, TCM1, TCM2, TCM3, TCM4, and TCM5 to the reallocation pool 620 page by page. When at least one TCM page (e.g., TCM0) is moved to the reallocatable pool 620, the memory management circuit 100 can perform a reallocation operation, replacing at least one DRAM page (e.g., DRAM page DRAM0) with at least one TCM page (e.g., TCM page TCM0) in the reallocatable pool 620. Compared to the embodiment of FIG. 5 (which illustrates a reallocation operation where at least one TCM page is replaced by at least one DRAM page), the embodiment of FIG. 6 illustrates a reallocation operation where at least one DRAM page is replaced by at least one TCM page. Specifically, the synchronization management circuit 131 can generate a status table 141S', which records the status of data cells to be moved from a withdrawn page (e.g., DRAM page DRAM0) of DRAM 122 to a reallocated page (e.g., TCM page TCM0 in the reallocatable pool 620) of TCM 121, and the proxy mapping table generator 132 can update the mapping table 141M based on the reallocation result of replacing the withdrawn page of DRAM 122 with a reallocated page of TCM 121. Those skilled in the art should understand how to construct and utilize the DRAM page state table 141S' according to the description in Figure 5 to complete the reallocation operation of replacing at least one DRAM page with at least one TCM page, and should understand how to perform address mapping operations during the movement of data units from DRAM pages to TCM pages according to the mapping table 141M and the state table 141S'. Therefore, for the sake of simplicity, other details of this embodiment are omitted here. This embodiment ensures the most efficient use of TCM storage resources and minimizes the waste of TCM storage resources.

[0020] Figure 7 illustrates the data isolation of TCM 121 according to an embodiment of the present invention. Figure 7 shows a specific application scenario of Figure 6. In this embodiment, task T4 can be an example of the previous task in the embodiment of Figure 6, and task T5 can be an example of the new task in the embodiment of Figure 6. That is, in this embodiment, when computing engine 50 requests storage space to execute task T5, TCM 121 is completely occupied by the previous task T4, so memory management circuit 100 allocates DRAM 122 (e.g., DRAM pages DRAM0, DRAM1, DRAM2, DRAM3, DRAM4, and DRAM5) as storage space initially serving task T5 (i.e., the new task). As described in the embodiments of Figures 6 and 7, TCM pages TCM0, TCM1, TCM2, TCM3, TCM4, and TCM5 can be progressively released to the reassignable pool 620 and reassigned to task T5. Before being reassigned to a new task (e.g., task T5 as shown in Figure 7), the data stored in TCM pages TCM0, TCM1, TCM2, TCM3, TCM4, and TCM5 related to the previous task (e.g., task T4 as shown in Figure 7) has been cleared. Therefore, the data related to task T4 belonging to the first context (labeled "context-1" in Figure 7 for simplicity) and the data related to task T5 belonging to the second context (labeled "context-2" in Figure 7 for simplicity) can be isolated, and additional data isolation encryption circuitry can be omitted, thereby saving circuit area. Furthermore, when TCM 121 is completely occupied by the previous task T4, the memory management circuit 100 allocates DRAM 122 (e.g., DRAM pages DRAM0, DRAM1, DRAM2, DRAM3, DRAM4 and DRAM5) as storage space to serve task T5, and then gradually increases the TCM resources allocated to task T5 (belonging to the second context) during the execution of task T5, thereby omitting the aforementioned silent time that prohibits the execution of task T5.

[0021] Figure 8 illustrates a workflow of a memory management method according to an embodiment of the present invention, wherein the memory management method is applicable to a memory management circuit (e.g., the memory management circuit 100 shown in Figure 1). It should be noted that the workflow shown in Figure 8 is for illustrative purposes only and does not imply limitation of the invention. For example, one or more steps may be added, deleted, or modified in the workflow shown in Figure 8. Furthermore, these steps need not be performed in the exact order shown in Figure 8 if the same result can be obtained.

[0022] In step S810, the memory management circuit uses the allocation control circuit (e.g., the synchronization management circuit 131 shown in FIG1) to move the data unit from the withdrawal page of the first memory device to the reallocation page of the second memory device to update the storage space provided for the ongoing task, and generates a status table to record the status of the data unit during the data unit movement process.

[0023] In step S820, the memory management circuit uses a mapping table control circuit (e.g., the proxy mapping table control circuit 132 shown in FIG1) to maintain the mapping table of the ongoing task during the data unit movement process, and updates the mapping table according to the data unit movement result, wherein the mapping table records the mapping relationship between the address used in the access command of the ongoing task and the address of the first memory device or the second memory device.

[0024] In step S830, the memory management circuit uses address mapping logic (e.g., address mapping logic 140 shown in Figure 1) to perform address mapping operations during data unit movement based on access commands, the mapping table maintained by the mapping table control circuit, and the status table.

[0025] The memory management circuit 100, electronic device 10 (including the memory management circuit 100), and memory management method provided in this embodiment of the invention utilize a page rollback state table to record the data movement state and a mapping table to record the mapping relationship between the address used by the access instruction and the memory device address, thereby achieving memory reallocation without interrupting the currently executing task. Furthermore, if an access instruction is received when the movement of a data unit has started but not yet completed (e.g., the data unit's state is "copying"), the memory management circuit 100, electronic device 10 (including the memory management circuit 100), and memory management method provided in this embodiment of the invention will temporarily hold the access instruction in a collision buffer to avoid erroneous address mapping and access operations. Therefore, this invention can dynamically configure the TCM / DRAM resources allocated to the computing engine without the computing engine detecting the memory reallocation, thereby ensuring that TCM / DRAM resources are effectively utilized. Furthermore, as shown in the embodiments in FIG.5-FIG.7 above, this invention can ensure the most efficient use of TCM storage resources and minimize the waste of TCM storage resources.

[0026] Those skilled in the art will readily observe that numerous modifications and alterations can be made to the apparatus and method while retaining the doctrine of the invention. Therefore, the above disclosure should be interpreted only according to the limits of the appended claims. The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made in accordance with the claims of the present invention shall be covered by the present invention.

[0027] 111, 112, 113: Accelerator 10: Electronic devices 100: Memory Management Circuit 140: Address Mapping Logic 142: Collision Buffer 130: Controller 141: Mapping table cache 141M: Mapping Table 132: Proxy Mapping Table Generator 141S,141S': State table 131: Synchronization Management Circuit 121:TCM 122:DRAM 20: Scheduler 110,50: Calculation Engine 620: Redistributable pool 610: Pool occupied S810, S820, S830: Steps

Claims

1. A memory management circuit, comprising: The allocation control circuit is configured to move a data unit from the withdrawal page of the first memory device to the reallocation page of the second memory device to update the storage space provided for the ongoing task, and generate a status table to record the status of the data unit during the movement of the data unit. A mapping table control circuit is configured to maintain a mapping table of the ongoing task during the movement of the data unit, and update the mapping table according to the movement result of the data unit, wherein the mapping table records the mapping relationship between the address used in the access command of the ongoing task and the address of the first memory device or the second memory device; and address mapping logic, coupled to the allocation control circuit and the mapping table control circuit, is configured to perform an address mapping operation according to the access command, the mapping table maintained by the mapping table control circuit, and the status table during the movement of the data unit.

2. The memory management circuit as claimed in claim 1, wherein when a page is selected as the revoked page, the allocation control circuit marks the state of all data units stored on the revoked page as a first state, wherein the page is selected as the revoked page when it is selected to be revoked from the storage space provided for the ongoing task, the allocation control circuit modifies the state of a specific data unit from the first state to a second state when the specific data unit is copied from the revoked page but not written to the reallocated page, and the allocation control circuit modifies the state of the specific data unit from the second state to a third state when the specific data unit is written to the reallocated page.

3. The memory management circuit as claimed in claim 2, wherein when the memory management circuit receives an access command corresponding to the specific data unit, when the status table indicates that the status of the specific data unit is the first state, the address mapping logic addresses the specific data unit in the revocation page according to the mapping table maintained by the mapping table control circuit, and causes the access command to access the specific data unit in the revocation page.

4. The memory management circuit as described in claim 2, wherein: When the memory management circuit receives an access command corresponding to a specific data unit, if the status table indicates that the specific data unit is in the second state, the allocation control circuit keeps the access command corresponding to the specific data unit in the collision buffer, and the address mapping logic prevents the access command from being executed; and when the status table indicates that the specific data unit's status changes from the second state to the third state, the allocation control circuit notifies the address mapping logic, and the address mapping logic addresses the specific data unit in the reallocation page according to the mapping table maintained by the mapping table control circuit and the status table, and enables the access command to access the specific data unit in the reallocation page.

5. The memory management circuit as claimed in claim 2, wherein when the memory management circuit receives an access command corresponding to the specific data unit, when the status table indicates that the specific data unit is in the third state, the address mapping logic addresses the specific data unit in the reallocation page according to the mapping table maintained by the mapping table control circuit and the status table, and causes the access command to access the specific data unit in the reallocation page.

6. The memory management circuit as claimed in claim 2, wherein the mapping table control circuit updates the mapping table based on the move result after all data units are marked as the third state.

7. The memory management circuit as claimed in claim 1, wherein when a previous task occupying a page of the second memory is completed or, although not completed, the data of the previous task stored in the occupied page has been used by the previous task, the allocation control circuit clears the data of the previous task stored in the occupied page to convert the occupied page into a blank page and moves the blank page to a reallocatable pool to allow the blank page to be selected as the reallocatable page.

8. The memory management circuitry as claimed in claim 7, wherein the allocation control circuitry clears the data of the previous task stored in the occupied page by filling the occupied page with zero or a default value.

9. An electronic device comprising: a computing circuit configured to perform an ongoing task; and a first memory device configured to selectively provide a first storage space for the computing circuit; A second memory device is configured to selectively provide a second storage space for the computing circuit; And memory management circuitry coupled to the computing circuitry, the first memory device and the second memory device, including a controller and address mapping logic, wherein the controller includes: allocation control circuitry configured to move a data unit from a withdrawn page of the first memory device to a reallocated page of the second memory device to update the storage space provided for the ongoing task, and to generate a status table to record the status of the data unit during the movement of the data unit; The system also includes a mapping table control circuit configured to maintain a mapping table for the ongoing task during the movement of the data unit, and to update the mapping table based on the movement result of the data unit. The mapping table records the mapping relationship between the address used in the access command of the ongoing task and the address of the first memory device or the second memory device. The address mapping logic couples the allocation control circuit and the mapping table control circuit, and is configured to perform an address mapping operation during the movement of the data unit based on the access command, the mapping table maintained by the mapping table control circuit, and the status table.

10. The electronic device of claim 9, wherein when a page is selected as the revoked page, the allocation control circuit marks the state of all data units stored on the revoked page as a first state, wherein the page is selected as the revoked page when it is selected to be revoked from storage space provided for the ongoing task, the allocation control circuit modifies the state of a specific data unit from the first state to a second state when the specific data unit is copied from the revoked page but not written to the reallocated page, and the allocation control circuit modifies the state of the specific data unit from the second state to a third state when the specific data unit is written to the reallocated page.

11. The electronic device of claim 10, wherein when the memory management circuit receives an access command corresponding to the specific data unit from the computing circuit, when the status table indicates that the specific data unit is in the first state, the address mapping logic addresses the specific data unit in the revocation page according to the mapping table maintained by the mapping table control circuit, and causes the access command to access the specific data unit in the revocation page.

12. The electronic device as claimed in claim 10, wherein: When the memory management circuit receives an access command corresponding to a specific data unit from the computing circuit, if the status table indicates that the specific data unit is in the second state, the allocation control circuit keeps the access command corresponding to the specific data unit in the collision buffer, and the address mapping logic prevents the access command from being executed; and when the status table indicates that the specific data unit's status changes from the second state to the third state, the allocation control circuit notifies the address mapping logic, and the address mapping logic addresses the specific data unit in the reallocation page according to the mapping table maintained by the mapping table control circuit and the status table, and causes the access command to access the specific data unit in the reallocation page.

13. A memory management method, applicable to a memory management circuit, comprising: moving a data unit from a revocation page of a first memory device to a reallocation page of a second memory device to update storage space provided for an ongoing task, and generating a status table to record the status of the data unit during the movement of the data unit; maintaining a mapping table of the ongoing task during the movement of the data unit, and updating the mapping table according to the result of the movement of the data unit, wherein the mapping table records the mapping relationship between addresses used in access commands of the ongoing task and addresses of the first memory device or the second memory device; and performing an address mapping operation during the movement of the data unit according to the access command, the maintained mapping table, and the status table.

14. The memory management method of claim 13, further comprising: when a page is selected as the revoked page, marking the state of all data units stored on the revoked page as a first state, wherein the page is selected as the revoked page when it is selected to be revoked from storage space provided for the ongoing task; when a specific data unit is copied from the revoked page but not written to the reallocated page, changing the state of the specific data unit from the first state to a second state; and when the specific data unit is written to the reallocated page, changing the state of the specific data unit from the second state to a third state.

15. The memory management method as claimed in claim 14, further comprising: when the memory management circuit receives an access command for a specific data unit, if the status table indicates that the status of the specific data unit is the first state, addressing the specific data unit in the revocation page according to the maintained mapping table, and causing the access command to access the specific data unit in the revocation page.

16. The memory management method of claim 14, further comprising: when the memory management circuit receives an access command for a specific data unit, if the status table indicates that the state of the specific data unit is the second state, retaining the access command for the specific data unit in a collision buffer and preventing the access command from being executed; and when the status table indicates that the state of the specific data unit changes from the second state to the third state, addressing the specific data unit in the reallocation page according to the maintained mapping table and the status table, and causing the access command to access the specific data unit in the reallocation page.

17. The memory management method of claim 14, further comprising: when the memory management circuit receives an access command for a specific data unit, if the status table indicates that the status of the specific data unit is the third state, addressing the specific data unit in the reallocation page according to the maintained mapping table and the status table, and causing the access command to access the specific data unit in the reallocation page.

18. The memory management method as described in claim 14, further comprising: updating the mapping table based on the move result after the state of all data units has been marked as the third state.

19. The memory management method of claim 13, further comprising: when a previous task occupying an occupied page of the second memory is completed or, although not completed, the data of the previous task stored in the occupied page has been used by the previous task, clearing the data of the previous task stored in the occupied page to convert the occupied page into a blank page, and moving the blank page to a reallocatable pool to allow the blank page to be selected as the reallocatable page.

20. The memory management method of claim 19, wherein clearing the data of the previous task stored in the occupied page to convert the occupied page to the blank page comprises: clearing the data of the previous task stored in the occupied page by filling the occupied page with zero or a default value.

Images