Memory resource processing method, apparatus, device, and program product

Abstract

The application provides a memory resource processing method, device and equipment and a program product, which can be applied to the technical field of databases. The method comprises the following steps: in the process that a first process calls a first memory space of a memory resource from a first cache pool to execute a first task, a second process is used to call a second memory space of the memory resource to execute a second task; in the case that the second task is executed, the second process is used to write second space attribute information of the second memory space into a preset first return chain table of the first process; the space attribute information in the first return chain table is detected to obtain a detection result; in response to the fact that the memory space capacity corresponding to the detection result and the second space attribute information satisfies a preset capacity condition, the second space attribute information is taken as idle resource information, and the idle resource information is written into the first cache pool, and the first process or the second process calls a memory space from the first cache pool according to the idle resource information.

Description

Technical Field

[0001] This application relates to the field of database technology, and more specifically to a method, apparatus, device, and program product for processing memory resources. Background Technology

[0002] MVCC (Multi-Version Concurrency Control) is a widely used concurrency control mechanism in database systems, especially in databases supporting high-concurrency transactions. It helps achieve data consistency and isolation when multiple transactions execute in parallel. With the evolution of MVCC in-memory technology, modern MVCC is implemented in memory. Traditional database memory experiences significant performance degradation and severe memory fragmentation as the number of memory blocks managed increases. Accessing the memory pool requires locking, resulting in low parallel performance. Therefore, an efficient memory resource management method is urgently needed to address the challenges of MVCC in-memory implementation. Summary of the Invention

[0003] In view of the above problems, this application provides a method, apparatus, device and program product for processing memory resources.

[0004] According to a first aspect of this application, a memory resource processing method is provided, comprising: during the process of a first process calling memory resources from a first cache pool to execute a first task, a second process calling memory resources from a second memory space to execute a second task; upon completion of the second task, the second process writing second space attribute information of the second memory space into a preset first return list of the first process, the first return list being used to store the first space attribute information of the first memory space upon completion of the first task; detecting the space attribute information in the first return list to obtain a detection result; in response to the detection result and the memory space capacity corresponding to the second space attribute information satisfying a preset capacity condition, the second space attribute information being used as free resource information and written into the first cache pool, and the first process or the second process calling memory space from the first cache pool according to the free resource information.

[0005] According to an embodiment of this application, in response to the detection result and the memory space capacity corresponding to the second spatial attribute information meeting a preset capacity condition, the second spatial attribute information is used as free resource information, and the free resource information is written into a first cache pool, including: when the memory space capacity is not lower than a preset cache threshold, determining free resource information from the second spatial attribute information based on a dynamic cache adjustment algorithm; wherein, the method further includes: determining remaining free resource information based on the difference between the second spatial attribute information and the free resource information, the remaining free resource information representing the portion of the memory space capacity corresponding to the second spatial attribute information that is unrelated to the free resource information; and writing the remaining free resource information into the first memory pool of the first process.

[0006] According to an embodiment of this application, the memory resource processing method further includes: when the second memory space in the first cache pool is insufficient, calling the remaining memory space corresponding to the remaining free resource information from the first memory pool to execute the second process.

[0007] According to an embodiment of this application, determining free resource information from second spatial attribute information based on a dynamic cache adjustment algorithm includes: obtaining the space capacity consumption rate corresponding to the type of second spatial attribute information; and fusing the space capacity consumption rate, a preset safety factor, and the basic resource cache amount to determine the target free resource information.

[0008] According to an embodiment of this application, a first cache pool is used to cache multiple memory resource types, and multiple memory spaces corresponding to the same memory resource type are associated with memory space indices in a space linked list.

[0009] According to an embodiment of this application, the memory resource processing method further includes: in response to a resource request initiated by a first process or a second process for a target table, obtaining memory resources matching the resource request from the cache resource amount of a first cache pool indicated by the target table; wherein the cache resource amount is determined based on the following method: obtaining the memory resource consumption rate corresponding to the target table; and combining the memory resource consumption rate and a preset safety coefficient to determine the amount of cache resources reserved for the target table in the first cache pool.

[0010] According to an embodiment of this application, the memory resource processing method further includes: when a unique index exists in the target table, merging the row information and index information stored in the memory space corresponding to the target table obtained from the first cache pool to generate a mapping entry, wherein the mapping entry is used to search for cached data related to the target table in the memory space corresponding to the amount of cached resources reserved for the target table in the first cache pool.

[0011] A second aspect of this application provides a memory resource processing apparatus, comprising: a task execution module, configured to, during the execution of a first task by a first process calling memory resources from a first cache pool into a first memory space, utilize a second process to call memory resources into a second memory space to execute a second task; an attribute information writing module, configured to, upon completion of the second task, utilize the second process to write the second space attribute information of the second memory space into a preset first return list of the first process, the first return list being used to store the first space attribute information of the first memory space upon completion of the first task; a detection module, configured to detect the space attribute information in the first return list and obtain a detection result; and a free information writing module, configured to, in response to the detection result and the memory space capacity corresponding to the second space attribute information satisfying a preset capacity condition, use the second space attribute information as free resource information and write the free resource information into the first cache pool, wherein the first process or the second process calls memory space from the first cache pool according to the free resource information.

[0012] A third aspect of this application provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the method described above.

[0013] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implements the above-described method.

[0014] The fifth aspect of this application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0015] According to an embodiment of this application, during the execution of a first task by a first process calling memory resources from a first cache pool into a first memory space, a second process can call memory resources from a second memory space to execute a second task. Upon completion of the second task, the second process writes the second space attribute information of the second memory space into a preset first return list of the first process, thus resolving the system overhead of repeated memory allocation and release between processes and saving memory scheduling efficiency. The space attribute information in the first return list is checked. If the memory space capacity corresponding to the second space attribute information meets a preset capacity condition based on the detection result, the second space attribute information can be used as free resource information and written into the first cache pool. When the second process returns memory resources, delayed return is adopted, eliminating the need to contend for resource locks and greatly improving concurrency. Furthermore, by caching memory resources through the first cache pool, memory allocation is not required for each resource request. Either the first or second process can call free memory space from the first cache pool without initiating memory allocation, reducing memory access latency and avoiding resource waste or imbalance. Attached Figure Description

[0016] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments of this application with reference to the accompanying drawings.

[0017] Figure 1 The illustration shows an application scenario diagram of a memory resource processing method, apparatus, device, and program product according to embodiments of this application.

[0018] Figure 2 A schematic diagram of a conventional memory resource processing method according to an embodiment of this application is shown.

[0019] Figure 3 A flowchart of a memory resource processing method according to an embodiment of this application is shown.

[0020] Figure 4 A schematic diagram of a memory resource processing method according to an embodiment of this application is shown.

[0021] Figure 5 A structural block diagram of a memory resource processing apparatus according to an embodiment of this application is shown.

[0022] Figure 6 A block diagram of an electronic device suitable for implementing a memory resource processing method according to an embodiment of this application is shown. Detailed Implementation

[0023] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0026] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0027] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0028] Figure 1 The illustration shows an application scenario diagram of a memory resource processing method, apparatus, device, and program product according to embodiments of this application.

[0029] like Figure 1As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0030] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0031] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0032] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0033] It should be noted that the memory resource processing method provided in this application embodiment can generally be executed by server 105. Correspondingly, the memory resource processing device provided in this application embodiment can generally be located in server 105. The memory resource processing method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the memory resource processing device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0034] It should be understood that Figure 1The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0035] Figure 2 A schematic diagram of a conventional memory resource processing method according to an embodiment of this application is shown.

[0036] MVCC (Multi-Version Concurrency Control) is a concurrency control mechanism widely used in database systems, especially in databases that support high-concurrency transactions. It helps achieve data consistency and isolation when multiple transactions execute in parallel. The basic idea of ​​MVCC is to maintain multiple versions of data, allowing multiple transactions to read and write data simultaneously without interfering with each other, thereby improving concurrency and system throughput.

[0037] With the evolution of MVCC in-memory technology, modern MVCC is implemented in memory, which greatly improves performance but also brings many problems. For example, in-memory MVCC involves a wide variety of resource types, each with different memory spaces, some of which are fixed-space and some are variable-space. In-memory MVCC places very high demands on memory performance, especially the speed of resource allocation and deallocation.

[0038] Traditional database memory pools often have several problems, such as... Figure 2 As shown, for example, there are four different memory structures: full version 201, incremental version 202, index information 203, and management structure 204. From Figure 2 As can be seen, memory pools require locked access. When the number of managed memory blocks increases, memory fragmentation becomes severe, resulting in poor parallel performance. In MVCC scenarios, memory usage is very frequent, and real-time requirements are also high. Traditional database memory pools are no longer adequate, necessitating an efficient memory resource management method to address the challenges of MVCC's in-memory nature.

[0039] To address at least one of the aforementioned problems, this application provides a memory resource processing method, apparatus, electronic device, and medium. The method includes: during the execution of a first task by a first process calling memory resources from a first cache pool to a first memory space, a second process calling memory resources from a second memory space to execute a second task; upon completion of the second task, the second process writing second space attribute information of the second memory space into a preset first return list of the first process, the first return list being used to store the first space attribute information of the first memory space upon completion of the first task; detecting the space attribute information in the first return list to obtain a detection result; in response to the detection result and the memory space capacity corresponding to the second space attribute information satisfying a preset capacity condition, the second space attribute information is used as free resource information and written into the first cache pool, and the first process or the second process calling memory space from the first cache pool according to the free resource information.

[0040] The following will be based on Figure 1 The described scene, through Figure 3 and Figure 4 The data processing of the disclosed embodiments will be described in detail.

[0041] Figure 3 A flowchart of a memory resource processing method according to an embodiment of this application is shown.

[0042] like Figure 3 As shown, the memory resource processing method of this embodiment includes operations S310 to S340.

[0043] During operation S310, while the first process is executing the first task by calling memory resources from the first cache pool in the first memory space, the second process is using the second memory space to call memory resources to execute the second task.

[0044] In operation S320, after the second task is completed, the second process writes the second space attribute information of the second memory space into the first process's preset first return list.

[0045] In operation S330, the spatial attribute information of the first return list is detected, and the detection result is obtained.

[0046] In operation S340, in response to the detection result and the memory space capacity corresponding to the second space attribute information meeting the preset capacity condition, the second space attribute information is used as idle resource information and written into the first cache pool.

[0047] While the first process is executing a first task by retrieving memory resources from the first cache pool in the first memory space, the second process can utilize the second memory space to execute a second task. The first and second memory spaces have the same resource type. Memory resource types may include full version, incremental version, index information, and management structure, but are not limited to these, and the embodiments of this application do not impose such limitations. For example, when the first process executes a first service and discovers the need to request an MVCC resource, it can obtain that type of resource from the header of the first cache pool on the first process. The first process can also execute other services. The second process also needs to use this type of resource and invokes the second memory space of the memory resources to execute the second task.

[0048] The first return list is used to store the second space attribute information of the second memory space after the second task has completed execution. When the second task completes, the second process can release the memory resource, and therefore can use the second process to write the second space attribute information of the second memory space into the first process's preset first return list. Each process prepares a preset return list for the corresponding memory resource type for other processes. Taking the first process as an example, it prepares a preset first return list for the second process, used to record the second space attribute information of the second memory space returned to the first process from the second process.

[0049] By checking the space attribute information in the first return list, a detection result can be obtained. The space attribute information in the first return list can be checked periodically to determine whether there is any returned memory space on the list. In response to the detection result and the memory space capacity corresponding to the second space attribute information meeting a preset capacity condition, the second space attribute information can be used as free resource information and written to the first cache pool. The first process or the second process then retrieves memory space from the first cache pool based on the free resource information.

[0050] During the execution of the first task in the first memory space, the second process can utilize the second memory space to execute the second task. Upon completion of the second task, the second process writes the second memory space's attribute information into the first process's pre-defined first return list, resolving the system overhead of repeated memory allocation and release between processes and improving memory scheduling efficiency. The space attribute information in the first return list is checked. If the memory space capacity corresponding to the second space attribute information meets the pre-defined capacity condition, the second space attribute information can be used as free resource information and written to the first cache pool. When the second process returns memory resources, delayed return is used, eliminating the need to contend for resource locks and significantly improving concurrency. Furthermore, by caching memory resources through the first cache pool, memory allocation is not required for each resource request. The first or second process can access free memory space from the first cache pool without initiating memory allocation, reducing memory access latency and avoiding resource waste or imbalance. Locking is not required during resource transfer. The delayed return method eliminates the need to contend for resource locks, further enhancing concurrency.

[0051] According to an embodiment of this application, in response to the detection result and the memory space capacity corresponding to the second spatial attribute information meeting a preset capacity condition, the second spatial attribute information is used as free resource information, and the free resource information is written into a first cache pool, including: when the memory space capacity is not lower than a preset cache threshold, determining free resource information from the second spatial attribute information based on a dynamic cache adjustment algorithm; wherein, the method further includes: determining remaining free resource information based on the difference between the second spatial attribute information and the free resource information, the remaining free resource information representing the portion of the memory space capacity corresponding to the second spatial attribute information that is unrelated to the free resource information; and writing the remaining free resource information into the first memory pool of the first process.

[0052] If the memory space capacity is lower than the preset cache threshold, the memory resources in the first return list can be written into the first buffer pool.

[0053] If the memory space capacity is not lower than the preset cache threshold, the free resource information can be determined from the second space attribute information based on the dynamic cache adjustment algorithm.

[0054] In MVCC, the frequency and amount of resource usage vary depending on the scenario. Consider the following business scenario: 1) First, transaction A inserts a large amount of data into table TA, which has an index; 2) Next, transaction B deletes a large amount of data into table TA, which also has an index; 3) Next, transaction C repeatedly inserts and deletes data into table TB, which has no index; 4) Finally, transaction D performs a large number of modifications on table TC, which has no index.

[0055] Transaction A inserts data into table TA, which has an index. The main resources needed are: a first type (small structure) for storing index information and a second type (small structure) for storing row information. These resources can be reclaimed during deletion. Transaction C inserts data into table TB, which does not have an index. The main resource needed is: a second type for storing row information. Transaction D modifies data in table TC, which does not have an index. The main resources needed are: a second type for storing row information and a third type (large structure) for storing the original row information.

[0056] Based on the above, the resource usage is as follows: Transaction A requests a large amount of Type 1 and Type 2; Transaction B returns a large amount of Type 1 and Type 2; Transaction C requests and returns a large amount of Type 2; Transaction D requests a large amount of Type 2 and Type 3.

[0057] During this process, it was discovered that the usage patterns of each resource differed:

[0058] The first type is used frequently and returned after use, and then not used again for a period of time. The second type is used relatively frequently, and is used again soon after being returned. The third type is used less frequently, but has a larger structure.

[0059] Therefore, for the first type, the resource pool capacity can be dynamically and appropriately reduced to free up memory for other services. For the second type, due to frequent use, the resource pool capacity needs to be appropriately increased. For the third type, due to infrequent use and large memory usage, the resource pool should not be too large.

[0060] According to the embodiments of this application, the space capacity consumption rate corresponding to the second space attribute information type can be obtained; the space capacity consumption rate, the preset security factor and the basic resource cache amount are fused to determine the target idle resource information.

[0061] The space capacity consumption rate can include short-term and long-term space capacity consumption rates. The target idle resource information (as shown in Table 1) is determined by combining the space capacity consumption rate, a preset safety factor, and the basic resource cache size. The formula is: Target Idle Resource Information = (Short-term Space Capacity Consumption Rate × 0.5 + Long-term Space Capacity Consumption Rate × 0.5) × Preset Safety Factor + (Basic Cache Size × Resource Type Weight). Here, the short-term space capacity consumption rate represents the short-term space capacity consumption rate (average over 10 minutes, unit: units / second), reflecting recent consumption fluctuations. The long-term space capacity consumption rate represents the long-term consumption rate (average over 1 hour, unit: units / second), reflecting long-term consumption trends. The preset safety factor is used to handle redundancy in case of sudden consumption, for example, 1.2~2.0. The basic cache size ensures a minimum available base, avoiding overly low calculated values. The resource type weight is inversely proportional to the actual memory space capacity of the resource.

[0062] Table 1 Target Idle Resource Information Table

[0063]

[0064] Based on the difference between the second-space attribute information and the idle resource information, the remaining idle resource information can be determined. This remaining idle resource information represents the portion of the memory space capacity corresponding to the second-space attribute information that is unrelated to the idle resource information. This remaining idle resource information can be written to the first memory pool of the first process. For example, assuming the initial total capacity of the first cache pool is 1GB, the first process uses 512MB of the first memory space to execute the main protection task, while the second process uses the remaining 512MB of the second memory space to execute the traffic cleaning subtask. After the second task is completed, the second process writes the second-space attribute information (corresponding to 512MB of capacity) of the second memory space into the first process's first return list. After system detection, it determines that this 512MB fully meets the preset capacity conditions, and only 256MB of it is written as idle resource information to the first cache pool. The remaining 256MB is considered the remaining idle resource information. Finally, this 256MB of remaining idle resource information is directly written to the first process's dedicated first memory pool, exclusively for the first process to use when executing the main protection task later, eliminating the need for cross-process cache pool allocation and further improving the memory scheduling efficiency of the main task.

[0065] According to the embodiments of this application, a preset cache threshold is used as the judgment boundary. The idle resource information is determined from the second space attribute information through a dynamic cache adjustment algorithm and included in the first cache pool shared across processes. The remaining idle resource information is directly allocated to the dedicated memory pool corresponding to the first process and the resource of that type. This not only ensures the unified scheduling of large-granularity memory resources that can be shared across processes and avoids the high overhead of repeatedly requesting memory from the system, but also allows the main task of the first process to prioritize the use of dedicated remaining memory, reducing the memory allocation latency of the main task. At the same time, it avoids the problem of cross-process memory fragmentation and improves memory utilization and operational stability in high-concurrency protection scenarios.

[0066] According to an embodiment of this application, the memory resource processing method further includes: when the second memory space in the first cache pool is insufficient, calling the remaining memory space corresponding to the remaining free resource information from the first memory pool to execute the second process.

[0067] If the second memory space in the first cache pool is insufficient, the remaining memory space corresponding to the remaining free resource information can be called from the first memory pool to execute the second process.

[0068] According to the embodiments of this application, when the second memory space in the first cache pool is insufficient, the remaining memory space can be called from the first memory pool to execute the second process. This establishes a resource hierarchical management system, which can perform more precise resource control. In addition, there is no need to request new memory from the system, thus improving resource utilization efficiency.

[0069] According to an embodiment of this application, a first cache pool is used to cache multiple memory resource types, and multiple memory spaces corresponding to the same memory resource type are associated with memory space indices in a space linked list.

[0070] The capacity of multiple memory spaces within a memory resource type is the same, and multiple memory spaces corresponding to the same memory resource type are associated based on the memory space index in the space linked list.

[0071] According to the embodiments of this application, memory spaces of the same type are associated through an indexed linked list, which reduces the time spent on searching and allocating memory of the same type within the cache pool. In addition, the connection through the space linked list reduces memory fragmentation and can also ensure parallel performance when the number of managed memory blocks increases.

[0072] According to an embodiment of this application, the memory resource processing method further includes: in response to a resource request initiated by a first process or a second process for a target table, obtaining memory resources matching the resource request from the cache resource amount of a first cache pool indicated by the target table; wherein the cache resource amount is determined based on the following method: obtaining the memory resource consumption rate corresponding to the target table; and combining the memory resource consumption rate and a preset safety factor to determine the amount of cache resources reserved for the target table in the first cache pool.

[0073] Because each table consumes resources at a different rate, in addition to resource pool-level dynamic resource management, table-level dynamic resource management can also be implemented. Due to the large number of tables, table-level resource caching only needs to consider the resource consumption rate of the target table, without requiring a base cache. In response to a resource request initiated by the first or second process for the target table, memory resources matching the resource request can be obtained from the cached resources in the first cache pool indicated by the target table. The formula for calculating the cached resource amount is shown below.

[0074] The cache resource amount for the table is calculated as follows: (Short-term memory consumption rate × 0.5 + Long-term memory consumption rate × 0.5) × Preset safety factor. This formula is based on the cache resource amount calculation.

[0075] Table 2: Cache Resource Quantity Table

[0076]

[0077] According to the embodiments of this application, cache pool resources are pre-allocated by taking into account the memory resource consumption rate of the target table and the safety factor, thereby achieving accurate reservation and on-demand allocation of cache resources, avoiding resource waste or contention, and improving cache hit rate and system stability during concurrent access by multiple processes.

[0078] Databases typically access row information by looking up index information. If the memory addresses of the row information and the index information are far apart, a cache miss is likely to occur.

[0079] According to an embodiment of this application, the memory resource processing method further includes: when a unique index exists in the target table, merging the row information and index information stored in the memory space of the corresponding target table obtained from the first cache pool to generate a mapping entry. The mapping entry is used to search for cached data related to the target table in the memory space corresponding to the amount of cached resources reserved for the target table in the first cache pool.

[0080] Because databases often contain tables with unique primary keys (a table has one unique index), as shown in Table 3.

[0081] Table 3 Primary Key Table

[0082]

[0083] If a unique index exists in the target table, the row information and index information stored in the memory space of the corresponding target table obtained from the first cache pool can be merged to generate a mapping entry. The mapping entry is used to find cached data related to the target table in the memory space corresponding to the amount of cache resources reserved for the target table in the first cache pool.

[0084] According to the embodiments of this application, after the above-described structural optimization, the content addresses of row information and index information are adjacent, which improves read and write efficiency. Furthermore, by requesting row information and index information together, the request efficiency and performance are improved, and the probability of cache misses is reduced.

[0085] Figure 4 A schematic diagram of a memory resource processing method according to an embodiment of this application is shown.

[0086] like Figure 4 As shown, each process contains a first cache pool, a memory pool, a return list, and a garbage collector.

[0087] The first cache pool primarily caches various types of pre-allocated resources. The memory for these resources has already been allocated and linked together in a linked list. When resources are requested, they are preferentially retrieved from this pool. When the cached resources in the first cache pool are insufficient, resources will be retrieved from the memory pool corresponding to the process. Each type of resource has its own independent memory pool.

[0088] When resources originate from the first process, the second process does not return them directly to the first process's resource pool or memory pool; instead, it returns them to the first process's return list. The first process's reclaimer then reclaims the resources. Due to the dedicated memory pool design, and the fact that each type of resource is almost always a fixed size, the memory pool is virtually free of fragmentation, reducing the time spent searching for free blocks and defragmenting the memory pool.

[0089] The garbage collector is a scheduled task of the first process, typically executing once per second. Its main function is to traverse the first process's return list and re-add resources received from the return list to the cache pool. If there are too many cached resources, they will be returned to a dedicated memory pool (the memory pool corresponding to the resource type).

[0090] Based on the above-described memory resource processing method, this application also provides a memory resource processing apparatus. The following will be combined with... Figure 5 The device is described in detail.

[0091] Figure 5 A structural block diagram of a memory resource processing apparatus according to an embodiment of this application is shown.

[0092] like Figure 5 As shown, the memory resource processing device 500 of this embodiment includes a task execution module 510, an attribute information writing module 520, a detection module 530, and a free information writing module 540.

[0093] The task execution module 510 is used to execute a second task by utilizing the second memory space of the second process to call memory resources during the execution of a first task by the first process calling memory resources from the first cache pool to the first memory space.

[0094] The attribute information writing module 520 is used to write the second space attribute information of the second memory space into the preset first return list of the first process when the second task is completed. The first return list is used to store the first space attribute information of the first memory space when the first task is completed.

[0095] The detection module 530 is used to detect the spatial attribute information in the first return list and obtain the detection result.

[0096] The idle information writing module 540 is used to respond to the detection result and the memory space capacity corresponding to the second space attribute information meeting the preset capacity condition, to use the second space attribute information as idle resource information, and to write the idle resource information into the first cache pool, and the first process or the second process calls memory space from the first cache pool according to the idle resource information.

[0097] According to an embodiment of this application, during the execution of a first task by a first process calling memory resources from a first cache pool into a first memory space, a second process can call memory resources from a second memory space to execute a second task. Upon completion of the second task, the second process writes the second space attribute information of the second memory space into a preset first return list of the first process, thus resolving the system overhead of repeated memory allocation and release between processes and saving memory scheduling efficiency. The space attribute information in the first return list is checked. If the memory space capacity corresponding to the second space attribute information meets a preset capacity condition based on the detection result, the second space attribute information can be used as free resource information and written into the first cache pool. When the second process returns memory resources, delayed return is adopted, eliminating the need to contend for resource locks and greatly improving concurrency. Furthermore, by caching memory resources through the first cache pool, memory allocation is not required for each resource request. Either the first or second process can call free memory space from the first cache pool without initiating memory allocation, reducing memory access latency and avoiding resource waste or imbalance.

[0098] The idle information writing module 540 includes: a determination unit.

[0099] The determination unit is used to determine free resource information from the second space attribute information based on the dynamic cache adjustment algorithm, provided that the memory space capacity is not lower than the preset cache threshold.

[0100] The memory resource processing device 500 in this embodiment further includes: a remaining free space determination module and a writing module.

[0101] The remaining free space determination module is used to determine the remaining free space information based on the difference between the second space attribute information and the free resource information. The remaining free space information represents the portion of the memory space capacity corresponding to the second space attribute information that is unrelated to the free resource information.

[0102] The write module is used to write the remaining free resource information into the first memory pool of the first process.

[0103] The memory resource processing device 500 in this embodiment further includes a calling module.

[0104] The calling module is used to call the remaining memory space corresponding to the remaining free resource information from the first memory pool to execute the second process when the second memory space in the first cache pool is insufficient.

[0105] The determination unit includes: a consumption rate acquisition subunit and a fusion subunit.

[0106] The consumption rate acquisition sub-unit is used to acquire the space capacity consumption rate corresponding to the second space attribute information type.

[0107] The fusion subunit is used to fuse space capacity consumption rate, preset safety factor and basic resource cache amount to determine target idle resource information.

[0108] The memory resource processing device 500 in this embodiment further includes: a memory resource acquisition module, a resource consumption rate acquisition module, and a fusion module.

[0109] The memory resource acquisition module is used to respond to a resource request initiated by the first process or the second process for a target table, and to acquire memory resources matching the resource request from the cached resource amount of the first cache pool indicated by the target table. The cached resource amount is determined based on the following method:

[0110] The resource consumption rate acquisition module is used to obtain the memory resource consumption rate corresponding to the target table.

[0111] The fusion module is used to fuse memory resource consumption rate and preset safety factor to determine the amount of cache resources reserved for the target table in the first cache pool.

[0112] The memory resource processing device 500 in this embodiment further includes a merging module.

[0113] The merging module is used to merge the row information and index information stored in the memory space of the corresponding target table obtained from the first cache pool when a unique index exists in the target table, and generate a mapping entry. The mapping entry is used to find cached data related to the target table in the memory space corresponding to the amount of cache resources reserved for the target table in the first cache pool.

[0114] The first cache pool is used to cache multiple memory resource types, and multiple memory spaces corresponding to the same memory resource type are associated with memory space indices in the space linked list.

[0115] Optionally, any multiple modules among the task execution module 510, attribute information writing module 520, detection module 530, and idle information writing module 540 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. Optionally, at least one of the task execution module 510, attribute information writing module 520, detection module 530, and idle information writing module 540 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in hardware or firmware, or in any one of the three implementation methods—software, hardware, and firmware—or in a suitable combination of any of these. Alternatively, at least one of the task execution module 510, attribute information writing module 520, detection module 530, and idle information writing module 540 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0116] Figure 6 A block diagram of an electronic device suitable for implementing a memory resource processing method according to an embodiment of this application is shown.

[0117] like Figure 6As shown, an electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a ROM 602 (Read-Only Memory) or a program loaded from a storage portion 608 into a RAM 603 (Random Access Memory). The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0118] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0119] Optionally, the electronic device 600 may also include an I / O interface 605, which is also connected to the bus 604. The electronic device 600 may also include one or more of the following components connected to the I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0120] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0121] Optionally, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, optionally, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0122] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the memory resource processing method provided in the embodiments of this application.

[0123] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this application embodiment. Optionally, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0124] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0125] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this application embodiment. Optionally, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0126] Optionally, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0129] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A method for processing memory resources, characterized in that, The method includes: While the first process is executing a first task by calling memory resources from the first cache pool into the first memory space, the second process calls the memory resources into the second memory space to execute a second task. When the second task is completed, the second process writes the second space attribute information of the second memory space into the first process's preset first return list. The first return list is used to store the first space attribute information of the first memory space when the first task is completed. The spatial attribute information in the first return list is detected to obtain the detection results; In response to the detection result and the memory space capacity corresponding to the second spatial attribute information meeting a preset capacity condition, the second spatial attribute information is used as idle resource information, and the idle resource information is written into the first cache pool, including: When the memory space capacity is not lower than a preset cache threshold, the free resource information is determined from the second space attribute information based on the dynamic cache adjustment algorithm, and the first process or the second process calls memory space from the first cache pool according to the free resource information; Based on the difference between the second spatial attribute information and the free resource information, the remaining free resource information is determined. The remaining free resource information represents the portion of the memory space capacity corresponding to the second spatial attribute information that is unrelated to the free resource information. The remaining free resource information is written to the first memory pool of the first process. The first memory pool is a special memory pool corresponding to the second space attribute information type.

2. The method according to claim 1, characterized in that, The method further includes: If the second memory space in the first cache pool is insufficient, the remaining memory space corresponding to the remaining free resource information is called from the first memory pool to execute the second process.

3. The method according to claim 1, characterized in that, The free resource information is determined from the second space attribute information based on the dynamic cache adjustment algorithm, including: Obtain the space capacity consumption rate corresponding to the second space attribute information type; By combining the space capacity consumption rate, the preset safety factor, and the basic resource cache amount, the target idle resource information is determined.

4. The method according to claim 1, characterized in that, The first cache pool is used to cache multiple memory resource types, and multiple memory spaces corresponding to the same memory resource type are associated with memory space indices in a space linked list.

5. The method according to claim 1, characterized in that, The method further includes: In response to a resource request initiated by a first process or a second process for a target table, memory resources matching the resource request are obtained from the cached resource amount of a first cache pool indicated by the target table; wherein, the cached resource amount is determined based on the following method: Obtain the memory resource consumption rate corresponding to the target table; By combining the memory resource consumption rate and the preset safety factor, the amount of cache resources reserved in the first cache pool for the target table is determined.

6. The method according to claim 5, characterized in that, The method further includes: If a unique index exists in the target table, the row information and index information stored in the memory space corresponding to the target table obtained from the first cache pool are merged to generate a mapping entry. The mapping entry is used to find cached data related to the target table in the memory space corresponding to the amount of cache resources reserved for the target table in the first cache pool.

7. A memory resource management device, characterized in that, The device includes: The task execution module is used to execute a second task by utilizing the second memory space of the memory resources called by the second process during the process of the first process calling the first memory space of the memory resources from the first cache pool to execute the first task. The attribute information writing module is used to write the second space attribute information of the second memory space into the preset first return list of the first process when the second task is completed. The first return list is used to store the first space attribute information of the first memory space when the first task is completed. The detection module is used to detect the spatial attribute information in the first return list and obtain the detection result; The idle information writing module is used to respond to the detection result and the memory space capacity corresponding to the second spatial attribute information meeting a preset capacity condition, to use the second spatial attribute information as idle resource information, and to write the idle resource information into the first cache pool, and the first process or the second process calls memory space from the first cache pool according to the idle resource information; the idle information writing module includes: a determination unit; The determining unit is configured to determine the free resource information from the second space attribute information based on a dynamic cache adjustment algorithm when the memory space capacity is not lower than a preset cache threshold. The remaining free space determination module is used to determine the remaining free space information based on the difference between the second space attribute information and the free resource information. The remaining free space information represents the portion of the memory space capacity corresponding to the second space attribute information that is unrelated to the free resource information. The writing module is used to write the remaining free resource information into the first memory pool of the first process, where the first memory pool is a dedicated memory pool corresponding to the second space attribute information type.

8. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images