Method and device for dynamically managing shared memory pool

Abstract

A method and apparatus for dynamically managing a shared memory pool are provided. The method determines a shared memory pool mechanism suitable for the current business scenario based on different business scenarios, and then dynamically adjusts the memory pool mechanism based on the determination result to improve system performance. The method for dynamically managing a shared memory pool includes: determining a first shared memory pool mechanism, wherein the first shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism (S401); determining a second shared memory pool mechanism suitable for a second business scenario based on a second business scenario, wherein the second shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism (S402); and, if the second shared memory pool mechanism is different from the first shared memory pool mechanism, adjusting the first shared memory pool mechanism to the second shared memory pool mechanism (S403).

Description

Technical Field

[0001] The present application relates to the field of storage, and more specifically, to a method and apparatus for dynamically managing a shared memory pool. Background Art

[0002] In recent years, with the popularization of deep learning and embedded neural network processing units (NPUs), deep learning technology has been adopted in an increasing number of application scenarios. To accommodate a wide range of application scenarios, a growing number of unified application platforms that meet both training and inference requirements have been launched. These platforms target inference-based application scenarios such as intelligent monitoring, facial recognition, image classification, traffic warnings, and tracking. Both edge computing and central computing require real-time transmission of large amounts of data from remote locations to chips with artificial intelligence (AI) processing capabilities for inference tasks. This large amount of data transmission not only consumes network bandwidth but also occupies significant memory resources. To address this latter issue, the industry currently offers a memory pool sharing mechanism that improves data transmission efficiency and conserves memory resources by reusing memory to avoid large amounts of allocation and release. Currently, there are two commonly used memory pool sharing mechanisms. One is the fixed memory pool mechanism. For scenarios with relatively stable memory usage, the memory pool size can be pre-allocated in advance to avoid unnecessary scaling and save a certain amount of memory. However, this method is not flexible enough and cannot be applied to scenarios with multiple threads or unstable data flows. The other is the dynamic memory pool mechanism. For scenarios with multiple threads and fluctuating data flows, this mechanism can maximize memory savings by dynamically scaling the memory pool and allocating memory on demand. However, frequent scaling can cause fluctuations in system performance. Summary of the Invention

[0003] The present application provides a method and apparatus for dynamically managing a shared memory pool, which determines a shared memory pool mechanism suitable for the current business scenario according to different business scenarios, and then dynamically adjusts the memory pool mechanism based on the determination result to improve system performance.

[0004] In a first aspect, a method for dynamically managing a shared memory pool is provided, the method comprising: determining a first shared memory pool mechanism, where the first shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism; determining a second shared memory pool mechanism suitable for a second business scenario based on a second business scenario, where the second shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism; and when the second shared memory pool mechanism is different from the first shared memory pool mechanism, adjusting the first shared memory pool mechanism to the second shared memory pool mechanism.

[0005] The method for dynamically managing a shared memory pool in an embodiment of the present application adopts different shared memory pool mechanisms for different business scenarios, determines the shared memory pool mechanism suitable for the current business scenario according to different business scenarios, and then dynamically adjusts the memory pool mechanism based on the determination result, giving full play to the advantages of different shared memory pool mechanisms to better meet the needs of business scenarios.

[0006] In combination with the first aspect, in a possible implementation, the first shared memory pool mechanism is a shared memory pool mechanism suitable for a first business scenario; before determining the second shared memory pool mechanism suitable for the second business scenario based on the second business scenario, it also includes: determining that the business scenario has switched from the first business scenario to the second business scenario.

[0007] The method for dynamically managing a shared memory pool according to an embodiment of the present application can be applied to a situation where, when a business scenario switches, if the shared memory pool mechanism applicable to the current business scenario also changes, the shared memory pool mechanism applicable to the current business scenario can be re-determined.

[0008] With reference to the first aspect, in a possible implementation manner, the first shared memory pool mechanism is an initialized shared memory pool mechanism.

[0009] The method for dynamically managing a shared memory pool according to an embodiment of the present application can be used to redetermine a shared memory pool mechanism suitable for the current business scenario when the shared memory pool initialized by the system is not suitable for the current business scenario.

[0010] In combination with the first aspect, in a possible implementation, determining a second shared memory pool mechanism suitable for the second business scenario based on the second business scenario includes: determining parameters of the second shared memory pool applicable to the second business scenario based on the second business scenario; and determining a second shared memory pool mechanism applicable to the second business scenario based on the parameters of the second shared memory pool.

[0011] With reference to the first aspect, in a possible implementation, the parameters of the second shared memory pool include a maximum value and a minimum value of the second shared memory pool size.

[0012] In the method for dynamically managing a shared memory pool according to an embodiment of the present application, the maximum and minimum sizes of the shared memory pool may be determined first, and the mechanism of the shared memory pool may be determined based on the maximum and minimum sizes of the shared memory pool.

[0013] In combination with the first aspect, in a possible implementation, determining the parameters of the second shared memory pool applicable to the second business scenario based on the second business scenario includes: obtaining memory usage information of a first time period in the second business scenario; and determining the parameters of the second shared memory pool based on the memory usage information.

[0014] The method for dynamically managing a shared memory pool according to an embodiment of the present application determines parameters of the shared memory pool based on actual memory usage information, and can derive shared memory pool parameters that are more suitable for the current business scenario.

[0015] In conjunction with the first aspect, in a possible implementation, determining the parameters of the second shared memory pool based on the memory usage information includes: obtaining a memory usage value within a second time period in the second business scenario; if the memory usage value is greater than the maximum value of the first shared memory pool size, using the memory usage value as the maximum value of the second shared memory pool size; or if the memory usage value is less than the maximum value of the first shared memory pool size, using the memory usage value as the maximum value of the second shared memory pool size; if the memory usage value is greater than the minimum value of the first shared memory pool size, using the memory usage value as the minimum value of the second shared memory pool size; or if the memory usage value is less than the minimum value of the first shared memory pool size, using the memory usage value as the minimum value of the second shared memory pool size.

[0016] The method for dynamically managing a shared memory pool in an embodiment of the present application provides a method for determining shared memory pool parameters based on actual memory usage values, and uses the actual memory usage values ​​as the size of the shared memory pool, so that the shared memory pool can be more suitable for the current business scenario, and is intuitive and convenient.

[0017] In combination with the first aspect, in a possible implementation, determining the second shared memory pool mechanism applicable to the second business scenario based on the parameters of the second shared memory pool includes: if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is less than a third threshold, determining that the second shared memory pool mechanism is a fixed memory pool mechanism; if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is greater than a fourth threshold, determining that the second shared memory pool mechanism is a dynamic memory pool mechanism.

[0018] In a second aspect, a device for dynamically managing a shared memory pool is provided, the device comprising: a determination unit, configured to determine a first shared memory pool mechanism, wherein the first shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism; the determination unit is further configured to determine a second shared memory pool mechanism suitable for a second business scenario based on a second business scenario, wherein the second shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism; and an adjustment unit, configured to adjust the first shared memory pool mechanism to the second shared memory pool mechanism when the second shared memory pool mechanism is different from the first shared memory pool mechanism.

[0019] An embodiment of the present application further provides a device for dynamically managing a shared memory pool, which is used to implement the above-mentioned method for dynamically managing a shared memory pool.

[0020] In combination with the second aspect, in a possible implementation, the first shared memory pool mechanism is a shared memory pool mechanism suitable for a first business scenario; before the determination unit determines the second shared memory pool mechanism suitable for the second business scenario based on the second business scenario, the determination unit is also used to: determine that the business scenario has switched from the first business scenario to the second business scenario.

[0021] In combination with the second aspect, in a possible implementation manner, the first shared memory pool mechanism is an initialized shared memory pool mechanism.

[0022] In combination with the second aspect, in a possible implementation, the determination unit determines a second shared memory pool mechanism suitable for the second business scenario based on the second business scenario, including: determining parameters of the second shared memory pool applicable to the second business scenario based on the second business scenario; determining the second shared memory pool mechanism applicable to the second business scenario based on the parameters of the second shared memory pool.

[0023] With reference to the second aspect, in a possible implementation, the parameters of the second shared memory pool include a maximum value and a minimum value of the second shared memory pool size.

[0024] In combination with the second aspect, in a possible implementation method, the determination unit determines the parameters of the second shared memory pool applicable to the second business scenario based on the second business scenario, including: obtaining memory usage information of a first time period in the second business scenario; and determining the parameters of the second shared memory pool based on the memory usage information.

[0025] With reference to the second aspect, in a possible implementation, the determining unit determines the parameters of the second shared memory pool according to the memory usage information, including: obtaining a memory usage value within a second time period in the second business scenario;

[0026] If the memory usage value is greater than the maximum value of the first shared memory pool size, the memory usage value is used as the maximum value of the second shared memory pool size; or, if the memory usage value is less than the maximum value of the first shared memory pool size, the memory usage value is used as the maximum value of the second shared memory pool size. If the memory usage value is greater than the minimum value of the first shared memory pool size, the memory usage value is used as the minimum value of the second shared memory pool size; or, if the memory usage value is less than the minimum value of the first shared memory pool size, the memory usage value is used as the minimum value of the second shared memory pool size.

[0027] In combination with the second aspect, in a possible implementation, the determination unit determines a second shared memory pool mechanism applicable to the second business scenario based on the parameters of the second shared memory pool, including: if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is less than a third threshold, determining that the second shared memory pool mechanism is a fixed memory pool mechanism; if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is greater than a fourth threshold, determining that the second shared memory pool mechanism is a dynamic memory pool mechanism.

[0028] In a third aspect, a computer program storage medium is provided, characterized in that the computer program storage medium has program instructions, and when the program instructions are executed by a processor, the processor executes the method of the first aspect and any one of the implementation methods of the first aspect.

[0029] In a fourth aspect, a chip is provided, characterized in that the chip includes at least one processor, and when program instructions are executed by the at least one processor, the at least one processor executes the method of the first aspect and any one of the implementation methods of the first aspect.

[0030] In a fifth aspect, a terminal device is provided, which includes the aforementioned apparatus for dynamically managing a shared memory pool. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a schematic block diagram of two existing memory pool sharing mechanisms in an embodiment of the present application;

[0032] Figure 2 is a schematic block diagram of possible application scenarios of the method for dynamically managing a shared memory pool according to an embodiment of the present application;

[0033] Figure 3 This is an application scenario of the method for dynamically managing a shared memory pool according to an embodiment of the present application;

[0034] Figure 4is a schematic flow chart of a method for dynamically managing a shared memory pool according to an embodiment of the present application;

[0035] Figure 5 This is a schematic block diagram of a specific implementation of the method for dynamically managing a shared memory pool according to an embodiment of the present application;

[0036] Figure 6 is a schematic architecture diagram of a system according to an embodiment of the present application;

[0037] Figure 7 is a schematic structural diagram of a device for dynamically managing a shared memory pool provided in an embodiment of the present application;

[0038] Figure 8 is a schematic structural diagram of another device for dynamically managing a shared memory pool provided in an embodiment of the present application;

[0039] Figure 9 This is a schematic structural diagram of another device for dynamically managing a shared memory pool provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solution in this application will be described below with reference to the accompanying drawings.

[0041] The shared memory pool is a pre-allocated memory pool of appropriate size, which is accessed by multiple processes. The multiple processes can be different processes of the same device or different processes of different devices. Figure 1 Two existing memory pool sharing mechanisms are shown: a fixed memory pool mechanism and a dynamic memory pool mechanism. The fixed memory pool mechanism pre-allocates a fixed, unchanging memory pool size. The memory pool does not dynamically scale based on business changes. For scenarios with relatively stable memory usage, this mechanism allows for pre-allocation of the memory pool size to avoid unnecessary scaling and save memory. However, this mechanism is inflexible and unsuitable for multi-threaded scenarios or scenarios with fluctuating data flows. In multi-threaded scenarios, the required memory pool size cannot be determined in advance, leading to rapid depletion of the fixed memory pool and subsequent memory requests being blocked. Even if physical memory is available, user services cannot request memory, resulting in performance bottlenecks. The dynamic memory pool mechanism allows the memory pool size to be adjusted on demand. Under this mechanism, the memory pool size is within a dynamic range [Min, Max]. For scenarios with multi-threaded or fluctuating data flows, dynamic scaling of the memory pool and on-demand memory allocation can maximize memory savings. However, memory scaling can also lead to performance fluctuations. Complex business scenarios such as multi-process / multi-threaded scenarios require frequent scaling, causing system performance fluctuations.

[0042] The application scenarios of the memory pool mechanism are diverse. Figure 2 Three possible application scenarios are shown, including:

[0043] Scenario A: Single-channel video stream, stable data input;

[0044] Scenario B: Single-channel video stream, data stream input fluctuates during the day and remains stable at night, and memory consumption has a certain pattern;

[0045] Scenario C: Multiple video streams, dynamic data input, and fluctuating memory consumption.

[0046] In the prior art, a single memory mechanism is typically determined to suit all scenarios. That is, scenarios A, B, and C all use a fixed memory pool, or scenarios A, B, and C all use a dynamic memory pool. In practice, scenario A is well-suited to a fixed memory pool mechanism, while scenario C is well-suited to a dynamic memory pool mechanism. In scenario B, a dynamic memory pool is suitable for daytime use, while a fixed memory pool is suitable for nighttime use. Furthermore, for both dynamic and fixed memory pools, key memory pool configuration parameters, such as memory size and range, are manually pre-set based on experience, making them incapable of truly adapting to different scenarios and needs. For open application scenarios, especially basic computing platforms, a variety of different business scenarios are inevitable. Currently, no solution can dynamically adapt to each application scenario. Pre-defined memory pool mechanisms each have their own advantages and disadvantages, making them inapplicable to all user business scenarios. The method for dynamically managing a shared memory pool in the embodiments of the present application can dynamically learn the matching relationships between various scenarios and memory pool mechanisms through pre-learning. Based on the learned results, the memory pool mechanism is then dynamically adjusted, ensuring that scenario A uses a fixed memory pool, scenario C uses a dynamic memory pool, and scenario B dynamically switches between a fixed and dynamic memory pool based on the business scenario, thereby achieving optimal business performance.

[0047] The present embodiment seeks the optimal match between different application scenarios and memory pool mechanisms, dynamically adjusting the memory pool mechanism based on the application scenario, leveraging the advantages of each memory pool mechanism while mitigating its disadvantages. By leveraging the fast, efficient, and stable advantages of fixed memory pools, and the flexible, economical, and on-demand advantages of dynamic memory pools, the present embodiment addresses the problem of a single memory pool mechanism being unable to dynamically adapt to different scenarios.

[0048] It should be understood that the above business scenarios A, B, and C are only examples of business scenarios and do not constitute a limitation on the embodiments of the present application. Figure 2 In addition to the business scenario in the embodiment of the present application, the method for dynamically managing the shared memory pool can also be applied to other business scenarios.

[0049] Figure 3An application scenario of the method for dynamically managing a shared memory pool according to an embodiment of the present application is shown. In the currently widely used application scenarios such as intelligent security and safe cities, a large number of videos or pictures need to be transmitted to the computing center through smart cameras for reasoning processing. Specific tasks include recognition (motor vehicles, non-motor vehicles, faces, etc.), image classification, retrieval and tracking (vehicles, pedestrians, etc.). Most of the above tasks rely on the processing capabilities of AI chips. With the continuous improvement of the processing capabilities of AI chips, the AI ​​computing capabilities of various types of AI NPUs currently on the market are also different. In order to give full play to the processing capabilities of different chips, a single chip will process multiple reasoning tasks at the same time, such as multiple videos being accessed at the same time, or multiple chips working together to process a single reasoning task, such as CPU-intensive retrieval. In the method for dynamically managing a shared memory pool according to an embodiment of the present application, when a variety of business applications transmit data streams, users do not need to analyze business flow behavior in advance. The system can dynamically adapt to the memory requirements under different scenarios to optimize system performance.

[0050] Figure 4 FIG. 4 shows a schematic flow chart of a method for dynamically managing a shared memory pool according to an embodiment of the present application. Figure 4 As shown, it includes steps 401 to 403, and these steps are introduced in detail below.

[0051] S401: Determine a first shared memory pool mechanism, where the first shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism.

[0052] Optionally, the first shared memory pool mechanism is an initialized shared memory pool mechanism. Specifically, the first shared memory pool mechanism can be a preset fixed memory pool mechanism or a dynamic memory pool mechanism, or a shared memory pool mechanism randomly initialized by the system. After determining a shared memory pool mechanism appropriate for the current business scenario, the current shared memory pool mechanism is adjusted.

[0053] Optionally, the first shared memory pool mechanism is a shared memory pool mechanism suitable for a first business scenario; before determining a second shared memory pool mechanism suitable for the second business scenario according to the second business scenario, the method further includes: determining that the business scenario has switched from the first business scenario to the second business scenario. Specifically, the business scenario can be Figure 2 The business scenarios A, B, and C listed as examples may also be other business scenarios.

[0054] S402: Determine a second shared memory pool mechanism suitable for the second business scenario according to the second business scenario, where the second shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism.

[0055] Specifically, first, parameters of a second shared memory pool applicable to the second business scenario are determined according to the second business scenario; and then, a second shared memory pool mechanism applicable to the second business scenario is determined according to the parameters of the second shared memory pool.

[0056] The parameters of the shared memory pool include a maximum value and a minimum value of the shared memory pool size. When the shared memory pool is a fixed memory pool, the difference between the maximum value and the minimum value of the shared memory pool size is less than a third threshold, wherein the third threshold may be a manually defined value.

[0057] In the method for dynamically managing a shared memory pool according to an embodiment of the present application, the parameters of the second shared memory pool applicable to the second business scenario are determined based on the second business scenario. The memory usage information for the first time period in the second business scenario can be first obtained, and then the parameters of the second shared memory pool can be determined based on the memory usage information. For example, the memory usage information can be a memory usage value. If the memory usage value is greater than the maximum value of the first shared memory pool size, the memory usage value is used as the maximum value of the second shared memory pool size. Alternatively, if the memory usage value is less than the maximum value of the first shared memory pool size, the memory usage value is used as the maximum value of the second shared memory pool size. If the memory usage value is greater than the minimum value of the first shared memory pool size, the memory usage value is used as the minimum value of the second shared memory pool size. Alternatively, if the memory usage value is less than the minimum value of the first shared memory pool size, the memory usage value is used as the minimum value of the second shared memory pool size. For another example, the memory usage information may be a memory usage rate (memory usage value / total memory pool size). If the memory usage rate is greater than a first threshold, the maximum value of the first shared memory pool size of the first shared memory pool mechanism is multiplied by a first coefficient to serve as the maximum value of the second shared memory pool size, and the first coefficient is greater than 1. Alternatively, the minimum value of the first shared memory pool size is multiplied by a second coefficient to serve as the minimum value of the second shared memory pool size, and the second coefficient is less than 1. If the memory usage rate is less than a second threshold, the maximum value of the first shared memory pool size is multiplied by a third coefficient to serve as the maximum value of the second shared memory pool size, and the third coefficient is less than 1. Alternatively, the minimum value of the first shared memory pool size is multiplied by a fourth coefficient to serve as the minimum value of the second shared memory pool size, and the first coefficient is greater than 1.

[0058] It should be understood that in addition to the above memory usage value and memory usage rate, the memory usage information in the business scenario may also include memory usage information such as the number of memory blocks that need to be added to the current memory pool and the number of memory blocks that need to be reduced in the current memory pool.

[0059] It should be understood that the above method for determining parameters of the second shared memory pool applicable to the second business scenario based on the second business scenario is merely an example of a method for dynamically managing a shared memory pool in an embodiment of the present application. In addition to the above method, embodiments of the present application may also employ a learned rule set to determine parameters of the second shared memory pool applicable to the second business scenario. The learned rule set may be a priori rules consisting of IF A THEN B.

[0060] In the method for dynamically managing a shared memory pool according to an embodiment of the present application, determining a second shared memory pool mechanism applicable to the second business scenario based on parameters of the second shared memory pool can be performed as follows: if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is less than a third threshold, determining that the second shared memory pool mechanism is a fixed memory pool mechanism; and if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is greater than a fourth threshold, determining that the second shared memory pool mechanism is a dynamic memory pool mechanism. The fourth threshold may be a manually defined value.

[0061] S403: When the second shared memory pool mechanism is different from the first shared memory pool mechanism, adjust the first shared memory pool mechanism to the second shared memory pool mechanism.

[0062] It should be understood that if the second shared memory pool mechanism is the same as the first shared memory pool mechanism, no adjustment is made.

[0063] It should be understood that the method for dynamically managing a shared memory pool in an embodiment of the present application is a dynamic loop method. After determining the shared memory pool mechanism applicable to the current business scenario based on the business scenario, the method of the embodiment of the present application also includes real-time monitoring of changes in the business scenario. For example, the parameters of the shared memory pool mechanism applicable to the current business scenario can be adjusted based on real-time memory usage information (such as memory usage value or memory usage rate) so that the size or size range of the shared memory pool is more suitable for the current business scenario; if the business scenario switches, making the current shared memory pool mechanism no longer suitable for the current business scenario, the shared memory pool mechanism applicable to the current business scenario can also be re-determined based on the business scenario after the switch.

[0064] It should be understood that the dynamic management shared memory pool method of the embodiment of the present application can dynamically switch the shared memory pool mechanism applicable to the current business scenario and dynamically adjust the parameters of the shared memory pool applicable to the current business scenario according to different business scenarios. However, since the actual business scenarios are diverse and cannot be exhaustive, the dynamic management shared memory pool method of the embodiment of the present application has no necessary connection with the actual business scenario, but dynamically switches the shared memory pool mechanism applicable to the current business scenario and dynamically adjusts the parameters of the shared memory pool mechanism applicable to the current business scenario based on the memory usage information of the actual business scenario.

[0065] Figure 5 A schematic block diagram showing a specific implementation of the method for dynamically managing a shared memory pool according to an embodiment of the present application is shown. Figure 5 As shown in the figure, it mainly includes learning rules, memory hyper-parameter, memory adapter and shared memory mechanism.

[0066] Two dynamic feedback loops are formed based on the data flow and the learning rule set: Loop A and Loop B. Loop A determines the first-stage memory hyperparameters based on the input data and the learning rule set. The shared memory mechanism that best matches the current business scenario is then determined based on the memory hyperparameters. Loop B adjusts the memory hyperparameters based on the learning rule set. If a business scenario changes, the current shared memory pool mechanism may no longer be suitable. In this case, the adjusted memory hyperparameters are used to determine the best shared memory pool mechanism for the current business scenario.

[0067] The learning rule set mainly includes a set of prior rules consisting of IF A THEN B, which are as follows:

[0068] 1. If UsedBlocks>Min&&Duration>TimeThreshold&&RepeatedTimes>TimesThreshold, Then Min=min(UsedBlocks);

[0069] 2. If MemUsageRate>95%&&Duration>TimeThreshold&&RepeatedTimes>TimesThreshold, Then Max=Max*(1+10%);

[0070] 3. If EnlargeCount < enThreshold && ReduceCount < reThreshold, then IsFixed = true;

[0071] 4. If Max - Min < LowThreshold && IsFixed == false, then IsFixed = true;

[0072] 5. If Max - Min > HighThreshold && IsFixed == true, then IsFixed = false;

[0073] 6. If Max - Min < LowThreshold, then FixedMemSize = Max.

[0074] Where:

[0075] 1. UsedBlocks: The number of used memory blocks; Duration > TimeThreshold: The used memory blocks continuously exceed a certain threshold time; RepeatedTimes > TimesThreshold: The number of occurrences of this situation is greater than a certain threshold number; where Min = min(UsedBlocks), and it can also be other limitations. For example, the value of Min can be set to a smaller value with the highest frequency among the used memory block numbers. This is just an example, and the embodiments of this application do not make specific limitations on this;

[0076] 2. MemUsageRate: The current memory usage rate, that is, the size of the used memory blocks / the total size of the memory pool; where, Max = Max * (1 + 10%) can also be other limitations. For example, it can be other percentages, or add a specific value such as 200M to the current value of Max. This is just an example, and the embodiments of this application do not make specific limitations on this;

[0077] 3. EnlargeCount < enThreshold: The number of memory blocks to be increased is less than the upper limit allowed by the system to increase; ReduceCount < reThreshold: The number of memory blocks to be reduced is less than the upper limit allowed by the system to reduce;

[0078] 4. Max - Min < LowThreshold: The interval between the maximum value and the minimum value of the dynamic memory pool range exceeds a certain minimum threshold;

[0079] 5. Max - Min > HighThreshold: The interval between the maximum value and the minimum value of the dynamic memory pool range exceeds a certain maximum threshold.

[0080] Memory hyperparameters include: Fixed memory pool mechanism parameter FixedMemSize, used to indicate the memory pool size; dynamic memory pool mechanism parameter Min, used to indicate the minimum value of the memory pool size; dynamic memory pool mechanism parameter Max, used to indicate the maximum value of the memory pool size; memory pool mechanism selection button IsFixed, used to indicate whether to switch the memory pool.

[0081] The memory adapter decides to switch between the dynamic and fixed memory pool mechanisms based on the value of IsFixed. During this switching process, the adapter also sets the current shared memory pool configuration, i.e., the memory pool size or size range, based on the learned memory hyperparameters.

[0082] The shared memory pool mechanism in the embodiments of the present application mainly includes a fixed memory pool mechanism and a dynamic memory pool mechanism. Specifically, the fixed memory pool mechanism has a fixed memory pool size that does not adjust with business changes, and all memory request operations are completed within this memory pool; the dynamic memory pool mechanism has a memory pool size that is dynamically adjusted according to business needs, and its adjustment range is generally a fixed size range [Min, Max].

[0083] The method for dynamically managing shared memory pools in the embodiments of the present application can be applied to scenarios such as inference computing platforms and deployed on computing nodes of related devices. By modifying the base platform software, it can achieve dynamic adaptation of different shared memory pool mechanisms in different business scenarios, maximizing the advantages of each shared memory pool mechanism. Specifically, when the shared memory pool mechanism is a dynamic memory pool mechanism, when the number of memory blocks used by the memory pool remains high for a long time, the minimum value Min of the memory pool size can be dynamically adjusted; when the memory pool usage rate is higher than a threshold (95%) for a long time, the maximum value Max of the memory pool size can be dynamically adjusted; when the minimum value Min of the memory pool size is approximately equal to the maximum value Max of the memory pool size, the dynamic memory pool can be adjusted to a fixed memory pool.

[0084] It should be understood that Figure 5The example in is only an example of the method for dynamically managing a shared memory pool in an embodiment of the present application, and does not constitute a limitation of the method for dynamically managing a shared memory pool in an embodiment of the present application. The above-mentioned learning rule set may also be other rules, or the limitation in the above-mentioned prior rules may also be other limitation methods, or the above-mentioned non-proper nouns may also be replaced by other nouns with the same meaning. Based on the same concept, those skilled in the art may make equivalent transformations in the above-mentioned examples to achieve the same or similar effects. The method for dynamically managing a shared memory pool obtained based on these equivalent transformations can still achieve the technical effects of the method for dynamically managing a shared memory pool in an embodiment of the present application. Therefore, these equivalent transformations should all fall within the scope of protection of the present application.

[0085] Figure 6 Schematic diagram of the system of the embodiment of the present application is shown. Figure 6 As shown, the system first acquires data from relevant applications, specifically memory usage information for the current business scenario. It then combines this data with learning rules to derive memory hyperparameters. Learning rules are a set of rules pre-set during system initialization that guide system learning. Memory hyperparameters determine the shared memory pool mechanism and the shared memory pool's memory size or range. A scenario-memory mechanism adapter switches between dynamic and fixed memory pool mechanisms based on the memory hyperparameters. After determining the appropriate shared memory pool mechanism for the current business scenario, the relevant application data is stored for subsequent processing.

[0086] Figure 7 FIG. 7 is a schematic structural diagram of an apparatus 700 for dynamically managing a shared memory pool provided in an embodiment of the present application. The apparatus 700 for dynamically managing a shared memory pool includes a determining unit 710 and an adjusting unit 720 .

[0087] The determining unit 710 is configured to determine a first shared memory pool mechanism, where the first shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism.

[0088] The determining unit 710 is further configured to determine, according to the second business scenario, a second shared memory pool mechanism suitable for the second business scenario, where the second shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism.

[0089] The adjusting unit 720 is configured to adjust the first shared memory pool mechanism to the second shared memory pool mechanism when the second shared memory pool mechanism is different from the first shared memory pool mechanism.

[0090] Optionally, the first shared memory pool mechanism is a shared memory pool mechanism suitable for a first business scenario.

[0091] Before the determining unit 710 determines the second shared memory pool mechanism suitable for the second business scenario according to the second business scenario, the determining unit 710 is further configured to: determine that the business scenario has switched from the first business scenario to the second business scenario.

[0092] Optionally, the first shared memory pool mechanism is an initialized shared memory pool mechanism.

[0093] Optionally, determining unit 710 determines a second shared memory pool mechanism suitable for the second business scenario based on the second business scenario, including: determining parameters of the second shared memory pool suitable for the second business scenario based on the second business scenario; and determining the second shared memory pool mechanism suitable for the second business scenario based on the parameters of the second shared memory pool.

[0094] Optionally, the parameters of the second shared memory pool include a maximum value and a minimum value of the second shared memory pool size.

[0095] Optionally, the determination unit 710 determines parameters of a second shared memory pool applicable to the second business scenario according to the second business scenario, including: obtaining memory usage information of a first time period in the second business scenario; and determining parameters of the second shared memory pool according to the memory usage information.

[0096] Optionally, the determining unit 710 determines parameters of the second shared memory pool according to the memory usage information, including:

[0097] Obtain a memory usage value within a second time period in the second business scenario; if the memory usage value is greater than the maximum value of the first shared memory pool size, use the memory usage value as the maximum value of the second shared memory pool size; or if the memory usage value is less than the maximum value of the first shared memory pool size, use the memory usage value as the maximum value of the second shared memory pool size; if the memory usage value is greater than the minimum value of the first shared memory pool size, use the memory usage value as the minimum value of the second shared memory pool size; or if the memory usage value is less than the minimum value of the first shared memory pool size, use the memory usage value as the minimum value of the second shared memory pool size.

[0098] Optionally, the determination unit 710 determines a second shared memory pool mechanism applicable to the second business scenario based on the parameters of the second shared memory pool, including: if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is less than a third threshold, determining that the second shared memory pool mechanism is a fixed memory pool mechanism; if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is greater than a fourth threshold, determining that the second shared memory pool mechanism is a dynamic memory pool mechanism.

[0099] It should be understood that Figure 7 The determination unit 710 in the embodiment can be used to perform the above Figure 4 In the method of S401 and S402, the adjustment unit 720 can be used to perform the above Figure 4 The method in S403 can be specifically referred to the above Figure 4 For the sake of brevity, the embodiments of the present application will not be described in detail here.

[0100] Figure 8 8 is a schematic structural diagram of an apparatus 800 for dynamically managing a shared memory pool, provided in accordance with one embodiment of the present application. Memory management apparatus 800 may include at least one processor 810 and a communication interface 820. Communication interface 820 may be used to enable memory management apparatus 800 to exchange information with other apparatuses. When program instructions are executed in at least one processor 810, memory management apparatus 800 implements the various steps, methods, operations, or functions described above for the apparatus for dynamically managing a shared memory pool.

[0101] Figure 9 Schematic diagram of a device for dynamically managing a shared memory pool according to an embodiment of the present application. The device 900 for dynamically managing a shared memory pool includes a communication module 910 and a memory management module 920.

[0102] The communication module 910 is used to exchange information with other modules.

[0103] The memory management module 920 is configured to determine a first shared memory pool mechanism, where the first shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism.

[0104] The memory management module 920 is further configured to determine, according to the second business scenario, a second shared memory pool mechanism suitable for the second business scenario, where the second shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism.

[0105] The memory management module 920 is further configured to, when the second shared memory pool mechanism is different from the first shared memory pool mechanism, adjust the first shared memory pool mechanism to the second shared memory pool mechanism.

[0106] Optionally, the first shared memory pool mechanism is a shared memory pool mechanism suitable for a first business scenario.

[0107] Before the memory management module 920 determines the second shared memory pool mechanism suitable for the second business scenario according to the second business scenario, the memory management module 920 is further configured to: determine that the business scenario has switched from the first business scenario to the second business scenario.

[0108] Optionally, the first shared memory pool mechanism is an initialized shared memory pool mechanism.

[0109] Optionally, the memory management module 920 determines a second shared memory pool mechanism suitable for the second business scenario based on the second business scenario, including: determining parameters of the second shared memory pool suitable for the second business scenario based on the second business scenario; and determining the second shared memory pool mechanism suitable for the second business scenario based on the parameters of the second shared memory pool.

[0110] Optionally, the parameters of the second shared memory pool include a maximum value and a minimum value of the second shared memory pool size.

[0111] Optionally, the memory management module 920 determines parameters of a second shared memory pool applicable to the second business scenario based on the second business scenario, including: obtaining memory usage information of a first time period in the second business scenario; and determining parameters of the second shared memory pool based on the memory usage information.

[0112] Optionally, the memory management module 920 determines parameters of the second shared memory pool according to the memory usage information, including:

[0113] Obtain a memory usage value within a second time period in the second business scenario; if the memory usage value is greater than the maximum value of the first shared memory pool size, use the memory usage value as the maximum value of the second shared memory pool size; or if the memory usage value is less than the maximum value of the first shared memory pool size, use the memory usage value as the maximum value of the second shared memory pool size; if the memory usage value is greater than the minimum value of the first shared memory pool size, use the memory usage value as the minimum value of the second shared memory pool size; or if the memory usage value is less than the minimum value of the first shared memory pool size, use the memory usage value as the minimum value of the second shared memory pool size.

[0114] Optionally, the memory management module 920 determines a second shared memory pool mechanism applicable to the second business scenario based on the parameters of the second shared memory pool, including: if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is less than a third threshold, determining that the second shared memory pool mechanism is a fixed memory pool mechanism; if the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is greater than a fourth threshold, determining that the second shared memory pool mechanism is a dynamic memory pool mechanism.

[0115] It should be understood that Figure 9 The memory management module 920 in the embodiment can be used to perform the above Figure 4 The methods in S401 to S403 can be specifically referred to the above Figure 4 For the sake of brevity, the embodiments of the present application will not be described in detail here.

[0116] An embodiment of the present application further provides a terminal device, which includes the aforementioned apparatus for dynamically managing a shared memory pool.

[0117] An embodiment of the present application also provides a computer program storage medium, characterized in that the computer program storage medium has program instructions, and when the program instructions are executed by a processor, the processor executes the method for dynamically managing a shared memory pool as described above.

[0118] An embodiment of the present application also provides a chip system, characterized in that the chip system includes at least one processor, and when program instructions are executed in the at least one processor, the at least one processor executes the method for dynamically managing the shared memory pool mentioned above.

[0119] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0122] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0124] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program codes.

[0125] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for dynamically managing a shared memory pool, characterized in that: include: Determining a first shared memory pool mechanism, where the first shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism; Determining a second shared memory pool mechanism suitable for the second business scenario according to the second business scenario, where the second shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism; When the second shared memory pool mechanism is different from the first shared memory pool mechanism, adjusting the first shared memory pool mechanism to the second shared memory pool mechanism; The determining, according to the second business scenario, a second shared memory pool mechanism suitable for the second business scenario includes: determining, according to the second business scenario, parameters of the second shared memory pool suitable for the second business scenario; and determining, according to the parameters of the second shared memory pool, a second shared memory pool mechanism suitable for the second business scenario. The determining, according to the second business scenario, parameters of the second shared memory pool applicable to the second business scenario includes: obtaining memory usage information of a first time period in the second business scenario; and determining parameters of the second shared memory pool according to the memory usage information. The determining of the parameters of the second shared memory pool according to the memory usage information includes: obtaining a memory usage value within a second time period in the second business scenario; if the memory usage value is greater than the maximum value of the first shared memory pool size, using the memory usage value as the maximum value of the second shared memory pool size; or, if the memory usage value is less than the maximum value of the first shared memory pool size, using the memory usage value as the maximum value of the second shared memory pool size; if the memory usage value is greater than the minimum value of the first shared memory pool size, using the memory usage value as the minimum value of the second shared memory pool size; or, if the memory usage value is less than the minimum value of the first shared memory pool size, using the memory usage value as the minimum value of the second shared memory pool size.

2. The method according to claim 1, characterized in that The first shared memory pool mechanism is a shared memory pool mechanism suitable for the first business scenario; Before determining the second shared memory pool mechanism suitable for the second business scenario according to the second business scenario, the method further includes: It is determined that the business scenario has switched from the first business scenario to the second business scenario.

3. The method according to claim 1, characterized in that The first shared memory pool mechanism is an initialized shared memory pool mechanism.

4. The method according to claim 1, wherein The parameters of the second shared memory pool include a maximum value and a minimum value of the second shared memory pool size.

5. The method according to claim 4, characterized in that The determining, according to the parameters of the second shared memory pool, a second shared memory pool mechanism applicable to the second business scenario includes: If a difference between a maximum value of the second shared memory pool size and a minimum value of the second shared memory pool size is less than a third threshold, determining that the second shared memory pool mechanism is a fixed memory pool mechanism; If the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is greater than a fourth threshold, it is determined that the second shared memory pool mechanism is a dynamic memory pool mechanism.

6. A device for dynamically managing a shared memory pool, characterized in that: include: a determining unit, configured to determine a first shared memory pool mechanism, where the first shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism; The determining unit is further configured to determine, according to the second business scenario, a second shared memory pool mechanism suitable for the second business scenario, where the second shared memory pool mechanism is a fixed memory pool mechanism or a dynamic memory pool mechanism; an adjusting unit, configured to adjust the first shared memory pool mechanism to the second shared memory pool mechanism when the second shared memory pool mechanism is different from the first shared memory pool mechanism; The determining unit determines, according to the second business scenario, a second shared memory pool mechanism suitable for the second business scenario, including: determining, according to the second business scenario, parameters of the second shared memory pool suitable for the second business scenario; and determining, according to the parameters of the second shared memory pool, the second shared memory pool mechanism suitable for the second business scenario. The determining unit determines parameters of the second shared memory pool applicable to the second business scenario according to the second business scenario, including: obtaining memory usage information of a first time period in the second business scenario; and determining parameters of the second shared memory pool according to the memory usage information; The determining unit determines the parameters of the second shared memory pool according to the memory usage information, including: obtaining a memory usage value within a second time period in the second business scenario; if the memory usage value is greater than the maximum value of the first shared memory pool size, using the memory usage value as the maximum value of the second shared memory pool size, or if the memory usage value is less than the maximum value of the first shared memory pool size, using the memory usage value as the maximum value of the second shared memory pool size; if the memory usage value is greater than the minimum value of the first shared memory pool size, using the memory usage value as the minimum value of the second shared memory pool size, or if the memory usage value is less than the minimum value of the first shared memory pool size, using the memory usage value as the minimum value of the second shared memory pool size.

7. The device according to claim 6, characterized in that The first shared memory pool mechanism is a shared memory pool mechanism suitable for the first business scenario; Before the determining unit determines the second shared memory pool mechanism suitable for the second business scenario according to the second business scenario, the determining unit is further configured to: It is determined that the business scenario has switched from the first business scenario to the second business scenario.

8. The device according to claim 6, characterized in that The first shared memory pool mechanism is an initialized shared memory pool mechanism.

9. The device according to claim 6, characterized in that The parameters of the second shared memory pool include a maximum value and a minimum value of the second shared memory pool size.

10. The device according to claim 9, characterized in that The determining unit determines, according to the parameters of the second shared memory pool, a second shared memory pool mechanism applicable to the second business scenario, including: If a difference between a maximum value of the second shared memory pool size and a minimum value of the second shared memory pool size is less than a third threshold, determining that the second shared memory pool mechanism is a fixed memory pool mechanism; If the difference between the maximum value of the second shared memory pool size and the minimum value of the second shared memory pool size is greater than a fourth threshold, it is determined that the second shared memory pool mechanism is a dynamic memory pool mechanism.

11. A computer program storage medium, characterized in that The computer program storage medium has program instructions, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 5.

12. A chip, characterized in that: The chip includes at least one processor. When program instructions are executed by the at least one processor, the at least one processor is caused to perform the method according to any one of claims 1 to 5.

Images