Application resource adaptive distribution management system for concentrator

Abstract

The invention discloses an application resource adaptive allocation management system for a concentrator, which relates to the technical field of concentrator management and comprises a monitoring module, a resource allocation module, a priority management module, a resource recovery module and a data storage module. According to the invention, the resource use conditions of the CPU, the memory, the bandwidth and the terminal equipment are monitored in real time, and the network flow and the terminal equipment demand data are collected. A resource allocation strategy is dynamically adjusted to ensure efficient utilization of resources. And according to the task type, the important value of the terminal equipment and the CRDI value, the priority is dynamically adjusted to ensure that the key task and the important terminal equipment preferentially obtain resources when the resources are insufficient. By monitoring the resource utilization rate and the task state, the recycling operation is triggered in real time, and the idle resources are redistributed to the current terminal equipment with the demand, so that the overall resource use efficiency is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of concentrator management, and in particular to an application resource adaptive allocation management system for a concentrator. Background Art

[0002] With the rapid development of the Internet of Things and large-scale data processing needs, concentrators, as core components in network terminal devices, usually need to manage and process data and requests from multiple terminal devices. However, with the increase in the number of terminal devices and the amount of data, the computing resources, storage resources, and bandwidth resources of the concentrator may face bottleneck problems. Therefore, how to efficiently allocate and manage the application resources of the concentrator to cope with the dynamically changing network environment and data traffic has become an urgent problem to be solved;

[0003] However, the existing concentrator resource allocation lacks precision and cannot accurately schedule resources according to the specific needs of applications and terminal devices. After resource allocation, the concentrator resource recovery mechanism is not flexible enough to release unused resources in a timely manner, resulting in long-term resource occupation. In addition, the concentrator priority management mechanism is relatively simple and cannot cope with diverse application scenarios and complex priority strategies.

[0004] In order to solve the above defects, a technical solution is now provided. Summary of the invention

[0005] The purpose of the present invention is to provide an application resource adaptive allocation management system for a concentrator to solve the problems mentioned in the above background technology. The present invention can dynamically adjust the resource allocation strategy of the concentrator according to the network load, the needs of the terminal equipment and the resource usage to improve the resource utilization and ensure the efficiency and accuracy of data processing.

[0006] The object of the present invention can be achieved by the following technical solutions: An application resource adaptive allocation management system for a concentrator, 1. The monitoring module is used to monitor the resource usage information of the concentrator in real time and send it to the resource allocation module;

[0007] The resource allocation module optimizes the allocation of computing resources, storage resources and bandwidth resources based on resource usage information. The specific process is: calculate the memory utilization rate according to the current memory usage data, and calculate the bandwidth utilization rate according to the current network traffic; calculate the comprehensive resource demand index of each terminal device, sort them from large to small according to the comprehensive resource demand index, and allocate computing resources, storage resources and bandwidth resources to the corresponding terminal device according to the order.

[0008] As a preferred implementation manner of the present invention, the resource usage information includes CPU usage data, memory usage data and bandwidth usage data.

[0009] As a preferred implementation of the present invention, the specific process of calculating the comprehensive resource demand index of each terminal device is as follows:

[0010] pass The comprehensive resource demand index CRDIi is obtained, where Ui = [Ucpui, Umemoryi, Ubandwidthi, Utraffici] is expressed as: the resource demand data vector of the i-th terminal device, W = [Wcpu, Wmemory, Wmandwidth, Wtraffic] represents the weight matrix, indicating the relative importance of different resources; b represents the bias vector; f(W×Ui+b) is the nonlinear activation function; represents the cross term, αj k is the cross-term coefficient, Uji and Uki represent the usage of the j-th and k-th resources of the ith terminal device.

[0011] As a preferred implementation of the present invention, it also includes a priority management module, which is used to allocate resources to terminal devices when resources are scarce. The specific steps are:

[0012] Step 1: Preliminary priority classification: including task type identification and terminal device type identification;

[0013] Step 2: Obtain the CRDIi of each terminal device based on the CRDIi value, task type, terminal device importance value and historical priority value;

[0014] Step 3: Substitute the preset formula to output the previous values ​​of the terminal device;

[0015] Step 4: Calculate the comprehensive excellence value corresponding to the priority of each terminal device through the comprehensive excellence value formula: comprehensive excellence value = w1×CRDIi+w2×task importance value+w3×terminal device importance value+w4×historical priority value+w5×static priority value; sort all terminal devices according to the comprehensive excellence value, and give priority to allocating resources to terminal devices with high comprehensive excellence values.

[0016] As a preferred embodiment of the present invention, the specific process of substituting the preset formula into the output terminal device's previous values ​​is:

[0017] Obtain the task start time, task end time, total number of tasks, maintenance times and maintenance time executed by the terminal device within the preset time range in the past, calculate the time difference between the task start time and the task end time to obtain the task processing time, obtain the preset time threshold corresponding to the task, if the task processing time is greater than the preset time threshold, the number of delays is increased once, and the difference between the two is calculated to obtain the delay time; if the task processing time is less than the preset time threshold, the number of advances is increased once, and the difference between the two is calculated to obtain the advance time; if the task processing time is equal to the preset time threshold, the standard number is increased once;

[0018] Substitute into the formula Output the historical values ​​RY of the terminal device, where RW1 is the total number of tasks, RWy is the total number threshold, gd1 is a preset fixed value, TQc is the number of advance times, YCc is the number of delays, TQz is the sum of all advance times, BZz is the standard number, YCz is the sum of all delays, and e1, e2, e3, e4 and e5 are all preset weight factors.

[0019] As a preferred embodiment of the present invention, it also includes a resource recovery module, which triggers resource recovery operations in real time or regularly, and automatically reallocates released resources to terminal devices that currently have needs, specifically:

[0020] When the resource recovery operation is triggered in real time or periodically, the released resources are automatically reallocated to the terminal devices that currently have demand; when the task of the terminal device is completed, the resource recovery module receives the end signal of the terminal device and recovers the occupied resources in time; when it is detected that the usage of various resources is close to the CPU bottleneck, memory bottleneck and bandwidth bottleneck, idle resources are actively recovered to make room.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. Based on the CRDIi index, the present invention integrates the usage of multiple resources, so as to more comprehensively evaluate the resource requirements of terminal devices. According to the CRDIi value, the terminal devices are sorted and more resources are allocated to terminal devices with large resource requirements first. This ensures that mission-critical or high-demand terminal devices can still operate stably under high load conditions.

[0023] 2. The present invention calculates the comprehensive merit value in real time and sorts all terminal devices according to the comprehensive merit value. The priority management module dynamically adjusts the allocation of resources. In the case of high load or resource shortage, the resource requirements of critical tasks or high-demand terminal devices can be met first. Resources are allocated according to the actual performance and requirements of the terminal devices. The priority management module can utilize system resources more effectively. It helps to reduce resource waste and improve the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0025] Figure 1 It is the overall block diagram of the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0028] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0029] See also Figure 1 As shown, an application resource adaptive allocation and management system for a concentrator includes a monitoring module, a resource allocation module, a priority management module, a resource recovery module and a data storage module, and each module is integrated in the concentrator.

[0030] The monitoring module is used to monitor the resource usage information of the concentrator in real time. The resource usage information includes CPU usage data, memory usage data, and bandwidth usage data. In addition, the monitoring module is also responsible for collecting and analyzing network traffic data and resource demand data of terminal devices, specifically:

[0031] CPU usage data is obtained by periodically obtaining CPU usage data from the operating system's performance monitoring API or tool, calculating the usage of each core (CPU usage = (total CPU time - idle CPU time) / total CPU time), obtaining the total CPU usage, and storing it as time series data. The real-time data and historical trends of CPU usage are displayed using charts, the changes in CPU usage are displayed using line charts, and the current CPU load is displayed using a dashboard. The usage of each core is displayed separately, for example, the usage of core 0 is 30% and the usage of core 1 is 45%;

[0032] Memory usage data includes the concentrator's total memory, used memory, free memory, and cache and buffer usage;

[0033] Bandwidth usage data obtains network traffic data through the network interface card driver or network management tool, records the inbound traffic JLL and outbound traffic CLL, and calculates the bandwidth usage rate;

[0034] Network traffic data uses network traffic monitoring tools to collect network packets, analyze traffic patterns, identify traffic peaks, detect abnormal traffic, and calculate key statistical indicators of traffic;

[0035] The resource demand data of the terminal device is collected through the API or monitoring tool of the terminal device, including CPU, memory, disk and network usage, and the resource demand data from different terminals is summarized and analyzed. Different terminals are named M1, M2, M3, M4, ..., and the terminal device load and demand distribution are saved. For example, the CPU utilization rate of the M1 terminal device is 85%, the memory usage reaches 12GB, and the network bandwidth usage peak reaches 50MB / s.

[0036] The resource allocation module dynamically adjusts the resource allocation strategy of the concentrator based on the resource usage information provided by the monitoring module, and uses an adaptive algorithm to optimize the allocation of computing resources, storage resources, and bandwidth resources according to the real-time network load and the needs of terminal devices to ensure efficient use of resources. Specifically:

[0037] According to the current memory usage data, the memory utilization of the network memory is calculated using the formula: memory utilization = current memory usage / available memory × 100%. When the memory utilization is greater than 85% and less than 100%, the corresponding period is identified as the memory peak period. When the bandwidth utilization is equal to 100%, it is identified as a memory bottleneck. According to the current network traffic, the network bandwidth utilization is calculated using the formula: bandwidth utilization = current bandwidth usage / available bandwidth × 100%. When the bandwidth utilization is greater than 85% and less than 100%, it is identified as a bandwidth peak period. When the bandwidth utilization is equal to 100%, it is identified as a bandwidth bottleneck. Similarly, when the CPU utilization is greater than 85% and less than 100%, it is identified as a CPU peak period. When the CPU utilization is equal to 100%, it is identified as a CPU bottleneck. The comprehensive resource demand index CRDIi is obtained, where Ui = [Ucpui, Umemoryi, Ubandwidthi, Utraffici] is expressed as: the resource demand data vector of the i-th terminal device, each component represents the usage of the terminal device on different resources, ranging from [0, 1]; W = [Wcpu, Wmemory, Wmandwidth, Wtraffic] represents the weight matrix, which represents the relative importance of different resources, and the specific value is customized by technical personnel in the field in specific use; b represents the bias vector, which is used to adjust the linear part of the model; f(W×Ui+b) is a nonlinear activation function, which is used to introduce nonlinear effects; considering the interaction between different resource usage, it is represented by cross terms: Here, the cross-term coefficient α jk reflects the interaction between resources Uji and Uki, for example, high bandwidth usage may have an impact on CPU usage; where α jk is the cross-term coefficient, which represents the impact of resource interaction on the overall demand; Uji and Uki represent the usage of the j-th and k-th resources of the ith terminal device, and their values ​​are obtained by multiplying the values ​​corresponding to the bandwidth utilization and CPU utilization by the corresponding preset weights and then summing them up; the terminal device with a higher CRDIi value has a greater resource demand; therefore, each terminal device is sorted from large to small according to the size of the CRDIi value, and more resources are allocated to the terminal device with a higher CRDIi value to meet its needs. This priority allocation strategy can ensure that mission-critical or high-demand terminal devices can still operate stably under high load conditions, thereby avoiding system bottlenecks and performance degradation.

[0038] In order to predict resource demand in the short term, time series analysis and machine learning models are used in combination with historical data to make predictions. This prediction method can help the system make resource allocation decisions in advance, thereby optimizing resource utilization efficiency.

[0039] Historical data use time series analysis: collect historical resource usage data, including but not limited to CPU usage, memory usage data, bandwidth usage data, network traffic data, and resource demand data of terminal devices, and set it according to actual conditions, such as resource usage per minute, hour, and day; use the autoregressive integrated moving average model (ARIMA) for stationary time series, which analyzes the autocorrelation and seasonal trends of resource usage for prediction, and divides the time series into trend, seasonal, and random components through seasonal decomposition, further improving the prediction accuracy;

[0040] Machine learning model prediction: Use short-term memory (LSTM) networks to process long time series data and capture complex time dependencies. Use a portion of historical data as a training set to train the model to predict future resource requirements. Use validation and test sets to evaluate model performance and adjust model parameters to improve prediction accuracy.

[0041] By combining historical data to make short-term forecasts of resource requirements, the system can allocate resources more proactively and accurately, avoid resource waste and performance bottlenecks, and improve overall efficiency.

[0042] The main function of the priority management module is to ensure that in the case of resource shortage, the priority identification process is a dynamic adjustment process that comprehensively considers the task type, terminal equipment importance value, resource demand (calculated by CRDIi value) and real-time data. The specific process is:

[0043] Step 1: Preliminary priority classification: including task type identification and terminal device type identification. First, identify the nature of each task, whether it is a critical task, security task, background task, user interaction task, etc., and the preset task importance value corresponding to each task nature; identify the purpose and importance of the terminal device, for example, core servers, switches, end-user terminal devices, monitoring terminal devices, etc., which correspond to a preset terminal device importance value respectively; according to the nature of the terminal device, give a preset preliminary static priority: critical tasks (such as system security monitoring) are set to high priority, corresponding to a high priority value, user interaction tasks (such as real-time communication) are set to medium priority, corresponding to a preset medium priority value, and background tasks (such as data synchronization) are set to low priority, corresponding to a low priority value;

[0044] Step 2: Obtain the CRDIi of each terminal device, based on the CRDIi value, task type, terminal device importance value and historical priority value. The historical priority value is determined according to the past situation of the terminal device. Specifically, obtain the task start time, task end time, total number of tasks, maintenance times and maintenance time executed by the terminal device within the preset time range in the past, calculate the time difference between the task start time and the task end time to obtain the task processing time, obtain the preset time threshold corresponding to the task, if the task processing time is greater than the preset time threshold, the delay times are increased once, and the difference between the two is calculated to obtain the delay time; if the task processing time is less than the preset time threshold, the advance times are increased once, and the difference between the two is calculated to obtain the advance time; if the task processing time is equal to the preset time threshold, the standard times are increased once;

[0045] Substitute into the formula Output the historical values ​​RY of the terminal device, where RW1 is the total number of tasks, RWy is the total number threshold, gd1 is a preset fixed value, TQc is the number of advance times, YCc is the number of delays, TQz is the sum of all advance times, BZz is the standard number, YCz is the sum of all delays, e1, e2, e3, e4 and e5 are all preset weight factors, and their sizes are custom settings, and e1>e2>e3>e3>e4>e5.

[0046] If the number of maintenance times is equal to zero, multiply the previous value by the preset conversion coefficient, and subtract the terminal equipment usage time multiplied by the preset conversion coefficient to obtain the historical priority value; if the number of maintenance times is greater than zero and less than the dimension threshold, multiply the previous value by the preset conversion coefficient and subtract the number of maintenance times multiplied by the preset conversion coefficient to obtain the historical priority value; if the number of maintenance times is greater than the dimension threshold, sort the maintenance moments in chronological order, calculate the difference between two adjacent maintenance moments to obtain the maintenance interval duration, calculate the average of all maintenance interval durations to obtain the average interval duration and mark it as PTa1, obtain the minimum value of the maintenance interval duration PTmin, mark the number of maintenance times as MW1, use the formula WJ=PTa1×es1+MW1×es2+10×es3 / PTmin to obtain the wiki value, multiply the previous value by the preset conversion coefficient, and subtract the wiki value multiplied by the preset conversion coefficient to obtain the historical priority value.

[0047] Through the comprehensive merit value formula: comprehensive merit value = w1×CRDIi+w2×task importance value+w3×terminal device importance value+w4×historical priority value+w5×static priority value, wherein w1, w2, w3, w4, and w5 are preset weights, and their sizes are custom settings. The comprehensive merit value corresponding to the priority of each terminal device is calculated, and all terminal devices are sorted according to the comprehensive merit value, and resources are preferentially allocated to terminal devices with high comprehensive merit values. It should be added that: for example, the terminal device M1 has the highest comprehensive merit value. By calculating the proportion of CPU usage, memory usage data, and bandwidth usage data, assuming that the CPU usage is 20%, the memory usage is 10%, and the bandwidth usage is 5%, the terminal device M1 is first allocated 20% of the total CPU resources, 10% of the total memory, and 5% of the total bandwidth.

[0048] The resource recovery module can trigger resource recovery operations in real time or periodically, and automatically reallocate released resources to terminal devices that currently have demand; specifically:

[0049] The resource recycling module has different recycling trigger mechanisms:

[0050] S1: When the utilization rate of resources (CPU, memory, bandwidth) allocated to a terminal device is lower than the set threshold (lower than 20%) for a long time, these resources will be regarded as idle resources.

[0051] S2: When a task of a terminal device is completed, the resource recovery module receives the end signal of the terminal device and recovers the occupied resources in time.

[0052] S3: When it is detected that the usage of various resources is approaching the CPU bottleneck, memory bottleneck, and bandwidth bottleneck, idle resources are actively reclaimed to free up space.

[0053] Then, according to the detection results of trigger mechanisms S1, S2 and S3, scan the underutilized resources on the terminal devices and mark them as "idle resources". These resources may include CPU, memory, bandwidth with a utilization rate below 20%, or resources occupied by completed tasks; when it is detected that the CPU, memory or bandwidth is close to a bottleneck, the resource usage of all current terminal devices will be evaluated, and those resources that have not reached the resource bottleneck but have a low utilization rate will be marked first; by comparing the importance of the terminal devices (such as high or low priority), the marked idle resources are classified, and the idle resources occupied by low-priority terminal devices are recycled first;

[0054] For terminal devices whose tasks in S2 have ended, the resource recovery module will first terminate the relevant processes and release CPU, memory and bandwidth resources. This process includes operations such as safely shutting down processes, releasing memory blocks, and closing network connections. By calling the operating system memory management interface, it releases long-unused memory blocks or caches and cleans up unnecessary memory usage so that it can be reallocated to other terminal device tasks. After the resources are released, the system will reallocate the released resources according to the priority and needs of the current terminal device.

[0055] The data storage module is used to store resource usage information as well as formulas and corresponding weight data.

[0056] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An application resource adaptive allocation management system for a concentrator, comprising a monitoring module and a resource allocation module, characterized in that: The monitoring module is used to monitor the resource usage information of the concentrator in real time and send it to the resource allocation module; The resource allocation module optimizes the allocation of computing resources, storage resources and bandwidth resources based on the resource usage information. The specific process is: calculating the memory utilization rate according to the current memory usage data, and calculating the bandwidth utilization rate according to the current network traffic; Calculate the comprehensive resource demand index of each terminal device, sort them from large to small according to the comprehensive resource demand index, and allocate computing resources, storage resources, and bandwidth resources to the corresponding terminal device in that order.

2. The application resource adaptive allocation management system for a concentrator according to claim 1, characterized in that: The resource usage information includes CPU usage data, memory usage data, and bandwidth usage data.

3. The application resource adaptive allocation management system for a concentrator according to claim 1, characterized in that: The specific process of calculating the comprehensive resource demand index of each terminal device is as follows: pass The comprehensive resource demand index CRDIi is obtained, where Ui = [Ucpui, Umemoryi, Ubandwidthi, Utraffici] is expressed as: the resource demand data vector of the i-th terminal device, W = [Wcpu, Wmemory, Wmandwidth, Wtraffic] represents the weight matrix, indicating the relative importance of different resources; b represents the bias vector; f(W×Ui+b) is a nonlinear activation function; represents the cross term, αj k is the cross-term coefficient, Uji and Uki represent the usage of the j-th and k-th resources of the ith terminal device.

4. The application resource adaptive allocation management system for a concentrator according to claim 1, characterized in that: It also includes a priority management module, which is used to allocate resources to terminal devices when resources are scarce. The specific steps are as follows: Step 1: Preliminary priority classification: including task type identification and terminal device type identification; Step 2: Obtain the CRDIi of each terminal device based on the CRDIi value, task type, terminal device importance value and historical priority value; Step 3: Substitute the preset formula to output the previous values ​​of the terminal device; Step 4: Calculate the comprehensive excellence value corresponding to the priority of each terminal device through the comprehensive excellence value formula: comprehensive excellence value = w1×CRDIi+w2×task importance value+w3×terminal device importance value+w4×historical priority value+w5×static priority value; sort all terminal devices according to the comprehensive excellence value, and give priority to allocating resources to terminal devices with high comprehensive excellence values.

5. The application resource adaptive allocation management system for a concentrator according to claim 4, characterized in that: The specific process of substituting the preset formula to output the previous values ​​of the terminal device is as follows: Obtain the task start time, task end time, total number of tasks, maintenance times and maintenance time executed by the terminal device within the preset time range in the past, calculate the time difference between the task start time and the task end time to obtain the task processing time, obtain the preset time threshold corresponding to the task, if the task processing time is greater than the preset time threshold, the number of delays is increased once, and the difference between the two is calculated to obtain the delay time; if the task processing time is less than the preset time threshold, the number of advances is increased once, and the difference between the two is calculated to obtain the advance time; if the task processing time is equal to the preset time threshold, the standard number is increased once; Substitute into the formula Output the historical values ​​RY of the terminal device, where RW1 is the total number of tasks, RWy is the total number threshold, gd1 is a preset fixed value, TQc is the number of advance times, YCc is the number of delays, TQz is the sum of all advance times, BZz is the standard number, YCz is the sum of all delays, and e1, e2, e3, e4 and e5 are all preset weight factors.

6. The application resource adaptive allocation management system for a concentrator according to claim 1, characterized in that: It also includes a resource recovery module, which triggers resource recovery operations in real time or regularly, and automatically reallocates released resources to terminal devices that currently have demand, specifically: When the resource recovery operation is triggered in real time or periodically, the released resources are automatically reallocated to the terminal devices that currently have demand; when the task of the terminal device is completed, the resource recovery module receives the end signal of the terminal device and recovers the occupied resources in time; when it is detected that the usage of various resources is close to the CPU bottleneck, memory bottleneck and bandwidth bottleneck, idle resources are actively recovered to make room.

Images