Edge resource automatic expansion and contraction system and method combining active and passive mechanisms

Abstract

The invention provides an edge resource automatic expansion and contraction system and method combining an active mechanism and a passive mechanism. The system comprises a custom collector, an edge information predictor, a management server and an automatic expansion and contraction decision maker. The custom collector collects edge system state information of the plurality of container units. The edge information predictor predicts a future state of the container unit according to the edge system state information acquired from the custom collector so as to output predicted edge information. And the management server performs centralized management on the edge system state information output by the definition collector and the predicted edge information output by the edge information predictor. The automatic scaling decision-making device determines the expected number of container units according to the predicted edge information. And the container arrangement server maintains or adjusts the number of the container units according to the expected number.

Description

Technical Field

[0001] This invention relates to edge resources, and more particularly to an automatic scaling system and method for edge resources that combines active and passive mechanisms. Background Technology

[0002] In today's microservice architecture, automatic resource scaling technology is widely used in containerized platforms. For example, the horizontal pod autoscaler (HPA) proposed by Kubernetes or K8s can be broadly divided into two categories: active and passive.

[0003] Passive methods typically adjust resource allocation in real time based on workload traffic, current system resource utilization, or their growth rate. However, they lack the ability to predict future trends and may lead to over- or under-allocation of resources when traffic suddenly changes. Proactive methods, on the other hand, make predictions based on historical data and plan resource allocation in advance. While this can improve system stability and efficiency, it may also lead to service failures or uneven resource allocation due to prediction errors, unclear resource contention, or expansion delays.

[0004] Especially in application scenarios where Artificial Intelligence (AI) microservices are deployed on edge nodes, resource scaling mechanisms become even more critical. Such environments need to simultaneously support the Quality of Service (QoS) requirements of multiple AI microservices, but also face challenges such as highly volatile traffic requests, potentially unstable node operating environments, and long startup delays often associated with resource scaling. These factors can all make it difficult to balance resource utilization, thereby affecting overall service quality and stability. Summary of the Invention

[0005] One embodiment of the present invention discloses an edge resource auto-scaling system combining active and passive mechanisms, comprising: a custom collector configured to collect edge system state information of multiple container units managed by a container orchestration server on a host; an edge information predictor configured to predict the future state of each container unit based on the edge system state information obtained from the custom collector, and output a predicted edge information; a management server configured to centrally manage the edge system state information output by the custom collector and the predicted edge information output by the edge information predictor; and an auto-scaling decision-maker configured to determine the desired number of container units based on the predicted edge information obtained from the management server and output a container orchestration instruction; wherein the container orchestration server is configured to maintain or adjust the number of container units based on the container orchestration instruction received from the auto-scaling decision-maker.

[0006] Optionally, the automatic scaling decision-maker is configured to: obtain the traffic of each container unit in historical real-world scenarios from the edge system state information as historical real-world traffic; and use artificial intelligence machine learning algorithms to learn the rules of historical real-world traffic changes over time in order to train an edge traffic prediction model.

[0007] Optionally, the edge resource automatic scaling system combining active and passive mechanisms further includes: a traffic generator configured to simulate traffic in usage scenarios other than historical real-world scenarios as simulated traffic; wherein the automatic scaling decision-maker is configured to use artificial intelligence machine learning algorithms to learn the rules of how the simulated traffic obtained from the traffic generator changes over time, in order to train an edge traffic prediction model.

[0008] Optionally, the edge information predictor is configured to: obtain the traffic requested by the user as the usage request traffic; and use an edge traffic prediction model to predict the traffic at future times based on the usage request traffic, which is included in the predicted edge information.

[0009] Optionally, the auto-scaling decision maker is configured to: obtain hardware status data and startup time of each container unit during host testing from edge system status information; and use artificial intelligence machine learning algorithms to learn the rules governing the changes in startup time of each container unit with hardware status data, in order to train a startup time prediction model.

[0010] Optionally, the edge information predictor is configured to use a startup time prediction model to predict the startup time of each container unit based on current hardware state data, wherein the startup time is included in the predicted edge information.

[0011] Optionally, the automatic scaling decision-maker includes: a first-order scaling decision-maker configured to train an edge traffic prediction model; and a second-order scaling decision-maker configured to train a startup time prediction model.

[0012] Optionally, the automatic scaling decision-maker is configured to: count the current number of multiple container units as the current number; determine whether the current number of multiple container units is less than or equal to the expected number; and when the current number is less than or equal to the expected number, output a container orchestration instruction to instruct the container orchestration server to expand the number of container units.

[0013] Optionally, the automatic scaling decision-maker is configured to: determine whether the resource utilization rate of multiple container units is greater than a first resource utilization threshold ratio when the expected number is not less than and not equal to the current number; and output container orchestration instructions to instruct the container orchestration server to expand the container units when the resource utilization rate is greater than the first resource utilization threshold ratio.

[0014] Optionally, the automatic scaling decision-maker is configured to: determine whether the resource utilization of multiple container units is less than or equal to a second resource utilization threshold ratio when the resource utilization rate is not greater than the resource utilization threshold ratio; and reduce the number of multiple container units when the resource utilization rate is less than or equal to the second resource utilization threshold ratio.

[0015] One embodiment of the present invention discloses an automatic scaling method for edge resources that combines active and passive mechanisms, comprising the following steps: collecting edge system state information of multiple container units managed by a container orchestration server from a host container orchestration server; predicting the future state of each container unit based on the edge system state information to output a predicted edge information; centrally managing the edge system state information and the predicted edge information of the edge information predictor; determining the desired number of container units based on the predicted edge information to output a container orchestration instruction; and maintaining or adjusting the number of container units based on the container orchestration instruction.

[0016] Optionally, the hybrid edge resource automatic scaling method further includes the following steps: obtaining the traffic of each container unit in a historical real-world context from the edge system state information, as a historical real-world traffic; and using artificial intelligence machine learning algorithms to learn the rules of historical real-world traffic changes over time, in order to train an edge traffic prediction model.

[0017] Optionally, the hybrid edge resource automatic scaling method further includes the following steps: simulating traffic in usage scenarios other than historical real-world scenarios as simulated traffic; and using artificial intelligence machine learning algorithms to learn the rules of how the simulated traffic changes over time to train an edge traffic prediction model.

[0018] Optionally, the hybrid edge resource automatic scaling method further includes the following steps: obtaining the traffic requested by the user as the usage request traffic; using an edge traffic prediction model, predicting future traffic based on the usage request traffic, and including it in the predicted edge information.

[0019] Optionally, the hybrid edge resource automatic scaling method further includes the following steps: obtaining hardware status data during host testing and the startup time of each container unit from the edge system status information; and using artificial intelligence machine learning algorithms to learn the rules governing the changes in startup time of each container unit with the hardware status data, so as to train a startup time prediction model.

[0020] Optionally, the hybrid edge resource auto-scaling method further includes the following steps: using a startup time prediction model to predict the startup time of each container unit based on current hardware status data, wherein the startup time is included in the predicted edge information.

[0021] Optionally, the hybrid edge resource automatic scaling method further includes the following steps: counting the current number of multiple container units as the current number; comparing the current number with the expected number to generate a container number comparison result; and maintaining or adjusting the number of container units based on the container number comparison result.

[0022] Optionally, the hybrid edge resource automatic scaling method further includes the following steps: determining whether the current number of multiple container units is less than or equal to the expected number; and when the current number is less than or equal to the expected number, increasing the number of container units.

[0023] Optionally, the hybrid edge resource automatic scaling method further includes the following steps: when the expected number is not less than and not equal to the current number, determining whether the resource utilization rate of multiple container units is greater than a first resource utilization threshold ratio; when the resource utilization rate is greater than the first resource utilization threshold ratio, expanding the container units.

[0024] Optionally, the hybrid edge resource automatic scaling method further includes the following steps: when the resource utilization rate is not greater than the resource utilization threshold ratio, determining whether the resource utilization rate of multiple container units is less than or equal to the second resource utilization threshold ratio; when the resource utilization rate is less than or equal to the second resource utilization threshold ratio, reducing the number of multiple container units.

[0025] In summary, this invention provides an edge resource auto-scaling system and method that combines active and passive mechanisms. It can predict the future state of hosts, including Pod traffic and startup time, through a custom collector and edge information predictor. A management server and auto-scaling decision-maker then generate container orchestration instructions based on this prediction information to maintain or adjust the number of Pods. Therefore, this invention simultaneously possesses the advantages of real-time response and forward-looking planning, effectively improving the Quality of Service (QoS / SLO) of edge AI microservices and enhancing the resource utilization of edge nodes, demonstrating significant practicality and industrial application value.

[0026] To further understand the features and technical content of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are for reference and illustration only and are not intended to limit the present invention. Attached Figure Description

[0027] Figure 1 The above are block diagrams of the edge resource automatic scaling system combining active and passive mechanisms according to the first to sixth embodiments of the present invention.

[0028] Figure 2 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the first embodiment of the present invention.

[0029] Figure 3 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the second embodiment of the present invention.

[0030] Figure 4 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the third embodiment of the present invention.

[0031] Figure 5 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the fourth embodiment of the present invention.

[0032] Figure 6 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the fifth embodiment of the present invention.

[0033] Figure 7 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the sixth embodiment of the present invention.

[0034] Figure 8 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the seventh embodiment of the present invention. Detailed Implementation

[0035] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of the present invention. Furthermore, the accompanying drawings of the present invention are for simple illustrative purposes only and are not depictions of actual dimensions, as stated in advance. The following embodiments will further describe the relevant technical content of the present invention in detail, but the disclosed content is not intended to limit the scope of protection of the present invention. In addition, the term "or" as used herein may, depending on the actual situation, include any or more combinations of the associated listed items.

[0036] Please see Figure 1 and Figure 2 ,in Figure 1 The above are block diagrams of the edge resource automatic scaling system combining active and passive mechanisms according to the first to sixth embodiments of the present invention. Figure 2 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the first embodiment of the present invention.

[0037] The hybrid edge resource automatic scaling system of the present invention, such as Figure 1The diagram includes a custom collector 100, an edge information predictor 200, a management server 300, and an autoscaling decision maker 400, and is suitable for performing steps S11 to S15 of the hybrid edge resource autoscaling method of the present invention.

[0038] The custom collector 100, edge information predictor 200, management server 300 and automatic scaling decision maker 400 can operate as functional modules carried by the host 1000, or they can be implemented as hardware devices, namely a custom collection circuit, an edge information prediction circuit, a management circuit, an automatic scaling decision circuit and an automatic scaling decision circuit.

[0039] The host 1000 may be a hardware platform, computing device, server, information processing device, or other electronic device. The host 1000 includes hardware and a container runtime management module 2000 running on the hardware, wherein the container runtime management module 2000 includes multiple container units 2100 and a container orchestration server 2200. The hardware of the host 1000 may include a processor such as a central processing unit (CPU), and may also include memory, a network interface, or other hardware components.

[0040] Container Pod 2100 is the execution unit of a microservice, deployed on an edge node to host the specific functions of the microservice. Each microservice is supported by at least one container Pod 2100, but to achieve high availability, load balancing, and dynamic scaling, a microservice can also be run by multiple container Pods 2100 simultaneously.

[0041] The container orchestration server 2200 is configured to dynamically manage the container units 2100. For example, the container orchestration server 2200 may be Kubernetes (K8s) or another container orchestration platform.

[0042] The container orchestration server 2200 has a variety of functions, including container deployment and scheduling, which can deploy container units 2100 to appropriate edge nodes according to workload requirements and dynamically adjust the schedule to optimize resource utilization; as well as fault tolerance and recovery mechanisms, which can automatically restart the container or migrate it to a healthy edge node when the container unit 2100 is abnormal or fails, to ensure service availability.

[0043] It is worth noting that the container orchestration server 2200 can integrate an automatic horizontal scaling (Horizontal PodAutoscaler) mechanism to support automatic scaling based on metrics or events. The hybrid edge resource autoscaling system of this invention provides further optimized resource monitoring and deployment capabilities, continuously monitoring the usage of each container unit 2100, predicting the future state of the container unit 2100, and updating resource allocation with a dynamic update strategy. The specific technical steps for implementing this function are described below.

[0044] Custom collector 100 is configured to collect edge system state information (such as...) from the container orchestration server 2200 hosted by host 1000, which manages multiple container units 2100. Figure 2 Step S11). In detail, the container orchestration server 2200 can passively or actively obtain information from multiple container units 2100 as edge system state information and provide it to the custom collector 100.

[0045] Edge information predictor 200 is configured to predict the future state of multiple container units 2100 based on edge system state information obtained from custom collector 100, to output predicted edge information (such as...). Figure 2 Step S12). The edge system state information may include historical information, real-time information, or a combination thereof, and the predicted future state is future information. This historical, real-time, and future information may include the hardware startup time of each container unit 2100 of the host 1000, the frequency and amount of read and write operations of each container unit 2100 to the storage device, the container event log of each container unit 2100, the workload and traffic, the resource utilization and activity level (i.e., busy level and idle level) of each container unit 2100 and the hardware carrying it, or any combination thereof.

[0046] Management server 300 is configured to centrally manage the edge system status information output by custom collector 100 and the predicted edge information output by edge information predictor 200 (e.g., Figure 2 (Step S13). In the overall architecture, in addition to playing the role of a data intermediary, the management server 300 also has authorization and coordination functions. It can aggregate relevant information from the custom collector 100, the edge information predictor 200, and the automatic scaling decision-maker 400, and transmit it to the automatic scaling decision-maker 400 to assist its decision-making.

[0047] The auto-scaling decision maker 400 is configured to determine the desired number of multiple container units 2100 for microservices based on predicted edge information obtained from the management server 300, and outputs a container orchestration instruction (such as...) based on this desired number. Figure 2 Step S14).

[0048] Container orchestration server 2200 is configured to maintain or adjust the number of container units 2100 (e.g., based on container orchestration instructions received from autoscaling decision 400) Figure 2 (Step S15). When the container orchestration server 2200 determines that the current number of multiple container units 2100 has reached the above-mentioned expected number, the container orchestration server 2200 maintains the number of container units 2100 and does not expand or reduce it.

[0049] Specifically, when the container orchestration server 2200 determines that the current number of multiple container units 2100 is less than (or equal to) the aforementioned expected number, the container orchestration server 2200 increases the number of container units 2100 to the expected number. Conversely, when the container orchestration server 2200 determines that the current number of multiple container units 2100 exceeds the expected number, the container orchestration server 2200 reduces the number of container units 2100 to the expected number.

[0050] It is worth noting that the key innovation of this invention lies in the fact that its hybrid edge resource auto-scaling system can predict the future demand of container units 2100 and thereby proactively perform auto-scaling. By combining real-time monitoring with forward-looking prediction, the system can adjust resources before traffic changes occur, achieving accurate and stable resource management.

[0051] To address the demands for service stability, performance, and energy efficiency in edge computing environments, this invention provides a hybrid edge resource auto-scaling system. This system can predict the resource requirements of container units 2100 and dynamically deploy them in advance. When the number of container units 2100 required by a microservice is insufficient, the system can automatically expand the number of container units in advance, reducing service latency, avoiding resource shortages caused by sudden surges in traffic, and ensuring sufficient computing resources during peak periods. Conversely, when the number of container units 2100 is excessive, the system can reduce redundant container units in real time, allocating resources to other applications, reducing computing and energy costs, and improving overall efficiency. Thus, this invention's hybrid edge resource auto-scaling system achieves flexible resource allocation and efficient utilization, ensuring stable operation of edge services under different loads, while effectively reducing costs and avoiding resource waste.

[0052] Please see Figure 1 and Figure 3 ,in Figure 3This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the second embodiment of the present invention.

[0053] The hybrid edge resource automatic scaling system of the present invention, such as Figure 1 In addition to the custom collector 100, edge information predictor 200, management server 300, and autoscaling decision maker 400, the system may also include a traffic generator 500, an information broker 600, or both, if needed. In the second embodiment, the autoscaling decision maker 400 may include a first-order scaling decision maker 410 and a second-order scaling decision maker 420.

[0054] The hybrid edge resource automatic scaling method of the present invention may also include, for example: Figure 3 The steps S211 to S213 (real traffic in historical scenarios), S221 to S223 (simulated virtual traffic), or combinations thereof, are shown, and step S23 (training the model) is also included. Steps S211 to S213, S221 to S223, and S23 can be derived from... Figure 1 The hybrid edge resource auto-scaling system is executed after step S11 and before step S12.

[0055] As described above, the custom collector 100 collects edge system state information (such as) from the container orchestration server 2200 from multiple container units 2100. Figure 2 and Figure 3 After step S11), the first-stage scaling decision 410 is configured to obtain edge system status information from the management server 300. This edge system status information may include the real traffic of each of the multiple container units 2100 in a historical real-world scenario, the virtual traffic in a virtual scenario generated in a simulated usage scenario, or a combination thereof.

[0056] In this embodiment, the traffic data used for training the model is divided into real traffic and virtual traffic. The two play different roles in model training and resource management and complement each other. Real traffic comes from user requests at past time edge nodes and can reflect the load characteristics of the system in a real environment, including user behavior patterns, periodic fluctuations and sudden events. It is mainly used to verify the prediction accuracy of the model in real scenarios and to provide practical experience for reinforcement learning decision models, so that the autoscaling strategy can cope with real loads.

[0057] Virtual traffic, on the other hand, generates and simulates various user requests using historical data, providing a large amount of diverse time-series data, including peaks, troughs, and sudden events. It also supplements the lack of rare or extreme scenarios in real traffic, improves the model's ability to generalize to unknown loads, and enables the startup time prediction model to learn the startup behavior of Pods under different system states.

[0058] In summary, historical traffic in real-world scenarios provides a reference for the real environment, while virtual traffic provides diverse training scenarios. When there are sufficient samples in real-world scenarios, virtual traffic can be omitted for model training. However, when there are insufficient samples in real-world scenarios, virtual traffic and historical traffic can be combined to train the model. This allows the model to accurately reflect real-world user behavior and cope with extreme or rare situations, thereby supporting proactive resource scaling and preparing resources in advance to maintain system performance.

[0059] In the process of using historical actual traffic to train the model, the first-order scaling decision maker 410 is configured to obtain the traffic of each of the multiple container units 2100 in the historical actual context from the edge system state information, or further sum the traffic of the multiple container units 2100 to obtain a total traffic as the historical actual traffic (e.g., Figure 3 Step S211). Next, the first-order scaling decision unit 410 is configured to use artificial intelligence (AI) machine learning algorithms (such as... Figure 3 Step S212) involves performing time series analysis on historical traffic data to learn the rules governing the changes in historical actual traffic over time (e.g., Figure 3 Step S213) or pattern and features, to train an edge traffic prediction model (such as Figure 3 Step S23).

[0060] In the process of using virtual traffic to train the model, the traffic generator 500 is configured to simulate traffic from usage scenarios other than historical real-world scenarios (e.g., ...). Figure 3 Step S221) can be transmitted to the container orchestration server 2200 via the information broker 600. The container orchestration server 2200 controls multiple container units 2100 to carry traffic equivalent to the generated simulated traffic. Next, the first-order scaling decision maker 410 is configured to use machine learning algorithms of artificial intelligence (such as...) Figure 3 Step S222) analyzes the simulated traffic obtained from the traffic generator 500 and learns the rules governing the changes in the simulated traffic obtained from the traffic generator 500 over time (e.g., Figure 3 Step S223) or pattern and features are used to train the above-mentioned edge traffic prediction model (e.g. Figure 3 Step S23) is used to further optimize the edge traffic prediction model.

[0061] For example, the traffic generator 500 can generate the simulated traffic described above based on a publicly available workload dataset. This workload dataset could include HTTP request workload datasets published by Wikimedia, recording the number of user requests for each project website at specific time intervals (e.g., per hour). To make the simulated traffic more realistic, the traffic generator 500 can preprocess the workload dataset, including using interpolation or other time-series refinement methods, and further scaling or normalizing the user request volume to a range suitable for simulation, so as to dynamically control the numerical scale according to simulation needs. The processed simulated traffic is then driven by multiple processors to simulate multiple users simultaneously issuing service requests, and continuously generating requests at a specified time granularity (e.g., per second), thereby reflecting the load changes that microservices may encounter in a real operating environment.

[0062] The management server 300 can centrally manage the edge traffic prediction model output by the first-order scaling decision maker 410. The edge information predictor 200 can predict the future state of the host 1000 based on the edge traffic prediction model obtained from the management server 300, including the future state of multiple container units 2100 (and hardware) of the host 1000, to output predicted edge information (such as...). Figure 2 Step S12).

[0063] Please see Figure 1 and Figure 4 ,in Figure 4 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the third embodiment of the present invention.

[0064] The hybrid edge resource automatic scaling method of the present invention may also include, for example: Figure 4 Steps S31 to S33 shown can be performed by Figure 1 The hybrid edge resource autoscaling system executes after step S23 described above. After training an edge traffic prediction model in test mode (steps S11 to S15, S211 to S213, S221 to S223, and S23), it can enter the actual use mode to use the edge traffic prediction model to predict whether the number of container units 2100 needs to be adjusted (steps S31 to S33). In the third embodiment, the edge information predictor 200 of the hybrid edge resource autoscaling system can be as follows: Figure 1 Includes a flow predictor 210.

[0065] Traffic predictor 210 is configured to obtain edge traffic prediction models (such as those from management server 300) Figure 4 Step S31), and obtaining the traffic requested by the user as the usage request traffic (e.g. Figure 4(Step S32). For example, the traffic predictor 210 receives user input or usage request traffic transmitted by a user device through a user interface (UI) 3000.

[0066] Traffic predictor 210 is configured to use an edge traffic prediction model to predict traffic for each time unit within any future period or time interval, based on usage request traffic, as a predicted future traffic (e.g., Figure 4 Step S33), included in the aforementioned predicted edge information, is output to the management server 300 for centralized management.

[0067] The second-stage scaling decision-maker 420 is configured to determine the desired number of multiple container units 2100 for microservices based on a predicted future traffic obtained from the management server 300, and outputs a container orchestration instruction to the container orchestration server 2200 based on this desired number (e.g., ...). Figure 2 Step S14) instructs the container orchestration server 2200 to maintain, reduce, or expand the number of container units 2100.

[0068] Please see Figure 1 and Figure 5 ,in Figure 5 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the fourth embodiment of the present invention.

[0069] The hybrid edge resource automatic scaling method of the present invention may include, for example: Figure 5 Steps S41 to S44 shown can be performed by Figure 1 The hybrid edge resource autoscaling system is executed after step S11 and before step S12. In the fourth embodiment, the edge information predictor 200 of the hybrid edge resource autoscaling system may include a startup time predictor 220.

[0070] In practice, the hybrid edge resource automatic scaling method of this invention can be trained. Figure 3 An edge traffic prediction model and Figure 5 Both startup time prediction models. In this case, in the third embodiment, the edge information predictor 200 of the hybrid edge resource autoscaling system may include a traffic predictor 210 and a startup time predictor 220.

[0071] As described above, the custom collector 100 collects edge system state information (such as) from the container orchestration server 2200 from multiple container units 2100. Figure 2 and Figure 3Following step S11), the first-stage scaling decision maker 410 is configured to obtain edge system status information from the management server 300, including data on the hardware of the host 1000 during testing in test mode as hardware status data, and the startup time of each of the multiple container units 2100 (e.g., ...). Figure 5 Step S41).

[0072] The first-order scaling decision 410 is configured to use machine learning algorithms of artificial intelligence (such as...) Figure 5 Step S42) analyzes the correlation between the startup time and hardware state data of multiple container units 2100, including the rules governing the change of startup time of each of the multiple container units 2100 with hardware state data (e.g., Figure 5 Step S43), to train a startup time prediction model (e.g. Figure 5 Step S44), output to the management server 300 for centralized management.

[0073] After training the startup time prediction model, the startup time predictor 220 can use the startup time prediction model obtained from the management server 300 to predict the startup time of each container unit 2100 based on the current / current hardware state data within a future timeframe (included in the aforementioned prediction edge information), and output a prediction edge information (such as...). Figure 2 Step S12).

[0074] To accurately assess the computing power of edge nodes, the first-order scaling decision unit 410 can be configured to integrate CPU utilization (i.e., raw utilization), dynamic CPU frequency, and the number of CPU cores to form adaptive CPU utilization as an indicator, thus more realistically reflecting the overall computing performance of edge nodes. The formula for calculating adaptive CPU utilization is: Rcpt = (Rcpu × Rfreq × ncore) / 1000, Where Rcpt represents CPU utilization (i.e., raw utilization), Rcpu represents dynamic CPU frequency, Rfreq represents dynamic CPU frequency, and ncore represents the number of CPU cores.

[0075] It should be understood that CPU utilization refers to the proportion of time the CPU on the edge node actually executes Pod 2100 or other system tasks during the statistical period, that is, the percentage of time the CPU is running. It is used to measure the busyness of the CPU on the edge node, and the value range can be 0% to 100%, where 0% can mean that the CPU is completely idle and 100% can mean that the CPU is running continuously.

[0076] Dynamic CPU frequency refers to the ability of the CPU on the edge node to automatically adjust its operating frequency based on the current workload of the running Pod 2100, the CPU's own temperature, or power-saving requirements. The number of CPU cores refers to the number of independent computing units (cores) contained in the CPU on the edge node. Each core can independently execute Pod 2100 or other tasks, so a multi-core CPU can handle more Pod 2100 tasks simultaneously, improving the overall computing power of the edge node. Through this integrated metric of adaptive CPU utilization, the startup time predictor can more accurately estimate the startup time of Pod 2100, thereby assisting in the dynamic adjustment and automatic scaling decisions of edge resources.

[0077] Please see Figure 1 and Figure 6 ,in Figure 6 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the fifth embodiment of the present invention.

[0078] The hybrid edge resource automatic scaling method of the present invention may also include, for example: Figure 6 Steps S51 to S54 shown can be performed by Figure 1 The hybrid edge resource autoscaling system executes after step S44 above. The startup time prediction model is trained in test mode. Figure 3 and Figure 6 After step S23), the startup time prediction model can be used to manage the number of container units 2100 in the actual use mode (e.g., ...). Figure 6 Steps S31 to S33 are explained in detail below.

[0079] Startup time predictor 220 is configured to obtain a startup time prediction model (e.g., from management server 300) Figure 6 Step S51), and hardware collection hardware status data (such as custom collector sensing or other sensor data) of the host 1000 obtained from the custom collector. Figure 6 Step S52).

[0080] Startup time predictor 220 is configured to use a startup time prediction model to predict the startup time of each or a subset of multiple container units 2100 in the future, based on current hardware state data. Figure 6 Step S53), as a prediction of the future container startup time, is included in the above-mentioned prediction edge information and output to the management server 300 for centralized management.

[0081] After the startup time predictor 220 predicts the future startup time of each container unit 2100, the traffic predictor 210 can be configured to use an edge traffic prediction model to predict the future traffic of each container unit 2100 after that future startup time, based on usage request traffic (e.g., ...). Figure 6 Step S54), included in the aforementioned predicted edge information, is output to the management server 300 for centralized management.

[0082] The second-order scaling decision 420 is configured to rely on the aforementioned predicted edge information obtained from the management server 300 (such as...). Figure 2 Step S13) determines the number of multiple container units 2100 used for microservices as the desired number, and outputs a container orchestration instruction to the container orchestration server 2200 based on this desired number (e.g., ...). Figure 2 Step S14) instructs the container orchestration server 2200 to maintain, reduce, or expand the number of container units 2100.

[0083] The second-stage scaling decision 420 can also calculate the time required for traffic to reach a bottleneck under the existing resource allocation based on the predicted future traffic. The bottleneck may include the upper limit of the service processing capacity supported by the container unit 2100, the available resources of the underlying hardware platform, or a combination thereof.

[0084] For example, the first-order scaling decision 410 can be implemented by executing a partially observable Markov Decision Process (POMDP), thereby enabling the first-order scaling decision 410 to learn the rules of historical actual traffic changes over time to train an edge traffic prediction model and to learn the rules of startup time changes of multiple container units 2100 with hardware state data to train a startup time prediction model, with only partial observation information. This improves the overall perception and prediction accuracy of the hidden state of the system and supports more accurate resource allocation and automatic scaling.

[0085] Finally, the second-stage scaling decision-maker 420 compares the predicted startup time of container unit 2100 with the calculated traffic bottleneck time to determine whether to add container units 2100 in advance for resource expansion. When the startup time of container unit 2100 is less than the bottleneck time, the second-stage scaling decision-maker 420 outputs container orchestration instructions to instruct the container orchestration server 2200 to expand container units 2100 in a timely manner to avoid resource shortage. If the startup time of container unit 2100 is greater than the bottleneck time, the container orchestration server 2200 is instructed to activate the expansion operation of container unit 2100 in advance to ensure that the expansion is completed before the traffic bottleneck is reached, so as to maintain service performance and stability.

[0086] Based on the above, the hybrid edge resource auto-scaling method of the present invention adopts the MAPE-K (Monitor-Analyze-Plan-Execute-Knowledge) control loop. It continuously monitors the hardware and container unit operating status and traffic information of the host, analyzes it to understand resource usage trends and potential load changes, and plans the optimal resource scheduling and scaling strategy based on the analysis results. Then, it executes the plan to automatically increase or decrease the number of Pods 2100 used for microservices. At the same time, it stores historical operating data, simulated traffic, and decision results as knowledge to support subsequent strategy optimization and learning. In this way, it combines active and passive scaling mechanisms to improve system resource utilization efficiency and elastic scaling capabilities.

[0087] In the hybrid edge resource automatic scaling method of the present invention, the edge system state information collected by the custom collector 100 can cover precise data on software platforms, hardware platforms, or combinations thereof. For example, software platform information may include historical traffic and container unit startup times; hardware platform information may include the usage and utilization rate of CPU, storage devices, or other hardware resources. The information collected in this way can support subsequent resource analysis and planning, and through repeated training, converge one or more optimized edge resource prediction models, such as... Figure 3 and Figure 4 The edge traffic prediction model shown Figure 5 and Figure 6 The startup time prediction models shown, or combinations thereof, can be used to predict the actual hardware and software resource usage during future runtime, thereby enabling precise and proactive dynamic resource deployment of Pods.

[0088] In implementing the aforementioned dynamic resource deployment, the hybrid edge resource auto-scaling method of this invention does not rely on third-party plugins for fine-grained decision-making. Instead, it directly analyzes and retrieves information through interfaces available at the system level, such as, but not limited to, Cgroups and Kubernetes APIs, to achieve more precise perception and control. These interfaces can be divided into two levels: low-level interfaces are mainly used for hardware resource control, such as monitoring and limiting the use of CPU, memory, and disk I / O, with Cgroups belonging to this category; high-level interfaces are used for container operation management, such as the lifecycle, scheduling, and auto-scaling of Pod 2100, with the Kubernetes API belonging to this category.

[0089] The aforementioned automatic scaling decisions can be categorized into low-order metrics and high-order metrics. Low-order metrics primarily reflect the utilization of hardware resources (e.g., CPU utilization, memory usage, disk I / O, network traffic), while high-order metrics focus on the performance of the application or service layer (e.g., Pod startup time, request latency, throughput, error rate). This invention focuses on using high-order functions as the primary monitoring target, while also utilizing low-order features to predict microservice startup time. The hybrid edge resource automatic scaling method of this invention can simultaneously obtain low-order hardware resource information and high-order application layer status, enabling accurate monitoring of system resources and improving the accuracy and real-time performance of automatic scaling decisions without relying on external tools.

[0090] In other words, this invention optimizes the edge resource management mechanism by integrating dynamic CPU frequency and CPU utilization into the monitoring of the computing performance of edge node CPU resources, and by combining proactive and passive hybrid autoscaling strategies, taking into account the startup time of AI microservices, observing and modeling unknown application resource competition, and predicting dynamic changes in expansion time.

[0091] Please see Figure 1 and Figure 7 ,in Figure 7 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the sixth embodiment of the present invention.

[0092] The hybrid edge resource automatic scaling method of the present invention may also include, for example: Figure 7 Steps S61 to S65 shown can be performed by Figure 1 The hybrid edge resource autoscaling system executes after step S14 above. After determining the desired number of container units 2100 as described above to output a corresponding container orchestration instruction (such as... Figure 1 After step S14), the number of container units 2100 is adjusted (e.g., reduced or expanded) according to container orchestration instructions. Figure 1 Step S15). This step S15 may include Figure 7 Steps S61 to S65.

[0093] The second-order scaling decision-maker 420 is configured to count the number of multiple container units 2100 currently used for microservices, as a current number (e.g., ...). Figure 7 Step S61).

[0094] The second-order scaling decision 420 is configured to compare the current quantity counted in step S61 with the expected quantity determined in step S14 (e.g., Figure 7Step S62) generates a container quantity comparison result. The second-stage scaling decision-maker 420 is configured to determine, based on this container quantity comparison result, whether to adjust the number of container units 2100 used for microservices (e.g., ...). Figure 7 Step S63) to output a corresponding container orchestration instruction to the container orchestration server 2200.

[0095] If the second-order scaling decision 420 decides not to adjust the number of container units 2100, the container orchestration server 2200 maintains the current number of container units 2100 for the microservices (e.g., ...). Figure 7 Step S64) enables the container orchestration server 2200 to maintain the existing microservice execution environment.

[0096] If the second-order scaling decision 420 decides to adjust the number of container units 2100, the container orchestration server 2200 reduces the number of container units 2100 used for microservices to avoid resource waste under low load, or expands more container units 2100 to support microservice operation, so that the system still has sufficient computing resources and service capabilities under high load (e.g., Figure 7 (Step S65). In this way, resource utilization and computing efficiency can be dynamically improved, and the system can be ensured to have sufficient flexibility to cope with different load requirements.

[0097] In practice, in the hybrid edge resource automatic scaling method of the present invention, the above-mentioned adjustment of the number of container units 2100 may further include the following steps: the second-stage scaling decision 420 is configured to calculate the difference between the current number and the expected number of multiple container units 2100, for example, subtracting the current number from the expected number; the second-stage scaling decision 420 outputs a container orchestration instruction containing the difference and quantity modulation information (e.g., expansion instruction information or reduction instruction information); when the container orchestration server 2200 receives the container expansion instruction information from the second-stage scaling decision 420, it expands the number of container units 2100 and the expanded number is equal to the difference; when it receives the container reduction instruction information, it reduces the number of container units 2100 and the reduced number is equal to the difference.

[0098] Please see Figure 1 and Figure 8 ,in Figure 8 This is a flowchart illustrating the steps of the automatic scaling method for edge resources that combines active and passive mechanisms according to the seventh embodiment of the present invention.

[0099] The hybrid edge resource automatic scaling method of the present invention may also include, for example: Figure 8 Steps S71 to S75 shown can be performed by Figure 1 The hybrid edge resource autoscaling system performs in Figure 7 After step S61. Figure 7Step S61 may include Figure 8 Steps S71 to S75.

[0100] The second-order scaling decision 420 is configured to determine whether the current number of container cells 2100 counted in step S61 is less than or equal to the expected number determined in step S14 (e.g., ...). Figure 8 Step S71).

[0101] If the second-order scaling decision 420 determines that the current number of container units 2100 is less than or equal to the expected number, the container orchestration server 2200 expands the number of container units 2100 in the manner described above, and the expanded number is equal to the difference between the current number and the expected number.

[0102] Conversely, if the second-order scaling decision-maker 420 determines that the current number of container units 2100 is not less than and not equal to the expected number, the second-order scaling decision-maker 420 further determines whether the resource utilization rate of the multiple container units 2100 is greater than the first resource utilization threshold ratio (e.g., Figure 8 Step S72).

[0103] If the second-order scaling decision maker 420 determines that the resource utilization rate of multiple container units 2100 is greater than the first resource utilization threshold ratio, the container orchestration server 2200 expands the number of container units 2100 to the desired number in the manner described above (e.g., Figure 8 Step S74).

[0104] Conversely, if the second-order scaling decision maker 420 determines that the resource utilization of multiple container units 2100 is not greater than the first resource utilization threshold ratio, the container orchestration server 2200 determines whether the resource utilization of the multiple container units 2100 is less than the second resource utilization threshold ratio (e.g., ...). Figure 8 (Step S75). For example, the first resource usage threshold ratio mentioned above can be 80%, and the second resource usage threshold ratio can be 20%. In practice, adjustments can be made according to actual needs.

[0105] If the container orchestration server 2200 determines that the resource utilization of multiple container units 2100 is less than the second resource utilization threshold ratio, the container orchestration server 2200 reduces / reduces the number of container units 2100 to the desired number in the manner described above (e.g., Figure 8 Step S76).

[0106] Conversely, if the container orchestration server 2200 determines that the resource utilization of multiple container units 2100 is not less than the second resource utilization threshold ratio, the container orchestration server 2200 maintains the number of container units 2100 and does not expand or shrink them.

[0107] In summary, this invention provides an edge resource auto-scaling system and method that combines active and passive mechanisms. It can predict the future state of hosts, including Pod traffic and startup time, through a custom collector and edge information predictor. A management server and auto-scaling decision-maker then generate container orchestration instructions based on this prediction information to maintain or adjust the number of Pods. Therefore, this invention simultaneously possesses the advantages of real-time response and forward-looking planning, effectively improving the Quality of Service (QoS / SLO) of edge AI microservices and enhancing the resource utilization of edge nodes, demonstrating significant practicality and industrial application value.

Claims

1. An automatic scaling system for edge resources that combines active and passive mechanisms, characterized in that, The edge resource auto-scaling system that combines active and passive mechanisms includes: A custom collector, configured to collect edge system state information from a host's container orchestration server, encompassing multiple container units managed by that container orchestration server; An edge information predictor is configured to predict the future state of each of the container units based on the edge system state information obtained from the custom collector, so as to output a predicted edge information. The management server is configured to centrally manage the edge system state information output by the defined collector and the predicted edge information output by the edge information predictor; and An automatic scaling decision-maker is configured to determine the desired number of container units based on the predicted edge information obtained from the management server, and output a container orchestration instruction. The container orchestration server is configured to maintain or adjust the number of container units based on the container orchestration instructions received from the autoscaling decision-maker.

2. The edge resource automatic scaling system combining active and passive mechanisms according to claim 1, characterized in that, The automatic scaling decision-maker is configured to: The flow rate of each container unit in the historical actual situation is obtained from the edge system state information and used as a historical actual flow rate. as well as Using artificial intelligence machine learning algorithms, a model for predicting edge traffic is trained to learn the rules governing how historical actual traffic changes over time.

3. The edge resource automatic scaling system combining active and passive mechanisms according to claim 2, characterized in that, The aforementioned automatic scaling system for edge resources, combining active and passive mechanisms, also includes: A traffic generator is configured to simulate traffic in usage scenarios other than the historical actual scenarios as a simulated traffic; The automatic scaling decision-maker is configured to use artificial intelligence machine learning algorithms to learn rules governing the changes in simulated traffic over time obtained from the traffic generator, in order to train the edge traffic prediction model.

4. The edge resource automatic scaling system combining active and passive mechanisms according to claim 2 or 3, characterized in that, The edge information predictor is configured to: Obtain the traffic requested by the user as the usage request traffic; and Using the edge traffic prediction model, based on the usage request traffic, the traffic at future times is predicted and included in the predicted edge information.

5. The edge resource automatic scaling system combining active and passive mechanisms according to claim 2, characterized in that, The automatic scaling decision-maker is configured to: From the edge system status information, obtain the hardware status data of the host during testing and the startup time of each container unit; as well as Using artificial intelligence machine learning algorithms, a startup time prediction model is trained to learn the rules governing the changes in the startup time of each container unit as a function of the hardware state data.

6. The edge resource automatic scaling system combining active and passive mechanisms according to claim 5, characterized in that, The edge information predictor is configured to use the startup time prediction model to predict the startup time of each of the container units based on the current hardware state data, wherein the startup time is included in the predicted edge information.

7. The edge resource automatic scaling system combining active and passive mechanisms according to claim 4, characterized in that, The automatic scaling decision-maker is configured to: A first-order scaling decision unit, configured to train the edge traffic prediction model; and The second-order scaling decision maker is configured to train the startup time prediction model.

8. The edge resource automatic scaling system combining active and passive mechanisms according to claim 1, characterized in that, The automatic scaling decision-maker is configured to: Count the current quantity of the multiple container units as a current quantity; Determine whether the current number of the plurality of container units is less than or equal to the expected number; and When the current quantity is less than or equal to the expected quantity, the container orchestration instruction is output to instruct the container orchestration server to increase the number of container units.

9. The edge resource automatic scaling system combining active and passive mechanisms according to claim 8, characterized in that, The automatic scaling decision-maker is configured to: When the expected quantity is not less than and not equal to the current quantity, determine whether the resource utilization rate of the multiple container units is greater than a first resource utilization threshold ratio; and When the resource utilization rate is greater than the first resource utilization threshold ratio, the container orchestration instruction is output to instruct the container orchestration server to expand the container unit.

10. The edge resource automatic scaling system combining active and passive mechanisms according to claim 9, characterized in that, The automatic scaling decision-maker is configured to: When the resource utilization rate is not greater than the resource utilization threshold ratio, determine whether the resource utilization rate of the plurality of container units is less than or equal to a second resource utilization threshold ratio; and When the resource utilization rate is less than or equal to the second resource utilization threshold ratio, the number of multiple container units is reduced.

11. A method for automatic scaling of edge resources combining active and passive mechanisms, characterized in that, The automatic scaling method for edge resources that combines active and passive mechanisms includes the following steps: Collect edge system status information from the host's container orchestration server, which manages multiple container units; Based on the edge system state information, predict the future state of each container unit to output predicted edge information; The edge system state information and the predicted edge information from the edge information predictor are centrally managed. Based on the predicted edge information, the desired number of container units is determined to output a container orchestration instruction; and The number of container units is maintained or adjusted according to the container arrangement instructions.

12. The automatic scaling method for edge resources combining active and passive mechanisms according to claim 11, characterized in that, The automatic scaling method for edge resources also includes the following steps: The flow rate of each container unit in the historical actual situation is obtained from the edge system state information and used as a historical actual flow rate. as well as Using artificial intelligence machine learning algorithms, a model for predicting edge traffic is trained to learn the rules governing how historical actual traffic changes over time.

13. The automatic scaling method for edge resources combining active and passive mechanisms according to claim 12, characterized in that, The automatic scaling method for edge resources also includes the following steps: Traffic in usage scenarios other than the historical actual scenarios is used as a simulated traffic; The edge traffic prediction model is trained by using machine learning algorithms based on artificial intelligence to learn the rules governing how the simulated traffic changes over time.

14. The automatic scaling method for edge resources combining active and passive mechanisms according to claim 12 or 13, characterized in that, The automatic scaling method for edge resources also includes the following steps: Obtain the traffic requested by the user as a usage request traffic; and Using the edge traffic prediction model, future traffic is predicted based on the usage request traffic and included in the predicted edge information.

15. The automatic scaling method for edge resources combining active and passive mechanisms according to claim 14, characterized in that, The automatic scaling method for edge resources also includes the following steps: From the edge system status information, obtain the hardware status data of the host during testing and the startup time of each container unit; as well as Using artificial intelligence machine learning algorithms, a startup time prediction model is trained to learn the rules governing the changes in the startup time of each container unit as a function of the hardware state data.

16. The automatic scaling method for edge resources combining active and passive mechanisms according to claim 15, characterized in that, The automatic scaling method for edge resources also includes the following steps: Using the startup time prediction model, the startup time of each container unit is predicted based on the current hardware status data; Using the edge traffic prediction model, based on the usage request traffic, the traffic after the predicted startup time is predicted and included in the predicted edge information.

17. The automatic scaling method for edge resources combining active and passive mechanisms according to claim 11, characterized in that, The automatic scaling method for edge resources also includes the following steps: Count the current number of the multiple container units, and use this as the current number; The current quantity is compared with the expected quantity to generate a container quantity comparison result; and Based on the comparison results of the container quantity, the quantity of the container units is maintained or adjusted.

18. The automatic scaling method for edge resources combining active and passive mechanisms according to claim 17, characterized in that, The automatic scaling method for edge resources also includes the following steps: Determine whether the current number of the plurality of container units is less than or equal to the expected number; and When the current quantity is less than or equal to the expected quantity, the number of container units is increased.

19. The automatic scaling method for edge resources combining active and passive mechanisms according to claim 18, characterized in that, The automatic scaling method for edge resources also includes the following steps: When the expected quantity is not less than and not equal to the current quantity, determine whether the resource utilization rate of the multiple container units is greater than a first resource utilization threshold ratio; and When the resource utilization rate is greater than the first resource utilization threshold ratio, the container unit is expanded.

20. The automatic scaling method for edge resources combining active and passive mechanisms according to claim 19, characterized in that, The automatic scaling method for edge resources also includes the following steps: When the resource utilization rate is not greater than the resource utilization threshold ratio, determine whether the resource utilization rate of the plurality of container units is less than or equal to the second resource utilization threshold ratio; as well as When the resource utilization rate is less than or equal to the second resource utilization threshold ratio, the number of multiple container units is reduced.

Images