Iot data access microservice elasticity scaling and load balancing method and system

Abstract

This invention discloses a method for elastic scaling and load balancing of IoT data access microservices in the field of IoT data processing technology, including steps such as monitoring data collection, health metric assessment, intelligent traffic scheduling, and macro-resource elastic scaling. This invention achieves accurate, efficient, and adaptive management of IoT data access microservice clusters by constructing a multi-dimensional dynamic health scoring model, an intelligent traffic scheduling mechanism, a fault self-healing state machine, and elastic scaling decisions based on time series prediction.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) data processing technology, specifically to a method for elastic scaling and load balancing of IoT data access microservices. Background Technology

[0002] The widespread adoption of IoT technology in key areas such as smart grids, industrial control, and smart cities has led to a continuous stream of high-concurrency real-time data from massive numbers of terminal devices, making microservice architecture the mainstream choice for IoT data access platforms. By breaking down monolithic applications into multiple fine-grained independent services, microservice architecture offers significant advantages in maintainability, scalability, and technological diversity, effectively addressing the challenges of diverse data types, complex protocols, and high concurrency in IoT scenarios.

[0003] However, existing technologies face several technical challenges when dealing with complex production environments:

[0004] Firstly, regarding load balancing, current load balancing mechanisms primarily rely on basic resource metrics such as CPU utilization and memory usage for traffic allocation. They fail to perceive the actual business processing status within microservice instances, such as message queue backlog, database access latency, and abnormal responses from downstream dependent services. This deficiency leads to situations where, under high load, a microservice instance, despite low CPU utilization, may be assigned a large number of new requests even if its message backlog is nearing saturation. This can cause localized failures to spread, ultimately impacting overall system stability.

[0005] Secondly, regarding elastic scaling, traditional elastic scaling solutions are mostly reactive, triggering expansion or contraction operations based on historical resource usage thresholds, resulting in significant decision-making lag. From the detection of threshold triggering to the full startup of the new microservice instance, completion of service registration, and entry into normal working state, there is a certain time window. During this period, the system's effective service capacity may not be able to meet the continuously influx of real-time data processing demands, especially struggling to adapt to the unique periodic fluctuations and sudden peak patterns of IoT data, easily leading to a decline in service quality or even request backlog.

[0006] Therefore, existing technologies urgently need an intelligent microservice elastic scaling and load balancing method that can sense the actual business processing status inside the service and proactively perform predictive resource scheduling. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention provides a method for elastic scaling and load balancing of IoT data access microservices.

[0008] The technical solution of this invention is as follows:

[0009] A method for elastic scaling and load balancing of microservices for IoT data access, characterized by the following steps:

[0010] S1. Monitoring data collection: A monitoring agent is embedded in each microservice instance of the IoT data access platform. The monitoring agent continuously collects multi-dimensional indicator data of each microservice instance and centralizes the multi-dimensional indicator data into a unified time-series database.

[0011] S2. Health Measurement and Assessment: Construct a health scoring model, use the health scoring model to calculate the comprehensive health score of each monitoring dimension in the time series database, and then obtain the real-time health status label of each microservice instance;

[0012] S3, Intelligent Traffic Scheduling: Construct a load balancer module, which maintains a dynamic routing table containing all available microservice instances and their overall health scores. When receiving data reporting requests from IoT devices, it executes a multi-stage selection strategy, first filtering out microservice instances with overall health scores below the threshold, and then distributing traffic among the remaining healthy microservice instances.

[0013] S4. Macro-level resource elastic scaling: Elastic scaling is performed on a microservice cluster consisting of multiple microservice instances as the unit of analysis, expanding or shrinking the capacity based on different needs.

[0014] According to the present invention based on the above scheme, the multi-dimensional indicator data in step S1 includes basic resource dimension indicators, service performance dimension indicators, and IoT data pipeline dimension indicators: the basic resource dimension indicators include the CPU utilization, memory usage, disk I / O, and network bandwidth throughput and latency of the container where the microservice instance resides; the service performance dimension indicators include the request processing rate, average response time, and high-order percentile latency data of each service interface; the IoT data pipeline dimension indicators include the rate of consuming messages from the message queue, the latency of the consumer group, and the backlog of messages to be processed.

[0015] According to the present invention based on the above scheme, the feature is that, in step S2, real-time data processing is completed before the health measurement assessment: the multi-dimensional indicator data in the time series database is cleaned and preprocessed, abnormal peaks caused by the startup or restart of microservice instances are filtered out, and missing short-time window data is interpolated and completed.

[0016] According to the present invention based on the above scheme, the feature is that, in step S2, the comprehensive health score of each monitoring dimension in the time series database is calculated using the health score model, and then the real-time health status label of each microservice instance is obtained. This means that the health score model assigns weights to each monitoring dimension, calculates the comprehensive health score by weighted summation, normalizes the comprehensive health score to a preset range, and uses it as the health status label of each microservice instance.

[0017] According to the present invention based on the above scheme, the feature is that, in step S3, traffic allocation among the remaining healthy microservice instances refers to: using an improved weighted random selection algorithm to allocate traffic among the remaining healthy microservice instances, wherein the weight of each microservice instance is positively correlated with its health score.

[0018] According to the present invention based on the above scheme, the feature is that, in step S3, after routing the request to the selected microservice instance, the load balancer module records the start time of the request and associates it with the processing completion time fed back by the microservice instance to continuously verify and calibrate the effectiveness of its routing decision.

[0019] According to the present invention based on the above scheme, step S3 further includes a fault recovery step: defining a microservice instance state machine, the microservice instance state includes a healthy state, a sub-healthy state, an isolated state, and an offline state. When the comprehensive health score of a microservice instance is continuously lower than the sub-healthy threshold for multiple consecutive detection periods, the traffic allocated to the microservice instance is gradually reduced until the score falls below the isolation threshold, at which point it is removed from the dynamic routing table and fault recovery processing is performed.

[0020] According to the above-described scheme, the present invention is characterized in that, in step S4, the microservice cluster is elastically scaled from two time scales: short-term real-time and long-term trend.

[0021] In the short-term real-time level, monitor cluster aggregation metrics and trigger immediate expansion when they exceed preset thresholds;

[0022] In terms of long-term trends, a time series analysis model is built to learn from historical load data, generate load prediction curves, and perform predictive scaling before load peaks. The resource scheduler is used to start new microservice instances on physical nodes with low load or idle resources. After warming up, the new instances officially take over the traffic.

[0023] According to the above-described scheme, the present invention is characterized by further including step S5, optimization and learning: collecting key business indicators before and after the execution of actions at each stage and storing them in an experience database, periodically analyzing historical experience and automatically generating parameter adjustment suggestions.

[0024] On the other hand, an IoT data access microservice elastic scaling and load balancing system is characterized by comprising:

[0025] The monitoring data acquisition module is configured to embed a monitoring agent in each microservice instance of the IoT data access platform, continuously collect multi-dimensional indicator data of each microservice instance, and aggregate the collected data into a time-series database.

[0026] The health metric assessment module is configured to calculate the overall health score for each microservice instance using a health score model and dynamically adjust the weights of each dimension through an adaptive mechanism.

[0027] The intelligent traffic scheduling module is configured to utilize the load balancer module to maintain a dynamic routing table containing all available microservice instances and their overall health scores, and to use an improved weighted random selection algorithm to schedule inbound requests.

[0028] The macro-resource elastic scaling module is configured to make elastic scaling decisions from two time scales: short-term real-time and long-term trend. It uses time series analysis models to identify load patterns and perform predictive scaling.

[0029] According to the above-described solution, the beneficial effects of this invention are as follows:

[0030] (1) This invention introduces a multi-dimensional dynamic health rating model, which incorporates various indicators such as basic resource indicators, business processing latency, message queue backlog status, and external dependent service health status into a unified evaluation system. This breaks through the limitation of existing technologies that rely solely on basic system indicators such as CPU and memory for load balancing, enabling the system to perceive the real business processing capabilities of each microservice instance in real time and accurately. This significantly improves the rationality of traffic distribution and the overall stability of the system, effectively avoiding the spread of faults caused by local overload.

[0031] (2) By constructing a microservice instance state machine and a corresponding smooth traffic switching strategy, the system can seamlessly or with low awareness complete traffic migration, instance isolation and recovery when instance performance degrades or fails, thus achieving high availability of services; after the isolated instance is diagnosed and recovered, it gradually resumes to bear traffic through a preheating mechanism, avoiding secondary impact caused by the sudden online of the instance.

[0032] (3) This invention effectively overcomes the lag defect of traditional reactive scaling through a predictive elastic scaling mechanism. It actively predicts the periodic and sudden load change trends of IoT data by using time series analysis, completes resource preparation before the traffic peak, completely eliminates the service capacity gap, and improves resource utilization and reduces operating costs through more accurate resource planning.

[0033] (4) This invention performs correlation analysis on infrastructure, application performance and business data indicators, providing end-to-end full-link observability. When a fault or performance degradation occurs, maintenance personnel can quickly locate the root cause, which greatly shortens the mean fault repair time and improves maintenance efficiency.

[0034] (5) The present invention has closed-loop self-optimization capability, which can continuously learn from historical operation data and automatically adjust key parameters such as load balancing strategy and elastic scaling threshold, so that the system has the ability to evolve and optimize itself, and continuously approach the optimal balance point in terms of stability, resource cost and performance, thereby reducing the complexity of operation and maintenance. Attached Figure Description

[0035] Figure 1 This is a flowchart of the IoT data access microservice elastic scaling and load balancing method in this invention;

[0036] Figure 2 This is a flowchart illustrating the specific process of health quantification assessment in this invention.

[0037] Figure 3 This is a flowchart illustrating the multi-stage selection decision process performed by the load balancer module in this invention.

[0038] Figure 4 This is the structural framework of the load balancer module in this invention;

[0039] Figure 5 This is a flowchart of the health sensing and scheduling unit in this invention;

[0040] Figure 6 This is a flowchart illustrating the specific process for verifying routing effectiveness in this invention.

[0041] Figure 7 This is a block diagram of the IoT data access microservice elastic scaling and load balancing system in this invention. Detailed Implementation

[0042] Existing IoT data processing methods suffer from drawbacks such as an inability to perceive the true internal business processing status of services, and lag in decision-making regarding scaling up or down. To address these issues, this invention aims to dynamically integrate key information such as business processing latency, message queue backlog, and dependent service status. This allows for precise scheduling of requests to the service instance with the strongest current processing capacity, avoiding localized failures and optimizing overall throughput. Furthermore, this invention addresses how to utilize time series analysis techniques to identify load patterns in IoT data and perform predictive scaling up and down accordingly. This ensures resource preparation before peak traffic periods, completely eliminating service capacity gaps and improving resource utilization.

[0043] The present invention will now be further described with reference to the accompanying drawings and embodiments:

[0044] Example 1: A Method for Elastic Scaling and Load Balancing of Microservices for IoT Data Access

[0045] This embodiment provides a method for elastic scaling and load balancing of microservices for IoT data access, which addresses the technical problems of existing technologies in IoT data access platforms under high concurrency and high dynamic load scenarios, such as the inability of load balancing to perceive the real business status of services and the decision-making lag in elastic scaling.

[0046] like Figure 1 As shown, the method includes the following implementation stages.

[0047] Phase 1: Monitoring Data Collection

[0048] In each microservice instance of the IoT data access platform, a lightweight monitoring agent is embedded. Designed to not consume business logic execution resources, the monitoring agent runs as an independent background thread or process, continuously collecting various metrics reflecting the operational status of the microservice instance.

[0049] The collected multi-dimensional indicator data is divided into three dimensions according to the monitoring focus:

[0050] Basic resource metrics include CPU utilization, memory usage, disk I / O rate, and network bandwidth throughput and latency of the container hosting the microservice instance. These metrics reflect the underlying hardware resource consumption of the microservice instance and are the foundational data for traditional monitoring systems.

[0051] Service performance metrics, at the business logic level, cover the request processing rate (RPS, i.e., requests per second), average response time, and high-percentile latency data (such as P95 and P99 latency) for each service interface. High-percentile latency data can reveal the processing status of long-tail requests and is an important reference for measuring the upper limit of service processing capacity.

[0052] The IoT data pipeline dimensional metrics are specifically designed for the characteristics of IoT data streams, including the rate at which messages are consumed from message queues (such as Kafka and RabbitMQ), consumer group latency, and the backlog of messages awaiting processing. These metrics directly reflect whether IoT device data is processed in a timely manner and are the most direct basis for evaluating the business processing capabilities of microservice instances.

[0053] In addition, the monitoring scope also covers external dependency resource indicators such as database connection pool utilization, number of active connections, and cache hit rate, which are used to perceive the impact of the health status of dependent services on the processing capacity of this microservice instance.

[0054] All the various metrics data collected by the monitoring agents are centrally pushed to a unified time-series database (such as InfluxDB, Prometheus, etc.) at fixed time intervals (e.g., every 5 seconds) through secure internal channels (such as gRPC or HTTP / 2 protocol) for centralized storage. This time-series database provides the data foundation for subsequent intelligent decision-making.

[0055] The monitoring agent reports collected multi-dimensional metric data to a unified time-series database asynchronously in batches via a dedicated internal network channel independent of the business request port. This internal channel uses an independent port isolated from business traffic, ensuring that the collection and reporting of monitoring data does not consume business processing resources or affect the normal service capabilities of microservice instances. In the event of temporary unavailability of the time-series database, the monitoring agent maintains a bounded buffer queue locally, prioritizing the resending of the latest data once the channel is restored, to ensure that critical metrics are not lost.

[0056] Phase Two: Real-time Data Processing and Metric Assessment of Health

[0057] 1. Real-time data processing

[0058] After data collection is complete, the data stream processing engine first cleans and preprocesses the indicator data in the time-series database, filtering out abnormal peaks caused by microservice instance startup or restart, and interpolating and completing missing short-time window data. Specifically, the cleaning steps include:

[0059] (a) Abnormal Peak Filtering: Filters out brief abnormal peak data generated when microservice instances start or restart, preventing them from interfering with subsequent health scores. This filtering is achieved by setting a startup protection period (e.g., data within 30 seconds of instance startup is not included in the score calculation).

[0060] (b) Missing data interpolation: missing short-time window data is interpolated and filled, for example by linear interpolation or the last valid value filling method, to ensure the integrity of time series data.

[0061] Data stream processing engines are mature stream processing technologies in the industry. For example, Apache Flink can be used for data processing. Through powerful stateful stream processing, it can be applied to production-level data processing processes.

[0062] 2. Quantitative assessment of health

[0063] After constructing a health score model and completing the cleaning and preprocessing steps, the calculation of the health score model is initiated. The health score model is a dynamic computational framework. Figure 2 As shown, its workflow is as follows:

[0064] First, assign a base weight (Wi) to each monitoring dimension, which reflects the importance of that dimension to the overall health of the microservice instance. For example, in an IoT data access scenario, the weight of the message queue backlog can be initially set to a high value, as it most directly reflects data processing capacity.

[0065] Secondly, the score (Si) for each dimension is calculated by mapping and comparing the current measured value of that dimension with multiple preset health thresholds for that dimension. For example, for the CPU utilization index, the following mapping rule can be defined: when the CPU utilization is below 0% to 60%, the score is 100 points, which is used to indicate that the CPU resources are sufficient; when the CPU utilization is between 60% and 75%, the score decreases linearly to 70 points; when the CPU utilization is between 75% and 90%, the score decreases linearly to 30 points; when the CPU utilization is between 90% and 100%, the score decreases to 0 points, which is used to indicate that the CPU utilization is critical.

[0066] Finally, a weighted summation method is used to calculate the overall health score: the calculated overall health score is normalized to the range of 0 to 100, and serves as a real-time health status label for each microservice instance. The higher the value, the stronger the overall processing capability and the healthier the running status of the microservice instance.

[0067] The overall health score is calculated using the following formula:

[0068]

[0069] Where H is the overall health score, ranging from 0 to 100, n is the total number of dimensions, Wi is the weight of dimension i, reflecting the importance of the indicator, and Si is the normalized score of dimension i. The initial value of Wi is set manually. For example, W1=0.25, representing the basic resource dimension indicator with a basic weight of 0.25, used as a basic guarantee and not directly determining business processing capacity; W2=0.35, representing the service performance dimension indicator with a basic weight of 0.35, directly reflecting the quality of business processing; W3=0.40, representing the IoT data pipeline dimension indicator with a basic weight of 0.40, which is the core indicator in the IoT scenario and is given the highest weight.

[0070] Health status tags are used to characterize the real-time health level of each microservice instance, uniformly representing the instance's real-time business processing capabilities and operational status. This tag serves as the core basis for subsequent intelligent traffic scheduling, fault self-healing, elastic scaling decisions, and closed-loop self-optimization, permeating the entire system operation process to achieve refined scheduling and management based on the instance's true health status. Different health status tags are defined according to different preset ranges. For example, H≥80 is defined as healthy, 80>H≥60 as sub-healthy, 60>H≥50 as critical, and H<50 as faulty.

[0071] The health scoring model also incorporates an adaptive mechanism: it periodically analyzes the correlation between various dimensional indicators and the final business results, and dynamically adjusts the weights of each monitoring dimension accordingly. Specifically, at regular intervals (e.g., hourly or daily), the system analyzes the correlation between various dimensional indicators and the final business results (such as actual throughput achievement rate and error rate), and dynamically adjusts the weights of each monitoring dimension accordingly.

[0072] Weights can be adjusted using a gradient increase or decrease. For example, if the accuracy of CPU utilization predictions has decreased over a period of time, the system will automatically decrease the weight of that metric (from 0.25 to 0.20) to ensure that the evaluation results always accurately reflect the true processing capacity and stability of the microservice instance.

[0073] This invention introduces a multi-dimensional dynamic health scoring model, enabling the system to perceive the real business processing capabilities of each microservice instance in real time and accurately, significantly improving the rationality of traffic distribution and the overall stability of the system.

[0074] Phase 3: Intelligent Traffic Scheduling and Fault Recovery

[0075] The intelligent traffic scheduling and fault recovery sub-functions work together to ensure the stable operation of the system.

[0076] 1. Intelligent traffic scheduling

[0077] A load balancer module is constructed, located at the entry point for all external IoT device data reporting requests. It performs traffic scheduling solely based on a real-time calculated and dynamically changing comprehensive health score (a health status label for microservice instances). The load balancer module maintains a dynamic routing table that records the service addresses of all available microservice instances and their current comprehensive health scores.

[0078] like Figure 3 As shown, when a new IoT device data reporting request arrives, the load balancer module performs the following multi-stage selection decision:

[0079] Phase 1: Candidate Pool Screening

[0080] The load balancer module quickly filters out microservice instances whose overall health score is below a preset threshold (e.g., below 20 points) based on the dynamic routing table. These instances are marked as unhealthy and temporarily excluded from the candidate pool to avoid routing requests to instances that are about to fail.

[0081] Phase 2: Weighted Random Selection

[0082] The load balancer module uses an improved weighted random selection algorithm to distribute traffic among the remaining healthy microservice instances.

[0083] In this algorithm, the weight of each microservice instance is positively correlated with its overall health score, but it is not a simple linear mapping. Instead, an elasticity factor is introduced. The elasticity factor allows microservice instances with high overall health scores (e.g., above 80) and idle status to receive a higher weight than a linear proportion, thus making more efficient use of high-quality resources. For microservice instances with decent but not optimal scores (e.g., 40 to 80) and slightly busy status, their weight will be reduced accordingly, but they still have a chance to receive a certain proportion of traffic to maintain their activity and avoid losing their warm-up status due to prolonged lack of traffic.

[0084] like Figure 4 As shown, based on the above scheduling strategy, the load balancer module in this invention includes a request receiving unit, a health-aware scheduling unit, a scheduling algorithm engine unit, an instance registry unit, and a rate limiting and degradation unit.

[0085] (1) Request receiving unit, used to receive data access requests from IoT devices or upstream services, supports HTTP / HTTPS protocol and MQTT protocol, and performs protocol parsing and session maintenance processing on the requests.

[0086] (2) A health-aware scheduling unit, used to obtain the comprehensive health score of each microservice instance in real time from the health scoring model, and to make status marking and traffic allocation decisions for each instance based on the comprehensive health score. Figure 5 As shown, the workflow of the health sensing scheduling unit includes the following steps:

[0087] A1. Subscribe to the health score model's score push messages. The health score model pushes an update of the comprehensive health score of each microservice instance every 10 seconds, and the trigger delay for scheduling weight updates does not exceed 1 second.

[0088] A2. Parse the received new rating and update the instance health status mapping table in memory.

[0089] A3. Mark the status of each instance according to the preset status threshold: instances with a comprehensive health score greater than or equal to 80 are marked as healthy, instances with a comprehensive health score greater than or equal to 60 and less than 80 are marked as sub-healthy, instances with a comprehensive health score greater than or equal to 50 and less than 60 are marked as isolated, and instances with a comprehensive health score less than 50 are marked as offline.

[0090] A4. Execute traffic allocation decisions based on status flag results: offline instances are removed from the scheduling pool, sub-healthy instances are enabled with rate limiting protection, isolated instances are enabled with circuit breaker protection, and healthy instances participate in traffic allocation normally.

[0091] A5. Synchronize the decision results to the scheduling algorithm engine unit to trigger the scheduling weight update.

[0092] (3) A scheduling algorithm engine unit, used to execute at least one scheduling algorithm among weighted round-robin scheduling, least-connection scheduling, or health-aware scheduling based on the decision results of the health-aware scheduling unit, to determine the target instance of the request to be forwarded. Wherein:

[0093] The weighted round-robin scheduling strategy refers to calculating the round-robin period based on the scheduling weight of each online instance, and allocating requests according to the proportion of each instance's weight to the total weight within each round-robin period; for example, if instance A has a weight of 7225 and instance B has a weight of 3844, then instance A will be allocated approximately 65.3% of the requests, and instance B approximately 34.7%.

[0094] The least-connection scheduling strategy refers to continuously counting the current active connections of each instance and selecting the instance with the fewest active connections as the target instance. The number of active connections includes the number of requests being processed and the number of long-lived connections waiting for a response.

[0095] The health-aware scheduling strategy refers to allocating requests only to instances whose health status is marked as "healthy". If all instances are in a sub-healthy or isolated state, requests are allocated according to weights across all online instances, and a degradation flag is appended to the response header.

[0096] The scheduling algorithm engine unit also supports dynamically switching scheduling strategies based on business load characteristics: when the proportion of long connections exceeds 60%, it automatically switches to the least connection scheduling strategy; when the request error rate exceeds 5%, it automatically switches to the health-aware scheduling strategy.

[0097] (4) Instance registry unit, which is used to store the registration information of microservice instances, form a service registry, and automatically remove faulty instances through a heartbeat detection mechanism.

[0098] (5) Rate limiting and degradation unit, used to perform degradation, rate limiting, or circuit breaking on instances at different levels based on the comprehensive health score. Specifically, for sub-healthy instances with a comprehensive health score between 60 and 80, the maximum request quota per second is automatically calculated and set; for isolated instances with a comprehensive health score between 50 and 60, the circuit breaker is opened and new data access requests are rejected, allowing only ongoing requests to complete, and after the circuit breaker ends, it enters a half-open state, allowing a small number of probe requests to pass; for offline instances with a comprehensive health score below 50, the instance is deregistered from the service registry and the container orchestration platform is notified to trigger the instance rebuild process.

[0099] Preferably, the rate limiting and degradation unit is linked with the health-aware scheduling unit: when an instance triggers rate limiting protection, the health-aware scheduling unit automatically reduces the scheduling weight of the instance; when the instance recovers to a healthy state, the scheduling weight gradually recovers to the normal level within 3 evaluation cycles.

[0100] In this invention, the load balancer module further includes a dynamic weight calculation unit, used to calculate the scheduling weight based on the overall health score of each instance. The dynamic weight calculation unit calculates the scheduling weight using the following formula:

[0101]

[0102] in, For scheduling weights, H is the overall health score of the instance; α is the health sensitivity coefficient of the instance, which defaults to 2, to achieve a squared amplification effect. The greater the difference in health scores, the more obvious the weight difference, thus achieving intelligent traffic distribution; C is the remaining capacity of the instance, which is the maximum capacity of the instance minus the current capacity; β is the capacity sensitivity coefficient, which defaults to 1.

[0103] In one embodiment, the present invention uses a health sensitivity coefficient α as an elasticity factor, which makes the scheduling weight have the characteristics of monotonically increasing (the higher the overall health score, the higher the scheduling weight), gap amplification (the exponential characteristic amplifies the advantages of high-scoring instances), and bottom protection (low-scoring instances retain the lowest priority and are not completely closed).

[0104] Let's take examples A, B, and C as examples to illustrate:

[0105] Traditional linear W=H Improved flexibility Instance A weights 85 85²= 7,225 Instance B weight 62 62²= 3,844 Instance C weight 40 40²= 1600 Weighted sum 187 12669 Instance A percentage 85 / 187 = 45.5% 7225 / 12669 = 57.0% Instance B percentage 62 / 187 = 33.2% 3844 / 12669= 30.3% Instance C percentage 40 / 187 = 21.3% 1600 / 12669= 12.7%

[0106] This shows that high-scoring instance A can obtain higher-quality traffic, unhealthy instance B is moderately diverted, and marginal instance C is suppressed but not completely eliminated, thus avoiding concentrating all the pressure on a few high-scoring instances.

[0107] In another embodiment, in addition to introducing a health sensitivity coefficient α, the present invention also considers an instance capacity sensitivity coefficient β, which is used to adjust the degree of influence of remaining capacity on weights.

[0108] In a specific application scenario, let's take Instance A and Instance B as examples for illustration:

[0109] H Current capacity Maximum capacity C Example A 85 (Health) 900 1000 100 Example B 62 (Sub-health) 200 1000 800

[0110] A comparison of two scenarios: using only the health sensitivity coefficient (α=2) as the elasticity factor, and using both the health sensitivity coefficient (α=2) and the capacity sensitivity coefficient (β=1) as elasticity factors.

[0111] H Example A 85 85²= 7,225 7225×100=722500 Example B 62 62²= 3,844 3844×800=3075200 Instance A percentage 85 / 147=57.8% 7225 / 11069=65.3% 722500 / 3797700=19.0% Instance B percentage 62 / 147=42.2% 3844 / 11069=34.7% 3075200 / 3797700=81.0%

[0112] Therefore, although instance A has a better health score, its capacity is smaller and it should not be allocated more traffic. Instance B has a slightly lower health score but has a higher capacity and can handle more traffic.

[0113] Phase 3: Routing Performance Verification

[0114] After routing a request to a selected microservice instance, the load balancer module records the request's start time and correlates it with the processing completion time reported by the microservice instance. This allows for real-time statistical analysis of the actual service latency and throughput of each instance, continuously verifying and calibrating the effectiveness of its routing decisions. Figure 6 As shown, the specific implementation steps are as follows:

[0115] (1) Record the start time when requesting routing.

[0116] When the load balancer routes an IoT device data reporting request to the target microservice instance, it generates a globally unique identifier for the request and records the request identifier, the target microservice instance identifier, the timestamp of the routing time, the comprehensive health score of the microservice instance at the routing time, and the scheduling weight in the routing record table. The routing record table is stored in an in-memory database or a distributed cache and has a preset expiration time.

[0117] (2) Callback after the microservice instance is processed.

[0118] After the target microservice instance finishes processing the request, it sends a processing completion message back to the load balancer through a callback channel. The callback message includes the identifier of the request, the processing completion timestamp, and the request processing status. The callback channel adopts an asynchronous callback mechanism or a synchronous callback mechanism.

[0119] (3) Actual processing time of association calculation.

[0120] The load balancer looks up the corresponding routing record based on the request identifier in the callback message, calculates the actual processing time of the request (i.e., the difference between the processing completion timestamp and the routing timestamp), and writes the calculation result, along with the microservice instance identifier, into the metrics time series database.

[0121] (4) Sliding window statistical analysis.

[0122] The performance evaluation unit aggregates and statistically analyzes the actual processing time of each microservice instance according to a preset duration sliding window, and calculates the average response time, 99th percentile latency, request success rate, and throughput within the window.

[0123] (5) Deviation detection.

[0124] The overall health score of each microservice instance at the routing time is compared with the actual statistical performance of the instance within the sliding window. A preset deviation threshold is used to determine whether there is a deviation. When the actual average response time is significantly higher than the expected response time corresponding to the overall health score, the overall health score is determined to be overestimated. When the actual average response time is significantly lower than the expected response time, the overall health score is determined to be underestimated.

[0125] (6) Deviation data is stored in the experience base to trigger self-optimization.

[0126] The comparison data between each routing decision and the actual execution result is stored in the experience base; when a serious deviation is detected, an emergency adjustment mechanism is triggered to temporarily reduce or increase the scheduling weight of the microservice instance; when a moderate deviation is detected, the microservice instance is marked for periodic review by the optimization and learning module.

[0127] 2. Fault Recovery

[0128] Define a microservice instance state machine, which includes four states: healthy, sub-healthy, isolated, and offline. Utilizing the microservice instance state machine, the failover and self-healing mechanisms are tightly coupled with the load balancer module, enabling them to work collaboratively.

[0129] When the overall health score of a microservice instance remains below the threshold for multiple consecutive detection cycles (e.g., 3 consecutive detection cycles, approximately 15 seconds), the load balancer module triggers a degradation operation. This involves linearly reducing the weight of the microservice instance in each scheduling cycle, gradually and smoothly reducing new traffic allocated to the microservice instance until the score falls below the isolation threshold. At this point, the microservice instance is removed from the dynamic routing table and fault recovery is performed, while allowing it to continue processing received requests, thus avoiding interruption of data transmission to IoT devices with established sessions.

[0130] Specifically, the state transition rules are as follows: When the overall health score of a microservice instance remains below the health threshold for multiple consecutive detection cycles, it transitions from a healthy state to a sub-healthy state; when the overall health score of a microservice instance remains below the sub-healthy threshold for multiple consecutive detection cycles, it transitions from a sub-healthy state to an isolated state; when the overall health score of a microservice instance falls below the isolation threshold or the heartbeat signal interruption exceeds the isolation heartbeat timeout threshold, it transitions from a sub-healthy state or an isolated state to an offline state; when the overall health score of a microservice instance in a sub-healthy state or an isolated state continues to rise and reaches the recovery threshold for multiple consecutive detection cycles, it gradually recovers to a healthy state in the opposite direction.

[0131] The method to gradually reduce the traffic allocated to microservice instances is as follows: linearly reduce the weight of the microservice instance in each scheduling cycle; after removing the microservice instance from the dynamic routing table, the traffic originally sent to the microservice instance is rerouted to other healthy microservice instances according to session consistency requirements; unprocessed requests rely on the retry mechanism of the upstream message queue (such as Kafka) and local checkpoints to ensure that IoT data is not lost or duplicated.

[0132] If the health status of the microservice instance deteriorates further, the overall health score falls below the isolation threshold, or the microservice instance completely loses its heartbeat for more than a set time (e.g., it has not reported a heartbeat for more than 10 seconds), the system will immediately put its status into isolation and remove it completely from the dynamic routing table.

[0133] The fault recovery process includes a self-healing process for isolated microservice instances. This could involve automatic restarts (container orchestration platforms like Kubernetes automatically trigger Pod restarts) or manual intervention by sending alerts to the operations platform. When a faulty microservice instance recovers and goes back online, it is not immediately loaded with full production traffic; instead, it enters a warm-up state. Specifically, the isolated microservice instance enters a diagnostic and recovery process. Upon re-launch after recovery, it enters a warm-up state where the load balancer module sends a small number of non-critical probe requests (e.g., starting with a weight of 5%) to monitor the trend of its overall health score in real time. Once its overall health score steadily recovers to a healthy range (e.g., above 70 points and maintained for more than two monitoring cycles), its weight is gradually restored to a normal level, eventually fully integrating into the microservice cluster and handling normal traffic load.

[0134] Phase 4: Macroeconomic Resource Elasticity

[0135] Elastic scaling is performed using a microservice cluster consisting of multiple microservice instances as the unit of analysis, rather than a single microservice instance, and scaling up or down is performed based on different needs.

[0136] This invention enables elastic scaling of microservice clusters across two time scales: short-term real-time and long-term trend.

[0137] 1. Short-term real-time level

[0138] Monitor cluster aggregation metrics and trigger immediate scaling when they exceed preset thresholds. These cluster aggregation metrics include the overall average health score of the microservice cluster (the weighted average of the health scores of all microservice instances), total request queue length (the sum of message queue backlogs for all instances), and average response time (the weighted average of the average response times for all instances).

[0139] (1) Expansion

[0140] When one or more of the above aggregated metrics exceed the preset scaling threshold (e.g., the cluster's average overall health score remains below 65 for more than 2 minutes), and this abnormal state persists for a period of time (to avoid frequent scaling triggers due to short-term fluctuations), an immediate scaling signal is triggered. Upon receiving the immediate scaling signal, the resource scheduler, based on the current resource distribution of the microservice cluster, prioritizes starting new microservice instances on physical nodes with low load or idle resources. The startup script pre-configures the monitoring agent's initialization parameters, the service registry address, and basic service configuration information.

[0141] After a new microservice instance starts up, it first registers with the service registry and reports its existence to the load balancer module, then begins receiving a small amount of pre-warm-up traffic allocated by the load balancer module. Its overall health score starts from an initial value and is rapidly calibrated as real traffic is processed. When its overall health score reaches the health standard, the load balancer module jointly confirms that the instance is fully ready, and then it officially assumes the traffic pressure equivalent to that of other microservice instances, completing the scaling operation.

[0142] (2) Volume reduction

[0143] The scaling-down process is more cautious: the system prioritizes microservice instances with high overall health scores but relatively low current traffic as scaling-down candidates, that is, it prioritizes healthy instances with relatively low resource utilization for scaling down to minimize the impact of scaling down on service quality: first, its weight in the dynamic routing table is gradually reduced to zero to ensure that it no longer receives new requests. After it has processed all ongoing tasks, its active connection count and the length of the pending message queue are monitored to determine whether it has been processed. Then, it is removed from the dynamic routing table and shut down to release physical node resources.

[0144] 2. Long-term trend level

[0145] A time-series analysis model is built to learn from historical load data, generate load forecast curves, and perform predictive scaling before load peaks. A resource scheduler is used to launch new microservice instances on physical nodes with low load or idle resources. These new instances are then warmed up before officially handling traffic. For example, ARIMA (Autoregressive Integral Moving Average) can be used to handle trends and seasonality in time series data; or Prophet (Facebook's open-source forecasting tool) can be used to handle complex data with multiple seasonalities.

[0146] IoT data access often exhibits clear periodic patterns. For example, industrial equipment reports significantly higher data density during production shifts than off-shift periods (i.e., daily cycle), and overall weekday traffic is significantly higher than weekend traffic (i.e., weekly cycle). Time series analysis models identify the daily and weekly load patterns of IoT data and, combined with external event calendars (such as holidays or specific business activities), generate a load prediction curve for a future period. Based on this prediction curve, the system performs predictive scaling: automatically triggering predictive scaling operations a certain lead time before the expected load peak, allowing sufficient time for the startup, initialization, and warm-up of new microservice instances. This ensures that all newly scaled microservice instances are in a healthy and ready state when the actual traffic peak arrives, completely eliminating service capacity gaps caused by the scaling time window.

[0147] This invention effectively overcomes the lag defects of traditional reactive scaling through a predictive elastic scaling mechanism. By using time series analysis to proactively predict load change trends, it can complete resource preparation before traffic peaks arrive. This not only eliminates service capacity gaps and ensures business continuity, but also improves resource utilization and reduces operating costs through more precise resource planning.

[0148] Phase 5: Optimization and Learning

[0149] The IoT data access microservice elastic scaling and load balancing method of the present invention also has optimization and learning steps to achieve closed-loop self-optimization function.

[0150] After a series of actions, including intelligent load balancing, failover, and elastic scaling, the performance evaluation unit begins its work, collecting a series of key business metrics data before and after the actions, including: changes in overall throughput, the degree of improvement in average response time, changes in resource utilization, and error rate fluctuations that may be caused by microservice instance switching. Specifically, changes in overall throughput are used to assess the extent of throughput improvement after scaling up; the degree of improvement in average response time is used to determine whether the latency perceived by users / devices has decreased; changes in resource utilization are used to assess whether there has been excessive scaling leading to resource waste; and error rate fluctuations are used to evaluate the stability of the failover strategy.

[0151] These action feedback data are stored in the system's experience base. The system's built-in optimization algorithm periodically analyzes these historical experiences to identify which parameter configurations achieve optimal results under which load scenarios. Based on this analysis, the system automatically generates parameter adjustment suggestions and applies fine-tuning under certain constraints. For example, automatic parameter adjustment is performed under the constraint that the adjustment range does not exceed 20% of the existing parameter value.

[0152] (a) Adjust the weights of each dimension in the health rating model, for example, increase the weight of message queue backlog after it was found to be more accurate in predicting failures.

[0153] (b) Optimize the elasticity factor value in the intelligent load balancing algorithm. For example, when it is found that too much traffic is concentrated in a few high-scoring instances, causing their load imbalance, the elasticity factor should be appropriately reduced.

[0154] (c) Adjust the threshold and timing of elastic scaling, for example, when the actual load peak is found to arrive 5 minutes earlier than predicted, adjust the lead time of predictive scaling accordingly.

[0155] Through the aforementioned closed-loop self-optimization mechanism, the system gradually acquires the ability to learn from history and adapt to the characteristics of specific business scenarios. It automatically adjusts key parameters such as load balancing strategies and elastic scaling thresholds, continuously approaching the optimal balance point in terms of stability, resource cost, and performance. This enables the system to have the ability to evolve and optimize itself, reducing the frequency and complexity of manual maintenance intervention.

[0156] This invention can use Bayesian optimization or reinforcement learning algorithms as the optimization algorithm. Bayesian optimization is suitable for adjusting continuous parameters such as elasticity factors and thresholds, and can find the optimal value in a small number of trials. Reinforcement learning algorithms are mostly used to deal with sequential decision problems, using the adjustment of current parameters to influence future states, thereby forming a long chain of causal relationships.

[0157] Example 2: IoT Data Access Microservice Elastic Scaling and Load Balancing System

[0158] like Figure 7 As shown, this embodiment provides an IoT data access microservice elastic scaling and load balancing system. The system includes a monitoring data acquisition module, a health measurement and assessment module, an intelligent traffic scheduling module, a fault recovery module, a macro-resource elastic scaling module, an optimization and learning module, and other components.

[0159] The monitoring data acquisition module is deployed within each microservice instance and runs as an independent process or monitoring agent. It continuously collects basic resource metrics (CPU utilization, memory usage, disk I / O, network throughput and latency), service performance metrics (request processing rate of each interface, average response time, high-order percentile latency), and IoT data pipeline metrics (message queue consumption rate, consumer group latency, and backlog of pending messages). The collected data is aggregated into a unified time-series database at fixed time intervals through a secure internal channel.

[0160] Among them, basic resource dimension indicators include CPU utilization, memory usage, disk I / O, network throughput and latency; service performance dimension indicators include request processing rate of each interface, average response time, high-order percentile latency, etc.; IoT data pipeline dimension indicators include message queue consumption rate, consumer group latency, and backlog of pending messages, etc.

[0161] The health measurement and assessment module continuously reads multi-dimensional indicator data from the time-series database. First, the data is cleaned and preprocessed by the data stream processing engine. Then, the health scoring model is run to calculate the comprehensive health score of each microservice instance. Through an adaptive mechanism, the correlation between each dimension indicator and business results is analyzed periodically, and the weight of each monitoring dimension is dynamically adjusted to ensure that the scoring results always accurately reflect the real processing capabilities of the microservice instance.

[0162] The intelligent traffic scheduling module is configured to utilize the load balancer module for intelligent traffic scheduling. The load balancer module is located at the entry point for all external IoT device data reporting requests. It maintains a dynamically updated routing table that records the addresses of all available microservice instances and their current overall health scores. When a request arrives, it performs a multi-stage selection decision, employing an improved weighted random selection algorithm with an added elasticity factor to schedule traffic. This ensures that overall traffic is tilted towards the healthiest microservice instances while avoiding concentrating all pressure on a few instances.

[0163] The multi-stage selection strategy consists of candidate pool screening → improved weighted random selection → routing effect verification.

[0164] The fault recovery module is configured to define and maintain the state machine of microservice instances, triggering degradation, isolation, and self-healing operations based on changes in the overall health score. Specifically, the fault recovery module is tightly coupled with the load balancer module, maintaining the state machine of microservice instances and triggering corresponding degradation, isolation, and self-healing operations based on changes in the overall health score; it achieves smooth degradation through linear weight reduction, ensures no data loss through message queue retry mechanisms and local checkpoints, and achieves safe recovery and online deployment of faulty instances through pre-warmed traffic allocation.

[0165] The macro-level resource elastic scaling module analyzes the microservice cluster as a whole, making elastic scaling decisions based on both short-term real-time and long-term trend time scales. It collaborates with the resource scheduler to launch new microservice instances on idle physical nodes, and the new instances officially take over the traffic after being warmed up. When scaling down, it completes resource release in an orderly manner by smoothly reducing the weight, clearing active connections, and closing instances.

[0166] The optimization and learning module collects key business metrics before and after each stage of action, stores them in an experience base, and then periodically analyzes them using a built-in optimization algorithm. This automatically generates and applies parameter adjustments under constraints, including the weights of each dimension of the health scoring model, load balancing elasticity factors, and elastic scaling thresholds and timing, enabling continuous self-optimization and evolution of the system. Key business metrics include changes in throughput, improvements in response time, changes in resource utilization, and fluctuations in error rate.

[0167] This embodiment systematically manages and transforms the volatility, uncertainty, and complexity faced during the IoT data access process, from micro-level microservice instance monitoring to macro-level microservice cluster scheduling, from real-time traffic distribution to long-term resource planning, and from passive fault response to proactive prediction and prevention. It constructs a highly resilient, efficient, and adaptive microservice architecture, providing effective technical support for the stable and efficient access and processing of massive IoT data.

[0168] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

[0169] The present invention has been described above with reference to the accompanying drawings. Obviously, the implementation of the present invention is not limited to the above-described manner. Any improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A method for elastic scaling and load balancing of microservices for IoT data access, characterized in that, Includes the following steps: S1. Monitoring data collection: A monitoring agent is embedded in each microservice instance of the IoT data access platform. The monitoring agent continuously collects multi-dimensional indicator data of each microservice instance and centralizes the multi-dimensional indicator data into a unified time-series database. S2. Health Measurement and Assessment: Construct a health scoring model, use the health scoring model to calculate the comprehensive health score of each monitoring dimension in the time series database, and then obtain the real-time health status label of each microservice instance; S3, Intelligent Traffic Scheduling: Construct a load balancer module, which maintains a dynamic routing table containing all available microservice instances and their overall health scores. When receiving data reporting requests from IoT devices, it executes a multi-stage selection strategy, first filtering out microservice instances with overall health scores below the threshold, and then distributing traffic among the remaining healthy microservice instances. S4. Macro-level resource elastic scaling: Elastic scaling is performed on a microservice cluster consisting of multiple microservice instances as the unit of analysis, expanding or shrinking the capacity based on different needs.

2. The IoT data access microservice elastic scaling and load balancing method according to claim 1, characterized in that, In step S1, the multi-dimensional indicator data includes basic resource dimension indicators, service performance dimension indicators, and IoT data pipeline dimension indicators: the basic resource dimension indicators include the CPU utilization, memory usage, disk I / O, and network bandwidth throughput and latency of the container where the microservice instance resides; the service performance dimension indicators include the request processing rate, average response time, and high-order percentile latency data of each service interface; the IoT data pipeline dimension indicators include the rate of consuming messages from the message queue, the latency of the consumer group, and the backlog of pending messages.

3. The IoT data access microservice elastic scaling and load balancing method according to claim 1, characterized in that, In step S2, real-time data processing is completed before the health measurement assessment: the multi-dimensional indicator data in the time series database is cleaned and preprocessed, abnormal peaks caused by the startup or restart of microservice instances are filtered out, and missing short-time window data is interpolated and completed.

4. The method for elastic scaling and load balancing of IoT data access microservices according to claim 1, characterized in that, In step S2, the comprehensive health score of each monitoring dimension in the time series database is calculated using the health score model, and then the real-time health status label of each microservice instance is obtained. The health score model assigns weights to each monitoring dimension, calculates the comprehensive health score by weighted summation, normalizes the comprehensive health score to a preset range, and uses it as the health status label of each microservice instance.

5. The IoT data access microservice elastic scaling and load balancing method according to claim 1, characterized in that, In step S3, traffic allocation among the remaining healthy microservice instances refers to: using an improved weighted random selection algorithm to allocate traffic among the remaining healthy microservice instances, where the weight of each microservice instance is positively correlated with its health score.

6. The method for elastic scaling and load balancing of IoT data access microservices according to claim 1, characterized in that, In step S3, after routing the request to the selected microservice instance, the load balancer module records the start time of the request and correlates it with the processing completion time reported by the microservice instance to continuously verify and calibrate the effectiveness of its routing decision.

7. The method for elastic scaling and load balancing of IoT data access microservices according to claim 1, characterized in that, Step S3 also includes a fault recovery step: Define a microservice instance state machine. The microservice instance state includes healthy state, sub-healthy state, isolated state, and offline state. When the overall health score of a microservice instance is continuously lower than the sub-healthy threshold for multiple consecutive detection periods, gradually reduce the traffic allocated to the microservice instance until the score falls below the isolation threshold. At that point, remove the microservice instance from the dynamic routing table and perform fault recovery processing.

8. The method for elastic scaling and load balancing of IoT data access microservices according to claim 1, characterized in that, In step S4, the microservice cluster is elastically scaled based on two time scales: short-term real-time and long-term trend. In the short-term real-time level, monitor cluster aggregation metrics and trigger immediate expansion when they exceed preset thresholds; In terms of long-term trends, a time series analysis model is built to learn from historical load data, generate load prediction curves, and perform predictive scaling before load peaks. The resource scheduler is used to start new microservice instances on physical nodes with low load or idle resources. After warming up, the new instances officially take over the traffic.

9. The method for elastic scaling and load balancing of IoT data access microservices according to claim 1, characterized in that, It also includes step S5, optimization and learning: collecting key business indicators before and after the execution of each stage of the action and storing them in the experience base, regularly analyzing historical experience and automatically generating parameter adjustment suggestions.

10. An IoT data access microservice elastic scaling and load balancing system, used to implement the IoT data access microservice elastic scaling and load balancing method according to any one of claims 1-9, characterized in that, include: The monitoring data acquisition module is configured to embed a monitoring agent in each microservice instance of the IoT data access platform, continuously collect multi-dimensional indicator data of each microservice instance, and aggregate the collected data into a time-series database. The health metric assessment module is configured to calculate the overall health score for each microservice instance using a health score model and dynamically adjust the weights of each dimension through an adaptive mechanism. The intelligent traffic scheduling module is configured to utilize the load balancer module to maintain a dynamic routing table containing all available microservice instances and their overall health scores, and to use an improved weighted random selection algorithm to schedule inbound requests. The macro-resource elastic scaling module is configured to make elastic scaling decisions from two time scales: short-term real-time and long-term trend. It uses time series analysis models to identify load patterns and perform predictive scaling.

Images