Elastic load balancing methods and systems for handling tens of millions of concurrent accesses

Abstract

This invention discloses a method and system for handling tens of millions of concurrent accesses in elastic load balancing. The key technical aspects include: a control unit, used to formulate semantic routing rules and caching rules for the data to be accessed, and send the caching rules to a gateway unit and the semantic routing rules to a service unit; the caching rules include the caching time for the data to be accessed; a gateway unit, used to send user access requests to edge modules in the service unit or data unit according to the user access request and the caching rules; a service unit, used to send user access requests to edge modules in the service instance or data unit according to the user access request and the semantic routing rules; and a data unit, used to store cached data and send the corresponding data to be accessed to the user terminal according to the user access request. This invention ensures stable system operation and improves user experience through a collaborative mechanism between the control unit, gateway unit, service unit, and data unit.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and more specifically to a method and system for handling tens of millions of concurrent accesses in an elastic load balancing manner. Background Technology

[0002] Driven by the wave of digital transformation, distributed access systems, including CDN, microservice architecture, and cloud storage systems, have become core infrastructure supporting internet applications. With the continuous expansion of business scale, the exponential growth of user traffic, and the increasing complexity of content and service formats, traditional architectural models are struggling to adapt to dynamic business changes, leading to increased user access latency and decreased service stability. Therefore, existing technologies have shortcomings. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for handling tens of millions of concurrent accesses in elastic load balancing. Through the collaborative mechanism among the control unit, gateway unit, service unit, and data unit, the system ensures stable operation and improves user experience.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] This invention provides a flexible load balancing system for handling tens of millions of concurrent accesses, comprising:

[0006] The control unit is used to formulate semantic routing rules and caching rules for at least part of the data to be accessed, and send the caching rules to the gateway unit and the semantic routing rules to the service unit. The semantic routing rules are determined according to the user identity and access type, and the caching rules include the caching time of the data to be accessed.

[0007] The gateway unit is used to send the user access request to the service unit according to the user access request and the caching rules;

[0008] A service unit is used to send the user access request to the edge module in the service instance or data unit according to the user access request and the semantic routing rules.

[0009] The data unit is used to store cached data and send the corresponding data to be accessed to the user terminal according to the user access request.

[0010] As a further improvement of the present invention, the control unit includes:

[0011] The configuration module is used to formulate the semantic routing rules and the caching rules, and send the caching rules to the gateway unit and the semantic routing rules to the service unit;

[0012] The management module is used to version-mark the caching rules defined by the configuration module and send the version marks to the data unit.

[0013] As a further improvement of the present invention, the gateway unit includes:

[0014] The parsing module is used to parse the user access request and add tags to the user access request.

[0015] The judgment module is used to determine whether the data to be accessed corresponding to the user access request is located in the local cache copy corresponding to the gateway unit;

[0016] The transmission module is configured to send the corresponding data to be accessed to the user terminal when the data to be accessed corresponding to the user access request is located in the local cache copy corresponding to the gateway unit, and to send the user access request and the tag to the service unit when the data to be accessed corresponding to the user access request is not located in the local cache copy corresponding to the gateway unit.

[0017] As a further improvement of the present invention, the service unit includes:

[0018] The pass-through module is used to determine the service cluster or the edge module based on the user access request and tag sent by the transmission module in the gateway unit.

[0019] The delivery module is configured to send the user access request to the service cluster after determining the service cluster, so that the service cluster selects a service instance from multiple service instances included in the service cluster according to the status of the service instance and sends the user access request to the service instance; or it is configured to send the user access request to the edge module after determining the edge module.

[0020] As a further improvement of the present invention, the data unit includes:

[0021] A storage module is used to store version tags sent by the management module in the control unit;

[0022] An edge module is used to send the data to be accessed corresponding to the received user access request to the user terminal. There are multiple edge modules.

[0023] This invention provides a method for handling tens of millions of concurrent accesses in elastic load balancing, implemented based on the aforementioned elastic load balancing system for handling tens of millions of concurrent accesses, comprising:

[0024] Obtain user access requests;

[0025] Based on the user access request and caching rules, the user access request is sent to a service instance or edge module in the service unit; so that the service instance or the edge module sends the data to be accessed corresponding to the user access request to the user terminal.

[0026] As a further improvement of the present invention, sending the user access request to a service instance or edge module in the service unit according to the user access request and caching rules includes:

[0027] Determine whether the data to be accessed is stored in the local cache copy corresponding to the gateway unit;

[0028] If not, the user access request will be sent to a service instance in the service unit or an edge module in the data unit;

[0029] If so, the data to be accessed corresponding to the user access request is sent to the user's terminal.

[0030] As a further improvement of the present invention, sending the user access request to a service instance or edge module in the service unit further includes:

[0031] The type of the data to be accessed is determined based on the user access request;

[0032] The user access request is sent to the service instance or the edge module according to the type.

[0033] As a further improvement of the present invention, sending the data to be accessed corresponding to the user access request to the user terminal includes:

[0034] Obtain semantic routing rules, the user access request, and tags;

[0035] The service cluster is determined based on the decision tree model, the user access request, and the tags, wherein the decision tree model is determined according to the semantic routing rules;

[0036] Based on the status of each service instance in the service cluster, a service instance is selected so that the service instance sends the data to be accessed corresponding to the user access request to the user terminal.

[0037] As a further improvement of the present invention, the step of selecting a service instance based on the status of each service instance in the service cluster includes:

[0038] Obtain the response score of the service cluster and each service instance, and the status score of each service instance;

[0039] Based on the response score of the service cluster and each service instance and the status score of each service instance, a comprehensive score for each service instance is obtained.

[0040] Select a service instance based on the overall score.

[0041] This invention automatically generates caching rules from data indicators collected by the control unit and sends them to the gateway unit. The gateway unit then sends user access requests to the service unit or appropriate edge nodes according to the caching rule mechanism. Simultaneously, a tag is sent to the service unit, ensuring that the user access request carries the tag into both the service unit and the data unit. This allows each layer to complete the user's access request based on the same tag. Through the interaction between the units in this invention, caching rules are sent and applied in a timely manner, avoiding the situation where the traditional gateway unit cannot promptly perceive the caching status and blindly sends user access requests. It also avoids problems such as unauthorized access or data inconsistency caused by inconsistent tags. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0043] Figure 2 This is a schematic diagram illustrating the transmission of caching rules and semantic routing rules;

[0044] Figure 3 This is a schematic diagram illustrating the interaction between the gateway unit and the service unit.

[0045] Figure 4 This is a schematic diagram illustrating the interaction between the gateway unit, service unit, and data unit.

[0046] Figure 5 This is a schematic diagram of the method steps of the present invention. Detailed Implementation

[0047] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof.

[0048] The term "and / or" in the following text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0049] like Figure 1 As shown in the figure, this application embodiment provides an elastic load balancing system for handling tens of millions of concurrent accesses, including:

[0050] The control unit is used to formulate semantic routing rules and caching rules for at least part of the data to be accessed, and send the caching rules to the gateway unit and the semantic routing rules to the service unit. The semantic routing rules are determined according to the user identity and access type, and the caching rules include the caching time of the data to be accessed.

[0051] The gateway unit is used to send user access requests to the service unit based on user access requests and caching rules;

[0052] Service units are used to send user access requests to edge modules in service instances or data units based on user access requests and semantic routing rules.

[0053] The data unit is used to store cached data and sends the corresponding data to be accessed to the user's terminal according to the user's access request.

[0054] Semantic routing rules refer to rules for sending requests based on semantic information in user access requests. Semantic information includes content semantics, context semantics, and tag semantics. Content semantics can include the type and keywords of the data requested by the user; context semantics can include user identity and region; and tag semantics can include data version number. For example, in a shopping scenario, the type of data requested includes product details and orders; keywords include promotional products; user identity includes member users and ordinary users; and the version number is data V2.0. Based on this semantic information, relevant semantic routing rules can be formulated. For instance, when the user's identity is an ordinary user, the user access request is sent to the ordinary user service cluster. Each service cluster is located within a service unit and is used to handle specific types of requests. The semantic information given in this embodiment is only a partial example, but this embodiment is not limited to this. Those skilled in the art can determine appropriate semantic information based on actual access situations.

[0055] Caching rules are a set of rules specifying cache location and validity period for different types of data. For example, one caching rule might set a certain image to be cached in a specific edge module for 24 hours, and also cache the same image in a local cache copy corresponding to the gateway unit for 1 hour. The validity period is the cache duration. There are multiple edge modules, each corresponding to a region. For instance, when a user accesses an image in Shanghai, the service unit will send the user's access request to the edge module closest to the user, i.e., the edge module corresponding to Shanghai. Furthermore, this embodiment only sets caching rules for a portion of the data to be accessed. For real-time interactive data, such as live stream comments or dynamically changing authentication information, caching rules are typically not set to avoid users obtaining outdated information due to caching.

[0056] This embodiment centrally formulates semantic routing rules and caching rules through the control unit. The gateway unit efficiently forwards user access requests based on the caching rules. The service unit accurately sends the user access request to the service instance or edge module based on the semantic routing rules. The data unit is responsible for cache data storage and response. Through the collaborative mechanism between multiple units, the caching efficiency and the accuracy of access request distribution are improved. At the same time, the user access experience is optimized and the backend pressure is reduced by leveraging the proximity service of the edge module.

[0057] Specifically, the control unit includes a configuration module and a management module; the gateway unit includes a parsing module, a judgment module, and a transmission module; the service unit includes a pass-through module and a transmission module; and the data unit includes a storage module and an edge module. The configuration module is used to formulate semantic routing rules and caching rules, and send the caching rules to the gateway unit and the semantic routing rules to the service unit. The management module is used to version-mark the caching rules formulated by the configuration module.

[0058] Version marking only applies to the currently changed parts. For example, suppose the current control unit detects that an image has been replaced, such as a merchant uploading a new product interface image. Based on the image's replacement frequency, a new caching rule needs to be configured for the replaced image. For instance, when the image is accessed too frequently, a lower cache time needs to be configured to ensure that users can obtain the latest image in a timely manner. This embodiment does not limit the access frequency or cache time. After the configuration module completes its cache time configuration, the management module marks the caching rule. For example, if the cache rule before the image was replaced was version 1.0, the configured cache rule would be recorded as 1.1, and 1.1 would be the version mark. Afterwards, as... Figure 2 As shown, the configuration module sends new caching rules to the judgment module in the gateway unit. Upon receiving the new caching rules, the judgment module needs to determine whether the local cache copy corresponding to the gateway unit stores the image before the change. If so, it deletes the image stored in the local cache copy. Furthermore, the management module needs to send the new caching rules, version flags, and the ID of the changed data to the storage module. This allows the storage module to push data from the origin server to the edge module via a distributed protocol (such as HTTP) according to the timeliness and edge module specified in the caching rules, ensuring users access data from the nearest location. Simultaneously, the corresponding caching rules should also be sent to the edge module when pushing data.

[0059] Furthermore, when pushing data, the storage module can consider the load of each edge module. Specifically, the storage module can receive data reported by each edge module, including current memory usage and CPU usage. Based on the data reported by each edge module, the storage module can generate a load heatmap. The load heatmap can intuitively display the load of each edge node. For example, the red area in the load heatmap has a higher load value than the green area. Each edge node reports data every preset time interval, and the load heatmap is updated every preset time interval. When pushing data, the storage module checks the edge modules specified in the caching rules according to the load heatmap. If the load value of an edge module specified in the caching rules is greater than the set load threshold, data will not be pushed to that edge module to avoid storage overflow or response delay. The storage module needs to return the updated load heatmap to each edge module. In this embodiment, the preset time and load threshold are not limited, and the step of generating the load heatmap is a technical means that can be implemented by those skilled in the art, and will not be described in detail here.

[0060] This embodiment updates the caching rules based on the changed data and sends the caching rules to the gateway unit and the data unit. This enables the gateway unit to delete data from its local cache copy in a timely manner, avoiding the accumulation of invalid cache. At the same time, it enables the storage module to push the updated data to the edge module in a timely manner, improving the timeliness of data updates and the efficiency of user access.

[0061] Based on the above analysis, it can be summarized that the control unit provided in this embodiment includes:

[0062] The configuration module is used to define semantic routing rules and caching rules, and send the caching rules to the gateway unit and the semantic routing rules to the service unit.

[0063] The management module is used to version-mark the caching rules defined by the configuration module and send the version marks to the data unit.

[0064] Furthermore, such as Figure 3As shown, the gateway unit receives user access requests. Upon receiving a user access request, the parsing module first parses the request and adds tags to it. These tags reflect the semantic information mentioned above. For example, after identifying the user's identity based on the access request, a tag named `user_role=VIP` can be added; a tag named `region=East China` can be added based on the request's source IP; and a tag named `resource_type=image` can be added when a request to access a product image is detected. Next, the judgment module determines whether the data corresponding to the user access request is located in the local cache copy of the gateway unit. Specifically, it first iterates through each caching rule to determine whether the data to be accessed corresponding to the user access request is located in the local cache copy. If so, it determines whether the data has expired. If not, the transmission module sends the data to be accessed from the local cache copy to the user. If the data has expired or is not located in the local cache copy, the user access request and tags are sent to the service unit.

[0065] After receiving the user access request and tag from the transmission module, the pass-through module in the service unit needs to select an appropriate service cluster for the user access request based on the semantic routing rules sent by the configuration module. For example, if the user is identified as a member (VIP) user based on the tag, the user access request is sent to the member user service cluster. This service cluster then forwards the user access request to the service instance or the edge module closest to the user. Specifically, the edge module, configured with caching rules, is used to store cached and reusable static data, such as images or static pages. The service instance is used to access dynamically changing data, such as authentication information, to avoid users retrieving outdated information due to caching. The service cluster needs to determine whether the data to be accessed is static or dynamically changing based on the user access request and tag, and sends the user access request accordingly. The data to be accessed is the data to be accessed corresponding to the user access request.

[0066] Each service cluster includes multiple service instances, each with the same or different functions. For example, a member user service cluster includes multiple service instances for querying user personal information, account information, and authentication information. The service cluster sends user access requests to the corresponding service instance. For service instances with high access frequency, in order to avoid a single service instance causing a response timeout due to excessive load, multiple service instances that implement this function need to be configured. In this case, the service cluster needs to select one of the multiple service instances that implement this function based on the status of each service instance and send the user access request to that service instance.

[0067] For example, to further explain the functions of the gateway unit and the service unit, let's take a scenario where a user in Beijing requests access to an image as an example. Assume that the image was replaced before the user's access. The control unit sends a new caching rule 1.1 to the judgment module in the gateway unit, instructing it to delete the image if the original image is still stored in its local cache copy. Specifically, the new caching rule caches the image to each edge module for 24 hours; the gateway unit's local cache copy caches the image for 1 hour. Simultaneously, the control unit sends a new caching rule to the storage module, instructing it to push data from the origin server to each edge module. Assume that the load value of each edge module is less than or equal to the aforementioned load threshold, and since the new caching rule has been received, the image needs to be temporarily stored in each edge module for 24 hours. It's important to note that at this point, the gateway unit only receives the caching rule "the local cache copy caches the image for 1 hour," but the updated image is not yet stored in the local cache copy.

[0068] Immediately afterwards, a user accesses the site. The gateway unit receives the user's access request, the parsing module parses the request and adds tags, and the judgment module determines whether the image is in the local cache copy corresponding to the gateway unit. It's important to note that at this point, the gateway unit only receives the caching rule "the local cache copy caches this image for 1 hour," but the updated image is not yet stored in the local cache copy. Therefore, if... Figure 4 As shown, the transmission module sends the user access request to the service unit. The pass-through module in the service unit determines the corresponding edge module in Beijing that needs to be accessed based on the user access request and tag. The transmission module then sends the user access request to the edge module. At this time, the edge module already has the image stored in it. The edge module directly sends the data required by the user to the user terminal. At the same time as sending the data to the user terminal, the edge module also needs to send the image to the gateway unit. The judgment module stores it in a local cache copy, and according to the caching rules, the storage time is 1 hour.

[0069] Suppose that 30 minutes later, another user accesses the image. Since the gateway unit's local cache copy contains the data, it sends the image to the user without going through the service unit and edge module. Suppose that one hour later, another user accesses the image. The image in the gateway unit's local cache copy has expired, but the image in the edge module has not. The steps from the first user's access must be repeated. If, after 24 hours, the image in the edge module has expired, and a new user accesses the image, the service unit sends the user access request to the edge module. The edge module retrieves the image from the origin server and sends it to the user, storing it within itself for 24 hours according to the caching rules. Simultaneously, the edge module sends the image to the gateway unit, where the decision module stores it in its local cache copy for 1 hour according to the caching rules. The origin server stores all current data.

[0070] In the example above, the user needs to access static images, so the edge module needs to be accessed. If the user needs to access dynamic data, the delivery module in the service unit needs to select a service instance. The service instance will then retrieve data from the database, other services, message queues, or external APIs according to the data type or real-time requirements and send it to the user. For example, structured dynamic data such as user orders can be retrieved from the database, product price change information can be retrieved from the message queue, and logistics information can be retrieved from the external API. However, this embodiment does not limit this, and the service instance can obtain dynamic information from other channels.

[0071] This embodiment sets different access paths for different types of data (dynamic and static data), and sets up a collaborative access mechanism between the gateway unit and the service unit when accessing static data. When a user accesses the data for the first time, the data is already cached in the edge module, and the data can be obtained directly from the edge module, improving access efficiency. On the second access, since the local cache copy already stores the data, there is no need to access the edge module through the service unit, which can further improve access efficiency. In this process, data needs to be obtained from the origin server only when the edge module data expires. Compared with the need to obtain data from the origin server every time, this greatly reduces the pressure on the origin server. Furthermore, this embodiment sets the local cache copy to passively receive and store data only when the gateway unit receives the user access request and the edge module sends data to the user. That is, the local cache copy does not store data when the user accesses the data for the first time. This embodiment can avoid users obtaining incorrect information due to improper active caching, such as prematurely caching of expired data, and will not occupy resources in advance, avoiding resource waste.

[0072] Based on the above analysis, it can be concluded that the gateway unit provided in this embodiment includes:

[0073] The parsing module is used to parse user access requests and add tags to them.

[0074] The judgment module is used to determine whether the data to be accessed corresponding to the user access request is located in the local cache copy corresponding to the gateway unit;

[0075] The transmission module is used to send the corresponding data to be accessed to the user terminal when the data to be accessed corresponding to the user access request is located in the local cache copy corresponding to the gateway unit, and to send the user access request and tag to the service unit when the data to be accessed corresponding to the user access request is not located in the local cache copy corresponding to the gateway unit.

[0076] The service units provided in this embodiment include:

[0077] The pass-through module is used to determine the service cluster or edge module based on the user access request and tag sent by the transmission module in the gateway unit.

[0078] The delivery module is used to send user access requests to the service cluster after determining the service cluster, so that the service cluster selects a service instance from multiple service instances included in the service cluster according to the status of the service instance and sends the user access request to the service instance; or it is used to send user access requests to the edge module after determining the edge module.

[0079] The data unit provided in this embodiment includes:

[0080] The storage module is used to store version tags sent by the management module in the control unit;

[0081] Edge modules are used to send the data to be accessed corresponding to the received user access request to the user terminal. There are multiple edge modules.

[0082] In this embodiment, the edge module can actively obtain data from the source station or receive source station data pushed by other modules. Other modules include the storage module and other edge modules. The above example only describes the cases where the edge module actively obtains data from the source station and the storage module pushes data.

[0083] Furthermore, based on the above example, data can be pushed between each edge module. Specifically, each edge module has its corresponding neighboring edge modules. The neighboring edge modules can be determined by the configuration module and sent to each edge module. For example, the configuration module can divide all edge modules into multiple regions based on the principle of clustering according to the region corresponding to each edge module. Each region includes multiple edge modules, and the edge nodes in each region correspond to regions that are close in location. At this time, the edge modules in each region are neighboring edge modules.

[0084] Each edge module can, at preset time windows, send a cache digest to its neighboring edge nodes whose load values ​​are less than or equal to the load threshold, based on the current load heatmap. The cache digest records the access count, data volume, and data ID of the priority data in its stored data. The data ID is a unique identifier for the data, and priority data refers to the data with the most accesses in its stored data. Upon receiving the cache digest, the neighboring edge module determines whether to store the data. If so, it sends a request to the edge module that sent the cache digest to obtain the priority data and its corresponding caching rules, and stores it according to the caching rules for the corresponding time. For example, the determination process can be as follows: first, it checks if it has already stored the data based on the data ID; then, it checks if its remaining storage capacity is sufficient to store the data. If not, it does not store it. If so, it determines the order of access counts of the priority data among the access counts of all its stored data. For example, it can sort the access counts of all its stored data in descending order and determine the quarter-digit number. If the access count of the priority data is greater than the quarter-digit number, it is stored; otherwise, it is not stored. If adjacent edge modules receive multiple cache digests with the same data ID at the same time, they can randomly select one edge module to send a request when retrieving data.

[0085] This embodiment adds a method for edge nodes to acquire data based on the above example. Through the communication mechanism between edge nodes, a new cache collaboration mechanism based on clustering to divide regions and combining load heatmaps and access frequency sorting is added. By dividing regions, the collaboration scope is narrowed, and priority data is transmitted based on load values ​​to ensure that only high-value data with high access volume is cached. This further increases the probability that the data to be accessed is located in the edge module and reduces the number of times the edge module calls data from the source station.

[0086] Furthermore, such as Figure 5 As shown, this application embodiment provides a method for handling tens of millions of concurrent accesses in elastic load balancing, applied to the aforementioned elastic load balancing system handling tens of millions of concurrent accesses, including:

[0087] Obtain user access requests;

[0088] Based on the user access request and caching rules, the user access request is sent to the service instance or edge module in the service unit; so that the service instance or edge module sends the data to be accessed corresponding to the user access request to the user terminal.

[0089] Specifically, similar to the analysis above, the gateway unit first receives the user access request. If the local edge copy of the gateway unit includes the data to be accessed corresponding to the user access request, the local edge copy sends the data to be accessed to the user. If the local edge copy does not include the data to be accessed corresponding to the user access request, the user access request is sent to the service unit. The service unit determines the type of the data to be accessed. If it is static data, the user access request is sent to the edge module. If it is dynamic data, the user access request is sent to the service instance. This embodiment will not elaborate on this.

[0090] This embodiment provides different access paths through the interaction settings of the gateway unit and the service unit, and improves access efficiency by combining caching rules, thereby reducing the pressure on the origin server.

[0091] Based on the above analysis, it can be summarized that the steps provided in this embodiment for sending user access requests to service instances or edge modules in the service unit according to user access requests and caching rules include:

[0092] Determine whether the data to be accessed is stored in the local cache copy corresponding to the gateway unit;

[0093] If not, the user access request will be sent to the service instance in the service unit or the edge module in the data unit;

[0094] If so, send the data to be accessed corresponding to the user's access request to the user's client.

[0095] The step of sending a user access request to a service instance or edge module in a service unit, as provided in this embodiment, further includes:

[0096] Determine the type of data to be accessed based on the user's access request;

[0097] Based on the type, the user access request is sent to the service instance or edge module.

[0098] Specifically, if the data is static, such as an image, it is sent to the edge module; if the data is dynamic, it is sent to the service instance.

[0099] Furthermore, this embodiment provides a step in which a service unit sends the data to be accessed corresponding to a user access request to the user terminal, including:

[0100] Obtain semantic routing rules, user access requests, and tags;

[0101] The service cluster is determined based on the decision tree model, user access requests, and tags. The decision tree model is determined based on semantic routing rules.

[0102] Based on the status of each service instance in the service cluster, a service instance is selected so that the service instance can send the data to be accessed corresponding to the user's access request to the user's client.

[0103] The step of determining the service cluster is performed in the pass-through module. The content of each node in the decision tree model can be determined according to the priority of the semantic routing rules. For example, the rule with the highest distinguishability among all rules can be used as the root node, such as the request path, because the request path has a high degree of distinguishability to the business domain, that is, the request paths of different business domains (such as products and orders) are significantly different. According to the request path, user access requests can be quickly divided into different branches. Then, all rules can be sorted according to priority, and child nodes can be built step by step according to priority order. Each child node corresponds to a judgment condition, such as whether the user tag is a member user. When building child nodes, the logical relationship between child nodes and parent nodes also needs to be considered. For example, the conditions of child nodes and parent nodes should belong to the attributes of the same object, and the conditions of parent nodes are the premise or upper-level classification of the conditions of child nodes. After all child nodes are obtained, the corresponding service cluster is determined as the leaf node, and the construction of the decision tree is completed. The priority can be determined according to importance (such as rules involving security information have higher priority) and rule update time.

[0104] When a user access request is received, the system starts from the root node and traverses the decision tree downwards based on the user access request and the tag. When a leaf node is reached, the corresponding service cluster is obtained, and the user request is sent to the determined service cluster for processing.

[0105] This embodiment transforms semantic routing rules into a tree-structured conditional judgment, which can quickly determine the service cluster corresponding to a user access request. Furthermore, the decision tree enables incremental updates; when the semantic routing rules change, only the corresponding nodes need to be modified. The method for constructing the decision tree provided in this embodiment is merely an example. The purpose of this embodiment is to provide a method for quickly determining service clusters by transforming semantic routing rules into a decision tree. Those skilled in the art can determine the content corresponding to each node based on semantic routing rules and labels, and select appropriate methods to determine the node's distinguishability and priority; this embodiment does not impose any limitations on this.

[0106] Furthermore, this embodiment provides a step for selecting a service instance based on the status of each service instance in the service cluster, including:

[0107] Obtain the response score of the service cluster and each service instance, and the status score of each service instance;

[0108] The overall score for each service instance is obtained based on the response score of the service cluster and each service instance, and the status score of each service instance.

[0109] Service instances are selected based on the overall score.

[0110] Specifically, when multiple service instances performing the same function exist, one needs to be selected from them. Assuming a user access request is currently located in service cluster A, and a choice needs to be made between service instances B and C, service cluster A needs to obtain the response records from both service instances and calculate the response score. For example, if service cluster A has three response records with service instance B: the first record indicates that service cluster A selected service instance B one hour ago, resulting in successful access; the second record indicates that service cluster A selected service instance B one day ago, resulting in access delay; and the third record indicates that service cluster A selected service instance B three days ago, resulting in access failure. Different scores and weights can be set according to the access type (success, failure, delay) of each record. The weighted score of each record is then used as the response score for service instance B. Preferably, since the time of each record is different, a time decay coefficient can be added to each weight, so that records closer to the current time have a greater impact on the response score. The response score for service instance C can be obtained using the same steps.

[0111] Next, obtain the status score for each service instance. For example, the status score can be determined based on the average response time, access failure rate, etc. of each service instance. The average response time, access failure rate, etc. can be divided into multiple levels, each level corresponds to a score, and different weights can be set for the average response time, access failure rate, etc. Finally, the weighted value is used as the status score.

[0112] This embodiment does not provide specific numerical values ​​for the score and weight. Those skilled in the art can set them according to actual conditions. This embodiment does not impose any restrictions on this. Furthermore, the calculation method for the state score in this embodiment is only an example. Those skilled in the art can choose other standards for calculation. This embodiment does not impose any restrictions on this.

[0113] This embodiment effectively filters out service instances with higher reliability by setting scores and weights based on access type. At the same time, it introduces a time decay coefficient to match dynamic changes in business, ensuring that the selection strategy closely follows the real-time status of service instances and can accurately select the service instance with the best running status to complete the user's access request.

[0114] Furthermore, the aforementioned comprehensive score can be calculated by the service instances themselves. Specifically, when the service cluster receives a user access request, it sends a request instruction to multiple service instances corresponding to the service cluster. Each service instance then calculates its comprehensive score based on its own response records with the service cluster, its average response time, access failure rate, etc. Each service instance then sends its comprehensive score to the service cluster, allowing the service cluster to select the service instance with the highest comprehensive score. Compared to the previous method, this distributes the computational burden across the service instances. The service cluster only needs to compare the final scores. Moreover, since the service instances calculate based on local real-time data, the time spent transmitting data such as average response time and access failure rate to the service cluster is reduced, making the score more relevant to the current state and improving real-time performance.

[0115] This application provides an elastic load balancing method and system for handling tens of millions of concurrent accesses. Through the collaborative cooperation between multiple units, it improves caching efficiency and the accuracy of access request distribution. At the same time, by leveraging the proximity service of the edge module, it optimizes the user access experience and reduces backend pressure.

[0116] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An elastic load balancing system for processing ten million concurrent access, characterized in that, include: The control unit is used to formulate semantic routing rules and caching rules for at least part of the data to be accessed, and send the caching rules to the gateway unit and the semantic routing rules to the service unit. The semantic routing rules are determined according to user identity and access type, and the caching rules include the caching time of the data to be accessed. The control unit includes a configuration module. The gateway unit is used to send the user access request to the service unit according to the user access request and the caching rules; A service unit is used to send the user access request to the edge module in the service instance or data unit according to the user access request and the semantic routing rules. The data unit is used to store cached data and send the corresponding data to be accessed to the user terminal according to the user access request. The gateway unit includes a transmission module, configured to send the corresponding data to be accessed to the user terminal when the data to be accessed corresponding to the user access request is located in the local cache copy corresponding to the gateway unit, and to send the user access request and tag to the service unit when the data to be accessed corresponding to the user access request is not located in the local cache copy corresponding to the gateway unit; the local cache copy stores data when the gateway unit receives the user access request and the edge module sends the data to be accessed corresponding to the user access request to the user terminal; The data unit includes an edge module and a storage module. The storage module pushes data to the edge module according to the load heatmap. Each edge module corresponds to multiple adjacent edge modules, which are determined by the configuration module. Every preset time window, each edge module sends a cache digest to its adjacent edge nodes whose load values ​​are less than or equal to the load threshold, based on the current load heatmap. The cache digest includes the access count, data volume, and data ID of the priority data stored by the edge module itself. This allows the adjacent edge module to first determine whether it has already stored the data based on the data ID, then determine whether its remaining storage capacity is sufficient to store the data. If so, it determines the order of the access count of the priority data among the access counts of all its stored data, and finally determines the priority data to be stored based on the order.

2. The elastic load balancing ten-million-level concurrent access processing system according to claim 1, wherein, The control unit includes: The configuration module is used to formulate the semantic routing rules and the caching rules, and send the caching rules to the gateway unit and the semantic routing rules to the service unit; The management module is used to version-mark the caching rules defined by the configuration module and send the version marks to the data unit.

3. The elastic load balancing hundred-million-level concurrent access processing system according to claim 1, wherein, The gateway unit includes: The parsing module is used to parse the user access request and add tags to the user access request. The judgment module is used to determine whether the data to be accessed corresponding to the user access request is located in the local cache copy corresponding to the gateway unit; The transmission module is configured to transmit the to-be-accessed data corresponding to the user access request to the user terminal when the to-be-accessed data corresponding to the user access request is located in the local cache copy corresponding to the gateway unit, and transmit the user access request and the tag to the service unit when the to-be-accessed data corresponding to the user access request is not located in the local cache copy corresponding to the gateway unit.

4. The elastic load balancing ten-million-level concurrent access processing system according to claim 3, wherein, The service unit comprises: The transmission module is configured to transmit the to-be-accessed data corresponding to the user access request to the user terminal when the to-be-accessed data corresponding to the user access request is located in the local cache copy corresponding to the gateway unit, and transmit the user access request and the tag to the service unit when the to-be-accessed data corresponding to the user access request is not located in the local cache copy corresponding to the gateway unit. The service unit comprises:

5. The elastic load balancing hundred-million-level concurrent access processing system according to claim 1, wherein, The storage module is configured to store the version mark transmitted by the management module of the control unit. The edge module is configured to transmit the to-be-accessed data corresponding to the user access request to the user terminal according to the received user access request, and the edge module is multiple. Comprise:

6. The method of claim 1-5, wherein the method is implemented by the system of claim 1-5. Obtain a user access request; According to the user access request and the cache rule, transmit the user access request to a service instance in a service unit or an edge module; So that the service instance or the edge module transmits the to-be-accessed data corresponding to the user access request to the user terminal. According to the user access request and the cache rule, transmitting the user access request to a service instance in a service unit or an edge module comprises:

7. The method of claim 6, wherein the method further comprises: Determine whether the to-be-accessed data is stored in the local cache copy corresponding to the gateway unit; If not, transmit the user access request to a service instance in a service unit or an edge module in a data unit; If yes, transmit the to-be-accessed data corresponding to the user access request to the user terminal. Transmitting the user access request to a service instance in a service unit or an edge module further comprises:

8. The method of claim 7, wherein the method further comprises: Determine the type of the to-be-accessed data according to the user access request; According to the type, transmit the user access request to the service instance or the edge module. Transmitting the to-be-accessed data corresponding to the user access request to the user terminal comprises:

9. The method of claim 7, wherein the method further comprises: Obtain a semantic routing rule, the user access request and a tag; According to a decision tree model, the user access request and the tag, determine a service cluster, the decision tree model is determined according to the semantic routing rule; According to the state of each service instance in the service cluster, select a service instance, so that the service instance transmits the to-be-accessed data corresponding to the user access request to the user terminal. The service cluster comprises:

10. The method of claim 9, wherein the method further comprises: Obtain the response score of the service cluster and each service instance and the state score of each service instance; According to the response score of the service cluster and each service instance and the state score of each service instance, obtain the comprehensive score of each service instance; According to the response score of the service cluster and each service instance and the state score of each service instance, obtain the comprehensive score of each service instance; selecting a service instance according to the composite score.

Images