Data processing method, electronic device, storage medium, and program product

Abstract

The application discloses a data processing method, an electronic device, a storage medium and a program product. The method comprises the following steps: in response to receiving a data request, obtaining a data amount of the data request and a current flow limit value of a flow limiter, wherein the current flow limit value is used to represent an upper limit value of a data amount currently processed by the flow limiter; determining a flow interaction state of the flow limiter based on the data amount and the current flow limit value, wherein the flow interaction state is used to represent a state of whether the flow limiter and a flow cache server perform flow interaction; and forwarding the data request to a data processing end based on the flow interaction state and the current flow limit value. The application solves the technical problem that the related art has a strong dependence risk on the flow cache server.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more specifically, to a data processing method, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid development of the aviation business, the number of users and business requests has increased dramatically, putting unprecedented pressure on the server cluster. To ensure service stability and response speed, effective control of request traffic has become crucial.

[0003] Traditional rate limiting techniques rely on third-party caching storage services for traffic allocation and control. However, each request requires interaction with the caching server, which not only increases network latency but may also cause the caching server to become a system bottleneck. Furthermore, the unavailability of the caching server will directly cause the rate limiting function to fail, thereby affecting the stable operation of the entire cluster. This leads to a strong dependence on traffic caching servers in related technologies.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a data processing method, electronic device, storage medium, and program product to at least address the technical problem of strong dependence on traffic caching servers in related technologies.

[0006] According to one aspect of the present invention, a data processing method is provided, comprising: in response to receiving a data request, acquiring the data volume of the data request and the current traffic limit of a rate limiter, wherein the current traffic limit is used to represent the upper limit of the data volume currently processed by the rate limiter; determining the traffic interaction state of the rate limiter based on the data volume and the current traffic limit, wherein the traffic interaction state is used to represent whether the rate limiter is interacting with a traffic caching server; and forwarding the data request to a data processing terminal based on the traffic interaction state and the current traffic limit.

[0007] Furthermore, based on the data volume and the current traffic limit, the traffic interaction state of the rate limiter is determined, including: determining the traffic request limit of the rate limiter based on the difference between the current traffic limit and a first preset value, wherein the traffic request limit is used to represent the limit for the rate limiter to request traffic from the traffic caching server; determining the traffic return limit of the rate limiter based on the difference between the current traffic limit and a second preset value, wherein the second preset value is greater than the first preset value, and the traffic return limit is used to represent the limit for the rate limiter to return traffic to the traffic caching server; and determining the traffic interaction state based on the data volume, the traffic request limit, and the traffic return limit.

[0008] Furthermore, the method also includes: determining a second preset value based on the product of a preset multiple and a flow interaction step size.

[0009] Furthermore, the traffic interaction state is determined based on the data volume, traffic request limit, and traffic return limit, including: in response to the data volume being greater than or equal to the traffic request limit, the traffic interaction state is determined to be a traffic request state, wherein the traffic request state is used to indicate the state of traffic interaction between the rate limiter and the traffic caching server, and the state of the rate limiter requesting traffic from the traffic caching server; in response to the data volume being less than or equal to the traffic return limit, the traffic interaction state is determined to be a traffic return state, wherein the traffic return state is used to indicate the state of traffic interaction between the rate limiter and the traffic caching server, and the state of the rate limiter returning traffic to the traffic caching server; in response to the data volume being less than the traffic request limit but greater than the traffic return limit, the traffic interaction state is determined to be a preset traffic state, wherein the preset traffic state is used to indicate the state of no traffic interaction between the rate limiter and the traffic caching server.

[0010] Furthermore, forwarding data requests to the data processing end based on the traffic interaction status and the current traffic limit includes: responding to the traffic interaction status being either a traffic request status or a traffic return status, updating the current traffic limit based on the traffic interaction step size to obtain the updated current traffic limit, and forwarding the data request to the data processing end based on the updated current traffic limit; and responding to the traffic interaction status being a preset traffic status, forwarding the data request to the data processing end based on the current traffic limit.

[0011] Furthermore, in response to the traffic interaction state being either a traffic request state or a traffic return state, the current traffic limit is updated based on the traffic interaction step size to obtain the updated current traffic limit. This includes: in response to the traffic interaction state being a traffic request state, controlling the rate limiter to request a preset traffic corresponding to the traffic interaction step size from the traffic caching server, and adding the preset traffic to the current traffic limit to obtain the updated current traffic limit; in response to the traffic interaction state being a traffic return state, controlling the rate limiter to return the preset traffic corresponding to the traffic interaction step size to the traffic caching server, and reducing the preset traffic to the current traffic limit to obtain the updated current traffic limit.

[0012] Furthermore, the method also includes: in response to receiving a fault instruction from the traffic caching server, forwarding the data request to the data processing end based on the current traffic limit.

[0013] According to another aspect of the present invention, a data processing apparatus is also provided, comprising: an acquisition module, configured to, in response to receiving a data request, acquire the data volume of the data request and the current traffic limit of a rate limiter, wherein the current traffic limit is used to represent the upper limit of the data volume currently processed by the rate limiter; a determination module, configured to determine the traffic interaction state of the rate limiter based on the data volume and the current traffic limit, wherein the traffic interaction state is used to represent whether the rate limiter and the traffic caching server are performing traffic interaction; and a forwarding module, configured to forward the data request to a data processing terminal based on the traffic interaction state and the current traffic limit.

[0014] According to another aspect of the present invention, an electronic device is also provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the above-described data processing method during runtime.

[0015] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the above-described data processing method.

[0016] According to another aspect of the present invention, a computer program product is also provided, including a computer program that implements the above-described data processing method when executed by a processor.

[0017] In this embodiment of the invention, when the system receives a data request, it first obtains the data volume of the data request and the current traffic limit of the rate limiter; then, based on the obtained data volume and the current traffic limit, it determines the traffic interaction state of the rate limiter; finally, based on the traffic interaction state and the current traffic limit, it forwards the data request to the data processing end. It is noteworthy that this application clarifies the upper limit of the current data volume that the rate limiter can process by obtaining the current traffic limit of the rate limiter; then, it compares the current traffic limit with the data volume to determine whether the rate limiter can currently process all the data, thereby obtaining the traffic interaction state of the rate limiter. Based on this traffic interaction state, it can determine whether the rate limiter needs to interact with the traffic caching server, achieving precise control over the interaction state between the rate limiter and the traffic caching server. This reduces dependence on the traffic caching server, improves the efficiency of traffic control and the processing capacity of concurrent requests, and solves the technical problem of strong dependence on traffic caching servers in related technologies. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present invention;

[0020] Figure 2 This is a flowchart of an optional data processing method according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of a data processing apparatus according to an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] According to an embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0025] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0026] Step S102: In response to receiving a data request, obtain the data volume of the data request and the current flow limit of the flow limiter, wherein the current flow limit is used to represent the upper limit of the current data volume processed by the flow limiter.

[0027] The aforementioned data request can refer to a data acquisition or processing request sent by a client or downstream service to a server or upstream service. The types of data requests can include, but are not limited to, data query requests and data acquisition requests. The specific data request needs to be determined according to the actual needs of the user or system, and is not limited here. Data requests can be used to obtain data required by the system or user.

[0028] The aforementioned data volume can refer to the amount of data that needs to be processed in a data request. The types of data volume can include, but are not limited to, the size and number of data packets and the frequency of data requests. The specific data volume needs to be determined based on the complexity of the data request, and is not limited here.

[0029] The aforementioned rate limiter can refer to a mechanism for controlling the flow of requests or data entering the system. The types of rate limiters can include, but are not limited to, local rate limiters, time window-based rate limiters, and request count-based rate limiters. The specific type of rate limiter needs to be determined according to actual needs and is not limited here. The number of rate limiters can include, but is not limited to, 1, 2, 3, 4, 5, etc. The specific number of rate limiters needs to be determined according to the data volume and system design and is not limited here. The functions of the rate limiter can include, but are not limited to, flow control, elastic scaling, fault isolation, and preventing system overload to ensure stable system operation.

[0030] The aforementioned current traffic limit refers to the upper limit of the amount of data requests that the rate limiter can process at the current moment. The current traffic limit can be used to restrict and manage the amount of data requests entering the data processing end, to prevent the server from being unable to respond or experiencing response delays due to overload, and to ensure the robustness and response speed of the system.

[0031] In one optional embodiment, when the system receives a data request from a user or downstream service, the rate limiter responds quickly and obtains information on the amount of data involved in the request, while also reading the current traffic limit. Through this mechanism, the rate limiter can monitor and control the amount of data entering the system in real time, avoiding server performance degradation or service interruption due to data request overload, thereby ensuring the stability and response speed of the ticket query service.

[0032] Step S104: Based on the data volume and the current traffic limit, determine the traffic interaction status of the rate limiter, wherein the traffic interaction status is used to indicate whether the rate limiter and the traffic caching server are interacting with each other.

[0033] The aforementioned traffic interaction status refers to whether the rate limiter needs to interact with the traffic caching server when processing data requests, based on the current traffic limit and the real-time data volume. The types of traffic interaction status may include, but are not limited to, no operation status, traffic request status, and traffic return status. The specific traffic interaction status needs to be determined based on the comparison result between the data volume and the current traffic limit, which is not limited here. The function of the traffic interaction status may include, but is not limited to, clarifying the needs of the rate limiter based on the current processing status of the rate limiter, and then making the next judgment.

[0034] The aforementioned traffic caching server can refer to the central node that stores and manages the traffic configuration of the entire cluster. The functions of the traffic caching server may include, but are not limited to, responding to traffic requests and return requests from rate limiters, dynamically adjusting the traffic quotas of each rate limiter, and maintaining the traffic balance of the cluster.

[0035] In one optional embodiment, the rate limiter assesses the relationship between the data volume and the current traffic limit to determine whether traffic interaction with the traffic caching server is necessary. The traffic caching server, acting as the central hub for cluster traffic configuration, responds to the rate limiter's requests to achieve intelligent traffic allocation and stable cluster operation. This process, through the cooperation of the rate limiter and the traffic caching server, reduces reliance on network communication, improves system response speed and resource utilization, and enhances the system's fault tolerance and scalability.

[0036] Step S106: Based on the traffic interaction status and the current traffic limit, forward the data request to the data processing end.

[0037] The aforementioned data processing end can refer to a backend component or service in the system responsible for receiving data requests forwarded by the rate limiter and executing specific business logic. The functions of the data processing end may include, but are not limited to, data querying and retrieval, data updating and maintenance, and service response generation.

[0038] In one optional embodiment, the rate limiter forwards data requests to the data processing end based on the traffic interaction status and the current traffic limit. The rate limiter's traffic control reduces invalid processing and waiting time at the data processing end, improving the efficiency of data retrieval and processing. Simultaneously, by dynamically adjusting the traffic limit, the data processing end can better adapt to changing business needs, avoiding resource waste or insufficiency, and enhancing the system's flexibility and performance. Furthermore, the decoupling between the rate limiter and the data processing end ensures that the system can still provide stable service even under poor network conditions or high server load, significantly improving user experience and overall system reliability.

[0039] In one alternative embodiment, Figure 2This is a flowchart of an optional data processing method according to an embodiment of the present invention, such as... Figure 2 As shown, firstly, multiple user requests are obtained; then, the user requests are evenly distributed to multiple rate limiters through load balancing; finally, the rate limiters renew traffic with the traffic caching server.

[0040] In the above process, the traffic limiter renews traffic with the traffic caching server based on the data volume and the current traffic limit. This determines the traffic interaction state of the rate limiter, and then determines how to renew the traffic with the traffic caching server based on this state. Renewal means that the rate limiter requests a certain amount of traffic limit from the traffic caching server (the current traffic limit), and then performs traffic calculation and control locally. All requests no longer interact with the caching server; calculations are only performed on the local server. When the local concurrency exceeds the current traffic limit, the rate limiter interacts with the caching server again to request another amount of traffic limit. This new amount is added to the local traffic limit to increase the total traffic limit, thus achieving a higher concurrency.

[0041] In this embodiment of the invention, when the system receives a data request, it first obtains the data volume of the data request and the current traffic limit of the rate limiter; then, based on the obtained data volume and the current traffic limit, it determines the traffic interaction state of the rate limiter; finally, based on the traffic interaction state and the current traffic limit, it forwards the data request to the data processing end. It is noteworthy that this application clarifies the upper limit of the current data volume that the rate limiter can process by obtaining the current traffic limit of the rate limiter; then, it compares the current traffic limit with the data volume to determine whether the rate limiter can currently process all the data, thereby obtaining the traffic interaction state of the rate limiter. Based on this traffic interaction state, it can determine whether the rate limiter needs to interact with the traffic caching server, achieving precise control over the interaction state between the rate limiter and the traffic caching server. This reduces dependence on the traffic caching server, improves the efficiency of traffic control and the processing capacity of concurrent requests, and solves the technical problem of strong dependence on traffic caching servers in related technologies.

[0042] Optionally, the traffic interaction state of the rate limiter is determined based on the data volume and the current traffic limit, including: determining the traffic request limit of the rate limiter based on the difference between the current traffic limit and a first preset value, wherein the traffic request limit is used to represent the limit for the rate limiter to request traffic from the traffic caching server; determining the traffic return limit of the rate limiter based on the difference between the current traffic limit and a second preset value, wherein the second preset value is greater than the first preset value, and the traffic return limit is used to represent the limit for the rate limiter to return traffic to the traffic caching server; and determining the traffic interaction state based on the data volume, the traffic request limit, and the traffic return limit.

[0043] The aforementioned first preset value can refer to a threshold standard used to trigger traffic request operations. The type of the first preset value can include, but is not limited to, a fixed first preset value, a dynamic first preset value, a proportional first preset value, etc. The specific first preset value needs to be determined according to the data volume type and actual situation. The size of the first preset value can include, but is not limited to, 5, 10, 15, etc. The specific first preset value needs to be determined according to the actual situation and is not limited here. The first preset value can be used as a judgment threshold for triggering traffic request operations, etc.

[0044] The aforementioned traffic request limit can refer to the specific value of the additional traffic quota requested by the rate limiter from the traffic caching server. The traffic request limit can be determined based on a comprehensive assessment of the system's processing capacity, business needs, and the status of the traffic caching server to ensure the balanced traffic of the cluster and the rational utilization of resources.

[0045] The aforementioned second preset value can refer to a threshold standard used to trigger traffic return operations. The type of the second preset value can include, but is not limited to, a fixed second preset value, a dynamic second preset value, a proportional second preset value, etc. The specific second preset value needs to be determined according to the data volume type and actual situation. The size of the second preset value can include, but is not limited to, 30, 35, 40, etc. The specific second preset value needs to be determined according to the actual situation and is not limited here. The function of the second preset value can include, but is not limited to, using it as a judgment standard to trigger traffic return operations, avoiding excessive concentration and waste of resources, and ensuring that the traffic resources of the entire cluster are optimally allocated.

[0046] The aforementioned traffic return limit can refer to the amount of traffic quota that the rate limiter considers returning to the traffic caching server when the data volume is lower than the difference between the current traffic limit and the second preset value. The traffic return limit can be used to release unused traffic quotas when the actual load of the rate limiter is much lower than its capacity, optimize resource allocation, and at the same time avoid the impact of rapid fluctuations in traffic quotas on system stability.

[0047] In one optional embodiment, a first preset value and a second preset value are first set. Then, based on the difference between the current traffic limit and the first preset value, the limit for the rate limiter to request traffic from the traffic caching server is determined. Based on the difference between the current traffic limit and the second preset value, the limit for the rate limiter to return traffic to the traffic caching server is determined. Through the above process, by setting the first preset value and the second preset value, the rate limiter can request additional traffic in a timely manner, ensuring the continuity and stability of data processing in high-concurrency scenarios. At the same time, it can also enable the rate limiter to return excess traffic, optimize resource allocation, and improve the processing efficiency and response time of the entire cluster.

[0048] Optionally, the method further includes: determining a second preset value based on the product of a preset multiple and a flow interaction step size.

[0049] The aforementioned preset multiple can refer to a fixed or dynamic multiplier used to calculate the second preset value when setting the traffic limiting rules. The preset multiple can include, but is not limited to, 2, 3, 4, etc. The specific preset multiple needs to be determined based on the current traffic limit and data volume, and is not limited here. The function of the preset multiple can include, but is not limited to, measuring the distance between the traffic limit and the traffic return trigger point.

[0050] In one optional embodiment, if the preset multiplier is 2, it means that when the difference between the current traffic limit of the flow limiter and its real-time data volume reaches twice the traffic interaction step size, a traffic return operation will be triggered to return part of the traffic quota to the traffic caching server.

[0051] The aforementioned traffic interaction step size refers to the fixed value of traffic requested or returned each time the rate limiter interacts with the traffic caching server. The type of traffic interaction step size can include, but is not limited to, fixed traffic interaction step size and dynamic traffic interaction step size. The specific traffic interaction step size needs to be determined according to the application scenario, and is not limited here. The traffic interaction step size can be used to adjust the allocation efficiency of traffic resources and the ability to dynamically adjust them.

[0052] In one optional embodiment, the second preset value is determined by the product of a preset multiple and a traffic interaction step size. This ensures that the rate limiter can effectively utilize resources and avoid traffic idleness when processing data requests, while also responding promptly to increased traffic demand to prevent resource strain. At the same time, by dynamically adjusting these parameters, the system can intelligently adjust traffic quotas according to changes in actual business scenarios and network environment, improve resource utilization, reduce server pressure, and ensure continuous, stable, and efficient service response.

[0053] In one optional embodiment, when the preset multiplier is 2 and the traffic interaction step size is 10, the second preset value is twice the traffic interaction step size, i.e., 20 traffic units. This means that when the data volume is lower than the difference between the traffic limit and the second preset value, the rate limiter will consider returning the traffic quota to the traffic caching server. This difference is dynamically calculated based on the traffic interaction step size and can be adjusted according to the real-time status of the system.

[0054] Optionally, the traffic interaction state is determined based on the data volume, traffic request limit, and traffic return limit, including: in response to the data volume being greater than or equal to the traffic request limit, determining the traffic interaction state as a traffic request state, wherein the traffic request state indicates that the rate limiter and the traffic caching server are interacting, and the rate limiter is requesting traffic from the traffic caching server; in response to the data volume being less than or equal to the traffic return limit, determining the traffic interaction state as a traffic return state, wherein the traffic return state indicates that the rate limiter and the traffic caching server are interacting, and the rate limiter is returning traffic to the traffic caching server; in response to the data volume being less than the traffic request limit and greater than the traffic return limit, determining the traffic interaction state as a preset traffic state, wherein the preset traffic state indicates that the rate limiter and the traffic caching server are not interacting.

[0055] The aforementioned traffic request status refers to a state in which the system needs to request additional traffic quotas from the traffic caching server when the amount of data monitored by the rate limiter reaches or exceeds its traffic request limit. The traffic request status can be used to ensure that the rate limiter can request additional traffic quotas in a timely manner to meet the sudden increase in business demand and avoid service interruption and response delay.

[0056] The aforementioned traffic return status refers to a state in which the system needs to return a portion of the traffic quota to the traffic caching server when the amount of data monitored by the rate limiter drops to equal to or below its traffic return limit. The traffic return status can be used to optimize resource allocation within the cluster, avoid excessive concentration and waste of traffic quotas, and improve the overall resource utilization and response speed of the system.

[0057] The aforementioned preset traffic state refers to a state in which the system does not need to interact with the traffic caching server when the data volume is lower than the traffic request limit and higher than the traffic return limit. The preset traffic state can be used to reduce the interaction between the system and the traffic caching server under stable traffic demand, reduce network communication latency and resource consumption, and improve system operating efficiency.

[0058] In one optional embodiment, the traffic interaction status between the rate limiter and the traffic caching server is determined by comparing the dynamically monitored data volume with preset traffic request limits and traffic return limits. When the data volume reaches or exceeds the traffic request limit, the system enters the traffic request state, and the rate limiter requests additional traffic quotas from the traffic caching server to cope with sudden high traffic demands, ensuring service continuity and response speed. Conversely, when the data volume drops to or equals the traffic return limit, the system enters the traffic return state, and the rate limiter returns unused traffic quotas to the traffic caching server, optimizing resource allocation, avoiding idle and wasted traffic resources, and ensuring that each node in the cluster can obtain reasonable traffic support. In the stable traffic range where the data volume is below the traffic request limit but above the traffic return limit, the system remains in the preset traffic state, and there is no traffic interaction between the rate limiter and the traffic caching server, reducing unnecessary network communication and resource overhead, and improving system operating efficiency and stability. This intelligent traffic control strategy based on real-time data volume not only enhances the system's flexibility and dynamic resource optimization capabilities, but also significantly reduces the dependence on traffic caching servers, improves the performance and reliability of ticket query services, and provides end users with a smoother and more efficient service experience.

[0059] Optionally, forwarding the data request to the data processing terminal based on the traffic interaction status and the current traffic limit includes: in response to the traffic interaction status being either a traffic request status or a traffic return status, updating the current traffic limit based on the traffic interaction step size to obtain the updated current traffic limit, and forwarding the data request to the data processing terminal based on the updated current traffic limit; in response to the traffic interaction status being a preset traffic status, forwarding the data request to the data processing terminal based on the current traffic limit.

[0060] The updated current traffic limit mentioned above refers to the upper limit of the traffic processing capacity that the rate limiter has adjusted according to the traffic interaction step size after the traffic request or traffic return operation. The updated current traffic limit can be used to change the rate limiter's processing capacity and efficiency for data requests, etc.

[0061] In one optional embodiment, different data forwarding processes are performed based on different traffic interaction states. When traffic demand increases, the rate limiter can promptly request more traffic quotas to avoid service interruptions and response delays; when traffic demand decreases, the rate limiter can return unused traffic quotas to avoid resource waste and optimize resource allocation within the cluster. Through the dynamic updating of the current traffic limit, the rate limiter can automatically adapt to real-time changes in traffic demand, improving resource utilization and system response speed.

[0062] Optionally, in response to the traffic interaction state being either a traffic request state or a traffic return state, the current traffic limit is updated based on the traffic interaction step size to obtain the updated current traffic limit. This includes: in response to the traffic interaction state being a traffic request state, controlling the rate limiter to request a preset traffic corresponding to the traffic interaction step size from the traffic caching server, and adding the preset traffic to the current traffic limit to obtain the updated current traffic limit; in response to the traffic interaction state being a traffic return state, controlling the rate limiter to return the preset traffic corresponding to the traffic interaction step size to the traffic caching server, and reducing the preset traffic to the current traffic limit to obtain the updated current traffic limit.

[0063] The aforementioned preset traffic can refer to a specific traffic value requested or returned when the rate limiter interacts with the traffic caching server during traffic request or return states. The types of preset traffic can include, but are not limited to, fixed preset traffic, dynamic preset traffic, and fine-tuning preset traffic. The specific type of preset traffic needs to be determined based on the current traffic state and is not limited here. The function of preset traffic can include, but is not limited to, dynamically adjusting the processing capacity of the rate limiter, balancing resources, and optimizing system performance.

[0064] In one optional embodiment, when the system detects a traffic request status, the rate limiter automatically requests a preset traffic amount corresponding to the traffic interaction step size from the traffic caching server. This operation increases the preset traffic amount based on the current traffic limit, forming an updated current traffic limit. This ensures that when traffic demand increases, the rate limiter can quickly increase its processing capacity, avoid service overload, and improve system response speed and user experience. Conversely, when a traffic return status is identified, the rate limiter returns the preset traffic amount to the traffic caching server, reducing the current traffic limit, avoiding resource waste, promoting balanced resource allocation within the cluster, and ensuring stable operation and efficient resource utilization of the overall system.

[0065] Optionally, the method further includes: in response to receiving a fault instruction from the traffic caching server, forwarding the data request to the data processing end based on the current traffic limit.

[0066] The aforementioned fault instruction can refer to a special instruction received by the system or rate limiter when the traffic caching server fails or becomes unavailable. The types of fault instructions may include, but are not limited to, hardware fault instructions, software exception instructions, and network interruption instructions. The specific fault instruction should be determined according to the actual situation of the system, and is not limited here. Fault instructions can be used for emergency handling based on fault instructions, thereby avoiding the avalanche effect caused by caching server failure and protecting the stability of the entire cluster.

[0067] In one optional embodiment, upon receiving a system fault command, the current traffic limit is forwarded to the data processing end. This ensures that the rate limiter can process data requests based on the existing traffic limit during a fault, thereby improving system stability and reliability, avoiding overall service interruption due to a single point of failure, and providing users with a more continuous and stable service experience. Simultaneously, the fault command can guide the system to promptly synchronize data and adjust policies after the fault is recovered, ensuring rapid service recovery and long-term stable operation.

[0068] According to another aspect of the present invention, a data processing apparatus is also provided, which can execute the data processing method of the above embodiments. The specific implementation method and preferred application scenarios are the same as those of the above embodiments, and will not be described in detail here.

[0069] Figure 3 This is a schematic diagram of a data processing apparatus according to an embodiment of the present invention, such as... Figure 3 As shown, the device includes the following: an acquisition module 302, a determination module 304, and a forwarding module 306.

[0070] The acquisition module 302 is used to acquire the data volume of the data request and the current traffic limit of the rate limiter in response to receiving a data request, wherein the current traffic limit is used to represent the upper limit of the data volume that the rate limiter can currently process; the determination module 304 is used to determine the traffic interaction status of the rate limiter based on the data volume and the current traffic limit, wherein the traffic interaction status is used to indicate whether the rate limiter is interacting with the traffic caching server; the forwarding module 306 is used to forward the data request to the data processing end based on the traffic interaction status and the current traffic limit.

[0071] Optionally, the determining module includes: a first determining unit, configured to determine a traffic request limit for the rate limiter based on the difference between the current traffic limit and a first preset value, wherein the traffic request limit represents the limit for the rate limiter to request traffic from the traffic caching server; a second determining unit, configured to determine a traffic return limit for the rate limiter based on the difference between the current traffic limit and the second preset value, wherein the second preset value is greater than the first preset value, and the traffic return limit represents the limit for the rate limiter to return traffic to the traffic caching server; and a third determining unit, configured to determine the traffic interaction state based on the data volume, the traffic request limit, and the traffic return limit.

[0072] Optionally, the device further includes: determining a second preset value based on the product of a preset multiple and a flow interaction step size.

[0073] Optionally, the third determining unit includes: a first determining subunit, configured to determine the traffic interaction state as a traffic request state in response to the data volume being greater than or equal to the traffic request limit, wherein the traffic request state indicates that the rate limiter and the traffic caching server are interacting with each other, and the rate limiter is requesting traffic from the traffic caching server; a second determining subunit, configured to determine the traffic interaction state as a traffic return state in response to the data volume being less than or equal to the traffic return limit, wherein the traffic return state indicates that the rate limiter and the traffic caching server are interacting with each other, and the rate limiter is returning traffic to the traffic caching server; and a third determining subunit, configured to determine the traffic interaction state as a preset traffic state in response to the data volume being less than the traffic request limit and greater than the traffic return limit, wherein the preset traffic state indicates that the rate limiter and the traffic caching server are not interacting with each other.

[0074] Optionally, the forwarding module includes: a first forwarding unit, configured to update the current traffic limit based on the traffic interaction step size in response to a traffic interaction state of traffic request or traffic return, obtain the updated current traffic limit, and forward the data request to the data processing end based on the updated current traffic limit; and a second forwarding unit, configured to forward the data request to the data processing end in response to a traffic interaction state of preset traffic state based on the current traffic limit.

[0075] Optionally, the first forwarding unit includes: a first control subunit, configured to, in response to a traffic interaction state of traffic request state, control the rate limiter to request a preset traffic corresponding to the traffic interaction step size from the traffic caching server, and add the preset traffic to the current traffic limit to obtain an updated current traffic limit; and a second control subunit, configured to, in response to a traffic interaction state of traffic return state, control the rate limiter to return the preset traffic corresponding to the traffic interaction step size to the traffic caching server, and reduce the preset traffic to the current traffic limit to obtain an updated current traffic limit.

[0076] Optionally, the device further includes: in response to receiving a fault instruction from the traffic caching server, forwarding the data request to the data processing end based on the current traffic limit.

[0077] According to another aspect of the present invention, an electronic device is also provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present invention during runtime.

[0078] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the execution of the methods of various embodiments of the present invention in a processor of the device.

[0079] The computer storage medium mentioned in the above steps can be a medium used in computer memory to store certain discontinuous physical quantities. Computer storage media mainly include semiconductors, magnetic cores, magnetic drums, magnetic tapes, and laser discs. Computer-readable storage media includes stored programs, which can be a set of instructions that a computer can recognize and execute, running on an electronic computer to meet certain human needs—an information tool.

[0080] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0081] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0085] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0086] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that, include: In response to receiving a data request, the system obtains the amount of data requested and the current traffic limit of the rate limiter, wherein the current traffic limit is used to represent the upper limit of the amount of data that the rate limiter is currently processing; Based on the data volume and the current traffic limit, the traffic interaction state of the rate limiter is determined. The traffic interaction state is used to indicate whether the rate limiter interacts with the traffic caching server. The traffic interaction state includes at least one of the following: preset traffic state, traffic request state, and traffic return state. The preset traffic state is used to indicate that the rate limiter does not need to interact with the traffic caching server. Based on the traffic interaction status and the current traffic limit, the data request is forwarded to the data processing terminal; In response to receiving a fault instruction from the traffic caching server, the data request is forwarded to the data processing terminal based on the current traffic limit, wherein the fault instruction is used to indicate an instruction issued when the traffic caching server fails or becomes unavailable.

2. The data processing method according to claim 1, characterized in that, Based on the data volume and the current traffic limit, the traffic interaction state of the flow limiter is determined, including: Based on the difference between the current traffic limit and the first preset value, the traffic request limit of the rate limiter is determined, wherein the traffic request limit is used to represent the limit of the rate limiter requesting traffic from the traffic caching server; Based on the difference between the current traffic limit and the second preset value, the traffic return limit of the traffic limiter is determined, wherein the second preset value is greater than the first preset value, and the traffic return limit is used to represent the limit of the traffic limiter returning traffic to the traffic caching server; The traffic interaction status is determined based on the data volume, the traffic request limit, and the traffic return limit.

3. The data processing method according to claim 2, characterized in that, The method further includes: The second preset value is determined based on the product of a preset multiple and a flow interaction step size.

4. The data processing method according to claim 2, characterized in that, Determining the traffic interaction status based on the data volume, the traffic request limit, and the traffic return limit includes: In response to the data volume being greater than or equal to the traffic request limit, the traffic interaction state is determined to be the traffic request state, wherein the traffic request state is used to indicate the state of traffic interaction between the rate limiter and the traffic caching server, and the state of the rate limiter requesting traffic from the traffic caching server; In response to the data volume being less than or equal to the traffic return limit, the traffic interaction state is determined to be the traffic return state, wherein the traffic return state is used to indicate the state of traffic interaction between the rate limiter and the traffic caching server, and the rate limiter returns traffic to the traffic caching server. In response to the data volume being less than the traffic request limit and greater than the traffic return limit, the traffic interaction state is determined to be the preset traffic state, wherein the preset traffic state is used to indicate that the rate limiter and the traffic caching server have not interacted in terms of traffic.

5. The data processing method according to any one of claims 1 to 4, characterized in that, Based on the traffic interaction status and the current traffic limit, the data request is forwarded to the data processing terminal, including: In response to the traffic interaction state being either a traffic request state or a traffic return state, the current traffic limit is updated based on the traffic interaction step size to obtain the updated current traffic limit, and the data request is forwarded to the data processing terminal based on the updated current traffic limit; In response to the traffic interaction state being a preset traffic state, the data request is forwarded to the data processing terminal based on the current traffic limit.

6. The data processing method according to claim 5, characterized in that, In response to the traffic interaction state being either the traffic request state or the traffic return state, the current traffic limit is updated based on the traffic interaction step size to obtain the updated current traffic limit, including: In response to the traffic interaction state being the traffic request state, the flow limiter is controlled to request a preset traffic corresponding to the traffic interaction step size from the traffic caching server, and the preset traffic is added to the current traffic limit to obtain the updated current traffic limit. In response to the traffic interaction state being the traffic return state, the flow limiter is controlled to return the preset traffic corresponding to the traffic interaction step size to the traffic caching server, and the preset traffic is reduced based on the current traffic limit to obtain the updated current traffic limit.

7. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 6.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images