Data processing method, electronic device, storage medium and program product

Abstract

The invention discloses a data processing method, electronic equipment, a storage medium and a program product. The method comprises the following steps: in response to a received data request, obtaining the data volume of the data request and the current traffic limit value of the current limiter, the current traffic limit value being used for representing the upper limit value of the current processing data volume of the current limiter; based on the data volume and the current traffic limit value, determining a traffic interaction state of the traffic limiter, the traffic interaction state being used for representing whether the traffic limiter and a traffic cache server perform traffic interaction; and forwarding the data request to a data processing end based on the traffic interaction state and the current traffic limit value. According to the method and the device, the technical problem of strong dependence risk on the traffic cache server in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data processing method, electronic equipment, storage medium and program product. Background Art

[0002] With the rapid development of aviation business, the number of users and business requests has increased dramatically, and the server cluster is under unprecedented pressure. In order to ensure the stability and response speed of the service, it is crucial to effectively control the request traffic.

[0003] Traditional flow limiting technology relies on third-party cache storage services to distribute and control traffic, but each request needs to interact with the cache server, which not only increases network latency, but may also cause the cache server to become a system bottleneck. In addition, the unavailability of the cache server will directly lead to the failure of the flow limiting function, thereby affecting the stable operation of the entire cluster, which in turn leads to the risk of strong dependence on traffic cache servers in related technologies.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a data processing method, an electronic device, a storage medium and a program product, so as to at least solve the technical problem that the related technology has the risk of strong dependence on the traffic cache server.

[0006] According to one aspect of an embodiment of the present invention, there is provided a data processing method, comprising: in response to receiving a data request, obtaining the data volume of the data request and the current flow limit of a flow limiter, wherein the current flow limit is used to indicate the upper limit of the current data volume processed by the flow limiter; based on the data volume and the current flow limit, determining the flow interaction state of the flow limiter, wherein the flow interaction state is used to indicate whether the flow limiter is performing flow interaction with a flow cache server; and forwarding the data request to a data processing end based on the flow interaction state and the current flow limit.

[0007] Furthermore, based on the data volume and the current traffic limit, the traffic interaction state of the limiter is determined, including: based on the difference between the current traffic limit and the first preset value, the traffic application limit of the limiter is determined, wherein the traffic application limit is used to indicate the limit for the limiter to apply for traffic from the traffic cache server; based on the difference between the current traffic limit and the second preset value, the traffic return limit of the limiter is determined, wherein the second preset value is greater than the first preset value, and the traffic return limit is used to indicate the limit for the limiter to return traffic to the traffic cache server; the traffic interaction state is determined based on the data volume, the traffic application limit and the traffic return limit.

[0008] Furthermore, the method also includes: determining a second preset value based on the product of a preset multiple and a flow interaction step length.

[0009] Furthermore, the traffic interaction state is determined based on the data volume, the traffic application limit and the traffic return limit, including: in response to the data volume being greater than or equal to the traffic application limit, determining the traffic interaction state as the traffic application state, wherein the traffic application state is used to indicate the state in which the flow limiter interacts with the flow cache server, and the flow limiter applies for traffic from the flow cache server; in response to the data volume being less than or equal to the traffic return limit, determining the traffic interaction state as the traffic return state, wherein the traffic return state is used to indicate the state in which the flow limiter interacts with the flow cache server, and the flow limiter returns traffic to the flow cache server; in response to the data volume being less than the traffic application limit and greater than the traffic return limit, determining the traffic interaction state as the preset traffic state, wherein the preset traffic state is used to indicate the state in which the flow limiter does not interact with the flow cache server.

[0010] Furthermore, the data request is forwarded to the data processing end based on the traffic interaction state and the current traffic limit, including: in response to the traffic interaction state being a traffic application state or a traffic return state, the current traffic limit is updated based on the traffic interaction step to obtain an updated current traffic limit, and the data request is forwarded to the data processing end based on the updated current traffic limit; in response to the traffic interaction state being a preset traffic state, the data request is forwarded to the data processing end based on the current traffic limit.

[0011] Further, in response to the traffic interaction state being a traffic application state or a traffic return state, the current traffic limit is updated based on the traffic interaction step to obtain an updated current traffic limit, including: in response to the traffic interaction state being a traffic application state, controlling the flow limiter to apply to the traffic cache server for a preset traffic corresponding to the traffic interaction step, and increasing the preset traffic on the basis of the current traffic limit to obtain an updated current traffic limit; in response to the traffic interaction state being a traffic return state, controlling the flow limiter to return the preset traffic corresponding to the traffic interaction step to the traffic cache server, and reducing the preset traffic on the basis of the current traffic limit to obtain an updated current traffic limit.

[0012] Furthermore, the method also includes: in response to receiving a fault instruction from the traffic cache server, forwarding the data request to the data processing end based on the current traffic limit.

[0013] According to another aspect of an embodiment of the present invention, a data processing device is also provided, including: an acquisition module, used to obtain the data volume of the data request and the current flow limit of the flow limiter in response to receiving a data request, wherein the current flow limit is used to indicate the upper limit value of the current data volume processed by the flow limiter; a determination module, used to determine the flow interaction state of the flow limiter based on the data volume and the current flow limit, wherein the flow interaction state is used to indicate whether the flow limiter and the flow cache server perform flow interaction; and a forwarding module, used to forward the data request to the data processing end based on the flow interaction state and the current flow limit.

[0014] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the above-mentioned data processing method is executed when the program is running.

[0015] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program runs, the device where the computer-readable storage medium is located is controlled to execute the above data processing method.

[0016] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and the computer program implements the above data processing method when executed by a processor.

[0017] In an embodiment of the present invention, first, when the system receives a data request, the data volume of the data request and the current flow limit of the flow limiter are obtained; then, based on the obtained data volume and the current flow limit, the flow interaction state of the flow limiter is determined; finally, based on the flow interaction state and the current flow limit, the data request is forwarded to the data processing end. It is easy to notice that the present application clarifies the upper limit of the current processing data volume of the flow limiter by obtaining the current flow limit of the flow limiter; then, the current flow limit is compared with the data volume to determine whether the flow limiter can currently process all the data volume, thereby obtaining the flow interaction state of the flow limiter, and then based on the flow interaction state, it can be determined whether the flow limiter needs to interact with the flow cache server for flow, thereby achieving precise control of the interaction state between the flow limiter and the flow cache server, achieving the technical effect of reducing the dependence on the flow cache server, improving the efficiency of flow control and the processing capacity of concurrent requests, thereby solving the technical problem that the related technology has a strong dependence risk on the flow cache server. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention;

[0020] Figure 2 is a flow chart of an optional data processing method according to an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a data processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] According to an embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0026] Step S102, in response to receiving a data request, obtaining the data volume of the data request and the current flow limit value of the flow limiter, wherein the current flow limit value is used to indicate the upper limit value of the data volume currently processed by the flow limiter.

[0027] The above-mentioned data request may refer to a data acquisition or processing request issued by a client or downstream service to a server or upstream service. The types of data requests may include but are not limited to data query requests, data acquisition requests, etc. The specific data request needs to be determined based on the actual needs of the user or system. There is no limitation here. The data request can be used to obtain data required by the system or user, etc.

[0028] The above-mentioned data volume may refer to the amount of data that needs to be processed in the data request. The type of data volume may include but is not limited to the size and quantity of data packets and the frequency of data requests. The specific data volume needs to be determined based on the complexity of the data request and is not limited here.

[0029] The above-mentioned rate limiter may refer to a mechanism for controlling the flow of requests or data entering the system. The types of rate limiters may include but are not limited to local rate limiters, rate limiters based on time windows, and rate limiters based on the number of requests. The specific rate limiter type needs to be determined based on actual needs and is not limited here. The number of rate limiters may include but is not limited to 1, 2, 3, 4, 5, etc. The specific number of rate limiters needs to be determined based on the amount of data and the system design and is not limited here. The functions of the rate limiter may include but are not limited to flow control, elastic scaling, fault isolation, and preventing system overload to ensure stable operation of the system.

[0030] The above-mentioned current traffic limit may refer to the upper limit of the amount of data requests that the limiter can process at the current moment. The current traffic limit may be used to limit and manage the amount of data requests entering the data processing end, thereby preventing the server from being unable to respond or responding late due to overload, and ensuring the robustness and response speed of the system.

[0031] In an optional embodiment, when the system receives a data request from a user or downstream service, the rate limiter will respond quickly and obtain the data volume information involved in the request, and read the current traffic limit at the same time. Through this mechanism, the rate limiter can monitor and control the amount of data entering the system in real time, avoiding server performance degradation or service interruption due to data request overload, thereby ensuring the stability and response speed of the ticket inquiry service.

[0032] Step S104, based on the data volume and the current traffic limit value, determining the traffic interaction state of the flow limiter, wherein the traffic interaction state is used to indicate whether the flow limiter and the traffic cache server are performing traffic interaction.

[0033] The above-mentioned traffic interaction status may refer to the decision of whether the flow limiter needs to interact with the traffic cache server for traffic data when processing a data request, based on the current traffic limit and the real-time data volume. The types of traffic interaction status may include but are not limited to no operation status, traffic application status, and traffic return status. The specific traffic interaction status needs to be determined based on the comparison result between the data volume and the current traffic limit, which is not limited here. The role of the traffic interaction status may include but is not limited to clarifying the needs of the flow limiter based on the current processing status of the flow limiter, and then making the next judgment.

[0034] The above-mentioned traffic cache server may refer to a central node that stores and manages the traffic configuration of the entire cluster. The functions of the traffic cache server may include but are not limited to responding to traffic application and return requests from the limiter, dynamically adjusting the traffic quota of each limiter, maintaining the traffic balance of the cluster, etc.

[0035] In an optional embodiment, the relationship between the data volume and the current traffic limit is evaluated by the flow limiter to determine whether traffic interaction with the traffic cache server is required; the traffic cache server, as the hub of cluster traffic configuration, responds to the request of the flow limiter to achieve intelligent distribution of traffic and stable operation of the cluster. The above process reduces the dependence on network communication through the cooperation between the flow limiter and the traffic cache server, improves the response speed and resource utilization of the system, and also enhances the fault tolerance and elastic scalability of the system.

[0036] Step S106: forwarding the data request to the data processing end based on the traffic interaction state and the current traffic limit.

[0037] The above-mentioned data processing end may refer to the backend component or service in the system that is responsible for receiving the data request forwarded by the rate limiter and executing specific business logic. The role of the data processing end may include but is not limited to data query and retrieval, data update and maintenance, service response generation, etc.

[0038] In an optional embodiment, the flow limiter forwards the data request to the data processing end according to the flow interaction state and the current flow limit, wherein the flow control of the flow limiter reduces the invalid processing and waiting time of the data processing end, and improves the efficiency of data retrieval and processing; at the same time, by dynamically adjusting the flow limit, the data processing end can better adapt to changing business needs, avoid waste or shortage of resources, and enhance the flexibility and performance of the system. In addition, the decoupling between the flow limiter and the data processing end enables the system to provide stable services even when the network conditions are poor or the server pressure is high, greatly improving the user experience and the overall reliability of the system.

[0039] In an optional embodiment, Figure 2is a flow chart of an optional data processing method according to an embodiment of the present invention. Figure 2 As shown, first, multiple requests from users are obtained; then, the user requests are evenly distributed to multiple limiters through load balancing; finally, the flow is renewed with the flow cache server through the limiter.

[0040] In the above process, the flow limiter renews the flow with the flow cache server, that is, based on the data volume and the current flow limit, determines the flow interaction state of the flow limiter, and then determines how to renew the flow with the flow cache server based on the flow interaction state. Renewal means that the flow limiter applies for a certain flow limit value, that is, the current flow limit, from the flow cache server, and then calculates and controls the flow in the local service. All requests no longer interact with the cache server, but are only calculated in the local server. When the local concurrency exceeds the current flow limit, the flow limiter interacts with the cache server again, and once again applies for a certain flow limit value, which is added to the local flow limit value to increase the total flow limit value of the flow limiter to achieve a higher concurrency.

[0041] In an embodiment of the present invention, first, when the system receives a data request, the data volume of the data request and the current flow limit of the flow limiter are obtained; then, based on the obtained data volume and the current flow limit, the flow interaction state of the flow limiter is determined; finally, based on the flow interaction state and the current flow limit, the data request is forwarded to the data processing end. It is easy to notice that the present application clarifies the upper limit of the current processing data volume of the flow limiter by obtaining the current flow limit of the flow limiter; then, the current flow limit is compared with the data volume to determine whether the flow limiter can currently process all the data volume, thereby obtaining the flow interaction state of the flow limiter, and then based on the flow interaction state, it can be determined whether the flow limiter needs to interact with the flow cache server for flow, thereby achieving precise control of the interaction state between the flow limiter and the flow cache server, achieving the technical effect of reducing the dependence on the flow cache server, improving the efficiency of flow control and the processing capacity of concurrent requests, thereby solving the technical problem that the related technology has a strong dependence risk on the flow cache server.

[0042] Optionally, based on the data volume and the current traffic limit, the traffic interaction state of the limiter is determined, including: based on the difference between the current traffic limit and a first preset value, the traffic application limit of the limiter is determined, wherein the traffic application limit is used to indicate the limit for the limiter to apply for traffic from the traffic cache server; based on the difference between the current traffic limit and a second preset value, the traffic return limit of the limiter is determined, wherein the second preset value is greater than the first preset value, and the traffic return limit is used to indicate the limit for the limiter to return traffic to the traffic cache server; the traffic interaction state is determined based on the data volume, the traffic application limit and the traffic return limit.

[0043] The above-mentioned first preset value may refer to a threshold standard for triggering a traffic application operation. The type of the first preset value may include but is not limited to a solid first preset value, a dynamic first preset value, a proportional first preset value, etc. The specific first preset value needs to be determined based on the data volume type and actual conditions. The size of the first preset value may include but is not limited to 5, 10, 15, etc. The specific first preset value needs to be determined based on actual conditions and is not limited here. The first preset value can be used as a judgment threshold for triggering a traffic application operation, etc.

[0044] The above-mentioned traffic application limit may refer to the specific value of the additional traffic quota requested by the traffic limiter to the traffic cache server. The traffic application limit may be determined based on a comprehensive evaluation of the system processing capacity, business requirements and the status of the traffic cache server to ensure traffic balance in the cluster and rational use of resources.

[0045] The above-mentioned second preset value may refer to a threshold standard for triggering a traffic return operation. The type of the second preset value may include but is not limited to a solid second preset value, a dynamic second preset value, a proportional second preset value, etc. The specific second preset value needs to be determined based on the data volume type and actual situation. The size of the second preset value may include but is not limited to 30, 35, 40, etc. The specific second preset value needs to be determined based on actual conditions and is not limited here. The role of the second preset value may include but is not limited to being used as a judgment criterion for triggering a traffic return operation, avoiding excessive concentration and waste of resources, and ensuring that the traffic resources of the entire cluster are optimally allocated.

[0046] The above-mentioned traffic return limit may refer to the amount of partial traffic quota that the limiter considers returning to the traffic cache server when the data volume is lower than the difference between the current traffic limit and the second preset value. The traffic return limit may be used to release unused traffic quota when the actual load of the limiter is far lower than its capacity, thereby optimizing resource allocation and avoiding the impact of rapid fluctuations in traffic quota on system stability.

[0047] In an optional embodiment, a first preset value and a second preset value are first set, and then based on the difference between the current traffic limit and the first preset value, the limit for the flow limiter to apply for traffic from the traffic cache server is determined; based on the difference between the current traffic limit and the second preset value, the limit for the flow limiter to return traffic to the traffic cache server is determined. In the above process, by setting the first preset value and the second preset value, the flow limiter can apply for additional traffic in a timely manner, thereby ensuring the continuity and stability of data processing in high-concurrency scenarios, and at the same time enabling the flow limiter to return excess traffic, thereby optimizing resource allocation and improving the processing efficiency and response time of the entire cluster.

[0048] Optionally, the method further includes: determining a second preset value based on the product of a preset multiple and a flow interaction step length.

[0049] The above-mentioned preset multiple may refer to a fixed or dynamic multiple coefficient used to calculate the second preset value when setting the flow limiting rules. The preset multiple may include but is not limited to 2, 3, 4, etc. The specific preset multiple needs to be determined based on the current flow limit and data volume, and is not limited here. The function of the preset multiple may include but is not limited to measuring the distance between the flow limit and the flow return trigger point.

[0050] In an optional embodiment, if the preset multiple is 2, it means that when the difference between the current traffic limit of the limiter and its real-time data volume reaches twice the traffic interaction step, the traffic return operation will be triggered to return part of the traffic quota to the traffic cache server.

[0051] The above-mentioned traffic interaction step may refer to a fixed value of traffic applied for or returned each time the flow limiter interacts with the traffic cache server. The types of traffic interaction step may include but are not limited to fixed traffic interaction step and dynamic traffic interaction step. The specific traffic interaction step needs to be determined according to the application scenario and is not limited here. The traffic interaction step can be used to adjust the allocation efficiency and dynamic adjustment capability of traffic resources.

[0052] In an optional embodiment, the second preset value is determined by multiplying the preset multiple by the product of the traffic interaction step length, which can ensure that the flow limiter can effectively utilize resources and avoid idle traffic when processing data requests, and can respond promptly to increases in traffic demand and prevent resource shortages. At the same time, by dynamically adjusting these parameters, the system can intelligently adjust traffic quotas according to changes in actual business scenarios and network environments, improve resource utilization, reduce server pressure, and ensure continuous stability and efficient response of services.

[0053] In an optional embodiment, when the preset multiple is 2 and the traffic interaction step is 10, the second preset value is twice the traffic interaction step, that is, 20 traffic units. This means that when the data volume is lower than the difference between the traffic limit value and the second preset value, the flow limiter will consider returning the traffic quota to the traffic cache server, and this difference is dynamically calculated based on the traffic interaction step and can be adjusted according to the real-time status of the system.

[0054] Optionally, the traffic interaction state is determined based on the data volume, the traffic application limit and the traffic return limit, including: in response to the data volume being greater than or equal to the traffic application limit, determining the traffic interaction state as the traffic application state, wherein the traffic application state is used to indicate the state in which the flow limiter interacts with the traffic cache server, and the flow limiter applies for traffic from the traffic cache server; in response to the data volume being less than or equal to the traffic return limit, determining the traffic interaction state as the traffic return state, wherein the traffic return state is used to indicate the state in which the flow limiter interacts with the traffic cache server, and the flow limiter returns traffic to the traffic cache server; in response to the data volume being less than the traffic application limit and greater than the traffic return limit, determining the traffic interaction state as the preset traffic state, wherein the preset traffic state is used to indicate the state in which the flow limiter does not interact with the traffic cache server.

[0055] The above-mentioned traffic application status may refer to a state in which the system needs to request additional traffic quota from the traffic cache server when the amount of data monitored by the limiter reaches or exceeds its traffic application limit. The traffic application status can be used to ensure that the limiter can apply for additional traffic quota in a timely manner to meet sudden growth in business demand and avoid service interruptions and response delays.

[0056] The above-mentioned traffic return state may refer to a state in which the system needs to return part of the traffic quota to the traffic cache server when the amount of data monitored by the traffic limiter drops to or below its traffic return limit. The traffic return state can be used to optimize resource allocation within the cluster, avoid excessive concentration and waste of traffic quotas, and improve the overall resource utilization and response speed of the system.

[0057] The above-mentioned preset traffic state may refer to a state in which the system does not need to interact with the traffic cache server for traffic when the data volume is lower than the traffic application limit and higher than the traffic return limit. The preset traffic state can be used to reduce the interaction between the system and the traffic cache server under stable traffic demand, reduce network communication delays and resource consumption, and improve system operation efficiency.

[0058] In an optional embodiment, the traffic interaction state between the flow limiter and the traffic cache server is determined by comparing the dynamically monitored data volume with the preset traffic application limit and the traffic return limit. When the data volume reaches or exceeds the traffic application limit, the system enters the traffic application state, and the flow limiter requests additional traffic quota from the traffic cache server to cope with sudden high traffic demands and ensure service continuity and response speed; conversely, when the data volume drops to or equals the traffic return limit, the system enters the traffic return state, and the flow limiter returns the unused traffic quota to the traffic cache server, optimizes resource allocation, avoids idleness and waste of traffic resources, and ensures that each node in the cluster can obtain reasonable traffic support; and in the stable traffic interval where the data volume is lower than the traffic application limit but higher than the traffic return limit, the system remains in the preset traffic state, and no traffic interaction is performed between the flow limiter and the traffic cache server, which reduces unnecessary network communications and resource overhead, and improves system operation efficiency and stability. This intelligent traffic control strategy based on real-time data volume not only enhances the flexibility of the system and the dynamic optimization capability of resources, but also significantly reduces the dependence on traffic cache servers, improves the performance and reliability of ticket query services, and provides end users with a smoother and more efficient service experience.

[0059] Optionally, the data request is forwarded to the data processing end based on the traffic interaction state and the current traffic limit, including: in response to the traffic interaction state being a traffic application state or a traffic return state, the current traffic limit is updated based on the traffic interaction step to obtain an updated current traffic limit, and the data request is forwarded to the data processing end based on the updated current traffic limit; in response to the traffic interaction state being a preset traffic state, the data request is forwarded to the data processing end based on the current traffic limit.

[0060] The above-mentioned updated current traffic limit may refer to the upper limit of the traffic processing capacity adjusted according to the traffic interaction step after the traffic limiter performs traffic application or traffic return operations. The updated current traffic limit may be used to change the processing capacity and efficiency of the traffic limiter for data requests.

[0061] In an optional embodiment, different data forwarding processes are performed based on different traffic interaction states. When traffic demand increases, the flow limiter can apply for more traffic quotas in a timely manner to avoid service interruption and response delay; when traffic demand decreases, the flow limiter can return unused traffic quotas to avoid resource waste and optimize resource allocation within the cluster. Through the dynamic update of the current traffic limit value, it is ensured that the flow limiter can automatically adapt to real-time traffic demand changes, thereby improving resource utilization and system response speed.

[0062] Optionally, in response to the traffic interaction state being a traffic application state or a traffic return state, the current traffic limit is updated based on the traffic interaction step to obtain an updated current traffic limit, including: in response to the traffic interaction state being a traffic application state, controlling the flow limiter to apply to the traffic cache server for a preset traffic corresponding to the traffic interaction step, and increasing the preset traffic on the basis of the current traffic limit to obtain an updated current traffic limit; in response to the traffic interaction state being a traffic return state, controlling the flow limiter to return the preset traffic corresponding to the traffic interaction step to the traffic cache server, and reducing the preset traffic on the basis of the current traffic limit to obtain an updated current traffic limit.

[0063] The above-mentioned preset traffic may refer to a specific traffic value applied for or returned by the flow limiter when interacting with the traffic cache server in the traffic application state or the traffic return state. The types of preset traffic may include but are not limited to fixed preset traffic, dynamic preset traffic, and fine-tuning preset traffic, etc. The specific type of preset traffic needs to be determined based on the current traffic state and is not limited here. The role of the preset traffic may include but is not limited to dynamically adjusting the processing capacity of the flow limiter, resource balancing, and system performance optimization, etc.

[0064] In an optional embodiment, when the system detects the traffic application status, the limiter will automatically apply to the traffic cache server for a preset traffic corresponding to the traffic interaction step. The above operation can increase the preset traffic on the basis of the current traffic limit to form an updated current traffic limit, ensuring that when the traffic demand increases, the limiter can quickly increase the processing capacity, avoid service overload, and improve the system response speed and user experience; on the contrary, when the traffic return status is identified, the limiter will return the preset traffic to the traffic cache server, reduce the current traffic limit, avoid resource waste, promote the balanced allocation of resources within the cluster, and ensure the stable operation of the overall system and efficient resource utilization.

[0065] Optionally, the method further includes: in response to receiving a fault instruction from the traffic cache server, forwarding the data request to the data processing end based on the current traffic limit.

[0066] The above-mentioned fault instruction may refer to a special instruction received by the system or the flow limiter when the traffic cache server fails or is unavailable. The types of fault instructions may include but are not limited to hardware fault instructions, software exception instructions, and network interruption instructions. The specific fault instructions should be determined based on the actual situation of the system and are not limited here. The fault instruction can be used for emergency processing based on the fault instruction, thereby avoiding the avalanche effect caused by the failure of the cache server and protecting the stability of the entire cluster.

[0067] In an optional embodiment, after receiving the system's fault instruction, the current flow limit is forwarded to the data processing end to ensure that the flow limiter can process data requests based on the existing flow limit during the fault period, thereby improving the stability and reliability of the system, avoiding overall service interruption caused by single point failures, and providing users with a more continuous and stable service experience. At the same time, the fault instruction can also guide the system to synchronize data and adjust policies in a timely manner after the fault is restored, ensuring rapid recovery of services and long-term stable operation.

[0068] According to another aspect of an embodiment of the present invention, a data processing device is further provided, which can execute the data processing method of the above embodiment. The specific implementation method and preferred application scenario are the same as those of the above embodiment and will not be described in detail here.

[0069] Figure 3 is a schematic diagram of a data processing device according to an embodiment of the present invention. Figure 3 As shown, the device includes the following: an acquisition module 302 , a determination module 304 , and a forwarding module 306 .

[0070] The acquisition module 302 is used to obtain the data volume of the data request and the current flow limit of the flow limiter in response to receiving the data request, wherein the current flow limit is used to indicate the upper limit of the current data volume processed by the flow limiter; the determination module 304 is used to determine the flow interaction state of the flow limiter based on the data volume and the current flow limit, wherein the flow interaction state is used to indicate whether the flow limiter and the flow cache server are conducting flow interaction; the forwarding module 306 is used to forward the data request to the data processing end based on the flow interaction state and the current flow limit.

[0071] Optionally, the determination module includes: a first determination unit, used to determine the flow application limit of the flow limiter based on the difference between the current flow limit and the first preset value, wherein the flow application limit is used to indicate the limit of the flow limiter applying for flow from the flow cache server; a second determination unit, used to determine the flow return limit of the flow limiter based on the difference between the current flow limit and the second preset value, wherein the second preset value is greater than the first preset value, and the flow return limit is used to indicate the limit of the flow limiter returning flow to the flow cache server; a third determination unit, used to determine the flow interaction state based on the data volume, the flow application limit and the flow return limit.

[0072] Optionally, the device also includes: determining a second preset value based on the product of a preset multiple and a flow interaction step length.

[0073] Optionally, the third determination unit includes: a first determination subunit, used to determine that the traffic interaction state is a traffic application state in response to the data volume being greater than or equal to the traffic application limit, wherein the traffic application state is used to indicate the state in which the flow limiter interacts with the flow cache server, and the flow limiter applies for traffic from the flow cache server; a second determination subunit, used to determine that the traffic interaction state is a traffic return state in response to the data volume being less than or equal to the traffic return limit, wherein the traffic return state is used to indicate the state in which the flow limiter interacts with the flow cache server, and the flow limiter returns traffic to the flow cache server; a third determination subunit, used to determine that the traffic interaction state is a preset traffic state in response to the data volume being less than the traffic application limit and greater than the traffic return limit, wherein the preset traffic state is used to indicate the state in which the flow limiter and the flow cache server do not interact with each other.

[0074] Optionally, the forwarding module includes: a first forwarding unit, used to update the current traffic limit based on the traffic interaction step in response to the traffic interaction state being a traffic application state or a traffic return state, obtain an updated current traffic limit, and forward the data request to the data processing end based on the updated current traffic limit; a second forwarding unit, used to forward the data request to the data processing end based on the current traffic limit in response to the traffic interaction state being a preset traffic state.

[0075] Optionally, the first forwarding unit includes: a first control subunit, used to control the flow limiter to apply for a preset flow corresponding to the flow interaction step from the flow cache server in response to the flow interaction state being a flow application state, and to increase the preset flow on the basis of the current flow limit to obtain an updated current flow limit; a second control subunit, used to control the flow limiter to return the preset flow corresponding to the flow interaction step to the flow cache server in response to the flow interaction state being a flow return state, and to reduce the preset flow on the basis of the current flow limit to obtain an updated current flow limit.

[0076] Optionally, the device further includes: in response to receiving a fault instruction from the traffic cache server, forwarding the data request to the data processing end based on the current traffic limit.

[0077] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present invention is executed when the program is running.

[0078] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored program. When the program is executed, a processor of a device is controlled to execute the methods of various embodiments of the present invention.

[0079] The computer storage medium in the above steps can be a medium used to store certain discontinuous physical quantities in a computer memory, and the computer storage medium mainly includes semiconductors, magnetic cores, magnetic drums, magnetic tapes, laser disks, etc. The stored program included in the computer-readable storage medium can be a set of instructions that can be recognized and executed by a computer, running on an electronic computer, and is an information tool that meets certain needs of people.

[0080] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0081] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0083] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0084] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0086] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: include: In response to receiving the data request, obtaining the data volume of the data request and the current flow limit value of the flow limiter, wherein the current flow limit value is used to indicate the upper limit value of the data volume currently processed by the flow limiter; Determine the flow interaction state of the flow limiter based on the data volume and the current flow limit value, wherein the flow interaction state is used to indicate whether the flow limiter and the flow cache server perform flow interaction; The data request is forwarded to a data processing end based on the traffic interaction state and the current traffic limit.

2. The data processing method according to claim 1, characterized in that: Determining a flow interaction state of the flow limiter based on the data volume and the current flow limit value includes: Determine a flow application limit of the flow limiter based on a difference between the current flow limit and a first preset value, wherein the flow application limit is used to indicate a flow limit that the flow limiter applies to the flow cache server; Determine a flow return limit of the flow limiter based on a difference between the current flow limit and a second preset value, wherein the second preset value is greater than the first preset value, and the flow return limit is used to indicate a limit value for the flow limiter to return the flow to the flow cache server; The traffic interaction state is determined based on the data volume, the traffic application limit and the traffic return limit.

3. The data processing method according to claim 2, characterized in that: The method further comprises: The second preset value is determined based on the product of the preset multiple and the flow interaction step length.

4. The data processing method according to claim 2, characterized in that: Determining the traffic interaction state based on the data volume, the traffic application limit, and the traffic return limit includes: In response to the data volume being greater than or equal to the traffic application limit, determining that the traffic interaction state is a traffic application state, wherein the traffic application state is used to indicate a state in which the flow limiter interacts with the traffic cache server, and the flow limiter applies for traffic from the traffic cache server; In response to the data volume being less than or equal to the traffic return limit, determining that the traffic interaction state is a traffic return state, wherein the traffic return state is used to indicate a state in which the flow limiter interacts with the traffic cache server, and the flow limiter returns traffic to the traffic cache server; In response to the data volume being less than the traffic application limit and greater than the traffic return limit, the traffic interaction state is determined to be a preset traffic state, wherein the preset traffic state is used to indicate a state in which the flow limiter and the traffic cache server do not perform traffic interaction.

5. The data processing method according to any one of claims 1 to 4, characterized in that: Forwarding the data request to a data processing end based on the traffic interaction state and the current traffic limit value includes: In response to the flow interaction state being a flow application state or a flow return state, updating the current flow limit value based on the flow interaction step length to obtain an updated current flow limit value, and forwarding the data request to the data processing end based on the updated current flow limit value; In response to the traffic interaction state being a preset traffic state, the data request is forwarded to the data processing end based on the current traffic limit.

6. The data processing method according to claim 5, characterized in that: In response to the flow interaction state being the flow application state or the flow return state, updating the current flow limit value based on the flow interaction step length to obtain an updated current flow limit value, including: In response to the traffic interaction state being the traffic application state, controlling the flow limiter to apply to the traffic cache server for the preset traffic corresponding to the traffic interaction step, and adding the preset traffic on the basis of the current traffic limit to obtain the updated current traffic limit; In response to the traffic interaction state being the traffic return state, the flow limiter is controlled to return the preset traffic corresponding to the traffic interaction step to the traffic cache server, and the preset traffic is reduced based on the current traffic limit to obtain the updated current traffic limit.

7. The data processing method according to claim 1, characterized in that: The method further comprises: In response to receiving a fault instruction from the traffic cache server, the data request is forwarded to the data processing end based on the current traffic limit.

8. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 7 when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images