A composite current limiting method and system for large-volume data transmission based on digital networking

Abstract

The present application provides a composite current limiting method and system for high-volume data transmission based on a digital networking system, which relates to the field of digital networking technology. The method includes: parsing the protocol header of the requested data to determine the target communication protocol for data transmission; determining the target data type and priority value of the digital object based on the metadata in the requested digital object; determining a corresponding first current limiting strategy based on the target communication protocol and the target data type; modifying the parameters of the first current limiting strategy based on the priority value to obtain a second current limiting strategy; and current limiting transmission of the digital object using the second current limiting strategy. The method aims to improve the stability of data transmission in a digital networking system.

Description

Technical Field

[0001] The present application relates to the field of digital networking technology, and in particular to a composite current limiting method and system for high-volume data transmission based on digital networking. Background Art

[0002] In the Internet of Things (IoT) environment, due to network fluctuations, resource constraints, high concurrency, and data diversity, existing current limiting methods (such as sliding window current limiting, token bucket algorithm, leaky bucket algorithm, etc.) have their own limitations and cannot fully and effectively address the challenges brought by large-scale data transmission. They are not flexible enough, which may cause the system to be overloaded due to too many requests, affecting system stability. Summary of the Invention

[0003] In view of this, the present application provides a composite current limiting method and system for high-traffic data transmission based on a data network, aiming to improve the stability of data transmission in a data network system.

[0004] In a first aspect of the present application, a composite current limiting method for high-traffic data transmission based on a data network is provided, the method comprising:

[0005] Parse the protocol header of the request data to determine the target communication protocol for data transmission;

[0006] Determining a target data type and a priority value of the digital object according to metadata in the requested digital object;

[0007] Determining a corresponding first current limiting strategy according to the target communication protocol and the target data type;

[0008] Modifying parameters of the first current limiting strategy according to the priority value to obtain a second current limiting strategy;

[0009] The digital object is subjected to current-limited transmission using the second current-limiting strategy.

[0010] Optionally, modifying parameters of the first current limiting strategy according to the priority value to obtain a second current limiting strategy includes:

[0011] Determining target parameters for parameter correction according to the first current limiting strategy;

[0012] According to the priority value and the preset correction rule, the target parameter is corrected to obtain a second current limiting strategy.

[0013] Optionally, modifying the target parameter according to the priority value and a preset modification rule to obtain a second current limiting strategy includes:

[0014] Determine a basic priority value based on the priority values ​​of all digital objects in the cache queue;

[0015] Determining a priority deviation value between the priority value of the digital object and the basic priority value;

[0016] The target parameter is modified according to the priority deviation value, the base value and adjustment step of the target parameter, and the target percentage value to obtain a second current limiting strategy, wherein the target percentage value is a value corresponding to both the priority value of the digital object and the type of the target parameter.

[0017] Optionally, before modifying parameters of the first current limiting strategy according to the priority value to obtain the second current limiting strategy, the method further includes:

[0018] Adjusting the priority value of the digital object according to the waiting time of the digital object in the cache queue;

[0019] The step of modifying parameters of the first current limiting strategy according to the priority value to obtain a second current limiting strategy includes:

[0020] According to the adjusted priority value, the parameters of the first current limiting strategy are modified to obtain a second current limiting strategy.

[0021] Optionally, before performing flow limiting transmission on the digital object using the second flow limiting strategy, the method further includes:

[0022] Determine actual measurements of network traffic during data transmission;

[0023] Dynamically adjusting the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network flow;

[0024] The limiting transmission of the digital object by using the second limiting strategy includes:

[0025] The digital object is subjected to current-limited transmission using the dynamically adjusted second current-limiting strategy.

[0026] Optionally, dynamically adjusting the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network flow includes:

[0027] In the case where the second current limiting strategy is a token bucket algorithm, dynamically adjusting the token bucket capacity or token generation rate in the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network traffic;

[0028] In the case where the second flow limiting strategy is a leaky bucket algorithm, dynamically adjusting the leaky bucket capacity or water outflow rate in the second flow limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network flow;

[0029] When the second current limiting strategy is a sliding window algorithm, the window size or request processing rate in the second current limiting strategy is dynamically adjusted through PID feedback regulation according to the actual measurement value and the set value of the network traffic.

[0030] Optionally, the method further includes:

[0031] When the priority values ​​of all digital objects in the cache queue are adjusted to the highest priority, resources are evenly allocated for data transmission of each digital object.

[0032] Optionally, when the target communication protocol is HTTP, the method further includes:

[0033] Filtering the request data through a regular rule library to determine whether the request data belongs to a known attack pattern;

[0034] If no rule is hit, the parameter semantics of the request data are analyzed by the AST parsing engine to determine whether the request data is a suspicious request;

[0035] If the request data is a suspicious request, direct the request data to an isolated environment for execution to monitor abnormal behavior;

[0036] If no abnormal behavior is detected, respond to the request data.

[0037] Optionally, when the target communication protocol is the gRPC protocol, the method further includes:

[0038] Verify the RPC method signature in the request data to determine whether the request data is legal, where the RPC method signature is a unique signature generated by the server for each RPC method;

[0039] If the request data is legal, determining whether the call chain of the request data is an abnormal call chain based on the request data and the service dependency graph;

[0040] In the case where the call chain is an abnormal call chain, the abnormal call chain is blocked, and the service dependency graph is a dependency relationship between services constructed based on information recorded for each data request; or

[0041] Determining whether the request data is legitimate based on a dynamic token in the request data, wherein the dynamic token is sent to the client after the server verifies the certificate and fingerprint of the connection request of the client sending the request data;

[0042] If the request data is legal, respond to the request data.

[0043] Optionally, when the target communication protocol is the WebSocket protocol, the method further includes:

[0044] Performing a binary magic number check on the request data to determine whether the request data is legal, and / or performing a malicious keyword scan on the request data to determine whether the request data is legal;

[0045] If the request data is valid, respond to the request data;

[0046] Adjust the number of connections for each client based on the server's CPU usage;

[0047] According to a preset heartbeat monitoring mechanism, a request connection that complies with the preset heartbeat monitoring mechanism is disconnected.

[0048] A second aspect of the present application provides a composite current limiting system for high-volume data transmission based on a digital network, the system comprising:

[0049] A target communication protocol determination module is used to parse the protocol header of the request data and determine the target communication protocol for data transmission;

[0050] A metadata parsing module, configured to determine a target data type and a priority value of the digital object according to metadata in the requested digital object;

[0051] A first current limiting strategy determining module, configured to determine a corresponding first current limiting strategy according to the target communication protocol and the target data type;

[0052] A second current limiting strategy determination module is configured to modify parameters of the first current limiting strategy according to the priority value to obtain a second current limiting strategy;

[0053] The current limiting control module is configured to perform current limiting transmission on the digital object using the second current limiting strategy.

[0054] Compared with the prior art, this application has the following advantages:

[0055] This application provides a composite current limiting method for high-volume data transmission based on a digital network. First, the protocol header of the requested data is parsed to determine the target communication protocol for data transmission; the target data type and priority value of the digital object are determined based on the metadata in the requested digital object; a corresponding first current limiting strategy is determined based on the target communication protocol and the target data type; the parameters of the first current limiting strategy are modified based on the priority value to obtain a second current limiting strategy; and the digital object is current-limited and transmitted using the second current limiting strategy. Thus, the application combines multiple current limiting strategies, selects a current limiting strategy that is adapted to the current data type and communication protocol from multiple current limiting strategies based on the data type recorded in the metadata of the digital object being transmitted and the communication protocol based on the data transmission, and performs current limiting processing on the digital object to adapt to the data transmission requirements in different scenarios, thereby improving the stability and reliability of the system. At the same time, based on the priority of the digital object, the corresponding current limiting strategy is adjusted, so that a lower current limiting degree is given to the digital object with a higher priority and a higher current limiting degree is given to the digital object with a lower priority, thereby optimizing resource utilization during data transmission.

[0056] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0058] Figure 1 A flow chart of a composite current limiting method for high-traffic data transmission based on a digital network provided in an embodiment of the present application;

[0059] Figure 2 Another flow chart of a composite current limiting method for high-volume data transmission based on a digital network provided in an embodiment of the present application;

[0060] Figure 3 A flow chart of the flow of a token bucket algorithm in a composite flow limiting method for high-traffic data transmission based on a data network provided in an embodiment of the present application;

[0061] Figure 4 A flow chart of the flow limiting of the leaky bucket algorithm in a composite flow limiting method for high-traffic data transmission based on a digital network provided in an embodiment of the present application;

[0062] Figure 5A flow chart of the current limiting of a sliding window algorithm in a composite current limiting method for high-traffic data transmission based on a data network provided in an embodiment of the present application;

[0063] Figure 6 A schematic diagram of a composite current limiting system for high-volume data transmission based on a digital network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings.

[0065] To facilitate understanding of the plan, the following terms are explained:

[0066] Internet of Data (IOD): It is a virtual data network based on the Internet. Through open software architecture and standardized protocols, it efficiently connects various data platforms and systems, supports the interconnection, interoperability and interoperability of heterogeneous, heterogeneous and heterogeneous data, and forms a data space with "data interconnection, on-demand scheduling, intra-domain autonomy and inter-domain collaboration".

[0067] Digital Object (DO): A data structure that abstractly describes data resources. It organizes and records data in a computer system according to basic elements such as identification, metadata, and data entities.

[0068] Figure 1 A flow chart of a composite current limiting method for large-volume data transmission based on a data network is provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0069] Step S1: Parse the protocol header of the request data to determine the target communication protocol for data transmission.

[0070] In this embodiment, the present application is applied to a data network built based on a digital object architecture. For ease of description, all subsequent references to data networks are based on the digital object architecture. The data types transmitted within this data network are diverse, including sensor data, medical emergency data, text data, video data, and so on. Different data types have different priorities during transmission, and a single rate limiting strategy cannot meet the transmission requirements of the diverse data types in this data network scenario. Therefore, this application introduces multiple rate limiting strategies and determines a matching rate limiting strategy based on the data type and communication protocol corresponding to the digital object, thereby meeting the transmission requirements of the digital object of that data type. For example, sensor data (such as a temperature sensor that only transmits approximately 10 bytes of data per second) has extremely high real-time requirements but a relatively small data volume. Given the frequent updates and time-sensitive nature of this type of data, a corresponding sliding window rate limiting algorithm is employed to ensure timely data transmission. The metadata of digital objects with strong timeliness records the data generation time and timeliness requirements. The timeliness of the digital object is determined based on these two. The generation time records the time point when the data is generated, which is extremely critical for data with timeliness requirements. The timeliness requirements will clearly define the data's validity period or the specified transmission completion time. The metadata of the digital object also records the data size.

[0071] Specifically, when a client requests a corresponding digital object from a server, it sends request data based on the requested digital object to the server. The server then parses the protocol header of the received request data and determines the communication protocol based on which the request data is sent from the client to the server. This communication protocol is the target communication protocol for data transmission of the request data. Subsequently, the digital object requested based on the request data is transmitted back to the client based on this target communication protocol. The target communication protocols based on which the client sends the request data to the server may include at least HTTP, gRPC, and WebSocket.

[0072] Step S2: Determine the target data type and priority value of the digital object according to the metadata in the requested digital object.

[0073] In this embodiment, the server determines the digital object requested by the received request data based on the request data. It then extracts the digital object's data type and the recorded priority value from the digital object's metadata. This data type serves as the target data type for the digital object. The data types of digital objects include, but are not limited to, text data, sensor data, image and video data, and structured data.

[0074] Step S3: Determine a corresponding first current limiting strategy according to the target communication protocol and the target data type.

[0075] In this embodiment, based on the target communication protocol based on which the request data is sent to the server and the target data type of the digital object requested by the request data, a current limiting strategy corresponding to the target communication protocol and the target data type is determined through a pre-established mapping relationship, and the current limiting strategy is determined as the first current limiting strategy to be adopted by the digital object. The pre-established mapping relationship is a mapping relationship that maps current limiting strategies to communication protocols and data types. An optional implementation of this mapping relationship is: when the data type is sensor data or the transmission protocol is HTTP, the corresponding current limiting strategy is a sliding window algorithm current limiting strategy; when the data type is file data with a data volume exceeding a set value or the transmission protocol is one of Grpc and WebSocket protocols, the corresponding current limiting strategy is a token bucket algorithm current limiting strategy; when the data type is background periodic task data, the corresponding current limiting strategy is a leaky bucket algorithm current limiting strategy. It should be understood that this mapping relationship can also record mapping relationships between other current limiting strategies, other communication protocols, and other data types. Only some mapping relationships are listed here.

[0076] Step S4: According to the priority value, modify the parameters of the first current limiting strategy to obtain a second current limiting strategy.

[0077] In this embodiment, after determining the first current limiting strategy corresponding to the digital object through step S3, the relevant parameters of the first current limiting strategy are corrected based on the priority value of the digital object to obtain a corrected first current limiting strategy, and the corrected first current limiting strategy is the second current limiting strategy to be adopted for the digital object. The parameter correction process is to correct the relevant parameters in the first current limiting strategy adopted by the digital object with a higher priority value, so that under the second current limiting strategy obtained after the correction of the relevant parameters, the current limiting degree of the digital object will be reduced compared to the first current limiting strategy before the correction. At the same time, the relevant parameters in the first current limiting strategy adopted by the digital object with a lower priority value are corrected, so that under the second current limiting strategy obtained after the correction of the relevant parameters, the current limiting degree of the digital object will be increased compared to the first current limiting strategy before the correction.

[0078] Step S5: performing current limiting transmission on the digital object using the second current limiting strategy.

[0079] In this embodiment, after obtaining the second rate limiting policy corresponding to each digital object requested by each client in step S4, the corresponding digital object is rate-limited and transmitted to the corresponding requesting client using the corresponding second rate limiting policy. For example, client A requests digital objects a1 and a2. Digital object a1 corresponds to the second rate limiting policy x1, and digital object a2 corresponds to the second rate limiting policy x2. Client B requests digital object b1. Digital object b1 corresponds to the second rate limiting policy y1. Digital object a1 is rate-limited and transmitted to client A using the second rate limiting policy x1. Digital object a2 is rate-limited and transmitted to client A using the second rate limiting policy x2. Digital object b1 is rate-limited and transmitted to client B using the second rate limiting policy y1.

[0080] In this embodiment, in the case where the digital object is composed of multiple data types, such as a digital object composed of video, attached text, and real-time barrage, the second flow limiting strategy corresponding to the digital object will include multiple types of second flow limiting strategies, and the multiple types of second flow limiting strategies correspond one-to-one to the multiple data types included in the digital object. For example, in the case where the digital object includes sensor data and file data whose data volume exceeds a set value, the corresponding second flow limiting strategy for the file data of the digital object whose data volume exceeds the set value is a sliding window algorithm flow limiting strategy, and the corresponding second flow limiting strategy for the sensor data of the digital object is a token bucket algorithm flow limiting strategy. During the transmission process of returning the digital object to the client, the corresponding second flow limiting strategy is called based on the data type currently transmitted by the digital object for flow limiting transmission. If the data type currently transmitted is sensor data, it is flow limited by the token bucket algorithm flow limiting strategy.

[0081] This application provides a composite current limiting method for high-volume data transmission based on a digital network. First, the protocol header of the requested data is parsed to determine the target communication protocol for data transmission; the target data type and priority value of the digital object are determined based on the metadata in the requested digital object; a corresponding first current limiting strategy is determined based on the target communication protocol and the target data type; the parameters of the first current limiting strategy are modified based on the priority value to obtain a second current limiting strategy; and the digital object is current-limited and transmitted using the second current limiting strategy. Thus, the application combines multiple current limiting strategies, selects a current limiting strategy that is adapted to the current data type and communication protocol from multiple current limiting strategies based on the data type recorded in the metadata of the digital object being transmitted and the communication protocol based on the data transmission, and performs current limiting processing on the digital object to adapt to the data transmission requirements in different scenarios, thereby improving the stability and reliability of the system. At the same time, based on the priority of the digital object, the corresponding current limiting strategy is adjusted, so that a lower current limiting degree is given to the digital object with a higher priority and a higher current limiting degree is given to the digital object with a lower priority, thereby optimizing resource utilization during data transmission. The solution of this application can effectively prevent the risk of system overload caused by massive requests, ensure the stability and availability of core services through an intelligent traffic control mechanism, maintain QoS indicators under sudden traffic impacts, achieve a dynamic balance between transmission efficiency and system reliability, and significantly improve user service experience in high-load scenarios.

[0082] In combination with the above embodiments, in one embodiment, the present application also provides a composite current limiting method for high-traffic data transmission based on a data network. In the composite current limiting method for high-traffic data transmission based on a data network, step S4 may include steps S41 to S42:

[0083] Step S41: Determine target parameters for parameter correction according to the first current limiting strategy.

[0084] In this embodiment, the parameter types modified in this application are different for different types of first rate limiting strategies. Specifically, when the first rate limiting strategy corresponding to the digital object is a sliding window algorithm rate limiting strategy, the target parameter for parameter modification is determined to be the window size. When the first rate limiting strategy corresponding to the digital object is a token bucket algorithm rate limiting strategy, the target parameter for parameter modification is determined to be the token generation rate. When the first rate limiting strategy corresponding to the digital object is a leaky bucket algorithm rate limiting strategy, the target parameters for parameter modification are determined to be the leaky bucket capacity and the leakage rate.

[0085] Step S42: According to the priority value and the preset correction rule, the target parameter is corrected to obtain a second current limiting strategy.

[0086] In this embodiment, the present application pre-establishes a preset correction rule, and based on the target parameter for parameter correction corresponding to the digital object and the priority value of the digital object, determines the correction amount corresponding to both the target parameter and the priority value from the preset correction rule, and uses the correction amount to perform parameter correction on the target parameter in the first current limiting strategy of the digital object, and obtains the second current limiting strategy corresponding to the digital object, and subsequently performs current limiting transmission on the digital object through the second current limiting strategy.

[0087] In combination with the above embodiments, in one embodiment, the present application also provides a composite current limiting method for high-traffic data transmission based on a digital network. In the composite current limiting method for high-traffic data transmission based on a digital network, step S42 may include steps S421 to S423:

[0088] Step S421: Determine a basic priority value according to the priority values ​​of all digital objects in the cache queue.

[0089] In this embodiment, a cache queue caches multiple digital objects to be returned to the requesting client. Based on the priority values ​​of all digital objects in the current cache queue, a basic priority value is determined. An optional implementation method is: sort the priority values ​​of all digital objects in the current cache queue according to the size of the values, and take the priority value corresponding to the median as the basic priority value. For example, the priority values ​​of all digital objects in the current cache queue are sorted by value size to obtain the result of 1, 3, 3, 5, 7. The middle value 3 is the median, so the basic priority value is determined to be 3; the priority values ​​of all digital objects in the current cache queue are sorted by value size to obtain the result of 1, 2, 3, 3, 5, 6. The two middle values ​​are 3 and 3 respectively, so the average of the two is taken as the median, that is, the basic priority value is determined to be 3.

[0090] Step S422: Determine a priority deviation value between the priority value of the digital object and the basic priority value.

[0091] In this embodiment, after determining a base priority value based on the priority values ​​of all digital objects to be transmitted in the cache queue in step S421, the priority value of the digital object to be transmitted in the current cache queue is subtracted from the base priority value to obtain a priority deviation value between the priority value of the digital object and the base priority value. Each digital object to be transmitted in the cache queue will obtain a corresponding priority deviation value through the same subtraction method as described above.

[0092] Step S423: According to the priority deviation value, the basic value and adjustment step of the target parameter, and the target percentage value, the target parameter is corrected to obtain a second current limiting strategy, and the target percentage value is a value corresponding to both the priority value of the digital object and the type of the target parameter.

[0093] In this embodiment, based on the priority value and the type of the target parameter of the digital object to be transmitted in the current cache queue, a target percentage value corresponding to both the priority value and the type of the target parameter is determined. That is, when determining the target percentage value of a digital object, based on the priority value of the digital object and the type of the target parameter corresponding to the digital object, a percentage value that matches both is determined as the target percentage value of the digital object. At the same time, the present application sets corresponding base values ​​and adjustment steps for different types of target parameters. After obtaining the priority deviation value and target percentage value corresponding to the digital object in the current cache queue through steps S421 to S423, the target parameter of the digital object is modified by a preset algorithm based on the priority deviation value, target percentage value, and the pre-set base value and adjustment step of the target parameter of the type, to obtain a second current limiting strategy for the digital object. The difference between the second current limiting strategy obtained after parameter modification and the first current limiting strategy before parameter modification is that the target parameter in the first current limiting strategy has changed, and has been modified from the initial base value through the calculation of the determined relevant parameters and the preset algorithm. The expression of the preset algorithm is: y=x+(am)×w×L, where y represents the value of the target parameter after parameter correction, x is the basic value of the target parameter, a is the priority value of the digital object, m is the priority deviation value, w is the target percentage value corresponding to both the priority value of the digital object and the type of the target parameter, and L is the adjustment step of the target parameter.

[0094] In conjunction with the above embodiments, in one embodiment, the present application also provides a composite current limiting method for high-traffic data transmission based on a digital network. In this composite current limiting method for high-traffic data transmission based on a digital network, before step S4, the method further includes step S04: adjusting the priority value of the digital object based on the waiting time of the digital object in the cache queue.

[0095] In this embodiment, new digital objects are constantly being stored in the cache queue, and some digital objects may remain in the cache queue for a long time. If a new digital object with a higher priority than the digital object that has remained in the cache queue for a long time is subsequently stored in the cache queue, and if a lower flow restriction is also given to the new digital object, and the digital object that has remained in the cache queue for a long time has a lower priority than the new digital object, and the digital object that has remained in the cache queue for a long time continues to be given a higher flow restriction, this will further aggravate the length of time that the digital object that has remained in the cache queue for a long time remains in the cache queue, resulting in unreasonable resource allocation for data transmission in the entire system, that is, some low-priority digital objects in the cache queue have not been allocated appropriate resources for data transmission for a long time. Therefore, this application will dynamically adjust the priority value of the digital object based on the waiting time of the digital object in the cache queue to avoid the above situation, thereby ensuring more reasonable resource allocation for data transmission in the entire system. One optional implementation involves multiplying the waiting time of a digital object in the cache queue by a preset percentage to obtain a priority correction for the digital object. This correction is then added to the priority value recorded in the metadata of the digital object to obtain the adjusted priority value for the digital object. The preset percentage can be set based on the actual application scenario and is not specifically limited here. For example, if digital object a in the cache queue has a priority of 3, the waiting time of digital object a in the cache queue is 200ms, and the preset percentage is 0.1%, the corresponding priority correction for digital object a is 200 × 0.1% = 2, resulting in an adjusted priority value of 5 for digital object a.

[0096] In this embodiment, the aforementioned method of dynamically adjusting the priority of digital objects in real time based on their waiting time in the cache queue can lead to excessively high frequency adjustments, which can easily waste system resources. Furthermore, the entire current limiting system frequently re-evaluates and processes digital objects in the cache queue based on the adjusted priority, resulting in frequent system responses and reduced processing efficiency. To address this issue, an alternative implementation of the present application, which adjusts the priority of digital objects based on their waiting time in the cache queue, is provided to reduce the frequency of priority adjustments and thus reduce system response time.

[0097] In this embodiment, another optional implementation is to determine whether the waiting time of the digital object in the cache queue is greater than or equal to a time threshold. The time threshold can be set according to actual application scenarios and is not specifically limited here, such as 100ms, 200ms, etc. If the waiting time of the digital object in the cache queue is not greater than or equal to the time threshold, the priority value of the digital object remains unchanged and the priority value of the digital object is not adjusted. If the waiting time of the digital object in the cache queue is greater than or equal to the time threshold, the priority value of the digital object is dynamically adjusted. The adjustment strategy is to increase the priority value recorded in the metadata of the digital object by a set percentage for each increase in the waiting time of the digital object in the cache queue. The set time can be set according to actual application scenarios and is not specifically limited here, such as 50ms, 100ms, etc. The set percentage can be set according to actual application scenarios and is not specifically limited here, such as 10%, 20%, etc. It should be understood that the priority value of a digital object has an upper limit, and after being adjusted to the highest priority during the dynamic adjustment process of the priority value of a digital object, the priority of the digital object will no longer be increased.

[0098] For example, the preset duration threshold is 200ms, the preset set duration is 50ms, and the preset set percentage is 10%. The waiting times of digital objects a and b in the cache queue are 150ms and 200ms, respectively, and the priority values ​​recorded in their metadata are 3 and 5. Since the waiting time of digital object a in the cache queue is less than the set duration, the priority value of digital object a remains unchanged. However, the waiting time of digital object b in the cache queue is equal to the set duration, so the priority value of digital object b is dynamically adjusted. Since the waiting time of digital object b in the cache queue is 200ms and the preset set duration is 50ms, the waiting time of digital object b in the cache queue increases by 4 set durations. Each set duration increases the set percentage by 10% based on the priority value recorded in the metadata of digital object b. Accordingly, the current priority value of digital object b is dynamically adjusted to 5×(1+40%)=7, and the priority value of digital object b is dynamically adjusted to 7. In this application, the larger the priority value, the less throttling will occur. If the waiting time for digital object b in the cache queue continues to 240ms, the priority value will not be adjusted, and will remain at the previously adjusted value of 7, because the set time has not yet been increased. However, if the waiting time for digital object b in the cache queue continues to 250ms, the set time has been increased, so the priority value will continue to adjust, and will be dynamically adjusted to 5 × (1 + 50%) = 7.5.

[0099] In the present application, when the composite current limiting method for high-traffic data transmission based on the digital network provided in the present application also includes step S04 before step S4, step S4 may include: according to the adjusted priority value, modifying the parameters of the first current limiting strategy to obtain the second current limiting strategy.

[0100] In this embodiment, when the composite current limiting method for high-traffic data transmission based on a digital network provided by the present application also includes step S04 before step S4, the priority value of the digital object used in the process of performing parameter correction on the first current limiting strategy corresponding to itself based on the priority value of the digital object in step S4 to obtain the corresponding second current limiting strategy is the priority value of the digital object after each dynamic adjustment.

[0101] Specifically: When a digital object is initially stored in a cache queue, if the waiting time of the digital object in the cache queue is not greater than or equal to the time threshold, the first current limiting strategy corresponding to the digital object will be modified based on the priority value recorded in the metadata of the digital object to obtain a second current limiting strategy, and then the digital object will be transmitted with limited current using the second current limiting strategy. If the digital object has not been transmitted with limited current for a long time or has not been fully transmitted to the requesting client, and the waiting time in the cache queue is greater than or equal to the time threshold, the priority value of the digital object will be dynamically adjusted in the same manner as in the above-mentioned implementation method, and then the parameters of the first current limiting strategy corresponding to the digital object will be modified with the adjusted priority value of the digital object to obtain a new second current limiting strategy, and then the subsequent data of the digital object will be transmitted with limited current using the new second current limiting strategy.

[0102] In combination with the above embodiments, in one embodiment, the present application also provides a composite current limiting method for high-traffic data transmission based on a data network. In the composite current limiting method for high-traffic data transmission based on a data network, before step S5, the method further includes steps S051 to S052:

[0103] Step S051: Determine the actual measured value of the network traffic during the data transmission process.

[0104] In this embodiment, in the process of determining the second flow limiting strategy for a digital object based on the data type, priority value, and communication protocol of the digital object, the present application only optimizes and adjusts the flow limiting strategy for the digital object based on the attribute data of the digital object itself to meet the transmission requirements of the digital object. However, this process does not take into account the network traffic of the entire system. This may lead to irrational network traffic distribution of the entire system, resulting in overload or excessive traffic surplus. For example, in the entire process of determining the second flow limiting strategy for the digital object, only the data type of the digital object is considered to meet the transmission requirements of the digital object. At the same time, the priority of the digital object is considered and the flow limiting strategy is appropriately adjusted to meet the requirements of the transmission of high-priority digital objects and the degree of flow limiting for low-priority digital objects. However, when each digital object adopts its corresponding second flow limiting strategy for flow limiting transmission, the actual network traffic of the entire system is not taken into account. This may lead to overload or excessive traffic surplus of the entire system. To solve this problem, before step S5, the present application further adjusts the second flow limiting strategy corresponding to each digital object in the cache queue based on the relationship between the actual measured value of the network traffic of the entire system during the data transmission process and the set value of the network traffic of the entire system. The set value of the network flow is the expected network flow value when the entire system is in an appropriate state. First, the actual measured value of the network flow during the current data transmission process is determined.

[0105] Step S052: dynamically adjusting the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network flow.

[0106] In this embodiment, based on the actual measured value of the network traffic during the current data transmission process and the preset network traffic setting value, the second current limiting strategy corresponding to each digital object in the cache queue is dynamically adjusted through PID feedback regulation.

[0107] In the present application, when step S051 to step S052 are included before step S5, step S5 may include: performing flow-limited transmission on the digital object by using the dynamically adjusted second flow-limiting strategy.

[0108] In this embodiment, when the present application includes dynamically adjusting the second current limiting policy corresponding to each digital object in the cache queue through steps S051 to S052, the implementation method of step S5 is as follows: for each digital object in the cache queue, current limiting transmission is performed on itself based on the second current limiting policy corresponding to itself after dynamic adjustment. For example, the cache queue includes digital object a1, and the second current limiting policy corresponding to the dynamically adjusted digital object a1 is z1; and includes digital object a2, and the second current limiting policy corresponding to the dynamically adjusted digital object a2 is z2; and includes digital object a3, and the second current limiting policy corresponding to the dynamically adjusted digital object a3 is z3. Accordingly, digital object a1 is current limited through z1, digital object a2 is current limited through z2, and digital object a3 is current limited through z3.

[0109] In combination with the above embodiments, in one embodiment, the embodiment of the present application further provides a composite current limiting method for high-traffic data transmission based on a digital network. In the composite current limiting method for high-traffic data transmission based on a digital network, step S052 may include: when the second current limiting strategy is a token bucket algorithm, dynamically adjusting the token bucket capacity or token generation rate in the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network traffic; when the second current limiting strategy is a leaky bucket algorithm, dynamically adjusting the leaky bucket capacity or water output rate in the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network traffic; when the second current limiting strategy is a sliding window algorithm, dynamically adjusting the window size or request processing rate in the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network traffic.

[0110] In this embodiment, the PID feedback control process includes proportional control (P), integral control (I), and differential control (D). Proportional control is used to adjust the data transmission rate or bandwidth allocation based on the deviation between the set value of network traffic and the actual measured value of network traffic. In other words, the controller output is adjusted according to the magnitude of the current network traffic error. This control determines the controller's response strength to the current deviation. Integral control is used to adjust the data transmission rate or bandwidth allocation based on the cumulative deviation between the set value of network traffic and the actual measured value of network traffic. In other words, the controller output is adjusted based on the accumulation of the current network traffic error over time. This control can eliminate the system's steady-state error, ensuring that the system ultimately stabilizes near the set value and avoids deviation. Differential control is used to adjust the data transmission rate or bandwidth allocation based on the rate of change between the set value of network traffic and the actual measured value of network traffic. In other words, the controller output is adjusted based on the rate of change of the current network traffic error. This control can predict future changes in network traffic, adjust the transmission rate in advance, reduce congestion and delays, and improve network performance. The PID feedback regulation adjusts the relevant parameters of the second flow limiting strategy for each digital object in the cache queue based on the relationship between the set value of the network traffic in each control item and the actual measured value of the network traffic, thereby adjusting the data transmission rate or bandwidth allocation, so as to keep the actual measured value of the network traffic as stable as possible near the set value. In the case where the second flow limiting strategy is a token bucket algorithm, based on the relationship between the actual measured value of the network traffic and the set value of the network traffic, when the PID feedback regulation adjusts the token bucket algorithm, it adjusts the token bucket capacity or token generation rate of the token bucket algorithm. In the case where the second flow limiting strategy is a leaky bucket algorithm, based on the relationship between the actual measured value of the network traffic and the set value of the network traffic, when the PID feedback regulation adjusts the leaky bucket algorithm, it adjusts the leaky bucket capacity or water output rate of the leaky bucket algorithm. In the case where the second flow limiting strategy is a sliding window algorithm, based on the relationship between the actual measured value of the network traffic and the set value of the network traffic, when the PID feedback regulation adjusts the sliding window algorithm, it adjusts the window size or request processing rate of the sliding window algorithm.

[0111] In this embodiment, if Figure 2As shown, the present application provides a composite current limiting method for high-volume data transmission based on a digital network. The method first receives request data initiated by a client and performs security rule checks. If the request data is determined to be qualified, the requested digital object is identified and then sent back to the client as the digital object to be transmitted. Metadata analysis is performed on the digital object to determine the target data type and priority value of the digital object. Communication protocol identification is performed on the request data corresponding to the digital object to determine the target communication protocol to be adopted. Based on the target data type and target communication protocol corresponding to the digital object, a current limiting strategy that matches the target data type and target communication protocol is determined and designated as the first current limiting strategy for the digital object. Target parameters of the first current limiting strategy for the digital object are then modified based on the priority value of the digital object to obtain a second current limiting strategy for the digital object. Next, based on the relationship between the actual measured value of the current network traffic and the set value of the network traffic, the corresponding parameters of the second current limiting strategy for the digital object are dynamically adjusted through PID feedback control to ensure that the actual measured value of the network traffic during the transmission of the digital object in the entire system remains stable near the set value. Finally, the digital object is transmitted with current limiting control using the second current limiting strategy for the digital object after PID feedback control.

[0112] In this embodiment, the flow limiting process of the token bucket algorithm is as follows: Figure 3 As shown, a token bucket is set up, initially containing a set number of tokens. The token generation rate is a dynamically adjustable parameter. When transmitting a digital object using this token bucket algorithm, the token bucket is checked for sufficient tokens. If sufficient tokens are available, the corresponding number of tokens are removed from the bucket and the digital object is allowed to be transmitted. If insufficient tokens are available, the data transmission of the digital object is temporarily suspended. During the rate limiting process, new tokens are continuously generated and placed in the token bucket to ensure timely response to sudden traffic demands.

[0113] In this embodiment, the flow limiting process of the leaky bucket algorithm is as follows: Figure 4 As shown, the leaky bucket drains digital object data at a fixed rate, which represents the average data transmission rate. When digital object data needs to be transmitted, it first enters the leaky bucket. Regardless of how quickly the digital object data flows in, the leaky bucket always drains the digital object data at a stable rate. This ensures that the average rate of digital object data transmission does not exceed a preset limit, ensuring long-term stable transmission. If the digital object data flows in too quickly during a certain period of time, the leaky bucket temporarily stores the excess digital object data until sufficient time is available for transmission at a stable rate.

[0114] In this embodiment, the current limiting process of the sliding window algorithm is as follows: Figure 5 As shown, the window size is set, which is the amount of data allowed to be sent within a set time range, as well as the maximum amount of data allowed to be sent per unit time. The set time range is divided into multiple time segments, and the amount of digital object data transmitted is recorded in each time segment. Throughout the process, the data transmission volume within the current window is checked to see if it exceeds the allowed data volume. If it does not exceed the allowed data volume, the digital object data transmission is allowed to continue. If it exceeds the allowed data volume, the transmission of the digital object data is temporarily denied and retried in the next window. The system continuously monitors the movement of the window to ensure the accuracy and real-time performance of the flow control.

[0115] In conjunction with the above embodiments, in one implementation, the present application also provides a composite current limiting method for high-traffic data transmission based on a digital network. In this composite current limiting method for high-traffic data transmission based on a digital network, the method further includes: when the priority values ​​of all digital objects in the cache queue are adjusted to the highest priority, evenly allocating resources for the data transmission of each digital object.

[0116] In this embodiment, after the priority values ​​of all digital objects in the cache queue reach the highest priority, resources are evenly allocated for data transmission of each digital object in the cache queue based on a set value of network traffic.

[0117] In combination with the above embodiments, in one embodiment, the embodiment of the present application also provides a composite current limiting method for high-traffic data transmission based on the digital network. In this composite current limiting method for high-traffic data transmission based on the digital network, when the target communication protocol is the HTTP protocol, the method further includes: filtering the request data through a regular rule library to determine whether the request data belongs to a known attack pattern; in the case where no rule is hit, analyzing the parameter semantics of the request data through an AST parsing engine to determine whether the request data is a suspicious request; in the case where the request data is a suspicious request, directing the request data to an isolated environment for execution to monitor abnormal behavior; and in the case where no abnormal behavior is detected, responding to the request data.

[0118] In this embodiment, limiting the flow during data transmission is not only a means of performance optimization, but also an important measure for security protection. This application will combine the security rules of the communication protocol to more effectively defend against various security threats.

[0119] Specifically: When the communication protocol for data transmission is HTTP, the client initiates a request and encapsulates the request into HTTP format and sends it to the server. After receiving the request data, the server uses the OWASP (Open Web Application Security Project) regular rule set (CRS, which contains more than 3,000 characteristic rules for various attacks) to check whether the request data contains specific XSS patterns, such as <script>.< / script> , and other known SQL injection or XSS attack patterns. If the request data matches a signature in the regularization rule base, the system will flag the request data as a potential attack and reject or perform further in-depth inspection of the request data. If the request data does not match a signature in the regularization rule base, the system will use an AST (Abstract Syntax Tree)-based parser to perform in-depth lexical and syntactic analysis of the request data parameters to identify more complex, non-standard character combinations or code structures that may constitute SQL injection or XSS attacks. After identifying potential attack code, the system will further mark the request data as suspicious and prepare for the next step of dynamic sandbox detection. For requests marked as suspicious, the system will redirect the request data to an isolated execution environment (i.e., a sandbox) for inspection. In the sandbox, the system simulates the execution of the code segments in the request data while closely monitoring for any unusual database operations or DOM tampering. If any unusual behavior is detected, the system will confirm that the request data is an attack request and immediately block further execution of the request data. If no unusual behavior is detected, the request data is deemed safe and processed normally.

[0120] In combination with the above embodiments, in one embodiment, the embodiment of the present application also provides a composite current limiting method for high-traffic data transmission based on the digital network. In the composite current limiting method for high-traffic data transmission based on the digital network, when the target communication protocol is the gRPC protocol, the method further includes: verifying the RPC method signature in the request data to determine whether the request data is legal, and the RPC method signature is a unique signature generated by the server for each RPC method; when the request data is legal, determining whether the call chain of the request data is an abnormal call chain based on the request data and the service dependency graph; when the call chain is an abnormal call chain, blocking the abnormal call chain, and the service dependency graph is a dependency relationship between services built based on the information recorded by each data request; or determining whether the request data is legal based on the dynamic token in the request data, and the dynamic token is sent to the client after the server verifies the certificate and fingerprint of the connection request of the client sending the request data; when the request data is legal, responding to the request data.

[0121] In this embodiment, when the data transmission protocol is HTTP, the client, based on business logic, prepares the RPC method to be called, the corresponding RPC method signature, and the required parameters for the RPC method, and encapsulates them into request data. The RPC method signature is a unique signature (e.g., SHA256) generated by the server for each RPC method and is used to sign the request data. Upon receiving the request data, the server first verifies whether the RPC method signature in the request data matches the signature pre-stored on the server. This ensures that the request data has not been tampered with and that it is indeed a call to the intended RPC method. If the request data passes the signature verification, the server analyzes the request data and the service dependency graph to determine whether the call chain is legitimate. The service dependency graph is constructed by the server based on a large amount of historical request data records and describes the dependency relationships between services. For example, a normal call chain might be "user service → order service → payment service", while the reverse call from "payment service → log service" might be abnormal. The server checks whether the call chain represented by the current request data conforms to the expected service dependency relationships. If the call chain of the request data is abnormal, the server will immediately block the request data to prevent potential security risks.

[0122] In this embodiment, another optional implementation involves the client establishing a connection with the server using mutual TLS (Transport Layer Security) encryption. During this process, the client and server perform mutual authentication during the connection establishment process. The client presents its client certificate to the server, and the server presents its server certificate to the client. After receiving the client certificate, the server verifies its validity, whether it is signed by a trusted certificate authority, and whether the certificate information (such as the client's public key and fingerprint) matches expectations. The server may also check other attributes in the client certificate, such as the certificate's validity period and the client's identity, to ensure that the client has permission to access the requested RPC service. After successfully verifying the client's legitimacy, the server generates a dynamic token and sends it to the client. This dynamic token is an encrypted string containing the client's identity information, the token's generation time, and its validity period. Dynamic tokens are time-sensitive and valid for a certain period of time after generation. After the validity period expires, the client must request a new dynamic token from the server. After receiving the dynamic token, the client saves it locally and records it in subsequent RPC requests. When the client calls an RPC method, it encapsulates the RPC method name, parameters, and the dynamic token as request data and sends it to the server.

[0123] In this embodiment, after receiving the client's request data, the server verifies whether the dynamic token in the request data is expired or has been tampered with, and whether the identity information in the dynamic token matches the client initiating the request. If the dynamic token verification passes, the request data is determined to be legal, and the corresponding RPC method is executed based on the legal request data. After execution is completed, the server generates response data.

[0124] In this embodiment, another optional implementation method is: based on the call frequency, parameter size and response error rate of the system's RPC method, determine whether it is currently in an abnormal request process, and if it is determined that it is in an abnormal request process, take corresponding security measures, such as recording logs, sending alarms or activating a fuse mechanism. Specifically: analyze the number of requests received by the system per unit time, determine whether the system has abnormal high-frequency requests, and monitor the size of the request parameters to determine possible parameter injection attacks or resource exhaustion attacks, for example, continuously transmitting a data stream greater than 10KB, and analyze the error rate of the system response to determine whether the system is under attack or is about to crash. When one of the call frequency, parameter size and response error rate exceeds its corresponding normal range, an abnormal judgment will be triggered, and corresponding security measures will be taken at this time, such as recording logs, sending alarms or activating a fuse mechanism.

[0125] In combination with the above embodiments, in one embodiment, the embodiment of the present application also provides a composite current limiting method for high-traffic data transmission based on the digital network. In the composite current limiting method for high-traffic data transmission based on the digital network, when the target communication protocol is the WebSocket protocol, the method further includes: performing a binary magic number check on the request data to determine whether the request data is legal, and / or performing a malicious keyword scan on the request data to determine whether the request data is legal; if the request data is legal, responding to the request data; adjusting the number of connections of each client according to the CPU usage of the server; and disconnecting the request connection that meets the preset heartbeat monitoring mechanism according to the preset heartbeat monitoring mechanism.

[0126] In this embodiment, the client first constructs a WebSocket request data and sends it to the server. After receiving the request data, the server performs a binary magic check, specifically checking whether the request data begins with a specific binary sequence (magic number). The magic number is part of the WebSocket protocol and is used to distinguish WebSocket requests from other network requests. If the beginning of the request data does not meet the magic number requirements of the WebSocket protocol, the server considers the request invalid and will reject the connection or return an error response. For text-formatted request data, the server uses the AC automaton algorithm to perform real-time malicious keyword scanning. If the request data contains any malicious keywords, the server determines the request data is invalid and will reject the connection, terminate the existing connection, or take other security measures. After establishing a connection with the client, the server dynamically adjusts the maximum number of connections per client based on the current CPU usage. For example, when CPU usage reaches 70%, the server may reduce the maximum number of connections per IP address from 100 to 30 to reduce server load and avoid performance issues. Once a WebSocket connection is established, the server and client will periodically send heartbeat frames (such as PING frames) to maintain the connection. The server implements a heartbeat detection mechanism, requiring the client to send a PING frame every 30 seconds. If the client fails to respond twice in a row (i.e., fails to send a PING frame or fails to correctly reply to a PONG frame), the server will consider the connection invalid and close the connection. The server also maintains a countdown for idle connections. If a connection remains inactive (i.e., no data is sent or received) for a preset period (e.g., 30 minutes), the server will automatically release the resources occupied by the connection. After the WebSocket connection is established and successfully passes the heartbeat detection, bidirectional communication between the client and server is possible. The client can send data to the server and the server can send data to the client. During communication, the server will continuously monitor CPU usage and connection status and adjust the number of connections as needed or close any connections that do not meet the requirements.

[0127] The composite current limiting method for high-traffic data transmission based on the digital network provided by this application not only enables the system to more effectively defend against security threats such as SQL injection, XSS attacks, and RPC abuse through specific security rules of the communication protocol, thereby improving the overall security. By applying multi-level current limiting and multi-strategy combinations, the system can automatically select the most appropriate current limiting strategy based on the metadata of the digital object, optimize traffic management, and improve the stability and response speed of the system. In addition, coherent data transmission and automated current limiting strategies ensure a smooth experience for users when sending different types of data. Whether it is video or text files, they will not be interrupted due to current limiting, which improves user satisfaction. At the same time, through metadata semantic parsing and the generation of dynamic policy chains, a deep coupling of current limiting rules and data transmission logic is achieved. For example, when transmitting a digital object composed of video, attached text and real-time barrage, the system will automatically assign independent current limiting strategies based on the metadata of each part, while maintaining the consistency of the overall transmission session.

[0128] In the overall system architecture of the present application, the digital object identification system creates a unique identification ID to ensure the identifiability of digital objects in the system, and works closely with the digital object registration system. The registration system extracts metadata to provide a decision-making basis for flow limiting. The digital object warehouse system is responsible for efficiently storing digital objects to facilitate quick retrieval and access. With flow limiting as the core, the flow limiting method is flexibly selected based on metadata, such as combining the advantages of sliding window, token bucket and leaky bucket algorithms, and dynamically switching or combining them according to different data scenarios. For example, when the network load is light and the priority of transmitted data is not high, the sliding window flow limiting threshold can be appropriately relaxed or the token generation rate of the token bucket algorithm can be reduced to improve transmission efficiency; when high-priority data appears, the strategy is quickly adjusted to ensure its bandwidth. This architecture effectively supports the implementation of a composite flow limiting method through the collaborative work of various components, ensuring that the system can respond flexibly to different situations and ensure the stability and efficiency of data transmission.

[0129] In the Internet of Things (IoT), a sector encompassing the digital network, a large number of sensors continuously generate data. For example, environmental monitoring sensors transmit a small amount of data per second, but with extremely high real-time requirements. This application, based on the characteristics of sensor data, uses sliding window throttling to ensure timely data transmission. Furthermore, it utilizes a dynamic feedback mechanism to promptly adjust sliding window parameters (such as shortening the window time or increasing the request threshold) when network conditions change (e.g., increased latency), ensuring data real-time transmission and system stability. For data transmission and control command issuance for production equipment within the industrial internet, which encompasses the digital network, equipment data transmission volumes are high and may experience bursts (e.g., large amounts of status data uploaded when a device suddenly fails). Furthermore, control commands have high real-time requirements. This application's token bucket algorithm effectively supports bursts, ensuring timely upload of device status data. By prioritizing transmission bandwidth for high-priority control commands (based on priority information in metadata), it ensures the continuity and stability of the production process. For example, in an automated production line, when an emergency occurs, high-priority fault alarm commands are prioritized for transmission. At the same time, the throttling strategy for other lower-priority data (such as equipment normal operation status monitoring data) is dynamically adjusted to ensure that critical commands are quickly transmitted to the control center, allowing timely measures to prevent losses. In big data analysis scenarios within the Internet of Things (IoT), data comes from a variety of sources, including real-time and batch data. For real-time data streams (such as real-time user behavior data for immediate analysis), sliding window throttling is used to ensure data freshness and timely analysis. For batch data transmission (such as historical data for model training), a token bucket algorithm is used to balance transmission efficiency and system resource utilization. Furthermore, based on data timeliness (e.g., the value of certain real-time analysis data decreases with aging) and importance (e.g., prioritizing core business data), personalized throttling is implemented using metadata to ensure efficient and stable operation of the big data analysis system under varying loads and data demands.

[0130] Based on the same inventive concept, the present application provides a composite current limiting system for large-volume data transmission based on a digital network, such as Figure 6 As shown, the system 600 includes:

[0131] The target communication protocol determination module 601 is used to parse the protocol header of the request data and determine the target communication protocol for data transmission;

[0132] The metadata parsing module 602 is used to determine the target data type and priority value of the digital object according to the metadata in the requested digital object;

[0133] A first current limiting strategy determining module 603 is configured to determine a corresponding first current limiting strategy according to the target communication protocol and the target data type;

[0134] A second current limiting strategy determining module 604 is configured to modify parameters of the first current limiting strategy according to the priority value to obtain a second current limiting strategy;

[0135] The current limiting control module 605 is configured to perform current limiting transmission on the digital object using the second current limiting strategy.

[0136] Optionally, the second current limiting strategy determination module 604 includes:

[0137] a target parameter determination module, configured to determine a target parameter for parameter correction according to the first current limiting strategy;

[0138] The parameter correction module is used to perform parameter correction on the target parameter according to the priority value and the preset correction rule to obtain a second current limiting strategy.

[0139] Optional parameter correction module, including:

[0140] A basic priority value determination module is used to determine a basic priority value based on the priority values ​​of all digital objects in the cache queue;

[0141] A priority deviation value determining module, configured to determine a priority deviation value between a priority value of a digital object and the basic priority value;

[0142] A parameter correction submodule is used to correct the target parameter according to the priority deviation value, the basic value and adjustment step of the target parameter, and the target percentage value to obtain a second current limiting strategy, where the target percentage value is a value corresponding to both the priority value of the digital object and the type of the target parameter.

[0143] Optionally, the system 600 further includes:

[0144] A priority adjustment module, configured to adjust the priority value of a digital object according to the waiting time of the digital object in the cache queue;

[0145] The second current limiting strategy determining module 604 is configured to modify parameters of the first current limiting strategy according to the adjusted priority value to obtain a second current limiting strategy.

[0146] Optionally, the system 600 further includes:

[0147] An actual measurement value determination module, used to determine the actual measurement value of network traffic during data transmission;

[0148] A dynamic adjustment module, configured to dynamically adjust the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network flow;

[0149] The current limiting control module 605 is configured to perform current limiting transmission on the digital object using the dynamically adjusted second current limiting strategy.

[0150] Optional, dynamic adjustment module, including:

[0151] A first dynamic adjustment module is configured to dynamically adjust the token bucket capacity or token generation rate in the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network traffic when the second current limiting strategy is a token bucket algorithm;

[0152] A second dynamic adjustment module is configured to dynamically adjust the leaky bucket capacity or water outflow rate in the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network flow when the second current limiting strategy is a leaky bucket algorithm;

[0153] The third dynamic adjustment module is used to dynamically adjust the window size or request processing rate in the second current limiting strategy through PID feedback adjustment according to the actual measurement value and the set value of the network traffic when the second current limiting strategy is a sliding window algorithm.

[0154] Optionally, the system 600 further includes:

[0155] The resource allocation module is used to evenly allocate resources for data transmission of each digital object when the priority values ​​of all digital objects in the cache queue are adjusted to the highest priority.

[0156] Optionally, the system 600 further includes:

[0157] a first security module, configured to filter the request data using a regular rule library to determine whether the request data belongs to a known attack pattern when the target communication protocol is the HTTP protocol;

[0158] A second security module is configured to analyze the parameter semantics of the request data through an AST parsing engine to determine whether the request data is a suspicious request if no rule is hit;

[0159] a third security module, configured to direct the request data to an isolated environment for execution and perform abnormal behavior monitoring if the request data is a suspicious request;

[0160] The first response module is used to respond to the request data when no abnormal behavior is detected.

[0161] Optionally, the system 600 further includes:

[0162] A fourth security module is configured to verify the RPC method signature in the request data to determine whether the request data is legal when the target communication protocol is the gRPC protocol, where the RPC method signature is a unique signature generated by the server for each RPC method;

[0163] A fifth security module is configured to determine, if the request data is legal, whether a call chain of the request data is an abnormal call chain based on the request data and a service dependency graph;

[0164] a sixth security module, configured to block the abnormal call chain if the call chain is an abnormal call chain, wherein the service dependency graph is a dependency relationship between services constructed based on information recorded for each data request;

[0165] a seventh security module, configured to determine whether the request data is legitimate based on a dynamic token in the request data, wherein the dynamic token is sent to the client after the server verifies the certificate and fingerprint of the connection request of the client sending the request data;

[0166] The second response module is configured to respond to the request data if the request data is legal.

[0167] Optionally, the system 600 further includes:

[0168] an eighth security module, configured to, when the target communication protocol is the WebSocket protocol, perform a binary magic number check on the request data to determine whether the request data is legal, and / or perform a malicious keyword scan on the request data to determine whether the request data is legal;

[0169] A third response module, configured to respond to the request data if the request data is legal;

[0170] The first connection adjustment module is used to adjust the number of connections of each client according to the CPU usage of the server;

[0171] The second connection adjustment module is configured to disconnect a request connection that complies with a preset heartbeat monitoring mechanism according to the preset heartbeat monitoring mechanism.

[0172] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0173] It should be noted that for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the order of the actions described, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0174] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0175] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0176] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0179] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0180] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0181] The above is a detailed introduction to a composite current limiting method and system for large-volume data transmission based on a digital network provided by this application. This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method of this application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this application.

Claims

1. A composite current limiting method for high-volume data transmission based on digital networking, characterized in that: The method comprises: Parse the protocol header of the request data to determine the target communication protocol for data transmission; Determining a target data type and a priority value of the digital object according to metadata in the requested digital object; Determining a corresponding first current limiting strategy according to the target communication protocol and the target data type; Modifying parameters of the first current limiting strategy according to the priority value to obtain a second current limiting strategy; Performing current-limited transmission on the digital object using the second current-limiting strategy; According to the priority value, the first current limiting strategy is parameter-corrected to obtain the second current limiting strategy, including: determining a target parameter for parameter correction according to the first current limiting strategy; and performing parameter correction on the target parameter according to the priority value and a preset correction rule to obtain the second current limiting strategy; Among them, according to the priority value and the preset correction rule, the target parameter is corrected to obtain a second current limiting strategy, including: determining a basic priority value according to the priority values ​​of all digital objects in the cache queue; determining a priority deviation value between the priority value of the digital object and the basic priority value; according to the priority deviation value, the basic value and adjustment step of the target parameter, and the target percentage value, the target parameter is corrected to obtain a second current limiting strategy, and the target percentage value is a value corresponding to both the priority value of the digital object and the type of the target parameter.

2. A composite current limiting method for high-volume data transmission based on a digital network according to claim 1, characterized in that: Before modifying parameters of the first current limiting strategy according to the priority value to obtain the second current limiting strategy, the method further includes: Adjusting the priority value of the digital object according to the waiting time of the digital object in the cache queue; The step of modifying parameters of the first current limiting strategy according to the priority value to obtain a second current limiting strategy includes: According to the adjusted priority value, the parameters of the first current limiting strategy are modified to obtain a second current limiting strategy.

3. The composite current limiting method for high-volume data transmission based on digital networking according to claim 1 is characterized in that: Before performing current limiting transmission on the digital object using the second current limiting strategy, the method further includes: Determine actual measurements of network traffic during data transmission; Dynamically adjusting the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network flow; The limiting transmission of the digital object by using the second limiting strategy includes: The digital object is subjected to current-limited transmission using the dynamically adjusted second current-limiting strategy.

4. A composite current limiting method for high-volume data transmission based on a digital network according to claim 3, characterized in that: According to the actual measurement value and the set value of the network flow, the second current limiting strategy is dynamically adjusted through PID feedback regulation, including: In the case where the second current limiting strategy is a token bucket algorithm, dynamically adjusting the token bucket capacity or token generation rate in the second current limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network traffic; In the case where the second flow limiting strategy is a leaky bucket algorithm, dynamically adjusting the leaky bucket capacity or water outflow rate in the second flow limiting strategy through PID feedback regulation according to the actual measurement value and the set value of the network flow; When the second current limiting strategy is a sliding window algorithm, the window size or request processing rate in the second current limiting strategy is dynamically adjusted through PID feedback regulation according to the actual measurement value and the set value of the network traffic.

5. The composite current limiting method for high-volume data transmission based on digital networking according to claim 2 is characterized in that: The method further comprises: When the priority values ​​of all digital objects in the cache queue are adjusted to the highest priority, resources are evenly allocated for data transmission of each digital object.

6. The composite current limiting method for high-volume data transmission based on digital networking according to claim 1 is characterized in that: In the case where the target communication protocol is the HTTP protocol, the method further includes: Filtering the request data through a regular rule library to determine whether the request data belongs to a known attack pattern; If no rule is hit, the parameter semantics of the request data are analyzed by the AST parsing engine to determine whether the request data is a suspicious request; If the request data is a suspicious request, direct the request data to an isolated environment for execution to monitor abnormal behavior; If no abnormal behavior is detected, respond to the request data.

7. The composite current limiting method for high-volume data transmission based on a digital network according to claim 1 is characterized in that: In the case where the target communication protocol is the gRPC protocol, the method further includes: Verify the RPC method signature in the request data to determine whether the request data is legal, where the RPC method signature is a unique signature generated by the server for each RPC method; If the request data is legal, determining whether the call chain of the request data is an abnormal call chain based on the request data and the service dependency graph; In the case where the call chain is an abnormal call chain, the abnormal call chain is blocked, and the service dependency graph is a dependency relationship between services constructed based on information recorded for each data request; or Determining whether the request data is legitimate based on a dynamic token in the request data, wherein the dynamic token is sent to the client after the server verifies the certificate and fingerprint of the connection request of the client sending the request data; If the request data is legal, respond to the request data.

8. The composite current limiting method for high-volume data transmission based on digital networking according to claim 1 is characterized in that: In the case where the target communication protocol is the WebSocket protocol, the method further includes: Performing a binary magic number check on the request data to determine whether the request data is legal, and / or performing a malicious keyword scan on the request data to determine whether the request data is legal; If the request data is valid, respond to the request data; Adjust the number of connections for each client based on the server's CPU usage; According to a preset heartbeat monitoring mechanism, a request connection that complies with the preset heartbeat monitoring mechanism is disconnected.

9. A composite current limiting system for high-volume data transmission based on digital networking, characterized in that: The system comprises: A target communication protocol determination module is used to parse the protocol header of the request data and determine the target communication protocol for data transmission; A metadata parsing module, configured to determine a target data type and a priority value of the digital object according to metadata in the requested digital object; A first current limiting strategy determining module, configured to determine a corresponding first current limiting strategy according to the target communication protocol and the target data type; A second current limiting strategy determination module is configured to modify parameters of the first current limiting strategy according to the priority value to obtain a second current limiting strategy; a current limiting control module, configured to perform current limiting transmission on the digital object using the second current limiting strategy; The second current limiting strategy determination module includes: a target parameter determination module for determining a target parameter for parameter correction according to the first current limiting strategy; a parameter correction module for performing parameter correction on the target parameter according to the priority value and the preset correction rule to obtain the second current limiting strategy; Among them, the parameter correction module includes: a basic priority value determination module, which is used to determine the basic priority value according to the priority values ​​of all digital objects in the cache queue; a priority deviation value determination module, which is used to determine the priority deviation value between the priority value of the digital object and the basic priority value; a parameter correction sub-module, which is used to perform parameter correction on the target parameter according to the priority deviation value, the basic value and adjustment step of the target parameter, and the target percentage value to obtain a second current limiting strategy, and the target percentage value is a value corresponding to both the priority value of the digital object and the type of the target parameter.

Images