Independent website access traffic control methods, devices, equipment, media, and products

Abstract

This application relates to a method, apparatus, device, medium, and product for controlling access traffic to an independent website. The method includes: receiving a data query request; initializing the flow control level corresponding to the request to the highest level among multiple member levels, where different member levels indicate data source operation permissions for different data scales; determining whether the total request traffic per unit time of the current independent website exceeds the independent website's flow control threshold, and if so, setting the flow control level to the second lowest level; recursively determining whether the total request traffic per unit time of the page corresponding to the request exceeds the page's flow control threshold, and if so, lowering the flow control level by one level if the flow control level has not reached the lowest level; determining whether the flow control level obtained by the data query request meets the level threshold required by the data source retrieval service corresponding to the request, and if so, controlling the data source retrieval service to execute the data query task according to the flow control level. This application improves the stability of independent websites by implementing flow control.

Description

Technical Field

[0001] This application relates to network security technology, and in particular to a method, apparatus, device, medium, or product for controlling access traffic to an independent website. Background Technology

[0002] E-commerce platforms based on independent websites need to provide network access security services for each independent website, including preventing saturation attacks or unauthorized access, to ensure that the independent websites can operate more stably with the support of the platform and to ensure that users have a good operating experience when accessing the independent websites.

[0003] In daily life, common problems that prevent users from accessing independent websites include malicious attacks, slow response times caused by web crawlers, surges in single-page traffic during flash sales, and partial or complete unavailability of data sources such as databases storing product information. These issues fall under the broad category of cybersecurity and require effective prevention and control measures to ensure the healthy operation of independent websites.

[0004] Traditional control methods do not take into account the relatively isolated maintenance of each independent website. They focus on the security maintenance of the data source retrieval service corresponding to the data source, and only implement appropriate control when the traffic of the data source retrieval service is too large. Although this method can achieve the effect of flow control, it is too brute-force and fails to take a systematic approach to security control from the perspective of the entire independent website. It cannot comprehensively regulate the traffic of the independent website and will still affect the operational quality of the independent website. Summary of the Invention

[0005] The purpose of this application is to solve the above-mentioned problems by providing a method for controlling the access traffic of an independent website and the corresponding apparatus, device, non-volatile readable storage medium, and computer program product.

[0006] According to one aspect of this application, a method for controlling access traffic to an independent website is provided, comprising the following steps:

[0007] Upon receiving a data query request, initialize the flow control level corresponding to the request to the highest level among multiple member levels. Different member levels indicate the data source operation permissions for different data sizes.

[0008] Execute the independent site flow control task, determine whether the total request traffic per unit time of the current independent site exceeds the independent site flow control threshold, and if it does, set the flow control level to the second lowest level among the multiple member levels;

[0009] Execute page flow control task, recursively determine whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold. If it exceeds the threshold, the flow control level is downgraded by one level if the flow control level has not reached the minimum level.

[0010] The system executes a data source flow control task, determining whether the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request. When the threshold is reached, the system controls the data source retrieval service to execute a data query task on the corresponding data source according to the flow control level.

[0011] Optionally, execute the independent site flow control task to determine whether the total request traffic per unit time of the current independent site exceeds the independent site flow control threshold. If it exceeds the threshold, set the flow control level to the second lowest level among the multiple member levels, including:

[0012] When the total request traffic per unit time of the current independent website exceeds the independent website's flow control threshold, the corresponding independent website flow control event is triggered;

[0013] In response to the independent site flow control event, the flow control level is set to the second lowest level among the multiple member levels. The second lowest level is used to indicate that the data query request is granted data source operation permissions with a minimum data size.

[0014] In response to the independent website flow control event, skip the step of executing the page flow control task and proceed to the step of executing the data source flow control task;

[0015] If the total request traffic per unit time of the current independent website does not exceed the independent website's flow control threshold, the flow control level will not be modified, and the steps of executing page flow control tasks will proceed sequentially.

[0016] Optionally, a page flow control task is executed, recursively determining whether the total request traffic per unit time for the page corresponding to the data query request exceeds the page flow control threshold, including:

[0017] Determine whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold. If the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold, trigger the corresponding page flow control event.

[0018] In response to the page flow control event, determine whether the flow control level is the lowest among the multiple member levels. If it is not the lowest level, lower the flow control level by one level.

[0019] The above process is iterated until the flow control level is the lowest level. At this point, the data service request is responded to by returning a null value, and the execution of subsequent steps is terminated, or until the data service request is asynchronously processed by the step of executing the data source flow control task.

[0020] Optionally, a data source flow control task is executed to determine whether the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request, including:

[0021] When the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request, a data query event is triggered;

[0022] In response to the data query event, determine whether the deviation ratio of the average request latency of the corresponding data source retrieval service relative to the set value reaches a preset threshold. When the preset threshold is reached, control the data source retrieval service to perform a data query task based on local data content query on the corresponding data source.

[0023] When the preset threshold is not reached, the data source retrieval service is controlled to perform a data query task based on querying all data content of the corresponding data source.

[0024] Optionally, when the preset threshold is not reached, the data source retrieval service is controlled to perform a data query task based on querying all data content of the corresponding data source, including:

[0025] Determine whether the total request traffic per unit time of the corresponding data source retrieval service has reached the set threshold or thrown an exception. When the set threshold is reached or an exception is thrown, trigger a circuit breaker event.

[0026] In response to the circuit breaker event, the data source retrieval service is controlled to perform a data query task based on local data content query on the corresponding data source;

[0027] When no circuit breaker event is triggered, the data source retrieval service is controlled to perform a data query task based on querying all data content of the corresponding data source.

[0028] Optionally, after executing the data source flow control task, the following may be included:

[0029] Determine whether the flow control level obtained by the data query request reaches the level threshold required by the data sorting service corresponding to the request. If the level threshold is reached, trigger a sorting event.

[0030] In response to the sorting event, the result data obtained after the data query task is executed by the data source retrieval service is sorted to obtain a sorting result;

[0031] The data query request is responded to with the sorting result.

[0032] Optionally, the flow control level includes the following membership levels:

[0033] The highest level indicates that the user has data source operation rights across the entire scope of the data content.

[0034] The second-highest level is used to indicate data source operation permissions within the scope of local data content;

[0035] The second lowest level is used to indicate that the user has data source operation permissions within a set range smaller than the size of the local data content.

[0036] The lowest level indicates that the user does not have data source operation permissions for any data content.

[0037] According to another aspect of this application, an independent website access traffic control device is provided, comprising:

[0038] The initialization module is set to receive data query requests and initialize the flow control level corresponding to the request to the highest level among multiple member levels. Different member levels indicate the data source operation permissions for different data sizes.

[0039] The in-site flow control module is configured to perform flow control tasks for independent sites, determine whether the total request traffic per unit time of the current independent site exceeds the flow control threshold of the independent site, and set the flow control level to the second lowest level among the multiple member levels when it exceeds the threshold.

[0040] The page flow control module is configured to perform page flow control tasks and recursively determine whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold. If it exceeds the threshold, the flow control level will be downgraded by one level if the flow control level has not reached the minimum level.

[0041] The retrieval flow control module is configured to perform data source flow control tasks, determine whether the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request, and when it does, control the data source retrieval service to perform data query tasks on the corresponding data source according to the flow control level.

[0042] According to another aspect of this application, a standalone website access traffic control device is provided, including a central processing unit and a memory, wherein the central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the standalone website access traffic control method described in this application.

[0043] According to another aspect of this application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the independent website access flow control method in the form of computer-readable instructions, wherein the computer program, when invoked by a computer, executes the steps included in the method.

[0044] According to another aspect of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0045] Compared to existing technologies, this application initializes a flow control level for an independent website upon receiving a data query request, assigning it the highest level among multiple member levels. Then, it can determine whether to adjust the flow control level not only based on the total request traffic per unit time at the website-wide level, but also at the page level corresponding to the data query request, adjusting the flow control level according to the page's total request traffic per unit time. Finally, it enters the data source flow control task execution process, where it controls the corresponding data source retrieval service to execute the data query task according to the flow control level. Thus, flow control is implemented at three levels: independent website, page, and data source. All parameters are associated with the same flow control level. By adapting to traffic changes at different levels, the flow control level of the data query request can be automatically and flexibly adjusted. When traffic overflows at any object, corresponding adjustments can be made to ensure that the data query request can be executed in the most efficient way possible. Furthermore, for the entire independent website, since all data query requests can be scheduled in this way, the flow rate of the entire independent website is organically regulated, and the robustness of the operation of the independent website, pages, data sources, etc. is stronger. This not only ensures the smooth operation of the independent website on a daily basis but also ensures that the independent website increases its throughput and effectively resists various illegal attacks. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the network architecture used in an exemplary application scenario for implementing the independent website access traffic control method of this application;

[0048] Figure 2 This is a flowchart illustrating one embodiment of the independent website access traffic control method of this application;

[0049] Figure 3 This is a flowchart illustrating the execution of independent station flow control tasks in an embodiment of this application;

[0050] Figure 4 This is a flowchart illustrating the execution of page flow control tasks in an embodiment of this application;

[0051] Figure 5 This is a flowchart illustrating the execution of the retrieval flow control task in an embodiment of this application;

[0052] Figure 6 This is a schematic diagram illustrating the process of implementing the circuit breaker mechanism in an embodiment of this application;

[0053] Figure 7 This is a flowchart illustrating the execution of the sorting mechanism in an embodiment of this application;

[0054] Figure 8 This is a schematic diagram of the independent website access traffic control device of this application;

[0055] Figure 9 This is a schematic diagram of the structure of a standalone website access traffic control device used in this application. Detailed Implementation

[0056] Please see Figure 1 The network architecture employed in an exemplary application scenario of this application includes a central control server 80, a data server 81, and a large number of independent websites 82. The central control server 80 can be used to deploy computer program products programmed according to the independent website access traffic control method of this application, thereby running the traffic control service of this application. The data server 81 can be used to deploy one or more data sources of this application. Each independent website also runs on a corresponding server. Each independent website is typically used to run an online store. Each independent website can install a corresponding plugin for the traffic control service to enable the invocation of the traffic control service. The central control server 80 can access the data sources in the data server 81 by executing a data retrieval service.

[0057] In some embodiments, the flow control service can run directly on each of the individual independent stations and directly access the data source in the data server 81. In other embodiments, the flow control service can also run on each of the data servers, providing corresponding interfaces for each independent station to call. Thus, it can be seen that the flow control service of this application can run in multiple ways to provide flow control services to each independent station.

[0058] The following will be illustrated by taking the independent website access traffic control method according to this application as a computer program product and running it on an independent website as an example.

[0059] Please see Figure 1 According to the independent website access traffic control method provided in this application, in one embodiment, it includes the following steps:

[0060] Step S1100: Receive a data query request and initialize the flow control level corresponding to the request to the highest level among multiple member levels. Different member levels indicate the data source operation permissions for different data scales.

[0061] When an independent site receives a data query request, by default, it first performs an initialization operation on the data query request. During the initialization operation, the flow control level Level 1 of the data query request is set to the highest level, Level 1_0.

[0062] The flow control levels may include multiple member levels, each indicating the data source operation permissions for different data sizes. For example, in one embodiment, four member levels may be set, as shown below:

[0063] The highest level, Level 1_3, indicates that the user has data source operation permissions across the entire data content. Correspondingly, for each data service request, the user executes all the logic corresponding to the data access service implemented on an independent site. The data access service includes a data retrieval phase and a result processing phase. The retrieval phase mainly executes the coarse retrieval process of the data access service. In the coarse retrieval process, multi-channel retrieval can be implemented. The result processing phase can be used to integrate the coarsely retrieved data, such as sorting the data and generating reports.

[0064] The second-highest level, Level l_2, is used to indicate that the user has data source operation permissions within the scope of local data content. Accordingly, the logic corresponding to the corresponding data access service can be adjusted to implement only part of the logic. For example, only some of the recall channels can be used in the coarse recall process, or only some data slices can be recalled. It can also be set to not execute the business logic corresponding to sorting.

[0065] The second lowest level, Level l_1, is used to indicate that data source operation permissions are enjoyed within a set range smaller than the local data content. Correspondingly, when providing data access services for the data query request, data access services can be provided according to the preset minimum data content size, so that the data sources that can be recalled only include a range smaller than the local data content, so that the data query request can only be queried within the specified smaller data source range.

[0066] The lowest level, Level l_0, indicates that no data source operation permissions are granted for any data content. Accordingly, the business logic of the data access service can be skipped for the data query request, no operation can be performed, and the data query request can be answered directly with a null value.

[0067] In other embodiments, multiple member levels, different in number than those in the above examples, can be set for the flow control level according to actual needs. This is achieved by establishing a correspondence between each member level and the business logic of a specific data access service, thereby ensuring that the data content relied upon by each member level is of different scales and exhibits a certain gradient relationship. Those skilled in the art can flexibly implement this based on the principles disclosed above in this application.

[0068] Step S1200: Execute the independent station flow control task, determine whether the total request traffic per unit time of the current independent station exceeds the independent station flow control threshold, and if it exceeds, set the flow control level to the second lowest level among the multiple member levels;

[0069] This step primarily implements traffic control at the independent website level, thus executing the corresponding independent website flow control task. In this task, it first determines whether the current independent website's total request traffic per unit time (QPS) exceeds a preset flow control threshold. If it does, it indicates that the independent website should no longer fully respond to the data query requests, and therefore the flow control level corresponding to the data query requests needs to be lowered. The flow control level is directly set to the second-lowest level, which is typically constrained to a minimum data content size. Therefore, this creates a front-end protection mechanism for the entire independent website, especially its backend services. This mechanism helps to resist saturation attacks from a large number of simultaneous data query requests, forming the first line of defense for the independent website.

[0070] In some embodiments, the independent station flow control threshold can be a measured threshold or an empirical threshold, which can be set as needed by those skilled in the art.

[0071] After setting the flow control level to the second lowest among the multiple member levels, step S1300 can be executed to further implement flow control based on the page corresponding to the data query request. However, in a preferred embodiment, considering that the total request traffic per unit time of the independent website has reached a high level, and the flow control level of the data query request has been reduced to the second lowest level corresponding to the minimum data content size, step S1300 can be skipped and step S1400 can be executed directly. This can reduce business steps by lowering the service level of the data query request and avoid increasing unnecessary system overhead on the independent website.

[0072] If the total request traffic per unit time of the current independent site does not exceed its independent site flow control threshold, it indicates that the current independent site is still in a stable operating state, and therefore can provide data access services for the data query request with all its might. Therefore, step S1300 can be directly continued.

[0073] Step S1300: Execute the page flow control task and recursively determine whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold. If it exceeds the threshold, the flow control level will be downgraded by one level if the flow control level has not reached the lowest level.

[0074] This step is mainly used to implement page-level traffic control. Therefore, the corresponding page is first determined based on the data query request, which can be obtained from the URL of the request. Then, it is determined whether the total request traffic per unit time for this page exceeds the page flow control threshold. When it exceeds the page flow control threshold, it indicates that the corresponding page is being accessed at a high frequency, and the existing access traffic will cause instability in the access of this page and even the independent website. Therefore, it is necessary to reduce the flow control level corresponding to the data query request. For example, if the flow control level of the data query request is the highest level, then after determining that it exceeds the page flow control threshold through the previous detection, the flow control level is set to the second highest level; if the flow control level of the data query request is the second lowest level, then if it is downgraded, the flow control level will be set to the lowest level.

[0075] In some embodiments, the page flow control threshold may be a measured threshold or an empirical threshold, which can be set as needed by those skilled in the art.

[0076] In some embodiments, the data query request is added to an asynchronous queue. Only when the request is dequeued and consumed does it proceed to step S1400 to execute the data source flow control task. During the queuing period, step S1300 can be executed iteratively. Through iteration, it checks whether the total request traffic per unit time of the page exceeds the page flow control threshold. In each iteration, if the total request traffic per unit time exceeds the page flow control threshold, the flow control level of the data query request is downgraded by one level, thus recursively adjusting the flow control level. It is easy to understand that in this case, when the flow control level reaches the lowest level, there is no need for further downgrading; the data access service business logic corresponding to the lowest level can be executed directly.

[0077] In some embodiments, if the total request traffic per unit time of the page does not exceed the page flow control threshold after multiple recursive checks, and if the flow control level has not reached the highest level at this time, the flow control level can be increased by one level to enhance the flexibility of the page-based flow control mechanism.

[0078] As can be seen, by combining this step S1300, the flow control level of the data query request can be dynamically set according to the real-time traffic of the page to which the data query request is directed, so that the flow control level can effectively reflect the busyness of the corresponding page and make responsive changes. This mechanism forms a second security shield for the independent website.

[0079] Step S1400: Execute the data source flow control task, determine whether the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request, and when it reaches the threshold, control the data source retrieval service to execute the data query task on the corresponding data source according to the flow control level.

[0080] This step is mainly used to implement flow control tasks at the data source level. Therefore, based on the flow control level finally determined by the data query request, the corresponding data source retrieval service is executed for the data query request so as to obtain the corresponding result data within the data content scale corresponding to the flow control level.

[0081] When performing data source flow control tasks, it is first determined whether the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the data query request. The level threshold can be set as needed by those skilled in the art, and should usually be lower than the highest level. For example, it can be set to the second lowest level, which means that the data source retrieval service will only be executed for the corresponding data query request when the flow control level reaches the second lowest level or above.

[0082] When the flow control level is determined to have reached the specified threshold, the data source retrieval service can be controlled to perform data query tasks on the corresponding data source according to the flow control level. The data source retrieval service is typically provided by a corresponding data source. For example, when executing a coarse data retrieval service corresponding to multiple retrieval channels, each retrieval channel can be encapsulated as a data source retrieval service. Each data source retrieval service retrieves data from its corresponding data source. Moreover, different data source retrieval services can rely on the same data source when performing coarse retrieval, such as the same database stored in a distributed system. Furthermore, multiple retrieval channels can also be encapsulated in a single data source retrieval service. Such flexible implementations are possible. The data source can refer to a complete database or a data slice in a distributed storage system.

[0083] The data source retrieval service is pre-implemented to execute corresponding data query tasks according to the specific member level specified by the flow control level, thereby enabling corresponding retrieval within the data content scale corresponding to the member level specified by the flow control level. The data content scale can be reflected by controlling the number of recall channels or by controlling the number of data sources it depends on, and so on, all of which can be flexibly implemented.

[0084] Once the data source retrieval service executes the corresponding data query task and obtains the coarse recall data, it can use the coarse recall data to respond to the data query request, thereby completing the entire business process. However, it is also possible that the data source retrieval service might throw an exception during the execution of the data query task, returning a null value to respond to the data query request.

[0085] In some embodiments, after obtaining result data from the data source retrieval service, the coarse recall data may be sorted before responding to the data query request, and the sorted result data may be used to respond to the data query request.

[0086] In some embodiments, during the operation of the data source retrieval service, the total request traffic per unit time corresponding to the data source can be further detected to adjust the flow control level, so as to build a third security barrier against traffic attacks for independent websites.

[0087] In some embodiments, the data source may be a product information database that specifically stores product information corresponding to product items, or a product image database that specifically stores product images corresponding to product items, but should not be limited thereto.

[0088] As can be seen from the above embodiments, this application initializes a flow control level for an independent website after a data query request arrives, making it hold the highest level among multiple member levels. Then, it can determine whether to adjust the flow control level not only based on the total request traffic per unit time at the website-wide level, but also specifically at the page level corresponding to the data query request, adjusting the flow control level according to the total request traffic per unit time of that page. Finally, it enters the data source flow control task execution process, where the corresponding data source retrieval service is controlled to execute the data query task according to the flow control level. Therefore, it can be seen that the flow control of objects at the independent website, page, and data source levels is all related to... Multiple security barriers are formed by using the same flow control level parameters. By adapting to traffic changes at different levels, the flow control level of the data query request can be automatically and flexibly adjusted. When traffic overflows at any object, corresponding adjustments can be made to ensure that the data query request can be executed in the most efficient way possible. Furthermore, for the entire independent website, since all data query requests can be scheduled in this way, the flow rate of the entire independent website is organically regulated, and the robustness of the operation of the independent website, pages, data sources, etc. is stronger. This not only ensures the smooth daily operation of the independent website, but also ensures that the independent website increases its throughput and effectively resists various illegal attacks.

[0089] Based on any embodiment of this application, please refer to Figure 3The system executes independent site flow control tasks, determining whether the total request traffic per unit time of the current independent site exceeds the independent site flow control threshold. If it does, the flow control level is set to the second lowest level among the multiple member levels, including:

[0090] Step S1210: When the total request traffic per unit time of the current independent site exceeds the independent site flow control threshold, trigger the corresponding independent site flow control event;

[0091] When the detection confirms that the total request traffic per unit time of the current independent site has exceeded the independent site flow control threshold, a corresponding independent site flow control event is triggered so that subsequent steps can be executed based on the event response mechanism. If the independent site flow control threshold has not been exceeded, the process can directly jump to step S1240.

[0092] Step S1220: In response to the independent station flow control event, set the flow control level to the second lowest level among the multiple member levels. The second lowest level is used to indicate that the data query request is granted data source operation permissions with the minimum data size.

[0093] In response to the independent site flow control event, the flow control level of the data query request is directly downgraded from the highest level to the second lowest level. Since the second lowest level only allows the data query request to have data source operation permissions for the smallest data content range, in fact, by setting the second lowest level, the subsequent business process of the data access service corresponding to the data query request is also controlled.

[0094] Step S1230: Respond to the independent site flow control event, skip the step of executing the page flow control task and proceed to the step of executing the data source flow control task;

[0095] After executing step S1220, continue to respond to the independent site flow control event. Since the flow control level has been reduced to the second lowest level, there is no need to enter step S1300 corresponding to the execution page flow control task. Instead, directly jump to step S1400 to execute the data source flow control task.

[0096] Step S1240: If the total request traffic per unit time of the current independent site does not exceed the independent site's flow control threshold, do not modify the flow control level, and proceed sequentially to the step of executing the page flow control task.

[0097] When the total request traffic per unit time of the current independent site does not exceed the independent site's flow control threshold, since the independent site has not yet entered the busy state represented by the flow control threshold, it can directly jump to step S1240 after step S1210 without modifying the flow control level, and directly enter step S1300 to execute the page flow control task.

[0098] As can be seen from the above embodiments, in this application, the process that the data query request needs to jump to is determined by comparing the total request traffic per unit time corresponding to the current time of the independent website with its corresponding set independent website flow control threshold. This allows the subsequent business process to be optimized according to the actual situation. Thus, not only can the load on the independent website be reduced by setting the flow control level when necessary, but the system overhead can also be saved by flexibly avoiding the flow control processing at the page level when necessary, thereby further reducing the load on the independent website.

[0099] Based on any embodiment of this application, please refer to Figure 4 The page flow control task is executed, recursively determining whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold, including:

[0100] Step S1310: Determine whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold. If the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold, trigger the corresponding page flow control event.

[0101] In the process of implementing flow control based on the page, it is first determined whether the total request traffic of the page corresponding to the data query request per unit time exceeds the pre-set page flow control threshold. When the page flow control threshold is exceeded, it indicates that the traffic of the corresponding page is overloaded. Thus, the corresponding page flow control event can be triggered so that subsequent steps can be executed based on the event response mechanism.

[0102] Step S1320: Respond to the page flow control event and determine whether the flow control level is the lowest level among the multiple member levels. If it is not the lowest level, lower the flow control level by one level.

[0103] In response to the page flow control event, it is further determined whether the flow control level has reached the lowest level among multiple member levels. When it is the lowest level, the iterative detection can theoretically be terminated. Otherwise, that is, when the flow control level is not the lowest level, the original member level can be downgraded by one level according to the gradient relationship of the member levels. For example, it can be downgraded from the highest level to the second highest level, from the second highest level to the second lowest level, and the second lowest level can be downgraded to the lowest level, etc.

[0104] Step S1330: Iterate the above process until the flow control level is the lowest level, then respond to the data service request and return a null value to terminate the execution of subsequent steps, or until the data service request is asynchronously processed by the step of executing the data source flow control task.

[0105] Since the data query request is usually waiting to be dequeued for consumption in an asynchronous mechanism, the above process can be iterated before it is consumed, and the execution can start again from step S1310, cyclically checking whether the flow control level needs to be downgraded, and continuously processing recursively.

[0106] During the above iterations, if the flow control level has reached the minimum level, it indicates that the page targeted by the data query request is too busy and should not be responded to. In this case, there is no need to execute subsequent business processes, and the data service request is directly answered with a null value. If the flow control level has not reached the minimum level during the above iterations, the data query request is dequeued and asynchronously consumed. At this time, the above iterations are naturally terminated, and step S1400 is executed.

[0107] As can be seen from the above embodiments, at the page level, the busy status of the corresponding page is monitored in real time by utilizing the time interval between the asynchronous queuing and exit of data query requests. When the page is too busy, its flow control level can be lowered in a timely manner, forming an adaptive dynamic adjustment mechanism. Under the action of this mechanism, the data query requests of the entire independent website can be efficiently scheduled according to the page traffic, thereby ensuring that the independent website achieves detailed and effective flow control as a whole.

[0108] Based on any embodiment of this application, please refer to Figure 5 The task involves executing data source flow control, determining whether the flow control level obtained by the data query request meets the required level threshold for the data source retrieval service corresponding to the request, including:

[0109] Step S1410: When the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request, a data query event is triggered;

[0110] During the execution of data source flow control tasks, when it is determined through detection that the flow control level ultimately obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the data query request, a corresponding data query event can be triggered so that subsequent business processes can be executed based on the event response mechanism.

[0111] Step S1420: In response to the data query event, determine whether the deviation ratio of the average request latency of the corresponding data source retrieval service relative to the set value reaches a preset threshold. When the preset threshold is reached, control the data source retrieval service to perform a data query task based on local data content query on the corresponding data source.

[0112] In response to the data query event, the access scale of the data query request to the corresponding data source can be further adjusted based on the specific performance of the average request latency of the data source retrieval service.

[0113] Specifically, a set value and a preset threshold are pre-set for the data source retrieval service. The set value represents the response latency threshold of the corresponding data source. When the average response latency generated by the data source retrieval service accessing the data source exceeds the response latency threshold, the data source is considered to be in a busy state; otherwise, it is not busy. The preset threshold is used to measure whether the magnitude of the average response latency exceeding the response latency threshold meets the necessary and sufficient condition for confirming that the data source is in a busy state. Accordingly, firstly, the difference between the average request latency of the data source retrieval service and the set value is divided by the set value to determine the deviation ratio. Then, the deviation ratio is compared with the preset threshold. When the deviation ratio is higher than the preset threshold, the data source is confirmed to be in a busy state; otherwise, the data source is confirmed to be in a non-busy state.

[0114] The busy status of the data source is confirmed by comparing the deviation ratio of the average request latency relative to the set value with the preset threshold. The deviation ratio is used to characterize the magnitude of the change. It has the ability to adaptively adjust in response to the set value and the preset threshold while maintaining an adjustment range within a certain range, thus making it more robust.

[0115] Once it is confirmed that the data source is in a busy state, the data source retrieval service can be controlled to enter the partial data content query business process, enabling the data source retrieval service to perform a data query task based on partial data content query on the corresponding data source. The partial data content can be a partial data slice generated by the data source stored in the distributed storage system, rather than a full data slice. Therefore, by querying partial data content, the load on the distributed storage coefficient can be reduced, which also reduces the load on the entire independent website.

[0116] Step S1430: When the preset threshold is not reached, control the data source retrieval service to perform a data query task based on querying all data content on the corresponding data source.

[0117] When step S1420 determines that the deviation ratio of the average request latency of the corresponding data source retrieval service from the set value has not reached the preset threshold, it indicates that the corresponding data source is in a non-busy state. At this time, the data source retrieval service can be controlled to perform a data query task based on the query of all data content on the corresponding data source in accordance with the normal business process. For example, the data query task can be performed in all data slices generated by the distributed storage system for the data source to perform a full query.

[0118] As can be seen from the above embodiments, after entering the business process of the data source retrieval service, the average request latency obtained after querying the data source by the data source retrieval service can be used to assess whether the corresponding data source is in a busy state. When the data source is in a busy state, the corresponding data query task can be controlled to narrow the data query scope, thereby reducing the load of the entire data service access of the independent website, improving the robustness of the entire independent website system, and ensuring a good access experience.

[0119] Based on any embodiment of this application, please refer to Figure 6 When the preset threshold is not reached, the data source retrieval service is controlled to perform a data query task based on querying all data content of the corresponding data source, including:

[0120] Step S1421: Determine whether the total request traffic per unit time of the corresponding data source retrieval service has reached the set threshold or thrown an exception. When the set threshold is reached or an exception is thrown, trigger the circuit breaker event.

[0121] In one embodiment, when the data source retrieval service confirms that the corresponding data source is in a non-busy state and a query process covering the entire data content needs to be initiated, the response capability of the data source can be further confirmed by detecting the total request traffic per unit time of the data source. Accordingly, it is first determined whether the total request traffic per unit time of the data source retrieval service has reached a set threshold. When the set threshold is reached, it indicates that it is processing a large number of requests, and therefore, a circuit breaker event is triggered to activate the circuit breaker mechanism.

[0122] In another embodiment, the circuit breaker event can also be triggered if the data source retrieval service throws an exception during the execution of a data query task.

[0123] Step S1422: In response to the circuit breaker event, control the data source retrieval service to perform a data query task based on local data content query on the corresponding data source;

[0124] In response to the circuit breaker event, the query is no longer performed based on global data content, but instead on local data content. That is to say, in the final stage, the system circuit breaker is further prevented based on the busy level of the data source retrieval service. The specific data query task is adjusted in a timely manner in response to the system circuit breaker event to ensure that the data query request can obtain the corresponding coarse recall data within a limited local data content range.

[0125] Step S1423: When no circuit breaker event is triggered, control the data source retrieval service to perform a data query task based on querying all data content of the corresponding data source.

[0126] If no circuit breaker event is triggered during the execution of a data query task by the data source retrieval service, it indicates that the traffic to the data source retrieval service is appropriate and the execution process is smooth. Therefore, the data source retrieval service continues to execute a data query task based on the full data content of the corresponding data source, in order to obtain coarsely retrieved data within the full data content scope.

[0127] As can be seen from the above embodiments, during the execution of the data source retrieval service, this application establishes a circuit breaker mechanism so that when the data source retrieval service is busy or throws an exception, timely adjustments can be made to control the query scale of the data query request in a timely manner, thereby effectively adjusting the load of the data source of the independent website and further ensuring the robustness of the independent website's operation.

[0128] Based on any embodiment of this application, please refer to Figure 7 After executing the data source flow control task, it includes:

[0129] Step S1500: Determine whether the flow control level obtained by the data query request reaches the level threshold required by the data sorting service corresponding to the request. When the level threshold is reached, trigger a sorting event.

[0130] After steps S1100 to S1400 of this application, the data query request will obtain corresponding coarse recall data after it is responded to normally. For these result data, according to the mechanism of the product recommendation system, a data sorting service can be used to sort and select the best results in order to produce the final sorting results.

[0131] Similarly, for the data sorting service, a suitable processing threshold can be set. When the data sorting service is invoked, it is first determined whether the flow control level obtained by the data query request meets the threshold required by the data sorting service. If the flow control level does not meet the threshold, it is easy to understand that the data sorting service will be skipped and execution will be terminated, and the data service request can be responded to with a null value. When the flow control level meets the threshold, it indicates that the data query request has obtained at least some coarsely recalled data from the data source as result data. Therefore, the data sorting service can be used, and a sorting event can be triggered to execute subsequent business processes based on the event response mechanism.

[0132] Step S1600: In response to the sorting event, sort the result data obtained after the data query task is executed by the data source retrieval service to obtain the sorting result;

[0133] In response to the sorting event, the data sorting service can sort the result data according to its preset sorting business logic. After sorting, the best data is selected as the sorting result, for example, the product data of several product items with the highest recommendation scores are selected to construct a product recommendation list.

[0134] Step S1700: Respond to the data query request with the sorting result.

[0135] After determining the sorting result, it can be pushed to the requester corresponding to the data query request, thus completing the response to the data query request. For example, the requester could be a user's terminal device on an independent website or any other device that initiated the data query request.

[0136] As can be seen from the above embodiments, this application, through traffic control, ensures that various data query requests can obtain corresponding query and sorting results with more stable reliability, ensuring the stable operation of the independent website and enabling users to obtain a good access experience.

[0137] Please see Figure 8 According to one aspect of this application, an independent website access traffic control device includes an initialization module 1100, an in-site flow control module 1200, a page flow control module 1300, and a retrieval flow control module 1400, wherein: the initialization module 1100 is configured to receive a data query request and initialize the flow control level corresponding to the request to the highest level among multiple member levels, with different member levels indicating data source operation permissions for different data scales; the in-site flow control module 1200 is configured to execute an independent website flow control task, determine whether the total request traffic per unit time of the current independent website exceeds the independent website flow control threshold, and set the flow control level when it exceeds the threshold. The page flow control module 1300 is set to the second lowest level among the multiple member levels. It is configured to execute a page flow control task, recursively determining whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold. If it does, the flow control level is lowered by one level if the flow control level has not reached the lowest level. The retrieval flow control module 1400 is configured to execute a data source flow control task, determining whether the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request. If it does, the data source retrieval service is controlled to execute a data query task on the corresponding data source according to the flow control level.

[0138] Based on any embodiment of this application, the site-wide flow control module 1200 includes: a site-wide event triggering unit, configured to trigger a corresponding site-wide flow control event when the total request traffic per unit time of the current independent site exceeds the site-wide flow control threshold; a site-wide setting unit, configured to respond to the site-wide flow control event by setting the flow control level to the second-lowest level among the multiple member levels, wherein the second-lowest level is used to indicate that the data query request is granted data source operation permissions with the minimum data size; a site-wide event response unit, configured to respond to the site-wide flow control event by skipping the step of executing the page flow control task and proceeding to the step of executing the data source flow control task; and a site-wide normal execution unit, configured to not modify the flow control level and sequentially proceed to the step of executing the page flow control task when the total request traffic per unit time of the current independent site does not exceed the site-wide flow control threshold.

[0139] Based on any embodiment of this application, the page flow control module 1300 includes: a page event triggering unit, configured to determine whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold, and trigger a corresponding page flow control event when the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold; a page event response unit, configured to respond to the page flow control event, determine whether the flow control level is the lowest level among the multiple member levels, and when it is not the lowest level, lower the flow control level by one level; and a page iteration processing unit, configured to iterate the above process until the flow control level is the lowest level, respond to the data service request and return a null value, and terminate the execution of subsequent steps, or until the data service request is asynchronously processed by the step of executing the data source flow control task.

[0140] Based on any embodiment of this application, the retrieval flow control module 1400 includes: a query event triggering unit, configured to trigger a data query event when the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request; a query event response unit, configured to respond to the data query event, determine whether the deviation ratio of the average request latency of the corresponding data source retrieval service relative to a set value reaches a preset threshold, and when the preset threshold is reached, control the data source retrieval service to perform a data query task based on partial data content query on the corresponding data source; and a normal query execution unit, configured to control the data source retrieval service to perform a data query task based on full data content query on the corresponding data source when the preset threshold is not reached.

[0141] Based on any embodiment of this application, the normal query execution unit includes: a circuit breaker event triggering unit, configured to determine whether the total request traffic per unit time of the corresponding data source retrieval service reaches a set threshold or throws an exception, and trigger a circuit breaker event when the set threshold is reached or an exception is thrown; a circuit breaker event response unit, configured to respond to the circuit breaker event and control the data source retrieval service to perform a data query task based on partial data content query on the corresponding data source; and a loop detection processing unit, configured to control the data source retrieval service to perform a data query task based on full data content query on the corresponding data source when no circuit breaker event is triggered.

[0142] Based on any embodiment of this application, the retrieval flow control module 1400 further includes: a sorting event triggering unit, configured to determine whether the flow control level obtained by the data query request reaches the level threshold required by the data sorting service corresponding to the request, and trigger a sorting event when the level threshold is reached; a sorting event response unit, configured to respond to the sorting event, sort the result data obtained after the data query task is executed by the data source retrieval service, and obtain a sorting result; and a sorting result push unit, configured to respond to the data query request with the sorting result.

[0143] Based on any embodiment of this application, the flow control level includes the following member levels: the highest level, which indicates that the user has data source operation rights within the scope of all data content; the second highest level, which indicates that the user has data source operation rights within the scope of a local data content; the second lowest level, which indicates that the user has data source operation rights within a set scope smaller than the local data content; and the lowest level, which indicates that the user does not have data source operation rights for any data content.

[0144] Another embodiment of this application also provides a standalone website access traffic control device. For example... Figure 9 The diagram shows the internal structure of a standalone website access flow control device. This device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable, non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database stores information sequences, and when executed by the processor, these computer-readable instructions enable the processor to implement a standalone website access flow control method.

[0145] The processor of the standalone station access flow control device provides computing and control capabilities to support the operation of the entire standalone station access flow control device. The memory of the standalone station access flow control device may store computer-readable instructions, which, when executed by the processor, cause the processor to perform the standalone station access flow control method of this application. The network interface of the standalone station access flow control device is used for communication with terminals.

[0146] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the independent site access traffic control device to which the present application is applied. The specific independent site access traffic control device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0147] In this embodiment, the processor is used to execute... Figure 8 The specific functions of each module are described, and the memory stores the program code and various data required to execute the above modules or sub-modules. The network interface is used to realize data transmission between user terminals or servers. The non-volatile readable storage medium in this embodiment stores the program code and data required to execute all modules in the independent station access flow control device of this application, and the server can call the server's program code and data to execute the functions of all modules.

[0148] This application also provides a non-volatile readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the stand-alone website access flow control method of any embodiment of this application.

[0149] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0150] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0151] In summary, this application enables the organic regulation of the traffic flow of independent websites on e-commerce platforms, making the operation of independent websites, pages, data sources, etc. more robust. It not only ensures the smooth daily operation of independent websites, but also ensures increased throughput and effectively resists various illegal attacks.

Claims

1. A method for controlling access traffic to an independent website, characterized in that, include: Upon receiving a data query request, initialize the flow control level corresponding to the request to the highest level among multiple member levels. Different member levels indicate the data source operation permissions for different data sizes. Execute the independent site flow control task, determine whether the total request traffic per unit time of the current independent site exceeds the independent site flow control threshold, and if it does, set the flow control level to the second lowest level among the multiple member levels; Execute page flow control task, recursively determine whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold. If it exceeds the threshold, the flow control level is downgraded by one level if the flow control level has not reached the minimum level. The system executes a data source flow control task, determining whether the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request. When the threshold is reached, the system controls the data source retrieval service to execute a data query task on the corresponding data source according to the flow control level.

2. The independent website access traffic control method according to claim 1, characterized in that, Execute the independent site flow control task, determine whether the total request traffic per unit time of the current independent site exceeds the independent site flow control threshold, and if it does, set the flow control level to the second lowest level among the multiple member levels, including: When the total request traffic per unit time of the current independent website exceeds the independent website's flow control threshold, the corresponding independent website flow control event is triggered; In response to the independent site flow control event, the flow control level is set to the second lowest level among the multiple member levels. The second lowest level is used to indicate that the data query request is granted data source operation permissions with a minimum data size. In response to the independent website flow control event, skip the step of executing the page flow control task and proceed to the step of executing the data source flow control task; If the total request traffic per unit time of the current independent website does not exceed the independent website's flow control threshold, the flow control level will not be modified, and the steps of executing page flow control tasks will proceed sequentially.

3. The independent website access traffic control method according to claim 1, characterized in that, Execute page flow control tasks, recursively determining whether the total request traffic per unit time for the page corresponding to the data query request exceeds the page flow control threshold, including: Determine whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold. If the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold, trigger the corresponding page flow control event. In response to the page flow control event, determine whether the flow control level is the lowest among the multiple member levels. If it is not the lowest level, lower the flow control level by one level. The above process is iterated until the flow control level is the lowest level. At this point, the data service request is responded to by returning a null value, and the execution of subsequent steps is terminated, or until the data service request is asynchronously processed by the step of executing the data source flow control task.

4. The independent website access traffic control method according to claim 1, characterized in that, Execute data source flow control tasks, and determine whether the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request, including: When the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request, a data query event is triggered; In response to the data query event, determine whether the deviation ratio of the average request latency of the corresponding data source retrieval service relative to the set value reaches a preset threshold. When the preset threshold is reached, control the data source retrieval service to perform a data query task based on local data content query on the corresponding data source. When the preset threshold is not reached, the data source retrieval service is controlled to perform a data query task based on querying all data content of the corresponding data source.

5. The independent website access traffic control method according to claim 4, characterized in that, When the preset threshold is not reached, the data source retrieval service is controlled to perform a data query task based on querying all data content on the corresponding data source, including: Determine whether the total request traffic per unit time of the corresponding data source retrieval service has reached the set threshold or thrown an exception. When the set threshold is reached or an exception is thrown, trigger a circuit breaker event. In response to the circuit breaker event, the data source retrieval service is controlled to perform a data query task based on local data content query on the corresponding data source; When no circuit breaker event is triggered, the data source retrieval service is controlled to perform a data query task based on querying all data content of the corresponding data source.

6. The method for controlling access traffic to an independent website according to any one of claims 1 to 5, characterized in that, After executing the data source flow control task, the following is included: Determine whether the flow control level obtained by the data query request reaches the level threshold required by the data sorting service corresponding to the request. If the level threshold is reached, trigger a sorting event. In response to the sorting event, the result data obtained after the data query task is executed by the data source retrieval service is sorted to obtain a sorting result; The data query request is responded to with the sorting result.

7. The independent website access traffic control method according to any one of claims 1 to 5, characterized in that, The flow control levels include the following membership levels: The highest level indicates that the user has data source operation rights across the entire scope of the data content. The second-highest level is used to indicate data source operation permissions within the scope of local data content; The second lowest level is used to indicate that the user has data source operation permissions within a set range smaller than the size of the local data content. The lowest level indicates that the user does not have data source operation permissions for any data content.

8. A device for controlling access traffic to independent websites, characterized in that, include: The initialization module is set to receive data query requests and initialize the flow control level corresponding to the request to the highest level among multiple member levels. Different member levels indicate the data source operation permissions for different data sizes. The in-site flow control module is configured to perform flow control tasks for independent sites, determine whether the total request traffic per unit time of the current independent site exceeds the flow control threshold of the independent site, and set the flow control level to the second lowest level among the multiple member levels when it exceeds the threshold. The page flow control module is configured to perform page flow control tasks and recursively determine whether the total request traffic per unit time of the page corresponding to the data query request exceeds the page flow control threshold. If it exceeds the threshold, the flow control level will be downgraded by one level if the flow control level has not reached the minimum level. The retrieval flow control module is configured to perform data source flow control tasks, determine whether the flow control level obtained by the data query request reaches the level threshold required by the data source retrieval service corresponding to the request, and when it does, control the data source retrieval service to perform data query tasks on the corresponding data source according to the flow control level.

9. A standalone website access flow control device, comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps included in the method as described in any one of claims 1 to 7.

10. A non-volatile readable storage medium, characterized in that, It stores a computer program in the form of computer-readable instructions, which, when invoked by a computer, performs the steps included in the method as described in any one of claims 1 to 7.

Images