High-concurrency condition processing system and method based on distributed system

Abstract

The application provides a high-concurrency working condition processing system and method based on a distributed system, and relates to the technical field of cloud computing. The method comprises the following steps: acquiring system running state parameter information of the distributed system; determining a current concurrency working condition level and a current high-concurrency response mode of the distributed system within a preset period according to the system running state parameter information and a high-concurrency identification threshold; inputting the system running state parameter information into a pre-trained concurrency event prediction model to obtain a high-concurrency working condition level change sensitivity of the distributed system, wherein the concurrency event prediction model is obtained by training historical concurrency event data of the distributed system; and determining a corresponding high-concurrency response mode of the distributed system within the preset period according to the high-concurrency working condition level change sensitivity, the current concurrency working condition level and the current high-concurrency response mode.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and in particular to a high-concurrency processing system and method based on a distributed system. Background Technology

[0002] With the advancement and development of cloud computing, more and more applications are being deployed and run in the cloud. One of the advantages of cloud computing over traditional data center mechanisms is the greater scalability and flexibility of cloud computing backend clusters, and the greater elasticity of cluster performance. However, the operational status of business application systems built on cloud computing clusters will exhibit unevenly distributed peak periods of business activity, depending on the characteristics of the business service objects. This can lead to high concurrency issues from time to time.

[0003] In real-world business applications, high concurrency scenarios can be categorized into both periodic and random high-concurrency events. Periodic high-concurrency events are characterized by concentration, planning, and a certain degree of periodicity. Examples include regular, time-distributed high-concurrency events such as annual shopping festivals or concentrated registration periods for social examinations, leading to a concentrated increase in system access pressure. Random high-concurrency events, on the other hand, are random and not pre-planned by the system operators, resulting from unforeseen and uncontrollable factors. Examples include high-concurrency events caused by natural disasters or war.

[0004] In existing technologies, whether vertical scaling improves the performance of a single machine or horizontal scaling addresses high concurrency through internet-based distributed cluster architecture, it results in a waste of resources caused by conventional accumulated hardware performance or conventional load balancing measures. Summary of the Invention

[0005] This invention provides a high-concurrency processing system and method based on a distributed system to solve the problems of high hardware costs and increased energy consumption in vertical scaling of traditional systems, as well as high technical and design implementation costs in horizontal scaling, and low utilization of expanded resources in non-high-concurrency operating conditions, resulting in equipment and energy waste.

[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0007] In a first aspect, embodiments of the present invention provide a method for handling high-concurrency conditions in a distributed system. The method includes: acquiring system operating status parameter information of the distributed system; determining the current concurrency level and current high-concurrency response mode of the distributed system within a preset period based on the system operating status parameter information and a high-concurrency identification threshold; inputting the system operating status parameter information into a pre-trained concurrency event prediction model to obtain the sensitivity of the high-concurrency level change of the distributed system, wherein the concurrency event prediction model is trained based on historical concurrency event data of the distributed system; and determining the corresponding high-concurrency response mode of the distributed system within the preset period based on the high-concurrency level change sensitivity, the current concurrency level, and the current high-concurrency response mode.

[0008] Further, the step of inputting the system operating status parameter information into a pre-trained concurrent event prediction model to obtain the sensitivity to changes in the high-concurrency operating condition level of the distributed system includes: inputting the system operating status parameter information into a first concurrent event model to obtain a first prediction result, wherein the first concurrent event prediction model is trained based on historical concurrent event data of the distributed system over a first time period; inputting the system operating status parameter information into a second concurrent event model to obtain a second prediction result, wherein the second concurrent event prediction model is trained based on historical concurrent event data of the distributed system over a second time period, wherein the first time period is longer than the second time period; and determining the sensitivity to changes in the high-concurrency operating condition level based on the first prediction result and the second prediction result.

[0009] Further, the first prediction result includes a first time distribution and a first level of concurrent events, and the second prediction result includes a second time distribution and a second level of concurrent events. The step of determining the sensitivity to changes in the high-concurrency operating condition level based on the first prediction result and the second prediction result includes: obtaining a set of time distribution prediction results based on the first time distribution and the second time distribution; obtaining a set of level prediction results based on the first level and the second level; combining the set of time distribution prediction results and the set of level prediction results to obtain a set of prediction results; and determining the sensitivity to changes in the high-concurrency operating condition level based on the consistency between the time distribution prediction results and the level prediction results in the prediction results.

[0010] Furthermore, the high concurrency identification threshold includes a first high concurrency threshold and a second high concurrency threshold. The step of determining the current concurrency level and current high concurrency response mode of the distributed system within a preset period based on the system operating status parameter information and the high concurrency identification threshold further includes: receiving the first high concurrency threshold and the second high concurrency threshold input by the user; and determining the current concurrency level and the current high concurrency response mode based on the first high concurrency threshold, the second high concurrency threshold, and the system operating status parameter information.

[0011] Further, the high-concurrency identification threshold includes a first high-concurrency operating condition threshold and a second high-concurrency operating condition threshold. The step of determining the current concurrency operating condition level and the current high-concurrency response mode of the distributed system within a preset period based on the system operating status parameter information and the high-concurrency identification threshold includes: obtaining the maximum and minimum number of accesses per unit time within a preset time period after the distributed system starts; determining the first high-concurrency operating condition threshold based on the maximum and minimum number of accesses; and determining the current concurrency operating condition level and the current high-concurrency response mode based on the first high-concurrency operating condition threshold, the second high-concurrency operating condition threshold, and the system operating status parameter information.

[0012] Further, determining the high-concurrency response mode corresponding to the distributed system within the preset period based on the high-concurrency operating condition level change sensitivity, the current concurrency operating condition level, and the current high-concurrency response mode includes: if the current concurrency level does not match the high-concurrency level corresponding to the current high-concurrency response mode, then determining the high-concurrency response mode based on the high-concurrency operating condition level change sensitivity, the current concurrency operating condition, and the current concurrency level.

[0013] Furthermore, the high-concurrency response mode includes one of the following: storing the business data and data replicas of the distributed system according to preset business logic and preset distributed storage cluster configuration; or, backing up the original data consistency configuration of the structured data storage system and the unstructured data storage system in the distributed storage cluster, and establishing a cache area in the remaining disk space of the distributed storage cluster; or, backing up the original data consistency configuration of the structured data storage system and the unstructured data storage system in the distributed storage cluster, controlling the number of data replicas stored in the distributed system, and establishing a cache area in the remaining disk space of the distributed storage cluster.

[0014] Secondly, embodiments of the present invention further provide a high-concurrency processing device based on a distributed system. The device includes: an acquisition module for acquiring system operating status parameter information of the distributed system; a first determination module for determining the current concurrency level and current high-concurrency response mode of the distributed system within a preset period based on the system operating status parameter information and a high-concurrency identification threshold; a processing module for inputting the system operating status parameter information into a pre-trained concurrency event prediction model to obtain the sensitivity of the high-concurrency level change of the distributed system, wherein the concurrency event prediction model is trained based on historical concurrency event data of the distributed system; and a second determination module for determining the high-concurrency response mode corresponding to the distributed system within the preset period based on the high-concurrency level change sensitivity, the current concurrency level, and the current high-concurrency response mode.

[0015] Thirdly, embodiments of the present invention further provide a high-concurrency processing system based on a distributed system. The system includes a system operation status information comprehensive acquisition module and a high-concurrency event analysis and control module. Specifically: the system operation status information comprehensive acquisition module acquires system operation status parameter information of the distributed system; the high-concurrency event analysis and control module determines the current concurrency level and current high-concurrency response mode of the distributed system within a preset period based on the system operation status parameter information and a high-concurrency identification threshold; the high-concurrency event analysis and control module inputs the system operation status parameter information into a pre-trained concurrency event prediction model to obtain the sensitivity to changes in the high-concurrency level of the distributed system, wherein the concurrency event prediction model is trained based on historical concurrency event data of the distributed system; and the high-concurrency event analysis and control module determines the corresponding high-concurrency response mode of the distributed system within the preset period based on the high-concurrency level change sensitivity, the current concurrency level, and the current high-concurrency response mode.

[0016] Fourthly, embodiments of the present invention further provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the high-concurrency processing method based on a distributed system as described in the first aspect.

[0017] Fifthly, embodiments of the present invention further provide a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the high-concurrency processing method based on a distributed system as described in the first aspect.

[0018] In this embodiment of the invention, system operation status parameter information of a distributed system is obtained; based on the system operation status parameter information and a high concurrency identification threshold, the current concurrency level and current high concurrency response mode of the distributed system within a preset period are determined; the system operation status parameter information is input into a pre-trained concurrency event prediction model to obtain the sensitivity to changes in the high concurrency level of the distributed system, wherein the concurrency event prediction model is trained based on historical concurrency event data of the distributed system; based on the sensitivity to changes in the high concurrency level, the current concurrency level, and the current high concurrency response mode, the corresponding high concurrency response mode of the distributed system within the preset period is determined. This embodiment of the invention determines the current concurrency level and current high concurrency response mode of the distributed system within a preset period based on the system operation status parameter information, then predicts the sensitivity to changes in the high concurrency level based on the system operation status parameter information, and corrects the current concurrency level and current high concurrency response mode of the distributed system based on the sensitivity to changes in the high concurrency level, thereby enhancing the ability of business application systems to timely and effectively identify high concurrency conditions and to automatically and proactively respond to high concurrency events. This avoids the problems of high hardware costs and increased energy consumption associated with vertical scaling of traditional systems, as well as high technical and design implementation costs for horizontal scaling, and low utilization of expanded resources under non-high-concurrency conditions, resulting in equipment and energy waste.

[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a high-concurrency handling method based on a distributed system in an embodiment of the present invention.

[0022] Figure 2 This is a schematic diagram illustrating an application scenario of a high-concurrency working condition processing method based on a distributed system, as described in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of the startup and initialization process of a distributed system according to an embodiment of the present invention;

[0024] Figure 4 This is a flowchart illustrating a method for determining a high-concurrency operating threshold in an embodiment of the present invention.

[0025] Figure 5 This is a schematic diagram of a high-concurrency threshold dynamic generation algorithm in an embodiment of the present invention;

[0026] Figure 6 This is a flowchart illustrating a high concurrency level identification method in an embodiment of the present invention;

[0027] Figure 7 This is a schematic diagram of the prediction process of a first concurrent event model with a long historical period in an embodiment of the present invention;

[0028] Figure 8 This is a schematic diagram of the prediction process of a short-term second concurrent event model in an embodiment of the present invention;

[0029] Figure 9 This is a flowchart illustrating a method for determining the sensitivity to changes in high-concurrency operating conditions according to an embodiment of the present invention.

[0030] Figure 10 This is a flowchart illustrating a method for determining the high concurrency level of a distributed system according to an embodiment of the present invention.

[0031] Figure 11 This is a schematic diagram illustrating a change in a high-concurrency response mode strategy in an embodiment of the present invention;

[0032] Figure 12 This is a schematic diagram of the structure of a high-concurrency processing device based on a distributed system according to an embodiment of the present invention;

[0033] Figure 13 This is a schematic diagram of the structure of a high-concurrency processing system based on a distributed system in an embodiment of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Example 1

[0036] According to embodiments of the present invention, a method for handling high-concurrency conditions in a distributed system is provided. This method is applied to a distributed system. Figure 1This is a flowchart illustrating a high-concurrency handling method based on a distributed system provided by an embodiment of the present invention, which may specifically include the following steps:

[0037] S101, Obtain system operation status parameter information of the distributed system;

[0038] Before introducing the method of this embodiment, the application scenario of this embodiment will be introduced first. For example... Figure 2 The diagram shows an application scenario of a high-concurrency handling method. This application scenario includes: a high-concurrency handling system 10, a distributed storage cluster resource layer 20, a client-side application 30, an application-side upper-layer business system 40, a distributed storage cluster gateway 50, and a high-concurrency response storage resource pool 60. The distributed storage cluster resource layer 20 and the high-concurrency response storage resource pool 60 are the structures used for storing and backing up data in the distributed system of this embodiment.

[0039] The high-concurrency handling method in this embodiment is executed by the high-concurrency handling system 10 and applied to a distributed system. Before implementing the method of this embodiment based on the high-concurrency handling system, it is necessary to complete the mounting and necessary interface connection between the high-concurrency handling system 10 and the distributed storage cluster resource layer 20 and the application-side upper-layer business system 40 to ensure smooth network and communication links. Appropriate system parameters and resource configurations for the distributed system are set in the distributed storage cluster resource layer 20 and the application-side upper-layer business system 40.

[0040] In this embodiment, the distributed system is a business system used to provide corresponding services to users and interact with them via data. Figure 2 In this embodiment, the distributed system includes a server-side application-side upper-layer business system 40 and a user-side client-side application 30.

[0041] In this embodiment, the system operation status information comprehensive acquisition module 100 in the high-concurrency working condition processing system 10 begins to collect system parameter information, including the number of requests received by the distributed system (application-side upper-layer business system 40), the number of queries (QPS), the number of concurrent tasks, and the read / write status information of the distributed storage cluster resource layer 20, etc., for the high-concurrency event analysis and control subsystem 110 in the high-concurrency working condition processing system 10 to judge the high-concurrency status of the storage cluster and make decisions on the response mode.

[0042] In this embodiment, the distributed storage system adopts general distributed storage based on the specific business application scenario requirements, including distributed databases, distributed file systems, distributed object storage, etc., and is not limited to a specific type. A distributed storage cluster layer storage system, such as distributed object storage, can be selected.

[0043] S102, Based on the system operating status parameter information and the high concurrency identification threshold, determine the current concurrency level and current high concurrency response mode of the distributed system within a preset period;

[0044] In this embodiment, the current concurrency level and current high-concurrency response mode of the distributed system within a preset period are determined based on the system operating status parameter information and the high-concurrency identification threshold.

[0045] Specifically, based on system parameter information such as the number of requests, query counts (QPS), and task concurrency received by the application-side upper-layer business system 40 from the front end, as well as the read / write status information of the distributed storage cluster resource layer 20, a high concurrency identification threshold is compared. If the conditions for a high concurrency event are met, the current concurrency level of the distributed system and the current high concurrency response mode of the current concurrency event are determined.

[0046] Optionally, in this embodiment, the high concurrency identification threshold includes a first high concurrency threshold and a second high concurrency threshold. Before determining the current concurrency status and current concurrency level of the distributed system based on the system operating status parameter information and the high concurrency identification threshold, the process further includes, but is not limited to: receiving the first high concurrency threshold and the second high concurrency threshold input by the user; and determining the current concurrency level and the current high concurrency response mode based on the first high concurrency threshold, the second high concurrency threshold, and the system operating status parameter information.

[0047] Specifically, the high-concurrency response mode setting module of the high-concurrency handling system 10 sends a requirement prompt to the system operation and maintenance personnel of the distributed system when the distributed system starts up. This prompt requires them to determine the first high-concurrency threshold H1 and the second high-concurrency threshold H2 for high-concurrency identification, the initial high-concurrency level of the system, and the initial high-concurrency response mode of the distributed storage cluster resource layer 20, based on the actual needs of the customer's business application system. If the administrators do not actively set the initialization parameters within the system's default time limit, the system will automatically adopt the default initial settings.

[0048] The initialization module for setting the high-concurrency response mode of the cluster needs to establish a connection with the high-concurrency event analysis and control module 110 of the high-concurrency working condition processing system 10, which includes the business system high-concurrency situation judgment and decision-making module and the high-concurrency response integrated communication module. Through the high-concurrency response integrated communication module, the module then performs joint debugging and system docking with the front-end business application system and the distributed cluster in the distributed storage cluster resource layer 20 to ensure smooth parameter acquisition and command transmission during system operation.

[0049] If system operations and maintenance personnel decide to manually set the initial high-concurrency response mode of the system based on business needs, they can set the required first high-concurrency threshold H1 and second high-concurrency threshold H2, as well as the initial high-concurrency response mode, according to user and business requirements. The weight set by the system operations and maintenance personnel for the initial high-concurrency response mode is higher than the weight configured for the default initial response mode.

[0050] In this embodiment, the high-concurrency operating conditions include three levels: primary, intermediate, and advanced.

[0051] Next, the method for predicting the current high concurrency status within the period includes comparing the system access status within a certain period of the day (which can be manually set, with the system default setting to identify once every 1 hour) with the high concurrency event thresholds, the second high concurrency status threshold H2 and the first high concurrency status threshold H1, to promptly determine whether a high concurrency event has occurred in the system and to determine its high concurrency status level.

[0052] If the average peak number of accesses within the time range used to identify high-concurrency conditions is R, then there are three cases:

[0053] High-concurrency operating condition level judgment case one:

[0054] R≤H1

[0055] In this case, the current high concurrency level of the system is determined to be "basic";

[0056] High-concurrency operating condition level judgment scenario two:

[0057] H1 < R < H2

[0058] In this case, the current high concurrency level of the system is determined to be "medium".

[0059] High-concurrency operating condition level judgment case three:

[0060] R≥H2

[0061] In this case, the current high concurrency level of the system is determined to be "high";

[0062] Then, the corresponding high-concurrency response mode is determined based on the current high-concurrency level of the system.

[0063] In one example, such as Figure 3 The diagram shown illustrates a process for the startup and initialization of a distributed system in this embodiment, which may include the following steps:

[0064] S301, System startup initialization;

[0065] S302, System initial high-concurrency response mode confirmed;

[0066] S303, determine whether the specified initial response mode is considered;

[0067] Specifically, if yes, proceed to S304; otherwise, proceed to S305.

[0068] S304, the system's initial default high-concurrency response mode;

[0069] Specifically, the initial default setting of the distributed system is "basic" for the current high concurrency level, and the distributed storage cluster in the distributed system is set to high concurrency response mode 1.

[0070] S305, Maintenance personnel set the initial high-concurrency response capability level;

[0071] Specifically, the initial high-concurrency response capability levels of a distributed system include "basic," "intermediate," and "high." The high-concurrency response mode of the distributed storage cluster in the distributed system is configured according to the settings instructions provided by operations and maintenance personnel with administrative privileges.

[0072] Furthermore, the high-concurrency response modes include High-concurrency response mode 1, which corresponds to the "basic" high-concurrency response capability level; High-concurrency response mode 2, which corresponds to the "intermediate" high-concurrency response capability level; and High-concurrency response mode 3, which corresponds to the "advanced" high-concurrency response capability level.

[0073] Through the above example, the system receives the first high concurrency threshold and the second high concurrency threshold input by the user; based on the first high concurrency threshold, the second high concurrency threshold, and system operating status parameters, it determines the current concurrency level and the current high concurrency response mode, thus realizing the rapid determination of the concurrency level and high concurrency response mode of high concurrency events.

[0074] Optionally, in this embodiment, the high concurrency identification threshold includes a first high concurrency threshold and a second high concurrency threshold. Before determining the current concurrency status and current concurrency level of the distributed system based on the system operating status parameters and the high concurrency identification threshold, the process includes, but is not limited to: obtaining the maximum and minimum number of accesses per unit time within a preset time period after the distributed system starts; determining the first high concurrency threshold based on the maximum and minimum number of accesses; and determining the current concurrency level and current high concurrency response mode based on the first high concurrency threshold, the second high concurrency threshold, and the system operating status parameters.

[0075] Specifically, if system operation and maintenance personnel do not actively set high concurrency identification thresholds and high concurrency response modes, or if the query waiting time exceeds the limit, the high concurrency event analysis and control subsystem 110 will automatically generate a second high concurrency threshold H2 and a first high concurrency threshold H1 for the high concurrency operating condition level. In this embodiment, the high concurrency operating condition level includes three levels: primary, intermediate, and advanced.

[0076] In one example, such as Figure 4 The diagram shown is a flowchart illustrating a high-concurrency threshold determination method in this embodiment, which may include the following steps:

[0077] S401, determine whether a manually set high concurrency threshold has been received;

[0078] Specifically, if yes, then jump to S403; if no, then jump to S402.

[0079] S402, automatically generates and updates high-concurrency thresholds;

[0080] S403, Get and update the high concurrency threshold.

[0081] like Figure 5 The diagram shown illustrates the dynamic threshold generation algorithm for high-concurrency operating conditions, and introduces the process of dynamic threshold generation for high-concurrency operating conditions:

[0082] For the distributed system, the peak hourly access count M2 within t hours after startup. t and the minimum number of visits per hour, M1 t For statistical analysis, t is a positive integer representing the number of hours the system has been running since startup, where t = 1, 2, 3...

[0083] Then obtain the peak access count M2 per hour within t hours after the distributed system starts. t Mean:

[0084]

[0085] Get the peak access count M1 per hour within n hours after the distributed system starts up. t Mean:

[0086]

[0087] Therefore, the value of the threshold H2 for judging the second high-concurrency condition, which is assigned the "Advanced" level of the high-concurrency condition classification, is:

[0088]

[0089] The value of the threshold H1 for judging the first high-concurrency condition, assigned to the "Medium" high-concurrency condition level, is:

[0090]

[0091] If the system maintenance personnel of the distributed system do not actively assign initial high-concurrency thresholds H2S and H1S within the default specified time limit, the system will calculate the initial high-concurrency threshold based on the statistical data of the peak and minimum hourly access times during the first 24 hours after system startup, according to the high-concurrency identification threshold dynamic generation algorithm.

[0092] Assigning the "Advanced" level to high-concurrency conditions sets the initial second-highest-concurrency threshold H2S as follows:

[0093]

[0094] The initial first high-concurrency threshold H1S value is assigned to the "Medium" level of the high-concurrency condition assessment:

[0095]

[0096] Among them, the peak number of accesses per hour M2 within 24 hours after the distributed system starts up. t Mean:

[0097]

[0098] The peak number of accesses per hour, M1, within 24 hours after the distributed system starts. t Mean:

[0099]

[0100] After obtaining the initial high-concurrency identification threshold, the high-concurrency handling system 10 will automatically generate new high-concurrency identification thresholds that better reflect the actual working conditions of the system within the default time interval according to the dynamic generation algorithm process. Alternatively, these thresholds can be manually set by operations and maintenance personnel.

[0101] The high-concurrency handling system 10 will automatically enable the default initial high-concurrency response mode, with the initial default high-concurrency level being primary. Correspondingly, the distributed system adopts the primary response mode, and the system will use the primary response mode as the default initial high-concurrency response mode for the distributed storage cluster.

[0102] Next, as Figure 6 The diagram shown is a flowchart of a high-concurrency level identification method in this embodiment, which specifically includes:

[0103] S601, obtain the threshold for high concurrency conditions;

[0104] Specifically, the method for predicting the current high concurrency status within a period includes comparing the system access data for a certain period of the day (which can be manually set, with the system default setting to identify once every 1 hour) with the high concurrency event thresholds, the second high concurrency status threshold H2 and the first high concurrency status threshold H1, to promptly determine whether a high concurrency event has occurred in the system and to determine its high concurrency level.

[0105] S602 determines the high-concurrency level of the classification system based on the average peak number of visits within a unit time range and the high-concurrency threshold.

[0106] Specifically, assuming the average peak number of accesses within a unit time range used to identify high-concurrency conditions is R, there are three cases:

[0107] High-concurrency operating condition level judgment case one:

[0108] R≤H1

[0109] In this case, the current high concurrency level of the system is determined to be "basic";

[0110] High-concurrency operating condition level judgment scenario two:

[0111] H1 < R < H2

[0112] In this case, the current high concurrency level of the system is determined to be "medium".

[0113] High-concurrency operating condition level judgment case three:

[0114] R≥H2

[0115] In this case, the current high concurrency level of the system is determined to be "high";

[0116] Then, the corresponding high-concurrency response mode is determined based on the current high-concurrency level of the system.

[0117] Using the above example, we obtain the maximum and minimum number of accesses per unit time within a preset time period after the distributed system starts up; based on the maximum and minimum number of accesses, we determine the first high concurrency threshold, the second high concurrency threshold, the current concurrency level, and the current high concurrency response mode.

[0118] S103, Input the system operating status parameter information into the pre-trained concurrent event prediction model to obtain the sensitivity of the high-concurrency operating level change of the distributed system. The concurrent event prediction model is trained based on the historical concurrent event data of the distributed system.

[0119] In practical applications, determining the high-concurrency level and corresponding high-concurrency response mode of a distributed system solely based on its operational status parameters is inaccurate. Therefore, in this embodiment, a concurrency event prediction model is trained using historical concurrent event data to determine the sensitivity to changes in the high-concurrency level. This model is then used to correct the current high-concurrency level and response mode of the current high-concurrency event in the distributed system.

[0120] Optionally, in this embodiment, system operating status parameter information is input into a pre-trained concurrent event prediction model to obtain the sensitivity to changes in the high-concurrency operating condition level of the distributed system. This includes, but is not limited to: inputting system operating status parameter information into a first concurrent event model to obtain a first prediction result, wherein the first concurrent event prediction model is trained based on historical concurrent event data of a first time period of the distributed system; inputting system operating status parameter information into a second concurrent event model to obtain a second prediction result, wherein the second concurrent event prediction model is trained based on historical concurrent event data of a second time period of the distributed system, wherein the first time period is longer than the second time period; and determining the sensitivity to changes in the high-concurrency operating condition level based on the first prediction result and the second prediction result.

[0121] Specifically, the first time period corresponding to the first concurrency event model is a long historical period, such as high-concurrency events that occurred in the past year; the main function of the first concurrency event model is to predict high-concurrency events and their levels within the prediction period based on historical statistical data and distribution data of historical high-concurrency events in the business application system.

[0122] In this embodiment, as Figure 7 The diagram shown illustrates the prediction process of a first concurrent event model with a long historical period in this embodiment, which may include the following steps:

[0123] S701, retrieves stored historical high-concurrency data;

[0124] Specifically, obtain historical statistics and distribution data of historical high-concurrency events based on business application systems;

[0125] S7011, acquires historical long-term high-concurrency distribution data of the system;

[0126] S7012, retrieves historical long-term high-concurrency system data;

[0127] S702, construct the first concurrent event model based on the distribution and level of high-concurrency events within the future prediction period;

[0128] Specifically, the first concurrency event model is based on historical high-concurrency event distribution data and corresponding high-concurrency level data over a longer period (by default, high-concurrency system data is recorded and retained for one year on a daily basis, but the time range can be set). It then establishes a mathematical model for the distribution of high-concurrency events and their corresponding high-concurrency levels. Finally, it predicts the distribution of potential high-concurrency events and their corresponding high-concurrency levels for the next seven days (the prediction time range can be set according to actual conditions).

[0129] S703, Prediction results of data storage read / write frequency and data volume within a future prediction period;

[0130] Specifically, the first concurrency event model uses high-concurrency data from the past year to model and predict whether high-concurrency events will occur in the following week, how they will be distributed over time, and the level of potential high-concurrency events. The daily prediction results within the prediction time frame are aggregated as follows:

[0131] (Date L n,Level L n)

[0132] Among them, Date L 'n' contains information about whether there will be enough high-concurrency events on a given day within the prediction period. Level L The information contained in 'n' indicates that if a high-concurrency event occurs on a certain day within the prediction period, the corresponding high-concurrency level needs to be predicted.

[0133] S704, based on the first concurrent event model, predicts the distribution of potential high-concurrency events and the level of high concurrency in the distributed system within a future prediction period;

[0134] Specifically, the output value of the first concurrent event model is Mode(L):

[0135] Mode(L) = {(Date L Level 1 L 1),Λ,(Date L n,Level L n)}

[0136] On the other hand, the second time period corresponding to the second concurrent event model is a recent short period of time, such as high-concurrency events that occurred in the past month.

[0137] In this embodiment, as Figure 8 The diagram shown illustrates the prediction process of a short-term second concurrent event model in this embodiment, which may include the following steps:

[0138] S801, retrieves stored historical high-concurrency data;

[0139] Specifically, obtain historical statistics and distribution data of historical high-concurrency events based on business application systems;

[0140] S8011, retrieves historical short-term high-concurrency distribution data of the system;

[0141] S8012, retrieves historical short-term high-concurrency system data;

[0142] S802, construct a second concurrent event model based on the distribution and level of high-concurrency events within the future prediction period;

[0143] Specifically, in this embodiment, the second concurrent event model predicts high-concurrency events and their levels within the prediction time range based on historical statistical data and distribution data of historical high-concurrency events in the business application system over a period of time relatively close to the start of the prediction period. It also predicts the distribution data of historical high-concurrency events and their corresponding high-concurrency levels over a shorter time range (by default, high-concurrency data for one month is recorded and retained in days; a time range can be set). A mathematical model is established for the distribution of high-concurrency events and their corresponding high-concurrency levels. Finally, it predicts the distribution of potential high-concurrency events and their corresponding high-concurrency levels for the next 8 days of the week (prediction time range).

[0144] S803, the prediction results of data storage read / write frequency and data volume within a future prediction period;

[0145] Specifically, the second concurrency event model uses high-concurrency data from the past month to model and predict whether high-concurrency events will occur in the following week, how they will be distributed over time, and the level of potential high-concurrency events. The daily prediction results for the prediction period are summarized as follows:

[0146] (Date S n,Level S n)

[0147] Among them, Date S 'n' contains information about whether there will be enough high-concurrency events on a given day within the prediction period. Level S The information contained in 'n' indicates that if a high-concurrency event occurs on a certain day within the prediction period, the corresponding high-concurrency level needs to be predicted.

[0148] S804, based on the second concurrent event model, predicts the distribution of potential high-concurrency events and the level of high concurrency in the distributed system within a future prediction period;

[0149] Specifically, the output value of the second concurrent event model is Mode(S):

[0150] Mode(S) = {(Date S Level 1 S 1),Λ,(Date S n,Level S n)}

[0151] Then, based on the first and second prediction results, the sensitivity to changes in high-concurrency operating conditions is determined.

[0152] Optionally, in this embodiment, the first prediction result includes a first time distribution and a first level of concurrent events, and the second prediction result includes a second time distribution and a second level of concurrent events. Determining the sensitivity to changes in the high-concurrency operating condition level based on the first and second prediction results includes, but is not limited to: obtaining a set of time distribution prediction results based on the first and second time distributions; obtaining a set of level prediction results based on the first and second levels; combining the set of time distribution prediction results and the set of level prediction results to obtain a set of prediction results; and determining the sensitivity to changes in the high-concurrency operating condition level based on the consistency between the time distribution prediction results and the level prediction results in the prediction results.

[0153] Specifically, such as Figure 9 The diagram shown is a flowchart illustrating a method for determining the sensitivity to changes in high-concurrency operating conditions in this embodiment. Specifically, it may include the following steps:

[0154] S901, Obtain the first prediction result of the first concurrent event model:

[0155] Mode(L) = {(Date L Level 1 L 1),Λ,(Date L n,Level L n)}

[0156] S902, Obtain the second prediction result of the second concurrent event model:

[0157] Mode(S) = {(Date S Level 1 S 1),Λ,(Date S n,Level S n)}

[0158] S903, Prediction result data extraction and aggregation processing;

[0159] S9031, Collection of prediction results based on the distribution of high-concurrency events using historical data;

[0160] By extracting and aggregating the first and second prediction results, a first time distribution prediction result set Date(L) based on the first prediction result and a second time distribution prediction result set Date(S) based on the second prediction result are generated:

[0161] Date(L) = (Date L 1. Date L 2,Λ,Date L n)

[0162] Date(S) = (Date S 1. Date S 2,Λ,Date S n)

[0163] S9032, Collection of prediction results for high-concurrency event levels based on historical data;

[0164] Generate a first-level prediction result set Level(L) and a second-level prediction result set Level(S):

[0165] Level(L) = (Level L Level 1 L 2,Λ,Level L n)

[0166] Level(S) = (Level S Level 1 S 2,Λ,Level S n)

[0167] S9033, the prediction results of high-concurrency event distribution correspond to a comparative analysis;

[0168] Specifically, the differences between the prediction result set based on the first time distribution and the prediction result set based on the second time distribution are analyzed by performing an XOR operation on the corresponding bits of the prediction results of the high-concurrency event distribution.

[0169] The algorithm formula is:

[0170]

[0171] Where 0 ≤ Date(T) ≤ n.

[0172] S9034, the prediction results for high-concurrency event levels correspond to a comparative analysis;

[0173] Specifically, the differences between the first-level and second-level prediction result sets are analyzed by performing an XOR operation on the corresponding bits of the high-concurrency event level prediction results. Algorithm formula:

[0174]

[0175] Where 0 ≤ Level(T) ≤ n.

[0176] S904, based on the consistency between the distribution prediction and the level prediction of high-concurrency events, analyzes the response sensitivity of high-concurrency events within the prediction period.

[0177] Specifically, the above results are combined into a prediction result set Mode(T). The consistency analysis results of the high-concurrency event distribution prediction and the level prediction are then used as the basis for judging the sensitivity of the response to high-concurrency events within the prediction period.

[0178] Mode(T) = (Date(T), Level(T))

[0179] The sensitivity level for high-concurrency event response within the prediction period is divided into the following four levels:

[0180] The high-concurrency duration threshold and the high-concurrency access peak interval range for high-concurrency events in this embodiment can be set based on practical experience. It should be noted that the values ​​in this embodiment are all default initial values.

[0181] (1) Sensitivity to changes in high concurrency level classification 1

[0182] Judgment conditions:

[0183]

[0184] In this case, the difference between the prediction results based on long-term historical data and the prediction results based on recent historical data regarding the distribution and grading of high-concurrency events within the prediction period is small. This indicates that the prediction results from both dimensions are relatively consistent, and the prediction reliability is high. In this situation, a higher sensitivity to changes in the high-concurrency grading can be assigned.

[0185] In this case, the final output of the high-concurrency operating condition level change sensitivity judgment module is "High-concurrency level classification change sensitivity 1": Mode1

[0186] Mode 1: When the high concurrency judgment node detects that the system has been in a high concurrency state for 1 hour, and the time interval between two high concurrency event access peaks is less than 30 minutes, the corresponding high concurrency response strategy of the system is activated according to the judged high concurrency event level.

[0187] (2) Sensitivity to changes in high-concurrency level classification 2

[0188] Judgment conditions:

[0189]

[0190] In this case, it indicates that the prediction results based on long-term historical data are relatively similar to the prediction results based on recent historical data regarding the distribution of high-concurrency events within the prediction period. However, the prediction results regarding the severity levels of high-concurrency events within the prediction period differ significantly.

[0191] This indicates that, based on both long-term and short-term historical data, the predictions of the temporal distribution of high-concurrency events within the prediction period have high reliability. However, the predictions of the intensity of high-concurrency events have lower reliability.

[0192] In this situation, to enable the system to take appropriate high-concurrency response strategies in a timely manner, a secondary level of high concurrency can be assigned a change sensitivity rating.

[0193] In this case, the final output of the high-concurrency operating condition level change sensitivity judgment module is "High-concurrency level classification change sensitivity 2": Mode2

[0194] Mode 2: When the high concurrency judgment node finds that the system has been in a high concurrency state for 2 hours, and the time interval between two high concurrency event access peaks is less than 30 minutes, the corresponding high concurrency response strategy of the system is activated according to the judged high concurrency event level.

[0195] (3) Sensitivity to changes in high concurrency level classification 3

[0196] Judgment conditions:

[0197]

[0198] In this case, it indicates a significant difference between the prediction results based on long-term historical data and the prediction results based on recent historical data regarding the distribution of high-concurrency events within the prediction period. However, the difference in the prediction results regarding the level of high-concurrency events within the prediction period is relatively small.

[0199] This indicates that, based on both long-term and short-term historical data, the prediction reliability for the temporal distribution of high-concurrency events within the prediction period is relatively low. However, the prediction reliability for the level of high-concurrency events is relatively high.

[0200] In this situation, to enable the system to take appropriate high-concurrency response strategies in a timely manner, and to avoid the system reacting too quickly due to short-duration high-concurrency events, thus wasting excessive resources, a lower level of high-concurrency sensitivity can be assigned.

[0201] In this case, the final output of the high-concurrency operating condition level change sensitivity judgment module is "High-concurrency level classification change sensitivity 3": Mode3

[0202] Mode 3: When the high-concurrency judgment node finds that the system has been in a high-concurrency state for 3 hours, and the time interval between two high-concurrency event access peaks is less than 30 minutes, the corresponding high-concurrency response strategy of the system is activated according to the judged high-concurrency event level.

[0203] (4) Sensitivity to changes in high concurrency level classification 4

[0204] Judgment conditions:

[0205]

[0206] In this case, it indicates a significant difference between the prediction results based on long-term historical data and the prediction results based on recent historical data regarding the distribution of high-concurrency events within the prediction period. Furthermore, there is a significant difference in the predicted levels of high-concurrency events within the prediction period.

[0207] This indicates that the prediction results based on both long-term and short-term historical data have low reliability in predicting the temporal distribution and level of high-concurrency events within the prediction period.

[0208] In this situation, to prevent the system from over-responding due to high-frequency but short-duration high-concurrency events within a certain period, thus affecting system resource allocation and usability, a minimum high-concurrency level can be assigned to change sensitivity.

[0209] In this case, the final output of the high-concurrency operating condition level change sensitivity judgment module is "High-concurrency level classification change sensitivity 4": Mode4

[0210] Mode4: When the high-concurrency judgment node finds that the system has been in a high-concurrency state for 4 hours and the time interval between two high-concurrency event access peaks is less than 30 minutes, the corresponding high-concurrency response strategy of the system is activated according to the judged high-concurrency event level.

[0211] The final output of the "High Concurrency Level Change Sensitivity Judgment Module" includes four sensitivity levels, ordered by weight from highest to lowest sensitivity:

[0212] Mode1 > Mode2 > Mode3 > Mode4

[0213] Based on the above method, the system operating status parameter information is input into the pre-trained first concurrent event model and the second concurrent event model, respectively, which can determine the sensitivity of the high-concurrency operating condition level change of the distributed system.

[0214] S104. Based on the sensitivity to changes in high-concurrency operating conditions, the current concurrency operating conditions level, and the current high-concurrency response mode, determine the corresponding high-concurrency response mode for the distributed system within a preset period.

[0215] In this embodiment, the current concurrency level and current high-concurrency response mode of the current high-concurrency event in the distributed system are corrected based on the sensitivity to changes in high-concurrency operating conditions, the current concurrency operating condition level, and the current high-concurrency response mode, so as to determine a high-concurrency response mode that conforms to the actual operating condition level of the current high-concurrency event in the distributed system.

[0216] In one example, such as Figure 10 The diagram shown is a flowchart illustrating a method for determining the high concurrency level of a distributed system in this embodiment. Specifically, it may include the following steps:

[0217] S1001, Obtain system operating status information data;

[0218] S1002, High-concurrency measurement parameter extraction and processing;

[0219] Specifically, from the comprehensive operational status parameters of the distributed system, extract the parameters needed to measure the system's high-concurrency operation. These may include, but are not limited to, QPS, TPS, and concurrency. Then, process the parameter information into a format that is easy to handle later.

[0220] S1003 determines the sensitivity of a distributed system to changes in high-concurrency operating conditions.

[0221] Specifically, based on historical statistics and distribution data of high-concurrency events in a distributed system, the sensitivity to changes in high-concurrency operating conditions within the prediction period is predicted.

[0222] S10031, High Concurrency Event Statistics;

[0223] Specifically, we need to statistically analyze historical high-concurrency events in distributed systems, and within a certain time range, analyze the time distribution of core parameters such as the occurrence time of high concurrency events and the intensity level of high concurrency at that time.

[0224] S10032, retrieve historical high-concurrency data from storage;

[0225] Specifically, high-concurrency measurement parameters extracted within a certain storage period, as well as statistically analyzed high-concurrency time distribution data, are stored. In this embodiment, the default storage period is set to one year, but it can also be set by system maintenance and management personnel according to the actual business application scenario.

[0226] S10033, prediction of the first concurrent event model;

[0227] Specifically, the first concurrent event model is used to predict the high-concurrency events and their levels within the prediction period based on historical long-term high-concurrency events, so as to obtain the first prediction result.

[0228] S10034, prediction of the second concurrent event model;

[0229] Specifically, the second concurrent event model is used to predict the high-concurrency events and their levels within the prediction period based on historical short-cycle high-concurrency events, so as to obtain the second prediction result.

[0230] S10035, Determine the sensitivity to changes in high-concurrency operating conditions;

[0231] Specifically, the sensitivity of the high-concurrency operating level of the distributed system to changes is determined based on the first and second prediction results.

[0232] S1004, dynamically generate high-concurrency recognition thresholds;

[0233] S1005 identifies the current high-concurrency operating conditions of the distributed system within the prediction period;

[0234] Specifically, the current high-concurrency status of the distributed system within the prediction period is determined based on the high-concurrency identification threshold. For example, by comparing the system access data within a certain period of the day (which can be manually set, with the system defaulting to identifying once every 1 hour) with the high-concurrency event threshold, it can be promptly determined whether a high-concurrency event has occurred in the system and its high-concurrency level.

[0235] S1006, High-concurrency operating condition identification result fusion and high-concurrency operating condition level change strategy generation.

[0236] Specifically, the system assesses the sensitivity of response to high-concurrency events within the prediction period, and determines whether and how high-concurrency events are currently occurring in the system. This assessment provides a comprehensive result indicating whether and how high-concurrency events are occurring in the distributed system.

[0237] Optionally, in this embodiment, the high-concurrency response mode corresponding to the distributed system is determined based on the sensitivity to changes in high-concurrency operating conditions, the current concurrency operating conditions level, and the current high-concurrency response mode. This includes, but is not limited to, determining the high-concurrency response mode based on the sensitivity to changes in high-concurrency operating conditions, the current concurrency operating conditions level, and the current high-concurrency response mode if the current concurrency level does not match the high-concurrency level corresponding to the current high-concurrency response mode.

[0238] In this embodiment, the prediction of the sensitivity to changes in the high-concurrency operating level of high-concurrency events within the prediction period, combined with the judgment of whether a high-concurrency event has occurred and its level in the current system, provides a comprehensive judgment result on whether a high-concurrency event has occurred in the current distributed system and its level. Based on whether the system is currently experiencing a high-concurrency event, and if so, combined with the current high-concurrency event level and the sensitivity to changes in the high-concurrency operating level output by the concurrency event prediction model, it is ultimately determined whether the current high-concurrency event level needs to be changed. The specific circumstances for changing the high-concurrency operating level can include the following two situations:

[0239] High-concurrency operating condition level change scenario one:

[0240] If the current high concurrency level assessment result matches the high concurrency level corresponding to the current high concurrency response mode of the system, then there is no need to change the current high concurrency level response mode strategy of the system.

[0241] High-concurrency operating condition level change scenario two:

[0242] If the current high-concurrency condition level assessment result does not match the high-concurrency level corresponding to the current high-concurrency response mode of the system, then the current high-concurrency condition level response mode of the system needs to be changed. First, obtain the sensitivity to high-concurrency condition level changes, and then change the high-concurrency response mode strategy according to the current high-concurrency condition level identified by the distributed system.

[0243] Specifically, such as Figure 11 The diagram shown illustrates a change in a high-concurrency response mode strategy in this embodiment, which may specifically include the following steps:

[0244] S1101, obtain the threshold for high concurrency conditions;

[0245] S1102, Time node control is determined based on high concurrency threshold;

[0246] S1103, Get the current system's high concurrency event level;

[0247] S1104, obtain the sensitivity to changes in high-concurrency operating conditions;

[0248] S1105, determine the current high-concurrency operating condition of the distributed system;

[0249] S1106, Based on the high-concurrency operating conditions and the sensitivity of high-concurrency operating condition level changes, determine whether to change the high-concurrency response mode strategy.

[0250] To effectively handle high-concurrency scenarios, this embodiment provides three high-concurrency response modes, allowing the distributed system to automatically activate the corresponding response mode based on the type of high-concurrency scenario. System administrators can also select and configure these modes according to user needs.

[0251] Further, optionally, in this embodiment, the high-concurrency response mode includes one of the following:

[0252] (1) High-concurrency response mode 1 - Basic response mode (corresponding to the "basic" level of high-concurrency operation in the distributed system):

[0253] Specifically, the business data and data copies of the distributed system are stored according to the preset business logic and the preset distributed storage cluster configuration.

[0254] In this mode, the distributed cluster in the distributed storage cluster resource layer 20 will store business data and data copies according to the usual business logic and the user's configuration of the distributed storage cluster, and perform regular redundancy disaster recovery backup.

[0255] In addition, it can be set to automatically switch to other high-concurrency response modes based on preset thresholds; it can also be set to not automatically switch to other high-concurrency response modes unless the operation and maintenance personnel or users actively switch to high-concurrency response modes according to actual business needs.

[0256] (2) High-concurrency response mode 2 - Intermediate response mode (corresponding to the "intermediate" level of high-concurrency operation in the distributed system):

[0257] Specifically, the original data consistency configuration of the structured data storage system and the unstructured data storage system in the distributed storage cluster is backed up, and a cache area is established in the remaining disk space of the distributed storage cluster.

[0258] Once this mode is activated, it signifies that the high concurrency pressure on the distributed system is continuously increasing. In this mode, the original data consistency configurations of the structured and unstructured data storage systems within the distributed storage cluster resource layer 20 are backed up. Furthermore, the data consistency settings for all storage systems are unified to ensure that the data synchronization process between the primary and backup replicas does not occur in concentrated or intensive bursts during peak system access periods.

[0259] In addition, this response mode utilizes a portion of the remaining disk space to create a cache area, thereby expanding the cluster's caching capacity. This expansion of the cache helps alleviate high concurrency pressure.

[0260] (3) High-concurrency response mode 3 - Advanced response mode (corresponding to the "Advanced" level of high-concurrency operation in distributed systems):

[0261] Specifically, the original data consistency configurations of the structured data storage system and the unstructured data storage system in the distributed storage cluster are backed up, the number of data replicas stored in the distributed system is controlled, and a cache area is established in the remaining disk space of the distributed storage cluster.

[0262] When the frequency of access requests from the front end exceeds a pre-set threshold, high-concurrency response mode 3 - advanced response mode will be activated. The activation of this mode signifies that the distributed system is under significant high concurrency pressure, and the system is experiencing substantial access load.

[0263] In this response mode, data consistency is ensured by backing up the original data consistency configurations of all data storage systems in the distributed storage cluster. Data consistency settings across all storage systems are unified to ensure that the synchronization process between non-critical primary and backup replicas avoids peak system access periods. The number of backup replicas in the storage systems is uniformly reduced. More storage space is allocated to the shared cache to improve system responsiveness and ensure smooth high-frequency user read / write operations, preventing system delays or outages due to sudden surges in concurrent data access.

[0264] In this embodiment, after the high-concurrency event ends, the cache space mounted and managed by the high-concurrency response storage resource pool 60 is released, and the original backup and configuration state of the distributed storage cluster before the high-concurrency event response is restored according to the original distributed storage cluster configuration mode file.

[0265] In addition, it should be noted that the high-concurrency handling storage resource pool 60 includes a cache construction module, a storage resource scheduling module, a backup distributed storage cluster, and a backup storage resource pool status control module.

[0266] The high-concurrency response storage resource pool 60 resides on the cluster server of the distributed storage cluster. Its main function is to activate "High-Concurrency Response Mode 2 - Medium Response Mode" or "High-Concurrency Response Mode 3 - Advanced Response Mode" when the system faces high-level high-concurrency events and significant high-concurrency pressure. It will establish the "High-Concurrency Response Storage Resource Pool" by starting a preset program, utilizing remaining space or releasing space from redundant storage copies, and centrally managing storage resources available for system cache expansion. As the system's high-concurrency pressure changes, it provides greater cache space for core business systems. This improves system access speed and increases system throughput, reduces direct interaction with the database during peak access periods, and enhances the business system's ability to handle high concurrency.

[0267] The caching module is used to expand storage capacity under high-concurrency conditions by utilizing storage resources released by the distributed storage cluster. It also acts as a system cache, supporting the system in effectively handling high-concurrency scenarios.

[0268] The storage resource scheduling module is used to control the expansion and release of cache in the distributed storage cluster based on the high-concurrency response strategy made by the high-concurrency processing system 10 according to the identification results of high-concurrency conditions.

[0269] A backup distributed storage cluster is essentially a simplified distributed storage cluster. Its main function is to uniformly mount and manage the redundant replica space that was cleared from the original distributed storage cluster and used for cache expansion, making it easier to manage, monitor, and reserve the storage space that can be used for cache expansion.

[0270] The backup storage resource pool status control module receives high-concurrency event judgment and decision information and distributed storage cluster response strategy information transmitted via the "high-concurrency response integrated communication module," and controls the construction and deployment of the backup distributed storage cluster, storage resource scheduling module, and cache construction module. This ensures the timely and orderly establishment and operation of the high-concurrency response storage resource pool, effectively improving the system's ability to handle high-concurrency events.

[0271] The backup storage resource pool status control module is also used to release the cache space mounted and managed by the high-concurrency response storage resource pool 60 after a high-concurrency event ends, and restore the original backup and configuration state of the cluster before the high-concurrency event response according to the original distributed storage cluster configuration mode file. This ensures that business system users are completely unaware of the high-concurrency event, while minimizing costs, and without affecting the original system performance or deployment configuration.

[0272] This embodiment enhances the ability of business application systems to promptly and effectively identify and proactively respond to high-concurrency events by reusing the system's own storage resources as much as possible, with minimal cost, energy consumption, resource, and manpower expenditure. It avoids the problems of high hardware costs and increased energy consumption associated with vertical scaling in traditional systems, as well as high technical and design implementation costs for horizontal scaling, and the waste of equipment and energy caused by low utilization of expanded resources outside of high-concurrency scenarios.

[0273] This embodiment can intelligently allocate, manage, and schedule redundant cloud resources in a unified and centralized manner, avoiding the waste of resources and manpower costs caused by conventional load balancing measures and architecture redesign, which are more costly.

[0274] This embodiment can address the randomness of most high-concurrency scenarios, improving upon existing cloud computing cluster-based elastic scaling capabilities. However, these systems often rely on predicting system traffic and high-concurrency patterns, leading to delays and reactive responses in identifying and handling sudden, random high-concurrency events. It also solves the problem of difficulty in predicting high-concurrency events caused by unforeseen external events such as natural disasters and wars. Furthermore, it supports cloud computing cluster operators in taking timely and effective countermeasures, ensuring smooth user access during peak hours, guaranteeing the continued usability of business functions under high-concurrency conditions, improving the user experience, and preventing unnecessary losses.

[0275] On the other hand, with business development, cloud computing platform-based application systems will face increasingly larger data storage volumes and more complex user structures and distributions. This will lead to a continuous increase in system read / write pressure, a sustained increase in the probability of high concurrency, and further increases in the complexity and randomness of the occurrence and distribution of high concurrency situations. This embodiment can effectively and as early as possible predict and intelligently respond to sudden high-concurrency business scenarios, enabling the early detection and handling of high-concurrency scenarios, and ensuring the sustainability of business operations for important business-bearing institutions or units.

[0276] This embodiment addresses the issue that the mobility of cloud computing operations and maintenance personnel leads to varying response strategies when facing the same high-concurrency scenarios, introducing uncertainty into system operation. By pre-configuring high-concurrency response modes and strategies, automatically generating high-concurrency scenario identification thresholds, and automatically executing policy formulations, the system effectively mitigates the uncertainties caused by personnel turnover, ensuring consistent and sustainable service performance.

[0277] Example 2

[0278] This invention provides a detailed description of a high-concurrency processing device based on a distributed system, as provided in the embodiments of the present invention.

[0279] Reference Figure 12 The diagram shows a structural schematic of a high-concurrency processing device based on a distributed system according to an embodiment of the present invention.

[0280] The high-concurrency processing device based on a distributed system according to this embodiment of the invention includes: an acquisition module 120, a first determination module 122, a processing module 124, and a second determination module 126.

[0281] The functions of each module and the interaction between them are described in detail below.

[0282] Module 120 is used to acquire system operating status parameter information of the distributed system.

[0283] The first determining module 122 is used to determine the current concurrency level and current high concurrency response mode of the distributed system within a preset period based on the system operating status parameter information and the high concurrency identification threshold.

[0284] The processing module 124 is used to input the system operating status parameter information into a pre-trained concurrent event prediction model to obtain the sensitivity of the high-concurrency operating condition level change of the distributed system, wherein the concurrent event prediction model is trained based on the historical concurrent event data of the distributed system.

[0285] The second determining module 126 is used to determine the high-concurrency response mode corresponding to the distributed system within the preset period based on the high-concurrency operating condition level change sensitivity, the current concurrency operating condition level, and the current high-concurrency response mode.

[0286] The high-concurrency processing device based on a distributed system proposed in this embodiment acquires system operation status parameter information of the distributed system; based on the system operation status parameter information and the high-concurrency identification threshold, it determines the current concurrency level and current high-concurrency response mode of the distributed system within a preset period; the system operation status parameter information is input into a pre-trained concurrency event prediction model to obtain the sensitivity to changes in the high-concurrency level of the distributed system, wherein the concurrency event prediction model is trained based on historical concurrency event data of the distributed system; based on the sensitivity to changes in the high-concurrency level, the current concurrency level, and the current high-concurrency response mode, it determines the corresponding high-concurrency response mode of the distributed system within a preset period. This embodiment of the invention enhances the ability of business application systems to timely and effectively identify high-concurrency conditions and to automatically and proactively respond to high-concurrency events. It avoids the problems of high hardware costs and increased energy consumption in vertical scaling of traditional systems, as well as high technical and design implementation costs in horizontal scaling, and the low utilization rate of expanded resources outside of high-concurrency conditions, resulting in equipment and energy waste.

[0287] Example 3

[0288] This invention provides a detailed description of a high-concurrency processing system.

[0289] Reference Figure 13 The diagram shows a structural schematic of a high-concurrency processing system according to an embodiment of the present invention.

[0290] The high-concurrency processing system of this invention includes: a system operation status information comprehensive acquisition module 130 and a high-concurrency event analysis and control module 132.

[0291] The functions of each module and the interaction between them are described in detail below.

[0292] Based on the system operation status information comprehensive acquisition module 130, the system operation status parameter information of the distributed system is obtained;

[0293] Based on the high-concurrency event analysis and control module 132, the current concurrency level and current high-concurrency response mode of the distributed system are determined within a preset period according to the system operation status parameter information and the high-concurrency identification threshold.

[0294] Based on the high-concurrency event analysis and control module 132, the system operating status parameter information is input into the pre-trained concurrent event prediction model to obtain the sensitivity of the high-concurrency operating condition level change of the distributed system. The concurrent event prediction model is trained based on the historical concurrent event data of the distributed system.

[0295] Based on the high-concurrency event analysis and control module 132, the high-concurrency response mode corresponding to the distributed system within the preset period is determined according to the high-concurrency operating condition level change sensitivity, the current concurrency operating condition level, and the current high-concurrency response mode.

[0296] The high-concurrency handling system proposed in this embodiment acquires system operation status parameter information of the distributed system; based on the system operation status parameter information and the high-concurrency identification threshold, it determines the current concurrency level and current high-concurrency response mode of the distributed system within a preset period; the system operation status parameter information is input into a pre-trained concurrency event prediction model to obtain the sensitivity of the high-concurrency level change of the distributed system, wherein the concurrency event prediction model is trained based on the historical concurrency event data of the distributed system; based on the high-concurrency level change sensitivity, the current concurrency level, and the current high-concurrency response mode, it determines the corresponding high-concurrency response mode of the distributed system within a preset period. This invention determines the current concurrency level and high-concurrency response mode of a distributed system within a preset period based on system operation status parameters. Then, it predicts the sensitivity to changes in the high-concurrency level based on these parameters. The current concurrency level and response mode are then corrected according to this sensitivity, thereby enhancing the ability of business application systems to promptly and effectively identify and proactively respond to high-concurrency events. This avoids the problems of high hardware costs and increased energy consumption associated with vertical scaling in traditional systems, as well as high technical and design costs for horizontal scaling, and the low utilization rate of expanded resources outside of high-concurrency scenarios, leading to equipment and energy waste.

[0297] Example 4

[0298] Preferably, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the high-concurrency processing method based on a distributed system as described above.

[0299] Optionally, in this embodiment, the memory is configured to store program code for performing the following steps:

[0300] S1, obtain system operating status parameter information of the distributed system;

[0301] S2, based on the system operating status parameter information and the high concurrency identification threshold, determine the current concurrency level and current high concurrency response mode of the distributed system within a preset period;

[0302] S3, input the system operating status parameter information into the pre-trained concurrent event prediction model to obtain the high-concurrency operating condition level change sensitivity of the distributed system, wherein the concurrent event prediction model is trained based on the historical concurrent event data of the distributed system;

[0303] S4. Based on the high concurrency condition level change sensitivity, the current concurrency condition level, and the current high concurrency response mode, determine the high concurrency response mode corresponding to the distributed system within the preset period.

[0304] Optionally, specific examples in this embodiment can refer to the examples described in Embodiment 1 above, and will not be repeated here.

[0305] Example 5

[0306] Embodiments of the present invention also provide a readable storage medium. Optionally, in this embodiment, the readable storage medium stores a program or instructions, which, when executed by a processor, implement the steps of the high-concurrency processing method based on a distributed system as described in Embodiment 1.

[0307] Optionally, in this embodiment, the readable storage medium is configured to store program code for performing the following steps:

[0308] S1, obtain system operating status parameter information of the distributed system;

[0309] S2, based on the system operating status parameter information and the high concurrency identification threshold, determine the current concurrency level and current high concurrency response mode of the distributed system within a preset period;

[0310] S3, input the system operating status parameter information into the pre-trained concurrent event prediction model to obtain the high-concurrency operating condition level change sensitivity of the distributed system, wherein the concurrent event prediction model is trained based on the historical concurrent event data of the distributed system;

[0311] S4. Based on the high concurrency condition level change sensitivity, the current concurrency condition level, and the current high concurrency response mode, determine the high concurrency response mode corresponding to the distributed system within the preset period.

[0312] Optionally, specific examples in this embodiment can refer to the examples described in Embodiment 1 above, and will not be repeated here. Optionally, the readable storage medium is also configured to store program code for performing the steps included in the method in Embodiment 1 above, and will not be repeated here.

[0313] Optionally, in this embodiment, the aforementioned readable storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0314] Optionally, specific examples in this embodiment can refer to the examples described in Embodiment 1 above, and will not be repeated here.

[0315] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0316] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0317] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

[0318] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0319] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0320] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0321] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0322] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0323] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0324] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for handling high-concurrency scenarios based on a distributed system, characterized in that, The method includes: Obtain system operating status parameter information of a distributed system; Based on the system operating status parameter information and the high concurrency identification threshold, determine the current concurrency level and current high concurrency response mode of the distributed system within a preset period; The system operating status parameter information is input into a pre-trained concurrent event prediction model to obtain the sensitivity of the high-concurrency operating condition level of the distributed system. The concurrent event prediction model is trained based on the historical concurrent event data of the distributed system. Based on the sensitivity to changes in the high-concurrency operating condition level, the current concurrency operating condition level, and the current high-concurrency response mode, determine the high-concurrency response mode corresponding to the distributed system within the preset period; The step of inputting the system operating status parameter information into a pre-trained concurrent event prediction model to obtain the sensitivity of the distributed system to changes in high-concurrency operating conditions includes: The system operating status parameter information is input into the first concurrent event model to obtain the first prediction result, wherein the first concurrent event prediction model is trained based on the historical concurrent event data of the distributed system in the first time period; The system operating status parameter information is input into the second concurrent event model to obtain a second prediction result. The second concurrent event prediction model is trained based on the historical concurrent event data of the distributed system in a second time period, wherein the first time period is longer than the second time period. Based on the first prediction result and the second prediction result, the sensitivity to changes in the high-concurrency operating condition level is determined.

2. The method according to claim 1, characterized in that, The first prediction result includes a first temporal distribution and a first level of concurrent events, and the second prediction result includes a second temporal distribution and a second level of concurrent events, wherein, The step of determining the sensitivity to changes in the high-concurrency operating condition level based on the first prediction result and the second prediction result includes: Based on the first time distribution and the second time distribution, obtain a set of time distribution prediction results; Based on the first level and the second level, obtain a set of level prediction results; The time distribution prediction result set and the grade prediction result set are combined to obtain a prediction result set; Based on the consistency between the time distribution prediction results and the level prediction results in the prediction results, the sensitivity of the high-concurrency operating condition level change is determined.

3. The method according to claim 1, characterized in that, The high-concurrency identification threshold includes a first high-concurrency threshold and a second high-concurrency threshold, wherein, The step of determining the current concurrency level and current high-concurrency response mode of the distributed system within a preset period based on the system operating status parameter information and the high-concurrency identification threshold also includes: Receive the first high-concurrency threshold and the second high-concurrency threshold input by the user; Based on the first high-concurrency threshold, the second high-concurrency threshold, and the system operating status parameter information, the current concurrency level and the current high-concurrency response mode are determined.

4. The method according to claim 1, characterized in that, The high-concurrency identification threshold includes a first high-concurrency threshold and a second high-concurrency threshold, wherein, The step of determining the current concurrency level and current high-concurrency response mode of the distributed system within a preset period based on the system operating status parameter information and the high-concurrency identification threshold includes: Obtain the maximum and minimum number of accesses per unit time within a preset time period after the distributed system starts. The first high-concurrency threshold is determined based on the maximum and minimum access counts. Based on the first high-concurrency threshold, the second high-concurrency threshold, and the system operating status parameter information, the current concurrency level and the current high-concurrency response mode are determined.

5. The method according to claim 1, characterized in that, The step of determining the high-concurrency response mode of the distributed system within the preset period based on the sensitivity to changes in the high-concurrency operating condition level, the current concurrency operating condition level, and the current high-concurrency response mode includes: If the current concurrency level does not match the high concurrency level corresponding to the current high concurrency response mode, then the high concurrency response mode is determined based on the high concurrency level change sensitivity, the current concurrency level, and the current concurrency level.

6. The method according to claim 1, characterized in that, The high-concurrency response mode includes one of the following: The business data and data copies of the distributed system are stored according to the preset business logic and the preset distributed storage cluster configuration. or, Back up the original data consistency configurations of the structured and unstructured data storage systems in the distributed storage cluster, and establish a cache area in the remaining disk space of the distributed storage cluster; or, The original data consistency configuration of the structured data storage system and the unstructured data storage system in the distributed storage cluster is backed up, the number of data replicas stored in the distributed system is controlled, and a cache area is established in the remaining disk space of the distributed storage cluster.

7. A high-concurrency processing device based on a distributed system, characterized in that, The device, applied to high-concurrency processing systems, includes: The acquisition module is used to acquire system operating status parameter information of the distributed system. The first determining module is used to determine the current concurrency level and current high concurrency response mode of the distributed system within a preset period based on the system operating status parameter information and the high concurrency identification threshold. The processing module is used to input the system operating status parameter information into a pre-trained concurrent event prediction model to obtain the sensitivity of the high-concurrency operating condition level of the distributed system. The concurrent event prediction model is trained based on the historical concurrent event data of the distributed system. The second determining module is used to determine the high-concurrency response mode corresponding to the distributed system within the preset period based on the high-concurrency operating condition level change sensitivity, the current concurrency operating condition level, and the current high-concurrency response mode; The module is used to input the system operating status parameter information into a first concurrent event model to obtain a first prediction result, wherein the first concurrent event prediction model is trained based on historical concurrent event data of the distributed system in a first time period; input the system operating status parameter information into a second concurrent event model to obtain a second prediction result, wherein the second concurrent event prediction model is trained based on historical concurrent event data of the distributed system in a second time period, wherein the first time period is longer than the second time period; and determine the sensitivity of the high-concurrency operating condition level change based on the first prediction result and the second prediction result.

8. A high-concurrency processing system based on a distributed system, characterized in that, The system includes a system operation status information comprehensive acquisition module and a high-concurrency event analysis and control module, wherein: Based on the system operation status information comprehensive acquisition module, the system operation status parameter information of the distributed system is obtained; Based on the high-concurrency event analysis and control module, the current concurrency level and current high-concurrency response mode of the distributed system are determined within a preset period according to the system operation status parameter information and the high-concurrency identification threshold. Based on the high-concurrency event analysis and control module, the system operating status parameter information is input into a pre-trained concurrent event prediction model to obtain the sensitivity of the high-concurrency operating condition level change of the distributed system. The concurrent event prediction model is trained based on the historical concurrent event data of the distributed system. Based on the high-concurrency event analysis and control module, the high-concurrency response mode corresponding to the distributed system within the preset period is determined according to the sensitivity of high-concurrency operating condition level changes, the current concurrency operating condition level, and the current high-concurrency response mode. The module is used to input the system operating status parameter information into a first concurrent event model to obtain a first prediction result, wherein the first concurrent event prediction model is trained based on historical concurrent event data of the distributed system in a first time period; input the system operating status parameter information into a second concurrent event model to obtain a second prediction result, wherein the second concurrent event prediction model is trained based on historical concurrent event data of the distributed system in a second time period, wherein the first time period is longer than the second time period; and determine the sensitivity of the high-concurrency operating condition level change based on the first prediction result and the second prediction result.

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the high-concurrency processing method for a distributed system as described in any one of claims 1 to 6.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the high-concurrency processing method based on a distributed system as described in any one of claims 1 to 6.

Images