Distributed system cluster fuse method and distributed system based on Hystrix

Abstract

The present invention provides a distributed system cluster fuse method and distributed system based on Hystrix, the method uses a sliding window for statistics, the sliding window is composed of a plurality of buckets, the buckets are stored as a hash structure using Redis, the key of the hash structure includes the bucket generation time and the bucket current value, the bucket current value is a binary integer, the bits of the binary integer include success, failure, rejection and unused bits and an anti-overflow bit set between the four bits, the success, failure and rejection bits respectively record the success, failure and rejected operations of the request, and if the anti-overflow bit has a value during statistics, the statistical process is terminated. The cluster fuse method and distributed system provided by the present invention can reduce redis cpu overhead and reduce redis io operations.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a cluster fusing method and a distributed system. Background Art

[0002] In distributed systems, multi-level upstream and downstream cascades formed by multi-level dependencies are common. For example, Figure 1 As shown in the figure, service a depends on service b and service c, and service b depends on service d. If service d has problems, a large number of requests from service b will time out, and a large number of connections from service b to service d will be exhausted, which may eventually cause service b to be unable to provide services to service a, causing a system avalanche. In fact, service d may not be an important service. In this case, we hope that service b can continue to provide services to service a without obtaining data from service d.

[0003] If an application cannot isolate failures from dependencies, the application itself is at risk of being dragged down. Therefore, in order to build a stable and reliable distributed system, services should have self-protection capabilities. When a dependent service is unavailable, the current service starts the self-protection function to avoid an avalanche effect.

[0004] Still Figure 1 For example, is there a fuse mechanism when service B requests service D? If statistics show that service D has problems, service D will not be requested anymore, but degraded data will be returned immediately, so that the connection will not be exhausted until each connection times out. Then, service D will be tried at a certain time, and if service D is considered to have recovered, service D will be requested again.

[0005] So the industry came up with Hystrix, and later Sentinel. For the self-protection of the PHP ecosystem where not all requests go through the gateway, the industry also has some solutions, such as odesk PHystrix, which uses apcu as a storage medium to implement the stand-alone version of the fuse capability. However, if the processing power of each machine is different, there will be multiple copies of the fuse parameter configuration, and once the load balancing strategy changes, the fuse parameter configuration of each machine must be adjusted. So in this case, the cluster version of the fuse solution is more suitable for this scenario.

[0006] At present, the industry has developed a clustered version of the circuit breaker solution for the PHP ecosystem, which is based on redis to solve the above problems, but this solution also has certain problems. For example, first, if redis's zset is used to store the number of three statistical requests, success, failure, and rejection, the purpose of statistical threshold judgment can be achieved. However, to clear expired points, the ZREMRANGEBYSCORE operation, which consumes a lot of redis performance, will drag down redis when operating a large number of elements; secondly, redis does not open long connections, and each operation requires establishing and ending connections; thirdly, redis does not have link tracking for each access, and redis cannot be perceived in the first time when reading and writing problems occur, which affects the judgment of the circuit breaker status; finally, the distributed system may have differences in the time obtained from the local machine. For a relatively small window, it is more sensitive to the difference in time. If there is no unified time, the accuracy of the count will be directly reduced.

[0007] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore it may contain information that does not constitute the prior art known to those skilled in the art. Summary of the invention

[0008] In order to at least solve one of the above problems existing in the prior art to a certain extent, the first object of the present invention is to provide a distributed system cluster fuse method based on Hystrix. The second object of the present invention is to provide a distributed system. Other objects of the present invention are to provide a computer-readable storage medium and a computer program product.

[0009] To achieve the above-mentioned purpose, according to one embodiment of the present invention, the first aspect of the present invention provides a distributed system cluster fuse method based on Hystrix, which uses a sliding window for statistics, and the sliding window is composed of multiple buckets, wherein: the bucket is stored as a hash structure using Redis, and the key of the hash structure includes the bucket generation time and the bucket current value; the bucket current value is a binary integer, and the bits of the binary integer include a success bit, a failure bit, a rejection bit, an unused bit, and an anti-overflow bit set between the four bits; the success bit, the failure bit, and the rejection bit respectively record the successful operation, the failed operation, and the rejected operation of the request; if the anti-overflow bit has a value during statistics, the statistical process is terminated.

[0010] The current value of the bucket may be a 64-bit integer, wherein the success bit may occupy 22 bits, the failure bit may occupy 16 bits, the rejection bit may occupy 18 bits, the unused bit may occupy 2 bits, and / or the anti-overflow bit may each occupy 2 bits.

[0011] According to a distributed system cluster circuit breaking method based on Hystrix according to an embodiment of the present invention, statistics are performed based on the time obtained from Redis; a long connection can be enabled for Redis and / or link tracking can be set; information can be written into a Lua command using a Lua script and sent to Redis; and notification information can be issued when the circuit breaker state changes and / or Redis has an abnormality.

[0012] To achieve the above-mentioned purpose, according to one embodiment of the present invention, the second aspect of the present invention provides a distributed system, which includes a cluster fuse module, and the cluster fuse module uses a sliding window for statistics based on Hystrix, and the sliding window is composed of multiple buckets, wherein: the bucket is stored as a hash structure using Redis, and the key of the hash structure includes the bucket generation time and the bucket current value; the bucket current value is a binary integer, and the bits of the binary integer include a success bit, a failure bit, a rejection bit, an unused bit, and an anti-overflow bit set between the four bits; the success bit, the failure bit, and the rejection bit respectively record the successful operation, the failed operation, and the rejected operation of the request; if the anti-overflow bit has a value during statistics, the statistical process is terminated.

[0013] The cluster fuse module included in the distributed system according to one embodiment of the present invention performs statistics based on the time obtained from Redis; a long connection can be opened for Redis and / or link tracking can be set; information can be written into Lua commands using Lua scripts and sent to Redis; and notification information can also be issued when the circuit breaker status changes and / or Redis has an abnormality.

[0014] To achieve the above-mentioned purpose, according to one embodiment of the present invention, the third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned distributed system cluster fuse method based on Hystrix are implemented.

[0015] To achieve the above-mentioned purpose, according to one embodiment of the present invention, a fourth aspect of the present invention provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the above-mentioned distributed system cluster fuse method based on Hystrix.

[0016] The distributed system cluster fusing method based on Hystrix and the distributed system thereof provided by the present invention use redis as a storage medium and utilize the sliding window and bucket of hystrix for counting, which can reduce the overhead of redis cpu and reduce redis io operations (input and output operations), so that the same redis can be counted for a larger concurrency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features of the present invention will be described in detail below with reference to specific exemplary embodiments shown in the accompanying drawings, which are given below only by way of illustration and therefore do not limit the present invention, in which:

[0018] Figure 1 A simplified architectural diagram showing a common distributed system.

[0019] Figure 2 The following is a basic logic flow chart showing how Hystrix performs availability checks before a request is made.

[0020] Figure 3 Show Figure 2 The specific operation flow chart of the flow chart shown.

[0021] Figure 4 The following is a basic logic flow chart showing how Hystrix judges the request results.

[0022] Figure 5 Shows Hystrix passing Figure 4 The logic flow chart shown is followed by a flow chart of related operations of the circuit breaker.

[0023] Figure 6 A schematic diagram of a sliding window and buckets contained therein according to an embodiment of the present invention is shown.

[0024] Figure 7 A schematic diagram showing the relationship between a sliding window, buckets contained therein, and time according to an embodiment of the present invention is shown.

[0025] Figure 8 A schematic diagram showing the bits of the current value of a bucket according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0026] The present invention is described in detail below through specific embodiments so that those skilled in the art can easily implement the present invention according to the contents disclosed in this specification. The embodiments described below are only some embodiments of the present invention, not all. Based on the embodiments described in this specification, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present invention. It should be noted that the embodiments in this specification and the features in the embodiments can be combined with each other without conflict.

[0027] The terms used herein are only for the purpose of illustrating specific embodiments and are not intended to limit the present invention. Unless the context otherwise specifically indicates, the singular forms "one", "a kind of" and "the" used herein also include the plural forms "multiple", "multiple" and "the". The words "first", "second" and the like used herein are only used to distinguish different features, steps, operations, elements and / or parts, etc., and neither represent any specific technical meaning nor the necessary logical order between them. The term "multiple" used herein may refer to two or more, and the term "at least one" may refer to one, two or more. For any feature, step, operation, element and / or part mentioned herein, it is generally understood to be one or more if the context is not otherwise specifically indicated. It should also be understood that the words "include" and / or "comprising" used herein refer to the presence of the features, steps, operations, elements and / or parts, but do not exclude the presence or increase of one or more other features, steps, operations, elements, parts and / or their collections. The term "and / or" used herein includes any or all combinations of one or more related listed items. The element suffixes “module” and “unit” herein are only used for convenience of description and, thus, can be used interchangeably without having any distinct meanings or functions.

[0028] When the prior art related to the description of the present invention is obvious to those skilled in the art, its detailed description will be omitted. It should also be understood that the description of each embodiment in this specification focuses on emphasizing the differences between the embodiments, and the same or similar aspects between the embodiments can be referenced to each other. For the sake of brevity, this specification does not repeat them one by one.

[0029] In the Internet distributed system, there are usually many ways to use Hystrix to implement fuses in the backend services of the PHP ecosystem, such as command mode and middleware mode. Both methods can be divided into two steps: the first step is to check the circuit breaker (Circuit, also known as "fuse") and the counter (metric, also known as "statistician") to determine whether a request is issued; the second step is to complete the counting (also known as "statistics") and calculate the circuit breaker status after the request is issued (or after the request is not issued). Since PHP ecological services use guzzle extensively to complete RPC (Remote Procedure Call, remote procedure call) requests, and guzzle provides a middleware method that can execute the code in the middleware before and after the request, the specific implementation method of this application is explained by taking the middleware method to implement fuses as an example. According to a specific embodiment of the present invention, the PHP composer package is used to implant a certain guzzle-based RPC logic in a middleware manner, thereby achieving at least one invention purpose of the present invention. It can be easily understood that those skilled in the art can easily implement the present invention in other ways based on this specification.

[0030] The following combination Figure 2 to Figure 5 A brief description of the overall logic and process of Hystrix and its circuit breakers. The role and basic working mechanism of the circuit breaker in Hystrix is ​​that during operation, Hystrix reports the status of request success, failure, and rejection to the circuit breaker corresponding to each commandKey. The circuit breaker maintains and counts these data, and decides whether to open the fuse switch based on these statistical information. If it is open, the subsequent requests will be disconnected and quickly returned (fallback). After a period of time, the circuit breaker tries to half open (half open) and let in some traffic requests, which is equivalent to a health check on the dependent service. If the request is successful, the circuit breaker is closed (close).

[0031] like Figure 2 As shown, in step S201, Hystrix performs an availability (isAvailable) judgment before the request is issued, that is, judging whether the request is available (the specific judgment method is, for example, Figure 3 As shown in the scheme in the middle, if the request is available, the request is continued to be processed in step S203, otherwise a downgrade fallback is performed in step S202.

[0032] Figure 3 The specific steps for performing availability judgment are shown, such as Figure 3As shown, in step S301, it is first determined whether the circuit breaker is in the closed state. If the answer is yes, that is, the circuit breaker is in the closed state, a request is issued in step S303, which is equivalent to Figure 2 In step S203, it is determined that the request is available and the request is continued to be processed; if the answer is negative, that is, the circuit breaker is not in the closed state, but in the open or half-open state, then in step S302, it is determined whether the time of the last failure (latest fail time) is within the observation range of the sliding window. If the answer is positive, that is, the time of the last failure is still within the observation range of the sliding window, then in step S305, the request is not allowed to be issued (rejected), the rejection is directly recorded, and then the downgrade rollback is performed, which is equivalent to Figure 2 In step S202, it is determined that the request is unavailable and the downgrade fallback is performed; if the answer is negative, that is, the time of the last failure is no longer within the observation range of the sliding window, then a request is issued in step S304, which is equivalent to Figure 2 In step S203, it is determined that the request is available and the request processing continues.

[0033] Figure 4 The following is a basic logic flow chart showing how Hystrix judges the request result after the request is issued. If the request is allowed after the above availability judgment, the following result will be obtained after the request is issued. Figure 4 The request result shown in S401 is initially divided into success shown in S402 and failure shown in S403. If the request result is success shown in S402, it is still necessary to undergo a validity check to distinguish again which request results are identified as success shown in S404 and failure shown in S405. The validity check here is a user-defined judgment and statistics on the returned request results for the convenience of counting. Therefore, there are certain differences in the identification criteria for the failure results shown in S403 and S405, but they are both classified as failures in the statistical caliber. In this sense, the two are essentially the same.

[0034] go through Figure 4 After the logical judgment shown, if the request result is judged to be successful, the success is recorded and the success count is increased. Figure 5 Perform circuit breaker related operations as shown. Figure 5 As shown, in step S501, the state of the circuit breaker is checked, and the circuit breaker state is divided into a half open state shown in S502, an open state shown in S504, and a closed state shown in S510. If the circuit breaker is in a half open state, the metric and circuit breaker state are reset to closed in step S503.

[0035] If the circuit breaker is in the open state, it is necessary to check whether the circuit breaker is closed or kept open in the current window. Specifically, in step S505, it is first determined whether the total number of failures is zero. If it is zero, the circuit breaker is closed in step S507; if it is not zero, it is determined in step S506 whether the number of failures is less than a specific first threshold value, and the first threshold value can be set as needed. If the number of failures is less than the first threshold value, the circuit breaker is closed in step S507, otherwise it is determined in step S508 whether the failure rate is less than a specific second threshold value, and similarly, the second threshold value can also be set as needed. If the failure rate is less than the second threshold value, the circuit breaker is closed in step S507, otherwise the circuit breaker is kept in the open state (no operation) in step S509, that is, the circuit breaker state does not change.

[0036] If it is necessary to count the failures, then the current state of the circuit breaker is closed. If the circuit breaker state is closed as shown in S510, it is necessary to check whether the circuit breaker is opened or kept closed in the current window. Specifically, in step S511, it is first determined whether the total number of failures is zero. If it is zero, the circuit breaker is kept in a closed state (no operation) in step S512, that is, the circuit breaker state does not change; if the number of failures is not zero, then in step S513, it is determined whether the number of failures is less than the above-mentioned specific first threshold value, and the first threshold value can be set as needed. If the number of failures is less than the specific threshold value, the circuit breaker is kept closed in step S515, and the failure count is increased (add one operation), otherwise, in step S514, it is determined whether the failure rate is less than the above-mentioned specific second threshold value, and similarly, the second threshold value can also be set as needed. If the failure rate is less than the second threshold value, the circuit breaker is kept closed in step S515, and the failure count is increased (add one operation), otherwise, the circuit breaker is opened in step S516, that is, the circuit breaker state is set to open.

[0037] There are many ways to implement counters in Hystrix. In this embodiment of the present invention, a sliding window (Rollingwindow) is used for counting. Figure 6-7 The sliding window of Hystrix is ​​explained. The sliding window records a circular array, and over time, it continuously covers the last bucket that needs to be eliminated. The bucket is the smallest unit when Hystrix counts the sliding window data. The size of the bucket multiplied by the number of buckets n is equal to the size of the sliding window, that is, the bucket is equivalent to the concept of one-nth of the window. The number of buckets n can be set according to the system configuration, or it can be a fixed value. However, in principle, the number of buckets n should not be set too large or too small. Too small will reduce the sensitivity of the counter, and too large will cause performance loss. The exemplary embodiment of this specification takes the sliding window containing 5 buckets as an example. Figure 6As shown, the buckets are marked with A, B, C, D, and E respectively. These five buckets are connected end to end to form a circle, indicating that the sliding window is filled with buckets A, B, C, D, and E.

[0038] The generation and expiration of buckets are divided into the following cases: (1) There is no bucket in the sliding window, which corresponds to the first running count or all buckets have expired. In this case, the count is recorded starting from bucket A; (2) There is an unexpired bucket in the sliding window. In this case, the sequence of the target bucket is calculated based on the latest bucket. Figure 7 This section explains how to calculate the sequence of target buckets. The shaded area in the figure represents the sliding window, the horizontal axis represents the timeline, and the vertical arrow represents the current moment. For ease of explanation, only buckets A, C, D, and the expired bucket C (represented as "front bucket C" in the figure) are shown in the figure. Figure 7 As shown, the nearest unexpired bucket in the sliding window to the current time is A. For example, if there is an interval of 2.7+ bucket times calculated based on the current time and the start time of bucket A, bucket C is directly generated and counted in bucket C. For example, if the window size is 10 seconds (10s), the size of each bucket is 2s. As time goes by, when the previous bucket C expires, a new bucket C is generated at the same time, and the data in the previous bucket C becomes invalid, and the new bucket C starts counting.

[0039] The implementation of the bucket in the present invention uses a data format stored in redis as a hash structure. The hash structure stores 10 keys. Buckets A, B, C, D, and E each have two keys, which are the generation time of each bucket and its current value. Taking bucket A as an example, its two keys are the generation time of bucket A (defined as bucket_A_start_time) and the current value of bucket A (defined as bucket_A_value). The two keys of buckets B, C, D, and E are similar to those of bucket A above and will not be repeated. In this way, there are a total of 10 keys in the hash structure. Still taking bucket A as an example (other buckets are similar to bucket A), bucket_A_start_time records the creation time of bucket A in seconds, accurate to 10 to the power of -6. This time is used to determine whether the bucket is expired. If it is expired, it will be destroyed and a new bucket will be created; Bucket_A_value records a 64-bit binary integer, and the use of each bit can be, for example, as follows Figure 8 The detailed description is listed in the following Table 1.

[0040] Table 1 Bucket A current value bit specific description

[0041]

[0042] Thus, corresponding to the above successful counting operation, only 1 needs to be added (1×2 0); corresponding to the operation of failure count, just add (1×2 24 ); corresponding to the rejection count operation, just add (1×2 42 ). Each time, only one of the three counting methods mentioned above is performed on the current bucket, which can be completed using the atomic operation of Redis Hincrby. If the overflow is added, each time the counter is used, it will be determined whether the overflow bit has a value. If there is a value, an exception will be thrown to terminate the counting process. This exception can be handled in the RPC request.

[0043] Through the above method, each time the count in each bucket is obtained, only one of the three counts of success, failure, and rejection is counted, instead of the three counts for each bucket in the prior art. In this way, each bucket can save the operation of reading two keys, and five buckets can save ten reads. And when the bucket expires, the count and time record of the entire bucket can be destroyed, without using the redis ZREMRANGEBYSCORE operation (the complexity of this operation is O(lgN+M), N is the number of elements in the set zset needs to find the start and end positions of this range of the jump table, M is the number of elements in this range, once the set is large, this operation will consume a lot of redis CPU and increase the overall time consumption), which greatly reduces the consumption of redis performance.

[0044] It should be noted that the bits of the current value of the bucket in the above embodiments, including the total number of bits and the bit range of each type of count, are exemplary and not restrictive. Those skilled in the art can easily change the bits of the current value of the bucket according to actual needs based on the above exemplary embodiments, such as changing it to 32 bits or 128 bits, and change the bit range of each type of count as needed, and can also implement the technical solution of the present invention, achieve the same technical effect, and realize the same purpose of the invention.

[0045] In a distributed system, although the server usually calibrates the time regularly, there will inevitably be a certain time difference. Therefore, the present invention can further include using centralized time to improve accuracy and eliminate distributed time differences. Specifically, if the server time is used as the time for recording and reporting, the statistical data reported at the same time may fall into different buckets. For example, assuming that server X is 1s faster than the reference time and server Y is 1s slower than the reference time, if the size of the bucket is 2s, server X calculates the current bucket as bucket B, and thus performs a counting operation on bucket B, and if server Y also has a request at the same time, it calculates the current bucket as bucket A, and thus performs a statistical operation in A, this scenario is bound to cause statistical confusion and affect the operation of the fuse. In order to eliminate this confusion caused by time differences, according to the technical solution of an embodiment of the present invention, each time the current bucket is calculated, the current time is obtained from redis in advance, and then the current bucket is calculated with this time. Although when obtaining the current time from redis, encountering network delays will also cause differences, it has been greatly improved compared to the solution of obtaining time locally from the server.

[0046] According to the technical solution of one embodiment of the present invention, a long connection is also enabled for redis and / or link tracking is added. In the actual application of high concurrency scenarios, enabling a long connection can establish a TCP channel between php-fpm and redis, eliminating many TCP establishment and closing operations, reducing waste, and saving time. Adding link tracking can facilitate querying whether the status of the request and write counter is correct in a certain request, so as to perceive whether the counter is working properly. The operation of link tracking may include the examples listed in Table 2 below.

[0047] Table 2 Link tracking operation examples and usage scenarios

[0048] operate Usage scenarios get Circuit breaker status acquisition set Circuit breaker status setting acquisition eval Get multiple values ​​or multiple operations at once hgetall Get counts of all buckets del Reset Circuit Breaker

[0049] According to the technical solution of one embodiment of the present invention, redis is used to run a Lua script, and information such as the obtained bucket, system time, latest failure time, and circuit breaker status are all written into the same Lua command, and then sent to redis. In this way, comprehensive information can be obtained and network IO overhead can be saved.

[0050] The technical solution according to one embodiment of the present invention also adds monitoring and alarm functions. Monitoring alarms can be divided into two categories: the first type of monitoring alarm is the monitoring alarm of the circuit breaker status. When the circuit breaker status changes, a notification will be issued to inform the developer that the circuit breaker has changed from closed to open. Generally, there are too many and too concentrated failures, and the fuse mechanism begins to trigger. Of course, there are also cases where the circuit breaker recovers. The second type of monitoring alarm is the reporting of link tracking, which can be reported to the existing monitoring system. Once redis has an abnormality, this type of alarm will be triggered. At this time, the developer can know that there is a problem with the circuit breaker itself, and then decide to close this mechanism or make a comprehensive judgment.

[0051] The above embodiment of the present invention optimizes the dependent storage medium redis, and while adding a fuse capability to the system, reduces the overhead of redis as much as possible, so that the same redis can be counted for a larger concurrency.

[0052] On the other hand, the present application also provides a distributed system, which, in addition to the hardware included in a conventional distributed system, also includes a cluster fuse module, which uses a sliding window for statistics based on Hystrix, and can implement the above-mentioned distributed system cluster fuse method based on Hystrix of the present invention. Wherein, the sliding window is composed of multiple buckets, and the buckets are stored as a hash structure using Redis. The key of the hash structure includes the bucket generation time and the bucket current value. The bucket current value can be a binary integer, and the bits of the binary integer include a success bit, a failure bit, a rejection bit, an unused bit, and an anti-overflow bit set between the four bits. The success bit, the failure bit, and the rejection bit respectively record the successful operation, the failed operation, and the rejected operation of the request. If the anti-overflow bit has a value during statistics, the statistical process is terminated.

[0053] According to an embodiment of the present invention, the cluster fuse module performs statistics based on the time obtained from Redis. In addition, the cluster fuse module can also open a long connection for Redis and / or set link tracking. In addition, the cluster fuse module can also use Lua scripts to write information into Lua commands and send them to Redis. Finally, when the circuit breaker state changes and / or Redis is abnormal, the cluster fuse module can also issue a notification message.

[0054] On the other hand, the present application also provides a computer-readable medium, which may be included in the system described in the above embodiment; or may exist independently without being assembled into the system. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the device can implement the steps of the above-mentioned distributed system cluster fusing method based on Hystrix of the present invention.

[0055] On the other hand, the present application also provides a computer program product, which includes computer instructions, which, when executed by a processor, implement the steps of the above-mentioned Hystrix-based distributed system cluster fuse method of the present invention.

[0056] In particular, the embodiments described above with reference to the flowcharts in the accompanying drawings can be implemented as computer software programs. For example, the embodiments disclosed in the specification of this application include a computer program product, which includes program instructions or codes for executing the distributed system cluster fuse method of Hystrix of the present invention shown in each flowchart in the accompanying drawings. In such an embodiment, the computer program can be downloaded and installed from the network through a communication unit, and / or installed from a removable medium. When the computer program is executed, the method of the present invention is implemented.

[0057] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: a computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0058] In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0059] The computer program code for performing the operation of the present application can be written by one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and also conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0060] The flow chart and block diagram in the accompanying drawings exemplarily illustrate the possible architecture, function and operation of the system, method and computer program product according to each embodiment of the present application. Each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should be noted that in some alternative embodiments, the functions marked in the box can also occur in a different order from that shown in the accompanying drawings. For example, two boxes represented in sequence can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0061] The units or modules involved in the embodiments of the present application may be implemented by software or hardware. The above units or modules may also be set in a processor, for example, it may be described as: a processor including a cluster fuse module. The names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0062] All documents mentioned in this specification are cited in this application as references, just as if each document was fully cited in this specification as references.

[0063] In addition, it should be understood that after reading the above description of the present invention, those skilled in the art may make various changes or modifications to the present invention, and these equivalent forms also fall within the protection scope of the present invention.

Claims

1. A distributed system cluster fuse method based on Hystrix, characterized in that: Statistics are performed using a sliding window, which consists of multiple buckets, where: The bucket is stored in a hash structure using Redis, and the key of the hash structure includes the bucket generation time and the bucket current value; The current value of the bucket is a binary integer, and the bits of the binary integer include a success bit, a failure bit, a rejection bit, an unused bit, and an anti-overflow bit set between the four bits; The success bit, failure bit and rejection bit record the successful operation, failed operation and rejected operation of the request respectively; If a value appears in the anti-overflow bit during statistics, the statistical process is terminated.

2. According to the distributed system cluster fusing method based on Hystrix according to claim 1, it is characterized in that: The current value of the bucket is a 64-bit integer.

3. According to the distributed system cluster fusing method based on Hystrix according to claim 2, it is characterized in that: The success bit occupies 22 bits, the failure bit occupies 16 bits, the rejection bit occupies 18 bits, the unused bit occupies 2 bits, and / or the anti-overflow bit occupies 2 bits each.

4. The distributed system cluster fusing method based on Hystrix according to claim 1, characterized in that: Statistics are calculated based on the time obtained from Redis.

5. The distributed system cluster fusing method based on Hystrix according to claim 1, characterized in that: Enable persistent connections and / or set up link tracking for Redis.

6. The distributed system cluster fusing method based on Hystrix according to claim 1, characterized in that: Use Lua scripts to write information into Lua commands and send them to Redis.

7. The distributed system cluster fusing method based on Hystrix according to claim 1, characterized in that: When the circuit breaker status changes and / or Redis encounters an exception, a notification message is sent.

8. A distributed system, characterized in that: Contains a cluster fuse module, which uses a sliding window for statistics based on Hystrix. The sliding window consists of multiple buckets, where: The bucket is stored in a hash structure using Redis, and the key of the hash structure includes the bucket generation time and the bucket current value; The current value of the bucket is a binary integer, and the bits of the binary integer include a success bit, a failure bit, a rejection bit, an unused bit, and an anti-overflow bit set between the four bits; The success bit, failure bit and rejection bit record the successful operation, failed operation and rejected operation of the request respectively; If a value appears in the anti-overflow bit during statistics, the statistical process is terminated.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images