Cache anomaly detection method and device, storage medium and electronic equipment
By performing exception detection and current limiting processing on the Redis cache server in the Internet financial payment system, the database crash problem caused by cache server exceptions is solved, and service stability and system availability are improved.
Patent Information
- Application Number
- CN202411034218.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-15
AI Technical Summary
In the field of Internet financial payment, abnormalities in Redis cache servers will cause database crashes, resulting in unsatisfactory service stability, and the existing technology lacks effective solutions.
The cache server is detected using a predetermined detection interval, and the number of interaction abnormalities is recorded. When the threshold is reached, the cache server is suspended, and the service is restricted until the cache server returns to normal and then restarts.
It effectively avoids database downtime caused by cache server exceptions, improves service stability, and ensures high availability and performance of the system.
Smart Images

Figure CN118964100A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of data processing and cache technology, and in particular to a cache anomaly detection method, device, storage medium and electronic device. Background Art
[0002] As the number of users increases day by day, the transaction scale in the field of Internet financial payment has exceeded hundreds of millions, and the server faces huge challenges. The concurrency of a single machine often exceeds 1,000 TPS (Transactions Per Second), and the demand for memory, CPU (Central Processing Unit) and network resources has increased sharply. When supporting such a high-load business, the performance bottleneck of traditional database clusters is prominent.
[0003] Therefore, the relevant technology adopts Redis cache technology (a cache database), which significantly reduces IO (Input / Output) by virtue of its memory storage advantage, thereby greatly improving concurrent processing capabilities and query efficiency, effectively enhancing application performance, and significantly improving stand-alone TPS. However, high dependence on Redis also has potential risks. Once the Redis service is interrupted, all traffic will directly hit the database, which can easily trigger a chain reaction, causing the database to crash or even the entire application service to be paralyzed, and there is a problem of unsatisfactory service stability.
[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0005] The embodiments of the present invention provide a cache anomaly detection method, device, storage medium and electronic device to at least solve the technical problem in the related art that cache server anomalies lead to unsatisfactory service stability.
[0006] According to one aspect of an embodiment of the present invention, a cache anomaly detection method is provided, comprising: performing anomaly detection on a cache server at a predetermined detection interval to obtain an anomaly detection result; when the anomaly detection result indicates that there is a data interaction anomaly, incrementing the number of interaction anomalies; wherein the number of interaction anomalies is used to record the cumulative number of data interaction anomalies detected on the cache server at the predetermined detection interval; when the incremented number of interaction anomalies reaches a first predetermined threshold, resetting the number of interaction anomalies and pausing the cache server; limiting the flow of a service, wherein when the cache server is paused, the service interacts with a database corresponding to the cache server, and when the cache server is turned on, the service interacts with the cache server, and data synchronization exists between the cache server and the corresponding database; until it is detected that there is no anomaly in the cache server at the predetermined detection interval, the cache server is turned on.
[0007] Optionally, the method of using a predetermined detection interval to perform anomaly detection on the cache server to obtain anomaly detection results includes: using the predetermined detection interval, according to a plurality of predetermined detection tasks, performing anomaly detection on the cache server, and obtaining sub-detection results corresponding to the plurality of detection tasks, wherein the plurality of detection tasks include at least an interaction detection task, and the interaction detection task is used to detect the existence of data interaction anomalies in the cache server; obtaining the anomaly detection result based on the sub-detection results corresponding to the plurality of detection tasks.
[0008] Optionally, the predetermined detection interval is used to perform abnormality detection on the cache server according to a plurality of predetermined detection tasks to obtain sub-detection results corresponding to the plurality of detection tasks, including: the predetermined detection interval is used to execute acquisition processing on the key value corresponding to the predetermined key included in the cache server to determine the sub-detection result corresponding to the interaction detection task; when the plurality of detection tasks also include at least one of the following: a performance detection task, a memory usage detection task, and a data integrity detection task, the predetermined detection interval is used to determine the response time of the cache server according to a predetermined measurement instruction; based on the response time, the sub-detection result corresponding to the performance detection task is determined; or, the predetermined detection interval is used to determine the sub-detection result corresponding to the memory usage detection task based on the memory occupancy status generated by the operation of the cache server; or, the predetermined detection interval is used to verify a predetermined data structure set in the cache server to determine the sub-detection result corresponding to the data integrity detection task.
[0009] Optionally, the method also includes: when the anomaly detection result indicates that other tasks have anomalies, incrementing the other anomaly times corresponding to the other tasks, wherein the other anomaly times are used to record the cumulative number of times other anomalies are detected in the cache server using the predetermined detection interval, and the other tasks are detection tasks other than the interactive detection task among the multiple detection tasks; when the other anomaly times are greater than a second predetermined threshold, determining an alarm strategy for the exception handling end based on the priority corresponding to the other tasks.
[0010] Optionally, the flow limiting processing of the service includes: when there are multiple services, determining service types corresponding to the multiple services respectively; and performing flow limiting processing on the services whose service types match the target type among the multiple services.
[0011] Optionally, before incrementing the number of interaction exceptions, the method further includes: detecting whether the cache server is turned on, and when the cache server is turned on, acquiring a synchronization lock for the cache server, wherein the synchronization lock is used to limit concurrent access to the cache server.
[0012] Optionally, the use of a predetermined detection interval to perform anomaly detection on the cache server to obtain anomaly detection results includes: determining the number of visits to the cache server; based on the number of visits, using a target model for processing to determine the predetermined detection interval, wherein the target model is trained based on the historical number of visits to the cache server and historical anomaly information, and the predetermined detection interval is used to perform anomaly detection on the cache server to obtain the anomaly detection result.
[0013] According to another aspect of an embodiment of the present invention, a cache anomaly detection device is provided, comprising: a detection module, configured to perform anomaly detection on a cache server at a predetermined detection interval to obtain an anomaly detection result; a number update module, configured to increment the number of interaction anomalies when the anomaly detection result indicates that there is a data interaction anomaly; wherein the number of interaction anomalies is used to record the cumulative number of data interaction anomalies detected on the cache server at the predetermined detection interval; a reset module, configured to reset the number of interaction anomalies and suspend the cache server when the incremented number of interaction anomalies reaches a first predetermined threshold; a current limiting module, configured to perform current limiting on a service, wherein when the cache server is paused, the service interacts with a database corresponding to the cache server, and when the cache server is turned on, the service interacts with the cache server, and the cache server and the corresponding database are in data synchronization; and a recovery module, configured to turn on the cache server until it is detected that there is no anomaly on the cache server at the predetermined detection interval.
[0014] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided, wherein the non-volatile storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing any one of the cache anomaly detection methods.
[0015] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the cache anomaly detection methods.
[0016] In the embodiment of the present invention, a heartbeat detection technology is adopted, and a cache server is detected for abnormality by using a predetermined detection interval to obtain an abnormality detection result; when the abnormality detection result indicates that there is a data interaction abnormality, the number of interaction abnormalities is incremented; wherein the number of interaction abnormalities is used to record the cumulative number of data interaction abnormalities detected in the cache server by using the predetermined detection interval; when the number of interaction abnormalities after increment reaches a first predetermined threshold, the number of interaction abnormalities is reset and the cache server is suspended; the service is subjected to flow limiting, wherein when the cache server is suspended, the service interacts with the database corresponding to the cache server, and when the cache server is turned on, the service interacts with the cache server, and the cache server and the corresponding database are synchronized in data; until the predetermined detection interval is used to detect that the cache server does not have an abnormality, the cache server is turned on. The purpose of avoiding database downtime due to cache server breakdown is achieved, and the technical effect of improving service stability is achieved, thereby solving the technical problem of cache server abnormality in the related technology, resulting in unsatisfactory service stability. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0018] Figure 1 is a schematic flow chart of an optional cache anomaly detection method provided according to an embodiment of the present invention;
[0019] Figure 2 is a first flow chart of an optional cache anomaly detection method provided according to an embodiment of the present invention;
[0020] Figure 3 is a second flow chart of an optional cache anomaly detection method provided according to an embodiment of the present invention;
[0021] Figure 4 It is a schematic diagram of an optional cache anomaly detection device provided according to an embodiment of the present invention. DETAILED DESCRIPTION
[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0024] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:
[0025] The heartbeat detection mechanism allows applications running on different nodes to sense whether applications on other nodes are running normally through network communication.
[0026] Redis, a high-performance key-value database based on cache.
[0027] According to an embodiment of the present invention, a method embodiment of cache anomaly detection is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0028] Figure 1 is a schematic flow chart of an optional cache anomaly detection method provided according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:
[0029] Step S102, performing anomaly detection on the cache server at a predetermined detection interval to obtain anomaly detection results;
[0030] It is understandable that a heartbeat detection mechanism may be used for detection, such as performing anomaly detection on the cache server (ie, Redis) at a predetermined detection interval (eg, every 1 second, every 5 minutes, every hour, etc.) to obtain anomaly detection results.
[0031] Optionally, the scheduled task is configured through SpringBoot's @Scheduled(cron="0 / 1****?") annotation. The cron expression indicates that the execution frequency of the scheduled task is 1 time / s. The Redis constant class configures the heartbeat detection constant key: REDIS_HEALTH_CHECK, the Redis constant class configures the detection threshold REDIS_FAIL_THRESHOLD=3, that is, the first predetermined threshold is set to 3 times, and configures the Redis switch RedisSwitch=OPEN (caching is enabled by default).
[0032] In an optional embodiment, a predetermined detection interval is used to perform anomaly detection on the cache server to obtain anomaly detection results, including: using a predetermined detection interval, according to a predetermined plurality of detection tasks, anomaly detection is performed on the cache server to obtain sub-detection results corresponding to the plurality of detection tasks, wherein the plurality of detection tasks include at least an interactive detection task, and the interactive detection task is used to detect the existence of data interaction anomalies in the cache server; anomaly detection results are obtained based on the sub-detection results corresponding to the plurality of detection tasks.
[0033] It can be understood that the anomaly detection process is performed according to a predetermined plurality of detection tasks, each detection task corresponds to a different aspect or performance indicator of the cache server, and provides a comprehensive anomaly detection capability. Among the multiple detection tasks, at least an interaction detection task is included, which specifically detects whether there is a data interaction anomaly in the cache server. Each detection task will generate a corresponding sub-detection result after execution, and the sub-detection result reflects the state or performance of the cache server in a specific aspect of the corresponding detection task evaluation. By combining all sub-detection results, the final anomaly detection result can be obtained.
[0034] Optionally, the above-mentioned multiple detection tasks can be multiple, for example: performance detection, resource utilization detection, network connection detection, security detection, etc. Each detection task has its specific purpose and detection standard, which is used to evaluate the health status of the cache server in this aspect. The performance detection task can generate sub-detection results about the processing speed of the cache server, and the resource utilization detection task can generate sub-detection results about the CPU and memory usage.
[0035] Through the above processing, possible problems of the cache server can be identified and located more accurately, so that corresponding measures can be taken. The fine-grained detection of multiple detection tasks helps to better understand the operating status of the cache server and perform targeted optimization and maintenance.
[0036] In an optional embodiment, a predetermined detection interval is used to perform an abnormality detection on the cache server according to a plurality of predetermined detection tasks, and sub-detection results corresponding to the plurality of detection tasks are obtained, including: using a predetermined detection interval, executing acquisition processing of a key value corresponding to a predetermined key included in the cache server, and determining a sub-detection result corresponding to the interaction detection task; when the plurality of detection tasks also include at least one of the following: a performance detection task, a memory usage detection task, and a data integrity detection task, using a predetermined detection interval, and determining a response time of the cache server according to a predetermined measurement instruction; based on the response time, determining a sub-detection result corresponding to the performance detection task; or, using a predetermined detection interval, based on the memory occupancy status generated by the operation of the cache server, determining a sub-detection result corresponding to the memory usage detection task; or, using a predetermined detection interval, verifying a predetermined data structure set in the cache server, and determining a sub-detection result corresponding to the data integrity detection task.
[0037] It can be understood that, using a predetermined detection interval, the key value corresponding to the predetermined key included in the cache server is obtained. If the key value can be successfully obtained, it means that there is no data anomaly. If the key value cannot be successfully obtained, it means that there is a data anomaly. If the interactive detection task is used to detect the basic ability of the cache server to realize its data interaction function, in addition to the interactive detection task, the multiple detection tasks can also include at least one of the following: performance detection task, memory usage detection task, data integrity detection task. Among the multiple detection tasks, the performance detection task adopts a predetermined detection interval to obtain the response time of the cache server according to the predetermined measurement instruction, and based on the response time, the sub-detection result corresponding to the performance detection task is determined, and then the performance of the cache server is evaluated. The memory usage detection task focuses on the memory resource utilization of the cache server, and also adopts a predetermined detection interval to evaluate based on the memory occupancy status generated by the operation of the cache server. The data integrity detection task adopts a predetermined detection interval to verify the predetermined data structure (such as a hash table, a linked list, etc.) set in the cache server. Based on the verification result, the sub-detection result corresponding to the data integrity detection task is determined, and the corresponding sub-detection result reflects the reliability of the data stored in the cache server. Through the above processing, a comprehensive evaluation method is provided for abnormal detection of the cache server, and the stability state of the cache server can be comprehensively judged based on the sub-detection results corresponding to various detection tasks.
[0038] Optionally, the predetermined key may be a plurality of predefined parameters, such as a key data item or a high-frequency access item.
[0039] Optionally, the measurement instructions may be of multiple types, for example, sending a specific type of request or executing a specific performance test script.
[0040] Optionally, the sub-detection result corresponding to the performance detection task may include, for example, information such as whether the response time is within an acceptable range and whether there is a trend of performance degradation.
[0041] Optionally, the memory usage detection task can obtain relevant data by querying the memory management information of the operating system, monitoring the memory usage of the cache server process, or using a dedicated memory analysis tool. The memory usage detection task corresponds to sub-detection results, including information such as memory usage rate, whether there is a memory leak, and whether there is enough memory available.
[0042] Optionally, the verification of the data integrity detection task may include multiple aspects, such as checking the consistency of the data structure (such as whether the key-value pairs of the hash table are correctly matched), integrity (such as whether the linked list forms a closed loop or whether there are breaks), and whether there is unexpected data modification or corruption.
[0043] In an optional embodiment, the method further includes: when the anomaly detection result indicates that other tasks have anomalies, incrementing other anomaly times corresponding to other tasks, wherein the other anomaly times are used to record the cumulative number of other anomalies detected in the cache server using a predetermined detection interval, and the other tasks are detection tasks other than the interactive detection tasks among multiple detection tasks; when the other anomaly times are greater than a second predetermined threshold, determining an alarm strategy for the anomaly handling end based on the priority corresponding to the other tasks.
[0044] It can be understood that it is expanded to handle multiple types of exceptions that may occur in the cache server, not just data interaction exceptions. When the exception detection result indicates that the cache server has exceptions in other tasks, that is, detection tasks other than interaction detection tasks, such as performance detection tasks, memory usage detection tasks, data integrity detection tasks, etc. If other tasks detect an exception, the number of occurrences of this exception will be recorded accordingly, and the corresponding other exception times will be incremented. Once the number of other exceptions of other tasks exceeds the corresponding second predetermined threshold, the alarm strategy for the exception handling end will be determined according to the preset priority of the other task. Through the above processing, the operating status of the cache server can be more comprehensively detected, and various types of exceptions can be discovered and responded to in a timely manner. Through a priority-based alarm strategy, it is ensured that high-priority exceptions are handled first, thereby minimizing the impact on the business.
[0045] Optionally, the above priorities may be set based on factors such as the business impact, urgency, and historical abnormality frequency of the task.
[0046] Optionally, the alarm strategy may include sending different types of alarm notifications (such as emails, text messages, system messages, etc.) and notifying exception handling terminals of different levels.
[0047] Optionally, taking into account the abnormal situations and priorities of multiple other tasks, determine whether to issue an alarm or suspend the cache server. For example, if the number of abnormalities of multiple other tasks exceeds their respective thresholds, more urgent response measures may be taken, such as immediately suspending the service of the cache server, starting a backup server, or performing a failover operation.
[0048] In an optional embodiment, a predetermined detection interval is used to perform anomaly detection on the cache server to obtain anomaly detection results, including: determining the number of visits to the cache server; based on the number of visits, using a target model for processing to determine the predetermined detection interval, wherein the target model is trained based on the historical number of visits to the cache server and historical anomaly information, and a predetermined detection interval is used to perform anomaly detection on the cache server to obtain anomaly detection results.
[0049] It can be understood that the number of visits to the cache server can be obtained in real time or regularly, and the target model is used to process the current number of visits. The target model is trained based on the historical number of visits to the cache server (i.e., the access data in the past period of time) and the historical abnormal information corresponding to these access numbers (such as the frequency and type of abnormal occurrence). The relationship between the number of visits and the probability of abnormal occurrence can be determined, and then the predetermined detection interval can be assisted to adjust the predetermined detection interval to find the most suitable detection interval under different access numbers. For example, when the number of visits is high, the detection interval may be shortened to respond to potential abnormalities more quickly, and when the number of visits is low, in order to save resources, the system may extend the detection interval. Once the current predetermined detection interval is determined by the target model, the cache server is detected for abnormalities according to the predetermined detection interval. Through the above processing, it is possible to more flexibly respond to changes in the number of visits to the cache server, which not only ensures that abnormalities can be detected in time when the number of visits is high, but also avoids unnecessary frequent detection when the number of visits is low, thereby improving the efficiency and accuracy of abnormality detection.
[0050] Optionally, the access quantity may be the number of access requests per second, per minute, or per hour, depending on the monitoring requirements and performance of the system.
[0051] Step S104, when the anomaly detection result indicates that there is a data interaction anomaly, the number of interaction anomalies is incremented; wherein the number of interaction anomalies is used to record the cumulative number of data interaction anomalies detected in the cache server at a predetermined detection interval;
[0052] It can be understood that when the anomaly detection result indicates that there is a data interaction anomaly, it means that it is now impossible to obtain data from the cache server. When the above anomaly occurs, the number of interaction anomalies is incremented. The above number of interaction anomalies can be obtained by a predetermined counter, which is used to record the cumulative number of data interaction anomalies detected in the cache server using a predetermined detection interval.
[0053] In an optional embodiment, before incrementing the number of interaction exceptions, the method further includes: detecting whether the cache server is turned on, and when the cache server is turned on, acquiring a synchronization lock for the cache server, wherein the synchronization lock is used to limit concurrent access to the cache server.
[0054] It is understandable that if the cache server is in the open state, a synchronization lock (also called a mutex lock) needs to be acquired to limit concurrent access to the cache server and ensure that only one process or thread can modify the state or data of the cache server at the same time, for example, the increment of the number of interaction exceptions. After successfully acquiring the synchronization lock, the number of interaction exceptions can be safely incremented. At this point, due to the limitation of the synchronization lock, any other operation that attempts to modify the state of the cache server must wait until the current operation is completed and the synchronization lock is released before it can be performed.
[0055] Optionally, after the number of interaction anomalies is incremented, it is determined whether the number of query interaction anomalies exceeds the threshold of the number of anomalies. If the number of query interaction anomalies is greater than the first predetermined threshold, it means that the number of interaction anomalies has reached the level that requires a fallback process, the number of query anomalies is reset, the service flow control is enabled, the service alarm window is called, and the synchronization lock is released. By obtaining the synchronization lock in advance, it is possible to ensure data consistency and system stability when operating the cache server (such as incrementing), while avoiding data errors and performance issues caused by concurrent access.
[0056] It should be noted that the use of synchronization locks is an important concept in concurrent programming, which helps to avoid race conditions and data inconsistency. Without synchronization locks, if multiple processes or threads try to update the number of interaction exceptions at almost the same time, the final result may be uncertain, resulting in data errors.
[0057] Step S106, when the incremented abnormal interaction times reaches a first predetermined threshold, the abnormal interaction times are reset and the cache server is suspended;
[0058] It is understood that if the number of interaction anomalies after increment reaches the first predetermined threshold (such as 3 times, 5 times, etc.), further processing measures are triggered. The number of interaction anomalies is reset (cleared or set to the initial value) to avoid the subsequent possible accumulation of anomalies affecting the judgment. The work of the cache server is suspended to avoid more serious problems or data inconsistencies that may be caused by continued anomalies.
[0059] Step S108, performing flow limiting processing on the service, wherein the service interacts with the database corresponding to the cache server when the cache server is suspended, and interacts with the cache server when the cache server is enabled, and the cache server and the corresponding database are synchronized;
[0060] It is understandable that during the suspension of the cache server, the business needs to be throttled. The data interaction requests processed by the cache server will now be directly transferred to the database corresponding to the cache server. Since the performance of the database is usually lower than that of the cache server, throttling helps protect the database from being overwhelmed by the sudden increase in load.
[0061] In an optional embodiment, the flow limiting processing is performed on the service, including: in the case of multiple services, determining the service types corresponding to the multiple services respectively; and performing flow limiting processing on the services whose service types match the target type among the multiple services.
[0062] It is understandable that in scenarios where there are multiple services, the specific type of each service will be identified and determined, and based on these service types, it is decided whether to perform flow control on specific services. Through the above processing, it is possible to perform flow control on multiple services more flexibly and accurately, thereby protecting the stability and availability of the cache server and improving the service support capabilities for core and important services.
[0063] Optionally, there can be multiple business types, such as user registration, data query, order processing, advertising, etc. When an abnormality occurs in the cache server and current limiting is required, the importance of the corresponding business can be determined according to the business type, such as giving priority to user registration, data query, order processing and other businesses, while limiting the business of advertising. The business type for current limiting can be determined based on factors such as the importance of the business, the load capacity of the cache server, and the resource allocation strategy. The above target type can be a single business type or a collection of multiple business types.
[0064] Optionally, there are multiple ways to perform flow control for services whose service types match the target types, such as limiting the request rate, such as only allowing a certain number of requests to pass through at a predetermined time interval, limiting the number of concurrent requests, such as only allowing a certain number of requests to be processed at the same time, and dynamically adjusting resource allocation (dynamically adjusting the resources allocated to each service according to the system load).
[0065] Optionally, log information of the current limiting process is recorded, for example, including the business being limited, the time of the current limiting, the parameters of the current limiting, etc.
[0066] Step S110 , when it is detected that there is no abnormality in the cache server at a predetermined detection interval, the cache server is started.
[0067] It is understandable that the cache server is detected at a predetermined detection interval, and once it is detected that the cache server can normally obtain data (i.e., restore normal interaction capability), the cache server is restarted. After the cache server is restarted, the business will resume normal interaction with the cache server to improve the response speed and throughput of the system.
[0068] Through the above step S102, a predetermined detection interval is used to perform an abnormality detection on the cache server to obtain an abnormality detection result; step S104, when the abnormality detection result indicates that there is a data interaction abnormality, the number of interaction abnormalities is incremented; wherein the number of interaction abnormalities is used to record the cumulative number of data interaction abnormalities detected in the cache server at the predetermined detection interval; step S106, when the number of interaction abnormalities after increment reaches the first predetermined threshold, the number of interaction abnormalities is reset and the cache server is suspended; step S108, flow limiting processing is performed on the service, wherein the service interacts with the database corresponding to the cache server when the cache server is suspended, and the service interacts with the cache server when the cache server is turned on, and the cache server and the corresponding database are synchronized for data; step S110, until the predetermined detection interval is used to detect that there is no abnormality in the cache server, the cache server is turned on. The purpose of detecting the abnormality of the cache server can be achieved, and the technical effect of avoiding the abnormality of the cache server and causing the service crash is achieved, thereby solving the technical problem of the abnormality of the cache server in the related technology and causing the service stability to be unsatisfactory.
[0069] Based on the above embodiments and optional embodiments, the present invention proposes an optional implementation mode: Figure 2 is a first flow chart of an optional cache anomaly detection method provided according to an embodiment of the present invention, such as Figure 2As shown, a scheduled task is configured in the application. The scheduled task includes the task name, execution frequency, Redis constant class, etc., and a Redis heartbeat check is performed on the cache database (i.e. Redis) every second. The scheduled task is configured through SpringBoot's @Scheduled(cron="0 / 1****?"") annotation. The cron expression indicates that the execution frequency of the scheduled task is 1 time / s. The Redis constant class configures the heartbeat detection constant key: REDIS_HEALTH_CHECK, the Redis constant class configures the detection threshold REDIS_FAIL_THRESHOLD=3, that is, the first predetermined threshold is set to 3 times, and the Redis switch RedisSwitch=OPEN (caching is enabled by default).
[0070] The heartbeat detection key (i.e., the predetermined key) is used to query whether the data interaction of Redis is normal. If an abnormality is detected in the data interaction and the Redis service is enabled, the synchronization lock is acquired and the number of abnormalities in the query interaction is recorded. The number of abnormal interactions is incremented each time an abnormality occurs, such as by 1.
[0071] Determine whether the number of interaction anomalies exceeds the first predetermined threshold. If the number of interaction anomalies found is greater than the first predetermined threshold, it means that the number of interaction anomalies has reached the level that requires a backup process. Execute a series of processes such as resetting the number of query anomalies, turning on business flow limiting, calling the business alarm window, and releasing the synchronization lock.
[0072] If the number of abnormal interactions is less than or equal to the first predetermined threshold, it means that the number of abnormal interactions has not reached the level that requires a fallback process, and no process is performed at this time.
[0073] If data interaction anomalies are detected and the Redis service is closed, it means that the Redis cache server has been shut down and no action is required.
[0074] If it is detected that there is no abnormality in data interaction and the Redis service is in a closed state, the number of query exceptions is reset, and the Redis service is updated to be set to an open state, the business current limiting is closed, and the synchronization lock is released.
[0075] If no abnormality is detected in data interaction and the Redis service is enabled, no action is required.
[0076] Figure 3 is a second flow chart of an optional cache anomaly detection method provided according to an embodiment of the present invention, such as Figure 3As shown in the figure, during the business processing, it is necessary to interact with the cache server to achieve high concurrency. First, determine the status of the Redis service. If the Redis service is enabled, pull data from the Redis cache. If the query result is not empty, perform business logic processing and return the business processing result. If the Redis service is disabled, or the result of pulling data from the Redis cache is empty, pull data directly from the database, perform business logic processing, and return the business processing result.
[0077] The above optional implementation method achieves the following effects: timely warning can be achieved in conjunction with the alarm mechanism, and the current limiting mechanism can be used as a disaster recovery solution for cache server (Redis) failure, which can effectively reduce the network load of the computer room, reduce invalid traffic, improve system performance and efficiency, and automatically monitor and process cache servers, which can quickly respond to and handle faults and restore to normal in time, avoiding long-term service interruptions, and improving the high availability and robustness of data services.
[0078] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0079] In this embodiment, a cache anomaly detection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module" and "device" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0080] According to an embodiment of the present invention, there is also provided an apparatus embodiment for implementing a cache anomaly detection method. Figure 4 is a schematic diagram of a cache anomaly detection device according to an embodiment of the present invention. Figure 4 As shown, the cache anomaly detection device includes: a detection module 402, a times updating module 404, a reset module 406, a current limiting module 408, and a recovery module 410. The device is described below.
[0081] The detection module 402 is used to perform anomaly detection on the cache server at a predetermined detection interval to obtain anomaly detection results;
[0082] The number updating module 404 is connected to the detection module 402 and is used to increase the number of abnormal interaction times when the abnormal detection result indicates that there is abnormal data interaction; wherein the number of abnormal interaction times is used to record the cumulative number of abnormal data interaction times detected in the cache server at a predetermined detection interval;
[0083] The reset module 406 is connected to the number update module 404 and is used to reset the number of abnormal interactions and suspend the cache server when the number of abnormal interactions after increment reaches a first predetermined threshold;
[0084] The current limiting module 408 is connected to the reset module 406 and is used to perform current limiting processing on the service, wherein the service interacts with the database corresponding to the cache server when the cache server is suspended, and interacts with the cache server when the cache server is enabled, and the cache server and the corresponding database are synchronized for data;
[0085] The recovery module 410 is connected to the current limiting module 408 and is used to enable the cache server until a predetermined detection interval is used and it is detected that there is no abnormality in the cache server.
[0086] In a cache anomaly detection device provided by an embodiment of the present invention, a detection module 402 is set to perform anomaly detection on a cache server at a predetermined detection interval to obtain an anomaly detection result; a number update module 404 is connected to the detection module 402 and is used to increase the number of interaction anomalies when the anomaly detection result indicates that there is a data interaction anomaly; wherein the number of interaction anomalies is used to record the cumulative number of data interaction anomalies detected in the cache server at a predetermined detection interval; a reset module 406 is connected to the number update module 404 and is used to reset the number of interaction anomalies and suspend the cache server when the number of interaction anomalies after the increase reaches a first predetermined threshold; a current limiting module 408 is connected to the reset module 406 and is used to perform current limiting processing on the service, wherein the service interacts with a database corresponding to the cache server when the cache server is suspended, and the service interacts with the cache server when the cache server is turned on, and the cache server and the corresponding database are synchronized; a recovery module 410 is connected to the current limiting module 408 and is used to turn on the cache server until it is detected that there is no abnormality in the cache server at a predetermined detection interval. The purpose of detecting the abnormality of the cache server is achieved, and the technical effect of avoiding the abnormality of the cache server and causing the service crash is realized, thereby solving the technical problem of the abnormality of the cache server in the related technology and causing the unsatisfactory service stability.
[0087] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.
[0088] It should be noted that the detection module 402, the number update module 404, the reset module 406, the current limiting module 408, and the recovery module 410 correspond to steps S102 to S110 in the embodiment, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the modules as part of the device can be run in a computer terminal.
[0089] It should be noted that the optional or preferred implementation of this embodiment can refer to the relevant description in the embodiment, which will not be repeated here.
[0090] The above-mentioned cache anomaly detection device may also include a processor and a memory, and the detection module 402, the number update module 404, the reset module 406, the current limiting module 408, the recovery module 410, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.
[0091] The processor includes a kernel, which retrieves the corresponding program unit from the memory. There can be one or more kernels. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.
[0092] An embodiment of the present invention provides a non-volatile storage medium on which a program is stored. When the program is executed by a processor, a cache anomaly detection method is implemented.
[0093] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and can be run on the processor. When the processor executes the program, the following steps are implemented: using a predetermined detection interval, performing an abnormality detection on a cache server to obtain an abnormality detection result; when the abnormality detection result indicates that there is a data interaction abnormality, the number of interaction abnormalities is incremented; wherein the number of interaction abnormalities is used to record the cumulative number of data interaction abnormalities detected in the cache server using a predetermined detection interval; when the number of interaction abnormalities after increment reaches a first predetermined threshold, the number of interaction abnormalities is reset and the cache server is suspended; current limiting is performed on the service, wherein when the cache server is suspended, the service interacts with the database corresponding to the cache server, and when the cache server is turned on, the service interacts with the cache server, and the cache server and the corresponding database are synchronized for data; until a predetermined detection interval is used to detect that there is no abnormality in the cache server, the cache server is turned on. The device in this article can be a server, a PC, etc.
[0094] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: using a predetermined detection interval to perform anomaly detection on a cache server to obtain anomaly detection results; when the anomaly detection results indicate that there is a data interaction anomaly, the number of interaction anomalies is incremented; wherein the number of interaction anomalies is used to record the cumulative number of data interaction anomalies detected in the cache server using a predetermined detection interval; when the number of interaction anomalies after increment reaches a first predetermined threshold, the number of interaction anomalies is reset and the cache server is paused; current limiting is performed on the service, wherein when the cache server is paused, the service interacts with a database corresponding to the cache server, and when the cache server is turned on, the service interacts with the cache server, and the cache server and the corresponding database are synchronized for data; until a predetermined detection interval is used to detect that there is no anomaly in the cache server, the cache server is turned on.
[0095] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0096] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0097] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0099] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0100] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0101] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0102] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0103] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0104] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims
1. A cache anomaly detection method, characterized in that: include: Using a predetermined detection interval, performing anomaly detection on the cache server to obtain an anomaly detection result; In the case where the abnormality detection result indicates that there is a data interaction abnormality, the number of interaction abnormalities is incremented; wherein the number of interaction abnormalities is used to record the cumulative number of data interaction abnormalities detected on the cache server using the predetermined detection interval; When the incremented number of abnormal interactions reaches a first predetermined threshold, resetting the number of abnormal interactions and pausing the cache server; Performing flow limiting processing on the service, wherein, when the cache server is suspended, the service interacts with the database corresponding to the cache server, and when the cache server is enabled, the service interacts with the cache server, and the cache server and the corresponding database are in data synchronization; The cache server is started until it is detected that there is no abnormality in the cache server within the predetermined detection interval.
2. The method according to claim 1, characterized in that The method of performing abnormality detection on the cache server at a predetermined detection interval to obtain an abnormality detection result includes: Using the predetermined detection interval, according to the predetermined multiple detection tasks, anomaly detection is performed on the cache server to obtain sub-detection results corresponding to the multiple detection tasks, wherein the multiple detection tasks include at least an interaction detection task, and the interaction detection task is used to detect whether there is data interaction anomaly in the cache server; The abnormality detection result is obtained based on the sub-detection results respectively corresponding to the multiple detection tasks.
3. The method according to claim 2, characterized in that The method of using the predetermined detection interval and performing abnormality detection on the cache server according to the predetermined multiple detection tasks to obtain sub-detection results corresponding to the multiple detection tasks respectively includes: Using the predetermined detection interval, executing acquisition processing of the key value corresponding to the predetermined key included in the cache server, and determining the sub-detection result corresponding to the interaction detection task; In the case where the multiple detection tasks also include at least one of the following: a performance detection task, a memory usage detection task, and a data integrity detection task, the predetermined detection interval is used to determine the response time of the cache server according to a predetermined measurement instruction; based on the response time, a sub-detection result corresponding to the performance detection task is determined; or, Using the predetermined detection interval, based on the memory occupancy status generated by the operation of the cache server, determining the sub-detection result corresponding to the memory usage detection task; or, The predetermined detection interval is used to verify the predetermined data structure set in the cache server to determine the sub-detection result corresponding to the data integrity detection task.
4. The method according to claim 2, characterized in that: The method further comprises: In the case where the abnormality detection result indicates that other tasks have abnormalities, the other abnormality times corresponding to the other tasks are incremented, wherein the other abnormality times are used to record the cumulative number of other abnormalities detected in the cache server at the predetermined detection interval, and the other tasks are the detection tasks other than the interactive detection task among the multiple detection tasks; When the number of other exceptions is greater than a second predetermined threshold, an alarm strategy for the exception handling end is determined based on the priorities corresponding to the other tasks.
5. The method according to claim 1, characterized in that The flow limiting processing of the service includes: In the case where there are multiple types of services, determining the service types corresponding to the multiple types of services respectively; For the services whose service types match the target type among the multiple services, current limiting processing is performed.
6. The method according to claim 1, characterized in that Before the process of incrementing the number of abnormal interactions, the method further includes: Detecting whether the cache server is turned on, and if the cache server is turned on, acquiring a synchronization lock for the cache server, wherein the synchronization lock is used to limit concurrent access to the cache server.
7. The method according to any one of claims 1 to 6, characterized in that: The method of performing abnormality detection on the cache server at a predetermined detection interval to obtain an abnormality detection result includes: Determining the number of visits to the cache server; Based on the number of visits, a target model is used for processing to determine the predetermined detection interval, wherein the target model is trained based on the number of historical visits and historical anomaly information of the cache server, and the predetermined detection interval is used to perform anomaly detection on the cache server to obtain the anomaly detection result.
8. A cache anomaly detection device, characterized in that: include: A detection module, used to perform anomaly detection on the cache server at a predetermined detection interval to obtain anomaly detection results; A times updating module, used for increasing the number of abnormal interactions when the abnormal detection result indicates that there is abnormal data interaction; wherein the abnormal interaction number is used to record the cumulative number of abnormal data interactions detected by the cache server at the predetermined detection interval; A reset module, configured to reset the number of abnormal interactions and suspend the cache server when the number of abnormal interactions reaches a first predetermined threshold after the increment; A current limiting module, used for current limiting processing of a service, wherein when the cache server is suspended, the service interacts with a database corresponding to the cache server, and when the cache server is enabled, the service interacts with the cache server, and data synchronization exists between the cache server and the corresponding database; The recovery module is used to start the cache server until it is detected that there is no abnormality in the cache server within the predetermined detection interval.
9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the cache anomaly detection method according to any one of claims 1 to 7.
10. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the cache anomaly detection method described in any one of claims 1 to 7.
Citation Information
Cited By
Advertisement delivery abnormity monitoring and alarm current limiting system and method
CN120951215A
