A method, device, equipment and storage medium for implementing a retry mechanism
Through the ELK log analysis system and the distributed publishing and subscription message system, collect and store communication log annotations of the distributed system, and configure the retry mechanism of the service governance platform based on preset rules, it solves the inflexible maintenance of retry strategies in distributed systems. Problem, an efficient monitoring and retry mechanism for inter-service calls is realized.
Patent Information
- Application Number
- CN202410446878.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-15
- Publication Date
- 2025-02-07
- Estimated Expiration
- 2044-04-15
AI Technical Summary
In distributed systems, the stateless characteristics of the HTTP protocol and the design limitations of Redis as the registry center make it impossible to directly intercept and retry service calls, resulting in the maintenance and configuration of retry policies being inflexible enough, thereby reducing the monitoring of inter-service calls efficiency.
By using the ELK log analysis system to collect parameters and results of communications of each system, generate log annotations, and store and index these annotations through the distributed publishing and subscription message system and the preset log analysis system. Then, based on preset monitoring and retry rules, the retry mechanism of the service governance platform is configured, and the log scanning component is used to scan log annotations to determine whether the data call request needs to be retryed.
It realizes monitoring and analysis of inter-service calls, simplifies the implementation of the retry mechanism, decouples code references between systems, allows custom retry strategies, reduces code redundancy and system maintenance costs, and improves system reliability. and monitoring efficiency.
Smart Images

Figure CN118606067B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of distributed system applications, and in particular to a method, device, equipment and storage medium for implementing a retry mechanism. Background Art
[0002] In the distributed system architecture environment, the communication between systems usually adopts the HTTP protocol (Hypertext Transfer Protocol), and uses Redis as a component for service registration and discovery. However, this configuration makes it quite complicated to monitor and analyze the calling relationship between services. In order to track the complete path of a service call, engineers must manually retrieve and analyze log files distributed on different servers, which is undoubtedly a time-consuming and inefficient task.
[0003] In addition, the current system's HTTP-based RPC (remote procedure call) communication mechanism does not support custom retry strategies. Taking the call from system A to system B as an example, system A integrates the Redis client to query the name of service B, thereby obtaining the address, port number, and method name of the target service, which are used to construct HTTP requests to implement service calls. However, due to the stateless nature of HTTP communication and the design limitations of Redis as a registration center, it is impossible to directly intercept the request from system A to service B in such an architecture. If the communication times out, system A must implement the retry logic of each RPC call in a hard-coded manner, which not only increases the complexity of coding, but also makes the maintenance and configuration of the retry strategy less flexible. Therefore, how to improve the monitoring efficiency of calls between services is a problem to be solved in this field. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide a retry mechanism implementation method, device, equipment and storage medium, which can conveniently monitor and analyze service call relationships by using the log analysis system ELK to collect the parameters and results of each system communication, and abstract the retry strategy through the data structure, simplify the retry mechanism, and decouple the code references between systems. The specific scheme is as follows:
[0005] In a first aspect, the present application provides a method for implementing a retry mechanism, including:
[0006] Determine a data call format of a data call request when a plurality of systems communicate according to a preset data format of a remote procedure call, and generate a log annotation of the data call request according to the data call format;
[0007] The log annotations are collected by using a distributed publish-subscribe message system and a preset log analysis system, and the collected log annotations are formatted and stored in a database of the preset log analysis system, and an index relationship between the log annotations and the data call request is established;
[0008] Configure a retry mechanism of a preset service governance platform according to a preset monitoring rule data structure and a preset retry rule data structure, and use a log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the retry mechanism;
[0009] Based on the scanning result obtained by scanning the log annotation, it is determined according to the retry mechanism whether to retry the current data call request.
[0010] Optionally, generating a log annotation of the data call request according to the data call format includes:
[0011] Based on the aspect-oriented programming method, a log aspect annotation of the data call request is generated according to the data call format, so as to standardize the log annotation of the data call request.
[0012] Optionally, before collecting the log annotations by using the distributed publish-subscribe message system and the preset log analysis system, the method further includes:
[0013] Determine the service-side system corresponding to the data call request, and add the log section annotation to the system interface of the service-side system, so as to obtain the log section annotation by using the system interface;
[0014] Correspondingly, after collecting the log annotations by using the distributed publish-subscribe message system and the preset log analysis system, the method further includes:
[0015] Use the AspectJ compiler to compile the aspect code corresponding to the log aspect annotation into a bytecode file;
[0016] Run the bytecode file, use a Java dynamic proxy or a CGLIB proxy to generate a proxy class corresponding to the bytecode file, and insert the proxy class into the aspect code to print the log aspect annotation.
[0017] Optionally, configuring a retry mechanism of a preset service governance platform according to a preset monitoring rule data structure and a preset retry rule data structure includes:
[0018] Determine a first data field corresponding to the preset monitoring rule data structure, and determine a second data field corresponding to the preset retry rule data structure, so as to configure the retry mechanism of the preset service governance platform based on the first data field and the second data field;
[0019] Correspondingly, before judging whether to retry the current data call request based on the scan result obtained by scanning the log annotation according to the retry mechanism, the method further includes:
[0020] A third data field corresponding to the record data structure of the retry record is determined to record the scanning result obtained by scanning the log annotation according to the third data field.
[0021] Optionally, the using the log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the retry mechanism includes:
[0022] Constructing a log query condition according to the first data field corresponding to the preset monitoring rule data structure;
[0023] A query time interval for the log annotation is set, and a query time is determined according to the current system timestamp and the query time interval, so as to utilize the log scanning component of the preset service governance platform to scan the log annotation in the database according to the index relationship at the query time based on the log query condition.
[0024] Optionally, judging whether to retry the current data call request based on the scan result obtained by scanning the log annotation according to the retry mechanism includes:
[0025] If it is determined according to the preset monitoring rule data structure in the retry mechanism that the judgment result of the data call request is manual processing, then the preset processing personnel are notified according to the notification method in the preset monitoring rule data structure;
[0026] If it is determined according to the preset monitoring rule data structure in the retry mechanism that the judgment result of the data call request is non-manual processing and non-retry, only the current judgment result of the data call request is recorded;
[0027] If it is determined that the judgment result of the data call request is a retry according to the preset monitoring rule data structure in the retry mechanism, a retry record and a retry task corresponding to the current data call request are created according to the preset retry rule data structure in the retry mechanism, so as to execute the retry task according to the retry rule in the retry record, and update the log annotation corresponding to the data call request in the database after the retry task is completed, and determine whether the retry task is successfully executed according to the preset retry rule data structure. If it is successfully executed within the number of retries in the preset retry rule data structure, stop the retry task, otherwise give an alarm according to the alarm method in the preset monitoring rule data structure.
[0028] Optionally, after executing the retry task according to the retry rule in the retry record, the method further includes:
[0029] Obtain the response code in the preset log analysis system, and count the retry records, actual number of retries and retry task response time corresponding to the data call request based on the response code; the response code is the response code synchronized from the service party system in the several systems to the preset log analysis system.
[0030] In a second aspect, the present application provides a retry mechanism implementation device, including:
[0031] An annotation generation module is used to determine the data call format of the data call request when several systems communicate according to the preset data format of the remote procedure call, and generate a log annotation of the data call request according to the data call format;
[0032] An annotation storage module, used to collect the log annotations by using a distributed publish-subscribe message system and a preset log analysis system, format the collected log annotations, store them in a database of the preset log analysis system, and establish an index relationship between the log annotations and the data call request;
[0033] An annotation scanning module, used to configure a retry mechanism of a preset service governance platform according to a preset monitoring rule data structure and a preset retry rule data structure, and use a log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the retry mechanism;
[0034] A request retry module is used to determine whether to retry the current data call request based on the scanning result obtained by scanning the log annotation according to the retry mechanism.
[0035] In a third aspect, the present application provides an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned retry mechanism implementation method.
[0036] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, which implements the aforementioned retry mechanism implementation method when executed by a processor.
[0037] In the present application, firstly, the data call format of the data call request when several systems communicate is determined according to the preset data format of the remote procedure call, and the log annotation of the data call request is generated according to the data call format, and then the distributed publish-subscribe message system and the preset log analysis system are used to collect the log annotation, and the collected log annotation is formatted and stored in the database of the preset log analysis system, and an index relationship between the log annotation and the data call request is established, and the retry mechanism of the preset service governance platform is configured according to the preset monitoring rule data structure and the preset retry rule data structure, and the log scanning component of the preset service governance platform is used to scan the log annotation in the database according to the index relationship based on the retry mechanism, so as to judge whether to retry the current data call request according to the retry mechanism based on the scanning result obtained by scanning the log annotation. In this way, by using ELK to centrally collect the parameters and results of each system communication, this application can easily monitor and analyze the call relationship between services, and define the retry strategy through data structure abstraction, simplify the implementation of the retry mechanism, and scan the data extracted from ELK. When the data that needs to be retried is matched, a retry request is initiated to the service party according to the configured retry rules. In this way, the code references between systems can be decoupled, and the retry strategy can be customized as needed, reducing code redundancy and system maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0039] Figure 1 A flow chart of a method for implementing a retry mechanism provided in this application;
[0040] Figure 2 A system topology diagram corresponding to a service governance platform provided for this application;
[0041] Figure 3A system data interaction flow chart provided for this application;
[0042] Figure 4 A flow chart of a specific retry mechanism implementation method provided in this application;
[0043] Figure 5 A schematic diagram of the structure of a retry mechanism implementation device provided in this application;
[0044] Figure 6 A structural diagram of an electronic device provided for this application. DETAILED DESCRIPTION
[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0046] The HTTP-based RPC communication mechanism in the current distributed system architecture environment does not support custom retry strategies. If a communication timeout occurs, the system must implement the retry logic for each RPC call in a hard-coded manner, which not only increases the complexity of coding, but also makes the maintenance and configuration of the retry strategy less flexible. In this application, by using ELK to centrally collect the parameters and results of each system communication, the call relationship between services can be easily monitored and analyzed, and the retry strategy can be defined through data structure abstraction, and the implementation of the retry mechanism can be simplified to scan the data extracted from ELK. The code references between systems can be decoupled, and the retry strategy can be customized as needed, reducing code redundancy and system maintenance costs.
[0047] See also Figure 1 As shown, an embodiment of the present invention discloses a method for implementing a retry mechanism, including:
[0048] Step S11, determining a data call format of a data call request when a plurality of systems communicate according to a preset data format of a remote procedure call, and generating a log annotation of the data call request according to the data call format.
[0049] It should be noted that the system architecture used to implement the retry mechanism in this application is as follows Figure 2As shown in the system topology diagram, it specifically includes external network module, network communication module, server module and data service module. The server module is connected with the ELK (Elasticsearch, Logstash and Kibana) log service module to retry through the communication logs of different systems through its own service governance platform combined with the ELK log service, and the generated data is stored in the relevant cluster of the data service module. The above system runs on the server. Because it involves the three major modules of retry, monitoring and statistics, it is named the service governance platform. The core function of retry is mainly implemented in the service governance platform.
[0050] In this embodiment, the system request is first standardized and defined, and the data call format of the data call request when several systems communicate is determined through the preset data format of the remote procedure call (RPC) request. For details, please refer to Figure 3 The RPC request data structure shown in the figure ensures the uniqueness of link tracking through a unified data format. Then, a log annotation of the data call request is generated according to the above data call format. In this embodiment, the RPC log aspect annotation of the data call request is generated according to the data call format in a manner of annotation based on aspect-oriented programming (AOP) to standardize the log annotation record of the data call request during system operation. It can be understood that the above data call request is only a communication request indicated in this embodiment, and other interactions between systems can also be used in this embodiment.
[0051] Step S12: using a distributed publish-subscribe message system and a preset log analysis system to collect the log annotations, formatting the collected log annotations, storing them in a database of the preset log analysis system, and establishing an index relationship between the log annotations and the data call request.
[0052] In this embodiment, a distributed publish-subscribe messaging system (Kafka) and a preset log analysis system (i.e., the above-mentioned ELK) can be used to collect log annotations, and the collected log annotations are formatted and stored in the database of the preset log analysis system, and an index relationship between log annotations and data call requests is established. In this embodiment, ELK and Kafka are connected to collect and index log data, so that Logstash can be used to collect logs, and the logs are formatted and output to the Elasticsearch cluster, so that Elasticsearch can be used to index and store the formatted data. In this way, in view of the efficiency and maintainability problems existing in the current distributed system communication mode, the introduction of a centralized log analysis tool can improve the monitoring efficiency of calls between services, so as to realize cross-machine log aggregation and search functions, thereby greatly simplifying the workflow of engineers and being able to track and analyze service calls more quickly and accurately.
[0053] And according to the log aspect annotation in the previous step, before collecting the log annotation in this embodiment, the service system corresponding to the data call request can be determined, and the log aspect annotation can be added to the system interface of the service system to obtain the log aspect annotation using the system interface. By adding aspect annotations to the service interface referenced in the system, it is helpful to track and manage the logs later. In this way, after collecting the log annotations, the aspect code corresponding to the log aspect annotation can be compiled into a bytecode file using the AspectJ compiler, and when running the bytecode file, the proxy class corresponding to the bytecode file can be generated using the Java dynamic proxy or CGLIB proxy, and the proxy class can be inserted into the aspect code to print the log aspect annotation. The above-mentioned process of inserting the aspect code can be inserted before and after the execution of the target object method of the above-mentioned data call request to realize the function of AOP log printing. By uniformly defining the log printing information of the system RPC (HTTP) communication and using ELK (Elastic Stack) to centrally collect the parameters and results of each system communication, the call relationship between services can be easily monitored and analyzed.
[0054] Step S13: configure the retry mechanism of the preset service governance platform according to the preset monitoring rule data structure and the preset retry rule data structure, and use the log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the retry mechanism.
[0055] In this embodiment, Figure 3 The monitoring rules and retry rules in the service governance platform shown above can be Figure 3The preset monitoring rule data structure and the preset retry rule data structure are used to configure the retry mechanism of the service governance platform, and the log scanning component of the preset service governance platform is used to scan the log annotations in the database based on the retry mechanism according to the previously established index relationship.
[0056] Step S14: Based on the scan result obtained by scanning the log annotation, determine whether to retry the current data call request according to the retry mechanism.
[0057] In this embodiment, the scanning results obtained based on the scanning log annotations can determine whether to retry the current data call request according to the retry mechanism, and count the data generated during the retry process. The results after platform statistics can present the health status of each system and the quality data of the services provided in the system, which is convenient for system analysis and optimization.
[0058] In this embodiment, the system request is firstly standardized and defined, and the data call format of the data call request when several systems communicate is determined by the preset data format of the remote procedure call request, and then the RPC log aspect annotation of the data call request is generated according to the data call format based on aspect-oriented programming, and the log annotation is collected by Kafka and ELK, and the collected log annotation is formatted and stored in the preset database to establish the index relationship between the log annotation and the data call request, and then the retry mechanism of the service governance platform is configured according to the preset monitoring rule data structure and the preset retry rule data structure, and the log scanning component of the preset service governance platform is used to scan the log annotation in the database based on the retry mechanism according to the previously established index relationship, so that the scanning result obtained based on the scanning log annotation can be used to determine whether to retry the current data call request according to the retry mechanism. In this way, a centralized log analysis tool is introduced in this embodiment, and log data is centrally collected and analyzed by ELK, which can quickly locate problems and service call situations, realize cross-machine log aggregation and search functions, greatly simplify the workflow of engineers, enable engineers to track and analyze service calls more quickly and accurately, and improve the monitoring efficiency of service calls.
[0059] Based on the previous embodiment, it can be seen that this application can use ELK to centrally collect the parameters and results of each system communication, and define the retry strategy through data structure abstraction. Next, this embodiment will describe in detail the process of retrying based on the retry mechanism. Figure 4 As shown, the embodiment of the present application discloses a specific method for implementing a retry mechanism, including:
[0060] Step S21: configure the retry mechanism of the preset service governance platform according to the preset monitoring rule data structure and the preset retry rule data structure.
[0061] In this embodiment, when configuring the retry mechanism of the preset service governance platform according to the preset monitoring rule data structure and the preset retry rule data structure, the first data field corresponding to the preset monitoring rule data structure can be determined first, and the second data field corresponding to the preset retry rule data structure can be determined, so as to configure the retry mechanism of the preset service governance platform based on the first data field and the second data field. For example, when defining the monitoring rule data structure, the fields include the monitoring system, method, whether to retry after triggering, warning method (SMS, email), whether to handle manually, etc.; when defining the retry rules, the fields include the number of retries, the retry frequency (i.e., how many minutes the retry is performed), and the result matching rules after the retry; in addition, in addition to the above-mentioned monitoring rules and retry rules, such as Figure 3 As shown, the record retry record can also be defined, and the fields include data compensation method, request address, request parameters, service provider, caller, response result, trigger time, number of retries, task ID (link ID), etc. It can be understood that the above rules can be added or deleted according to actual needs. In this way, by abstracting the retry mechanism into a data structure, hard-coded logic can be reduced, the maintainability of the code can be improved, and the retry strategy can be customized to automatically retry when the network is unstable or the service fails, thereby improving the reliability of the system.
[0062] Step S22: Utilize the log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the retry mechanism.
[0063] In this embodiment, when the log scanning component is used to scan the log annotations in the database according to the index relationship based on the retry mechanism, the log query conditions are first constructed according to the first data field corresponding to the preset monitoring rule data structure, and then the query time interval of the log annotations is set, and the query time is determined according to the current system timestamp and the query time interval, so that the log scanning component of the preset service governance platform is used to scan the log annotations in the database according to the index relationship based on the log query conditions at the query time. In this way, in this embodiment, the Elasticsearch data structure can be polled regularly, for example, according to the current system timestamp, the query is queried once every minute with the defined monitoring rules, and the query conditions can be set to the monitoring system and method. The above query conditions can also be changed according to actual needs based on the preset fields in the above steps.
[0064] Step S23: Based on the scan result obtained by scanning the log annotation, determine whether to retry the current data call request according to the retry mechanism.
[0065] According to the above steps in this embodiment, before determining whether to retry the current data call request based on the scan result, the third data field corresponding to the record data structure of the retry record can also be determined to record the scan result obtained by the scan log annotation according to the third data field. Then, when judging whether to retry the current data call request, if the judgment result of the data call request is determined to be manual processing according to the preset monitoring rule data structure in the retry mechanism, the preset processing personnel are notified according to the notification method in the preset monitoring rule data structure; if the judgment result of the data call request is determined to be non-manual processing and non-retry according to the preset monitoring rule data structure in the retry mechanism, only the current judgment result of the data call request is recorded; if the judgment result of the data call request is determined to be a retry according to the preset monitoring rule data structure in the retry mechanism, a retry record and a retry task corresponding to the current data call request are created according to the preset retry rule data structure in the retry mechanism, so as to execute the retry task according to the retry rule in the retry record, and update the log annotation corresponding to the data call request in the database after the retry task is executed, and judge whether the retry task is successfully executed according to the preset retry rule data structure. If it is successfully executed within the number of retries in the preset retry rule data structure, the retry task is stopped, otherwise an alarm is issued according to the alarm method in the preset monitoring rule data structure.
[0066] That is to say, after this embodiment retrieves the corresponding data in the database according to the query conditions, it can make a judgment according to the monitoring rules. In a specific embodiment, if the rules configure manual processing, a text message notification will be sent. If the automatic retry is configured to be no and it is not manual processing, only data recording (i.e., the retry record is recorded) is performed for statistical usage.
[0067] In another specific embodiment, after retrieving the data, if the field of whether to retry after triggering in the rule is "retry after triggering", a retry record is created according to the retry rule and a corresponding scheduled task is created at the same time, where one link ID corresponds to one task, and the request address, request system, request method, request parameters, maximum number of retries, number of retries, retry return parameters, etc. are recorded. Then the scheduled task attempts the http request according to the request address, request parameters, service system, and service method in the retry record, and updates the number of retries in the data table for each attempt and records the retry return result. If the result returned by the retry is consistent with the result rule in the rule definition, the retry is stopped and the corresponding result is returned. If the maximum number of retries still fails, a text message notification is sent according to the alarm method configured in the rule. In this way, through the configurable retry strategy, developers are allowed to customize the number of retries, intervals, and other related parameters as needed, thereby providing a more flexible and reliable service-to-service calling experience, simplifying the implementation of the retry mechanism and improving the robustness of the system.
[0068] In this embodiment, after executing the retry task according to the retry rules in the retry record, the response code in the preset log analysis system can also be obtained, and the relevant parameters of the retry task such as the retry record corresponding to the request, the actual number of retries and the retry task response time can be called according to the above response code statistical data; wherein, the above response code is the data structure system response code synchronized from the service party system of several systems in communication to the Elasticsearch in the preset log analysis system. By querying and counting the abnormal statistics, retry count statistics, and average response time of each system method of each system, the health status of each system and the quality data of the services provided in the system can be presented, which is convenient for system analysis and optimization.
[0069] In this embodiment, the first data field corresponding to the preset monitoring rule data structure is first determined, and the second data field corresponding to the preset retry rule data structure is determined, so as to configure the retry mechanism of the preset service governance platform based on the first data field and the second data field, and then the log query condition is constructed according to the first data field corresponding to the preset monitoring rule data structure, the query time interval of the log annotation is set, and the query time is determined according to the current system timestamp and the query time interval, so as to use the log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the log query condition at the query time, so as to determine whether to retry the current data call request according to the retry mechanism based on the scanning result obtained by scanning the log annotation, and execute the corresponding retry task according to the retry mechanism. Through the above technical scheme, an abstract retry strategy configuration is realized, which can reduce hard-coded logic and improve the maintainability of the code. Through the customized retry strategy, it can automatically retry when the network is unstable or the service fails, improve the reliability of the system, and reduce the system code redundancy and maintenance cost by decoupling the code references between systems. In this way, when the data that needs to be retried is matched, a retry request is initiated to the service provider according to the configured different retry rules, which can decouple the code references between systems, improve the performance and reliability of the distributed system, reduce maintenance costs and complexity, and achieve a more flexible and efficient service-to-service calling experience.
[0070] See also Figure 5 As shown, the embodiment of the present application also discloses a retry mechanism implementation device, including:
[0071] An annotation generation module 11 is used to determine the data call format of the data call request when several systems communicate according to the preset data format of the remote procedure call, and generate a log annotation of the data call request according to the data call format;
[0072] An annotation storage module 12 is used to collect the log annotations by using a distributed publish-subscribe message system and a preset log analysis system, format the collected log annotations, store them in a database of the preset log analysis system, and establish an index relationship between the log annotations and the data call request;
[0073] An annotation scanning module 13, used to configure a retry mechanism of a preset service governance platform according to a preset monitoring rule data structure and a preset retry rule data structure, and use a log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the retry mechanism;
[0074] The request retry module 14 is used to determine whether to retry the current data call request based on the scanning result obtained by scanning the log annotation according to the retry mechanism.
[0075] This embodiment first determines the data call format of data call requests when several systems communicate according to the preset data format of remote procedure calls, and generates log annotations for data call requests according to the data call format, and then uses a distributed publish-subscribe message system and a preset log analysis system to collect log annotations, and stores the collected log annotations in the database of the preset log analysis system after formatting, and establishes an index relationship between log annotations and data call requests, and configures a retry mechanism of a preset service governance platform according to a preset monitoring rule data structure and a preset retry rule data structure, and uses a log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the retry mechanism, so as to determine whether to retry the current data call request according to the retry mechanism based on the scanning results obtained by scanning the log annotations. Through the above technical solution, ELK can be used to centrally collect the parameters and results of communication between each system, so that the calling relationship between services can be easily monitored and analyzed. The retry strategy can be defined through data structure abstraction to simplify the implementation of the retry mechanism. The data extracted from ELK can be scanned. When the data that needs to be retried is matched, a retry request is initiated to the service provider according to the configured retry rules. In this way, the code references between systems can be decoupled, and the retry strategy can be customized as needed, reducing code redundancy and system maintenance costs.
[0076] In some specific embodiments, the annotation generation module 11 specifically includes:
[0077] An annotation generating unit is used to generate a log aspect annotation of the data call request according to the data call format based on aspect-oriented programming to standardize the log annotation of the data call request.
[0078] In some specific embodiments, the annotation generating module 11 further includes:
[0079] An annotation adding unit, used for determining the service-side system corresponding to the data call request, and adding the log section annotation to the system interface of the service-side system, so as to obtain the log section annotation by using the system interface;
[0080] Correspondingly, the annotation storage module 12 further includes:
[0081] An annotation compilation unit, used to compile the aspect code corresponding to the log aspect annotation into a bytecode file using the AspectJ compiler;
[0082] An annotation printing unit is used to run the bytecode file, use a Java dynamic proxy or a CGLIB proxy to generate a proxy class corresponding to the bytecode file, and insert the proxy class into the aspect code to print the log aspect annotation.
[0083] In some specific embodiments, the annotation scanning module 13 specifically includes:
[0084] A mechanism configuration unit, configured to determine a first data field corresponding to the preset monitoring rule data structure, and to determine a second data field corresponding to the preset retry rule data structure, so as to configure the retry mechanism of the preset service governance platform based on the first data field and the second data field;
[0085] Correspondingly, the request retry module 14 further includes:
[0086] The field determination unit is used to determine a third data field corresponding to the record data structure of the retry record, so as to record the scanning result obtained by scanning the log annotation according to the third data field.
[0087] In some specific embodiments, the annotation scanning module 13 specifically includes:
[0088] A condition building unit, used to build a log query condition according to the first data field corresponding to the preset monitoring rule data structure;
[0089] An annotation scanning unit is used to set a query time interval for the log annotations, and determine the query time according to the current system timestamp and the query time interval, so as to scan the log annotations in the database according to the index relationship at the query time based on the log query conditions and using the log scanning component of the preset service governance platform.
[0090] In some specific embodiments, the request retry module 14 specifically includes:
[0091] A result notification unit, configured to notify a preset processing personnel according to a notification method in the preset monitoring rule data structure if the judgment result of the data call request is determined to be manual processing according to the preset monitoring rule data structure in the retry mechanism;
[0092] A result recording unit, configured to record only the current judgment result of the data call request if it is determined that the judgment result of the data call request is non-manual processing and non-retry according to the preset monitoring rule data structure in the retry mechanism;
[0093] A task execution unit is used to create a retry record and a retry task corresponding to the current data call request according to the preset retry rule data structure in the retry mechanism if the judgment result of the data call request is determined to be a retry according to the preset monitoring rule data structure in the retry mechanism, so as to execute the retry task according to the retry rule in the retry record, and update the log annotation corresponding to the data call request in the database after the execution of the retry task is completed, and determine whether the retry task is successfully executed according to the preset retry rule data structure; if the retry task is successfully executed within the number of retries in the preset retry rule data structure, stop the retry task, otherwise issue an alarm according to the alarm method in the preset monitoring rule data structure.
[0094] In some specific embodiments, the request retry module 14 further includes:
[0095] A data statistics unit is used to obtain the response code in the preset log analysis system, and to count the retry records, actual number of retries and retry task response time corresponding to the data call request according to the response code; the response code is the response code synchronized from the service party system of the several systems to the preset log analysis system.
[0096] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.
[0097] Figure 6 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the retry mechanism implementation method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0098] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0099] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0100] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the retry mechanism implementation method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.
[0101] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed retry mechanism implementation method is implemented. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiment, and will not be repeated here.
[0102] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0103] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0104] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0105] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0106] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A method for implementing a retry mechanism, characterized in that: include: Determine a data call format of a data call request when a plurality of systems communicate according to a preset data format of a remote procedure call, and generate a log annotation of the data call request according to the data call format; The log annotations are collected by using a distributed publish-subscribe message system and a preset log analysis system, and the collected log annotations are formatted and stored in a database of the preset log analysis system, and an index relationship between the log annotations and the data call request is established; Configure a retry mechanism of a preset service governance platform according to a preset monitoring rule data structure and a preset retry rule data structure, and use a log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the retry mechanism; Based on the scanning result obtained by scanning the log annotation, judging whether to retry the current data call request according to the retry mechanism; The configuration of the retry mechanism of the preset service governance platform according to the preset monitoring rule data structure and the preset retry rule data structure includes: Determine a first data field corresponding to the preset monitoring rule data structure, and determine a second data field corresponding to the preset retry rule data structure, so as to configure the retry mechanism of the preset service governance platform based on the first data field and the second data field; Correspondingly, before judging whether to retry the current data call request based on the scan result obtained by scanning the log annotation according to the retry mechanism, the method further includes: Determine a third data field corresponding to the record data structure of the retry record, so as to record the scanning result obtained by scanning the log annotation according to the third data field; Furthermore, the log scanning component using the preset service governance platform scans the log annotations in the database according to the index relationship based on the retry mechanism, including: Constructing a log query condition according to the first data field corresponding to the preset monitoring rule data structure; Setting a query time interval for the log annotation, and determining a query time according to a current system timestamp and the query time interval, so as to scan the log annotation in the database according to the index relationship using the log scanning component of the preset service governance platform based on the log query condition at the query time; And, judging whether to retry the current data call request based on the scanning result obtained by scanning the log annotation according to the retry mechanism includes: If it is determined according to the preset monitoring rule data structure in the retry mechanism that the judgment result of the data call request is manual processing, then the preset processing personnel are notified according to the notification method in the preset monitoring rule data structure; If it is determined according to the preset monitoring rule data structure in the retry mechanism that the judgment result of the data call request is non-manual processing and non-retry, only the current judgment result of the data call request is recorded; If it is determined that the judgment result of the data call request is a retry according to the preset monitoring rule data structure in the retry mechanism, a retry record and a retry task corresponding to the current data call request are created according to the preset retry rule data structure in the retry mechanism, so as to execute the retry task according to the retry rule in the retry record, and update the log annotation corresponding to the data call request in the database after the retry task is completed, and determine whether the retry task is successfully executed according to the preset retry rule data structure. If it is successfully executed within the number of retries in the preset retry rule data structure, stop the retry task, otherwise give an alarm according to the alarm method in the preset monitoring rule data structure.
2. The method for implementing the retry mechanism according to claim 1, characterized in that: Generating a log annotation of the data call request according to the data call format includes: Based on the aspect-oriented programming method, a log aspect annotation of the data call request is generated according to the data call format, so as to standardize the log annotation of the data call request.
3. The method for implementing the retry mechanism according to claim 2, characterized in that: Before collecting the log annotations by using the distributed publish-subscribe message system and the preset log analysis system, the method further includes: Determine the service-side system corresponding to the data call request, and add the log section annotation to the system interface of the service-side system, so as to obtain the log section annotation by using the system interface; Correspondingly, after collecting the log annotations by using the distributed publish-subscribe message system and the preset log analysis system, the method further includes: Use the AspectJ compiler to compile the aspect code corresponding to the log aspect annotation into a bytecode file; Run the bytecode file, use a Java dynamic proxy or a CGLIB proxy to generate a proxy class corresponding to the bytecode file, and insert the proxy class into the aspect code to print the log aspect annotation.
4. The method for implementing the retry mechanism according to claim 1, characterized in that: After executing the retry task according to the retry rule in the retry record, the method further includes: Obtain the response code in the preset log analysis system, and count the retry records, actual number of retries and retry task response time corresponding to the data call request based on the response code; the response code is the response code synchronized from the service party system in the several systems to the preset log analysis system.
5. A retry mechanism implementation device, characterized in that: include: An annotation generation module is used to determine the data call format of the data call request when several systems communicate according to the preset data format of the remote procedure call, and generate a log annotation of the data call request according to the data call format; An annotation storage module, used to collect the log annotations by using a distributed publish-subscribe message system and a preset log analysis system, format the collected log annotations, store them in a database of the preset log analysis system, and establish an index relationship between the log annotations and the data call request; An annotation scanning module, used to configure a retry mechanism of a preset service governance platform according to a preset monitoring rule data structure and a preset retry rule data structure, and use a log scanning component of the preset service governance platform to scan the log annotations in the database according to the index relationship based on the retry mechanism; A request retry module, used to determine whether to retry the current data call request according to the retry mechanism based on the scan result obtained by scanning the log annotation; Wherein, the annotation scanning module specifically includes: A mechanism configuration unit, configured to determine a first data field corresponding to the preset monitoring rule data structure, and to determine a second data field corresponding to the preset retry rule data structure, so as to configure the retry mechanism of the preset service governance platform based on the first data field and the second data field; Correspondingly, the request retry module further includes: a field determination unit, configured to determine a third data field corresponding to a record data structure of a retry record, so as to record the scanning result obtained by scanning the log annotation according to the third data field; Furthermore, the annotation scanning module specifically includes: A condition building unit, used to build a log query condition according to the first data field corresponding to the preset monitoring rule data structure; An annotation scanning unit, used to set a query time interval for the log annotation, and determine a query time according to a current system timestamp and the query time interval, so as to scan the log annotation in the database according to the index relationship using the log scanning component of the preset service governance platform based on the log query condition at the query time; And, the request retry module specifically includes: A result notification unit, configured to notify a preset processing personnel according to a notification method in the preset monitoring rule data structure if the judgment result of the data call request is determined to be manual processing according to the preset monitoring rule data structure in the retry mechanism; A result recording unit, configured to record only the current judgment result of the data call request if it is determined that the judgment result of the data call request is non-manual processing and non-retry according to the preset monitoring rule data structure in the retry mechanism; A task execution unit is used to create a retry record and a retry task corresponding to the current data call request according to the preset retry rule data structure in the retry mechanism if the judgment result of the data call request is determined to be a retry according to the preset monitoring rule data structure in the retry mechanism, so as to execute the retry task according to the retry rule in the retry record, and update the log annotation corresponding to the data call request in the database after the execution of the retry task is completed, and determine whether the retry task is successfully executed according to the preset retry rule data structure; if the retry task is successfully executed within the number of retries in the preset retry rule data structure, stop the retry task, otherwise issue an alarm according to the alarm method in the preset monitoring rule data structure.
6. An electronic device, characterized in that: The electronic device comprises a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the retry mechanism implementation method as claimed in any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the retry mechanism implementation method as described in any one of claims 1 to 4.
Citation Information
Patent Citations
Service gateway and log record retrieval device based on Elasticsearch
CN108712329A
A highly reliable real-time log collection and visual retrieval method
CN109542733A