API (Application Program Interface) high-availability configuration method for flexibly managing flow

CN119052071APending Publication Date: 2024-11-29QMAI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411074141.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

[0004](1)研发效率低下:需要投入较多的后端研发人力进行接口开发,同时需要多工种联合进行协作进行研发提供数据接口;

Benefits of technology

[0037] (1) Through traffic distribution, data cluster switching and interface current limiting, it is ensured that the system maintains high availability in the face of high traffic, failures or abnormal situations, improves the stability and reliability of the system, and ensures high system availability;

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119052071A_ABST
    Figure CN119052071A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computer data processing, and discloses an API (Application Program Interface) high-availability configuration method for flexibly managing flow, which comprises the following steps of: configuring interface information, setting request parameters, writing SQL (Structured Query Language), and finishing API configuration through an online interface after debugging is finished; performing data caching on the data through redis and realizing a distributed lock; distributing flow to different data clusters in a targeted manner according to defined role distribution; analyzing the processing request through an SQL analyzer, and processing the request parameter and the SQL into an executable SQL; executing the executable SQL, returning data, and adding the returned data to a cache at the same time; abnormities of a complete request link are captured, all nodes are covered, and appearing task abnormities can be recorded to an access log. A high-availability, flexible and stable data processing solution is provided, and powerful support is provided for successful operation of services.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and in particular to a high-availability configuration method for an API interface for flexibly managing traffic. Background Art

[0002] With the rapid development of information technology and cloud computing, data management and processing have become an important part of enterprise and organizational architecture. Especially in the era of big data, data collection, storage, processing and analysis have become increasingly important. Currently, many organizations rely on complex database systems and interfaces to manage their huge data sets.

[0003] Although existing technologies provide basic functions for data management and processing, they still face many challenges when processing large-scale, high-concurrency data requests. These challenges include but are not limited to:

[0004] (1) Low R&D efficiency: A large amount of back-end R&D manpower is required to develop interfaces, and multiple types of work need to collaborate to develop and provide data interfaces;

[0005] (2) Poor performance: Existing systems may be inefficient when processing a large number of data requests, especially when accessing a large amount of data in a high-concurrency situation, which may cause the system to crash;

[0006] (3) Inadequate caching mechanism: The lack of an effective caching mechanism leads to frequent database access, increasing the database load;

[0007] (4) Traffic management and restriction: The lack of flexible traffic management and restriction means makes the system vulnerable to traffic peaks;

[0008] (5) Insufficient traffic diversion control: Without a good traffic diversion mechanism, requests are concentrated on one data node, which can easily cause problems in the data cluster and lead to service unavailability.

[0009] (6) High availability capability is poor, and high availability switching cannot be achieved at the interface level;

[0010] (7) It is difficult to distinguish between offline and real-time data before the offline data is batched and the offline data cannot be viewed in a timely manner on the business terminal.

[0011] Therefore, designing an API interface configuration method to solve the above problems has become an urgent issue to be solved in today's data management and processing. Summary of the invention

[0012] The purpose of the present invention is to solve the shortcomings of the above-mentioned prior art, and to solve a series of challenges in the existing data processing system by integrating multiple key technologies, so as to meet the growing and changing data needs and provide a high-availability configuration method for an API interface that flexibly manages traffic, including:

[0013] The steps include:

[0014] S1: Configure interface information, set request parameters, and write SQL. After debugging, complete the API interface configuration through the online interface.

[0015] S2: caches data and implements distributed locks through redis;

[0016] S3: distributes traffic to different data clusters in a targeted manner based on defined roles;

[0017] S4: Parsing the request through a SQL parser, processing the request parameters and the SQL into executable SQL;

[0018] S5: executing the executable SQL, returning data, and adding the returned data to a cache;

[0019] S6: Exception capture is performed on the entire request chain, covering all nodes, and any task exceptions that occur are recorded in the access log.

[0020] Wherein, step S1 specifically includes:

[0021] Design an API interface management page to manage all interfaces. The management page provides a new interface function, and then configures the request path, request parameters, data source selection, and cache time setting of the interface;

[0022] Provide SQL writing function to debug the written SQL and publish it online.

[0023] Furthermore, in step S1, after configuring the API interface, determine whether the current interface needs current limiting by obtaining the API configuration, and at the same time determine whether the current request needs current limiting according to the current limiting logic. If current limiting is required, directly return a null value; if current limiting is not required, continue to execute step S2.

[0024] Furthermore, in step S2, redis is used as a cache component, and each interface can customize the cache time to improve the interface response speed. At the same time, redis is used as a distributed lock to prevent the same request from accessing the database multiple times, thereby reducing the database pressure.

[0025] Preferably, by caching data in redis, configuring redis information, generating a unique cache key according to the request parameters and the API information; when querying the cache, if the corresponding cache exists, the data is returned, if not, continuing to execute step S3;

[0026] At the same time, customize the cache time configuration and set the cache time to be as dispersed as possible to avoid large-scale cache failure.

[0027] Furthermore, step S3 specifically includes:

[0028] Through the defined role diversion, the judgment logic of the role is encapsulated, and the role corresponding to the request is judged according to the specific request. By obtaining the data source bound to the role and checking the corresponding data cluster, the purpose of diversion is achieved.

[0029] Among them, flexible diversion is performed according to different access traffic, and a current limiting function corresponding to the interface level is provided to cope with high access volume requirements.

[0030] Furthermore, in step S4, the SQL parser is customized to process the SQL, and the parameters and the written SQL are processed into a complete executable SQL statement. During this period, additional business-related logic can be added, including data permissions, SQL assembly, and the function of modifying the SQL content is provided to meet special business needs.

[0031] Further, in step S5, the executable SQL is executed by a SQL execution engine, specifically including:

[0032] Use native jdbc to execute the executable SQL, obtain the execution result, return data, set cache, add the returned data to the cache, and record it in the access log.

[0033] Preferably, the access log records the response time, status and traffic trend of the interface, and step S6 further includes:

[0034] An access log will be recorded asynchronously for each request, and exceptions will be captured for the complete request link, covering all nodes. Any task exceptions will be recorded in the access log, and the scheduled task scans the exception information to issue an alarm, promptly respond to system problems, and provide assistance for subsequent interface analysis.

[0035] Furthermore, in response to service abnormalities, the traffic distribution and abnormal causes are analyzed, and traffic switching between different data clusters is performed through manual operation.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) Ensure that the system remains highly available in the face of high traffic, failures, or abnormal situations through traffic distribution, data cluster switching, and interface current limiting, thereby improving system stability and reliability and ensuring high system availability;

[0038] (2) The role-specified traffic allocation mechanism allows flexible configuration of traffic distribution rules based on user roles, business function modules, or other types, improving the configurability and flexibility of the system;

[0039] (3) Flexible traffic scheduling and manual switching mechanisms enable the system to adapt to the needs of different business scenarios, providing a highly versatile and scalable solution to adapt to different business scenarios;

[0040] (4) By recording detailed access logs, the present invention aims to provide strong support for system monitoring, performance analysis and troubleshooting, so as to timely discover and solve potential problems and improve the maintainability and debugging efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0042] In the attached picture:

[0043] Figure 1 This is a flow chart of a high-availability configuration method for an API interface that flexibly manages traffic according to the present invention;

[0044] Figure 2 This is an API interface configuration flow chart of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0047] The present invention aims to distribute traffic to different data clusters in a targeted manner according to user roles or access characteristics. This method is particularly suitable for distinguishing between core and non-core businesses, ensuring the stability and response speed of important businesses; for each API interface, configurable traffic restrictions are implemented to ensure that the system can still stably respond to key business requests under high traffic conditions.

[0048] The specific implementation of the present invention is described below with reference to the accompanying drawings and embodiments.

[0049] Example 1

[0050] like Figure 1 As shown, it is a flowchart of a method for configuring a high-availability API interface for flexible traffic management provided in this embodiment, including the following steps:

[0051] S1: Configure interface information, set request parameters, and write SQL. After debugging, complete the API interface configuration through the online interface.

[0052] S2: caches data and implements distributed locks through redis;

[0053] S3: distributes traffic to different data clusters in a targeted manner based on defined roles;

[0054] S4: Parsing the request through a SQL parser, processing the request parameters and the SQL into executable SQL;

[0055] S5: executing the executable SQL, returning data, and adding the returned data to a cache;

[0056] S6: Exception capture is performed on the entire request chain, covering all nodes, and any task exceptions that occur are recorded in the access log.

[0057] Wherein, step S1 is as follows: Figure 2 As shown, specifically including:

[0058] Design an API interface management page to manage all interfaces. The management page provides a new interface function, and then configures the request path, request parameters, data source selection, and cache time setting of the interface;

[0059] Provide SQL writing function to debug the written SQL and publish it online.

[0060] Preferably, in step S1, after configuring the API interface, determine whether the current interface needs current limiting by obtaining the API configuration, and determine whether the current request needs current limiting according to the current limiting logic. If current limiting is required, directly return a null value; if current limiting is not required, continue to execute step S2.

[0061] Furthermore, in step S2, redis is used as a cache component, and each interface can customize the cache time to improve the interface response speed. At the same time, redis is used as a distributed lock to prevent the same request from accessing the database multiple times, thereby reducing the database pressure.

[0062] Specifically, in this embodiment, redis caches data, configures redis information, and generates a unique cache key according to the request parameters and the API information; when querying the cache, if the corresponding cache exists, the data is returned, and if not, step S3 is continued;

[0063] At the same time, customize the cache time configuration and set the cache time to be as dispersed as possible to avoid large-scale cache failure, which will cause service abnormalities.

[0064] Step S3 specifically includes:

[0065] Through the defined role diversion, the judgment logic of the role is encapsulated, and the role corresponding to the request is judged according to the specific request. By obtaining the data source bound to the role and checking the corresponding data cluster, the purpose of diversion is achieved.

[0066] Flexible traffic diversion is performed according to different access traffic, and a current limiting function corresponding to the interface level is provided to cope with high access volume requirements.

[0067] In step S4, the customized SQL parser processes the SQL, and processes the parameters and the written SQL into a complete executable SQL statement. During this period, additional business-related logic can be added, including data permissions, SQL assembly, and the function of modifying the SQL content is provided to meet special business needs.

[0068] Further, in step S5, the executable SQL is executed by a SQL execution engine, specifically including:

[0069] Use native jdbc to execute the executable SQL, obtain the execution result, return data, set cache, add the returned data to the cache, and record it in the access log.

[0070] Specifically, in this embodiment, the access log records the response time, status, and traffic trend of the interface, and step S6 further includes:

[0071] Each request will asynchronously record an access log, and the complete request link will be captured for exceptions, covering all nodes. Any task exceptions will be recorded in the access log to help with subsequent interface analysis.

[0072] Each node needs to capture exceptions and record exception information in the log for subsequent log analysis. Scheduled tasks scan exception information for alarms to promptly respond to system problems and facilitate system stability construction.

[0073] In response to service anomalies, the traffic distribution and the cause of the anomaly are analyzed. If a problem is found in a data cluster, the traffic between different data clusters is switched manually to restore system stability and increase system flexibility and availability.

[0074] In general, the purpose of this invention is to provide enterprises and organizations with an intelligent, efficient, flexible and stable data processing solution to cope with today's growing and diverse data needs and provide strong support for the successful operation of the business.

[0075] First, by configuring interface request parameters, SQL, Reddit cache, real-time and offline time point segmentation access and other technologies, data engineers who do not have back-end development capabilities can quickly provide interfaces and achieve high-performance access, thereby accelerating data response speed.

[0076] Then, multiple data clusters, distributed locks, failover mechanisms and other mechanisms are configured through the interface. The present invention aims to ensure that the system maintains high availability in the face of high access volume, failures or abnormal situations, and improve the stability and reliability of the system.

[0077] Secondly, through the role-specified traffic allocation mechanism, the present invention aims to achieve flexible traffic management for different clients, roles or business functions, and at the same time can automatically allocate traffic to different data clusters for a set of interfaces according to the amount of accessed data, so as to meet the requirements of high concurrency and high performance in different business scenarios.

[0078] In addition, by recording detailed access logs, the present invention aims to provide strong support for system monitoring, performance analysis and troubleshooting, so as to timely discover and solve potential problems and improve the maintainability and debugging efficiency of the system.

[0079] More preferably, through features such as manually operable traffic switching between different data clusters and configurable interface-level current limiting, the present invention aims to adapt to different business scenarios, including high-concurrency access, large-scale data sets, etc., and provide a more flexible solution.

[0080] Specifically, in this embodiment, not only can traffic be diverted according to user roles, but roles can also be defined according to specific business function modules that users are responsible for. In this way, each role will be responsible for a specific function module, and traffic diversion can be more business-specific.

[0081] In addition to business function modules, roles can also be defined based on other user attributes such as geographic location, user type, etc. In this way, the system can allocate traffic to different types of users in a more granular manner to meet personalized needs.

[0082] At the same time, a current limiting mechanism is implemented for the entire data cluster to ensure that each data cluster can handle its maximum load. This can be done by monitoring the load of each data cluster and dynamically adjusting the amount of incoming requests to avoid overloading a data cluster. Within each data cluster, current limiting can be further implemented for a single interface. This helps ensure that even within a data cluster, core business interfaces are given priority processing to prevent non-core businesses from affecting critical businesses.

[0083] Finally, it should be noted that the above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0084] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A high-availability configuration method for an API interface that flexibly manages traffic, characterized in that: The steps include: S1: Configure interface information, set request parameters, and write SQL. After debugging, complete the API interface configuration through the online interface. S2: Use redis to cache data and implement distributed locks; S3: distributes traffic to different data clusters in a targeted manner based on defined roles; S4: Parsing the request through a SQL parser, processing the request parameters and the SQL into executable SQL; S5: executing the executable SQL, returning data, and adding the returned data to a cache; S6: Exception capture is performed on the entire request chain, covering all nodes, and any task exceptions that occur are recorded in the access log.

2. The method for configuring an API interface with high availability for flexible traffic management according to claim 1, characterized in that: Step S1 specifically includes: Design an API interface management page to manage all interfaces. The management page provides a new interface function, and then configures the request path, request parameters, data source selection, and cache time setting of the interface; Provide SQL writing function to debug the written SQL and publish it online.

3. The method for configuring an API interface with high availability for flexible traffic management according to claim 2, characterized in that: In step S1, after configuring the API interface, determine whether the current interface needs current limiting by obtaining the API configuration, and determine whether the current request needs current limiting based on the current limiting logic. If current limiting is required, return a null value directly; if current limiting is not required, continue to execute step S2.

4. The method for configuring an API interface with high availability for flexible traffic management according to claim 2, characterized in that: In step S2, the redis is used as a cache component, and each interface can customize the cache time to improve the interface response speed. At the same time, the redis is used as a distributed lock to prevent the same request from accessing the database multiple times, thereby reducing the database pressure.

5. The method for configuring an API interface with high availability for flexible traffic management according to claim 4, characterized in that: Cache data through redis, configure redis information, and generate a unique cache key according to the request parameters and the API information; when querying the cache, if the corresponding cache exists, return the data, if not, continue to execute step S3; At the same time, customize the cache time configuration and set the cache time to be as dispersed as possible to avoid large-scale cache failure.

6. The method for configuring an API interface with high availability for flexible traffic management according to claim 1, characterized in that: Step S3 specifically includes: Through the defined role diversion, the judgment logic of the role is encapsulated, and the role corresponding to the request is judged according to the specific request. By obtaining the data source bound to the role and checking the corresponding data cluster, the purpose of diversion is achieved.

7. The method for configuring an API interface with high availability for flexible traffic management according to claim 6, characterized in that: Flexible traffic diversion is performed according to different access traffic, and a current limiting function corresponding to the interface level is provided to cope with high access volume requirements.

8. The method for configuring an API interface with high availability for flexible traffic management according to claim 1, characterized in that: In step S4, the customized SQL parser processes the SQL, and processes the parameters and the written SQL into a complete executable SQL statement. During this period, additional business-related logic can be added, including data permissions, SQL assembly, and the function of modifying the SQL content is provided to meet special business needs.

9. The method for configuring an API interface with high availability for flexible traffic management according to claim 1, characterized in that: In step S5, the executable SQL is executed by the SQL execution engine, specifically including: Use native jdbc to execute the executable SQL, obtain the execution result, return data, set cache, add the returned data to the cache, and record it in the access log.

10. The method for configuring an API interface with high availability for flexible traffic management according to claim 9, characterized in that: The access log records the response time, status and traffic trend of the interface. Step S6 further includes: An access log will be recorded asynchronously for each request, and exceptions will be captured for the complete request link, covering all nodes. Any task exceptions will be recorded in the access log, and the scheduled task scans the exception information to issue an alarm, promptly respond to system problems, and provide assistance for subsequent interface analysis.

11. The method for configuring an API interface with high availability for flexible traffic management according to claim 1, characterized in that: In response to service anomalies, the traffic distribution and anomaly causes are analyzed, and traffic switching between different data clusters is performed through manual operation.