API gateway regulation and control method and device based on log, equipment and storage medium

Abstract

The invention provides a log-based API gateway regulation and control method and device, equipment and a storage medium, and the method comprises the steps: obtaining the log data of each API in an API gateway, determining the label of each API according to a preset rule, carrying out the training of a regulation and control model according to the labels and the log data, obtaining the real-time log data of each API in the API gateway, and obtaining the real-time log data of each API in the API gateway. According to the label, the regulation and control model and the real-time log data, the API gateway is regulated and controlled, the real-time log data is analyzed based on the regulation and control model to regulate and control the API gateway, the real-time performance is high, human errors can be avoided, and the management effect is enhanced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a log-based API gateway control method, device, equipment and storage medium. Background Art

[0002] In modern microservice architecture, the performance management of API gateway is the key to ensure service stability and reliability. Today, the performance management of API gateway is usually: 1. Static configuration: The configuration is set when the system is deployed and remains unchanged during operation; 2. Manual monitoring: The operation and maintenance team identifies potential performance problems by manually monitoring the performance indicators of the API gateway, such as response time and error rate. 3. Set fixed thresholds to trigger alarms. When the monitoring indicators exceed these thresholds, the alarm system is triggered to notify the operation and maintenance personnel to intervene; 4. Log analysis: Use log analysis tools such as ELK to collect and analyze the log data of the API gateway to identify problems and performance bottlenecks. Therefore, the performance management of the existing API gateway lacks dynamism and cannot adapt to the dynamic changes of the system load. The management effect is poor, the risk of human error is increased, and the analysis of logs can only find problems after the fact, and the real-time performance is poor. Summary of the invention

[0003] The embodiment of the present application provides a log-based API gateway control method, device, equipment and storage medium to solve at least one problem existing in the related technology. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a method for controlling an API gateway based on a log, including:

[0005] Get the log data of each API in the API gateway;

[0006] Determine a label for each of the APIs according to a preset rule;

[0007] According to the label and the log data, a control model is obtained by training;

[0008] The real-time log data of each API in the API gateway is obtained, and the API gateway is regulated according to the label, the regulation model and the real-time log data.

[0009] In one embodiment, determining the label of each of the APIs according to a preset rule includes:

[0010] In response to a configuration instruction, configuring a tag for the API when the API is registered in the API gateway;

[0011] or,

[0012] Based on the preset list, when the API is registered in the API gateway, the corresponding tag is matched from the preset list.

[0013] In one embodiment, training a control model according to the label and the log data includes:

[0014] Extract key performance indicators of log data of each of the APIs;

[0015] Classifying the key performance indicators according to the tags, and determining target key performance indicators corresponding to different tags;

[0016] Training a random forest model according to the labels and the corresponding target key performance indicators;

[0017] During the training process, the number of trees, the maximum depth of the trees, and the minimum number of sample segmentation leaf nodes are adjusted to train and obtain the regulation model.

[0018] In one embodiment, during the training process, adjusting the number of trees, the maximum depth of the trees, and the minimum number of sample segmentation leaf nodes to train the regulation model includes:

[0019] During the training process, the number of trees, the maximum depth of the trees, and the minimum number of sample segmentation leaf nodes are adjusted to obtain initial parameters and the corresponding initial model;

[0020] The initial model is evaluated through cross-validation and performance indicators to obtain an evaluation result;

[0021] When the evaluation result meets a preset threshold, the initial model is used as the control model;

[0022] When the evaluation result does not meet the preset threshold, the initial parameters are tuned through grid search until the evaluation result corresponding to the tuned initial model meets the preset threshold, and the tuned initial model is used as the regulation model; the regulation model is deployed in the API gateway.

[0023] In one implementation, regulating the API gateway according to the tag, the regulation model, and the real-time log data includes:

[0024] Extract real-time key performance indicators of real-time log data of each API;

[0025] Predicting different performance thresholds of the real-time key performance indicators through the control model, labels of different APIs, and corresponding real-time key performance indicators;

[0026] The API gateway is regulated according to the regulation model, the performance threshold, and the real-time key performance indicators.

[0027] In one embodiment, regulating the API gateway according to the regulation model, the performance threshold, and the real-time key performance indicator includes:

[0028] The real-time key performance indicators are analyzed through the control model; wherein the performance thresholds include the number of requests threshold, the first time threshold, the second time threshold, the first timeout threshold, the second timeout threshold, the failure threshold, the concurrency threshold, the frequency threshold, and the usage threshold; the real-time key performance indicators include the number of API requests, the average response time, the timeout rate, the failure rate, the number of concurrent connections, the error code, and the resource utilization rate;

[0029] When the number of API requests is greater than the request number threshold, if the tag of the API is high priority, increase the allocated server resources; if the tag of the API is not high priority, reduce the allocated server resources;

[0030] If the label of the API is high priority, the average response time of the control API is the first time threshold, otherwise the average response time of the control API is the second time threshold, and the first time threshold is less than the second time threshold;

[0031] If the tag of the API is high priority, when the timeout rate is greater than the first timeout threshold, a troubleshooting notification message is sent; if the tag of the API is not high priority, when the timeout rate is greater than the second timeout threshold, a troubleshooting notification message is sent, and the first timeout threshold is less than the second timeout threshold;

[0032] When the failure rate exceeds the failure threshold, locate the failure problem and implement current limiting of the corresponding API;

[0033] When the number of concurrent connections exceeds the concurrent threshold, enhancing the concurrent processing capability of the API gateway;

[0034] When the occurrence frequency of the error code exceeds the frequency threshold, strengthen the alarm of the API where the error code occurs;

[0035] When the resource usage rate exceeds the usage threshold, expanding the server instance;

[0036] If the label of the API is high traffic, distribute the traffic of the API to multiple servers through load balancing;

[0037] If the tag of the API is resource-intensive, allocating dedicated server resources to the API or reserving server resources for the API;

[0038] If the label of the API is security-sensitive, encrypt communication and add access control for the API;

[0039] If the tag of the API is a content type, configure serialization and deserialization mechanisms for the API;

[0040] If the label of the API is a user group, provide corresponding differentiated services for the API based on different preset groups;

[0041] If the label of the API is degradable, when server resources are insufficient, the response to the API is reduced and the update is delayed.

[0042] In one embodiment, the method further comprises:

[0043] Detect abnormal fluctuations of the real-time key performance indicators, and when abnormal fluctuations occur, send notification information and provide solutions;

[0044] or,

[0045] The control model is optimized according to the real-time log data.

[0046] In a second aspect, an embodiment of the present application provides a log-based API gateway control device, including:

[0047] The acquisition module is used to obtain the log data of each API in the API gateway;

[0048] A determination module, used to determine a label of each of the APIs according to a preset rule;

[0049] A training module, used for training a control model according to the label and the log data;

[0050] The control module is used to obtain the real-time log data of each API in the API gateway, and control the API gateway according to the label, the control model and the real-time log data.

[0051] In one embodiment, the control module is further used to:

[0052] Deploy the control model to the API gateway;

[0053] The performance thresholds of different key performance indicators are predicted by using the control model and the key performance indicators corresponding to different tags.

[0054] In one embodiment, the control module is further used to:

[0055] Detect abnormal fluctuations of the real-time key performance indicators, and when abnormal fluctuations occur, send notification information and provide solutions;

[0056] or,

[0057] The control model is optimized according to the real-time log data.

[0058] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement a method in any one of the above-mentioned embodiments.

[0059] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, the method in any one of the above-mentioned embodiments is implemented.

[0060] The beneficial effects of the above technical solution include at least:

[0061] By obtaining the log data of each API in the API gateway, determining the label of each API according to preset rules, training a control model based on the label and log data, obtaining the real-time log data of each API in the API gateway, and regulated the API gateway according to the label, control model and real-time log data. The API gateway is regulated based on the real-time log data analyzed by the control model. This has strong real-time performance, is conducive to avoiding human errors, and enhances management effectiveness.

[0062] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0064] Figure 1 This is a schematic diagram of the steps of the log-based API gateway control method according to an embodiment of the present application;

[0065] Figure 2 This is a flow chart of a log-based API gateway control method according to an embodiment of the present application;

[0066] Figure 3This is a structural block diagram of a log-based API gateway control device according to an embodiment of the present application;

[0067] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0069] Glossary:

[0070] API Gateway: A server that serves as a hub for inter-service communication in a microservices architecture and is responsible for request routing, load balancing, security control, etc.

[0071] Pandas: is an open source data analysis and manipulation library for the Python programming language. It provides high-performance, easy-to-use data structures and data analysis tools. It is very suitable for processing tabular data (such as CSV files, SQL query results, etc.) and time series data.

[0072] Traffic control strategy: a strategy used to manage the traffic and resource allocation of different APIs in the API gateway;

[0073] Scikit-learn: is an open source machine learning library for the Python programming language. It is built on top of scientific computing libraries such as NumPy, SciPy, and matplotlib, and provides a range of powerful tools for machine learning and statistical modeling, including tasks such as classification, regression, clustering, and dimensionality reduction.

[0074] The method of the embodiment of the present application can solve the problems of staticness and response delay of the existing API gateway performance management solution. These problems limit the performance and stability of the API gateway in the face of dynamically changing network environments, especially during high-load periods, such as batch calls at the beginning and end of the month. The relevant technology cannot adjust resource allocation in time, resulting in performance bottlenecks and reduced user experience. The solution of the present application can monitor key performance indicators in real time, predict performance bottlenecks, and dynamically adjust flow control strategies and resource allocation to adapt to changing load requirements.

[0075] Reference Figure 1 , showing a flow chart of a log-based API gateway control method according to an embodiment of the present application, the log-based API gateway control method may at least include steps S100-S400:

[0076] S100. Obtain log data of each API in the API gateway.

[0077] S200: Determine a label for each API according to a preset rule.

[0078] S300: Train and obtain a control model based on the labels and log data.

[0079] S400: Obtain real-time log data of each API in the API gateway, and regulate the API gateway according to the label, regulation model and real-time log data.

[0080] The technical solution of the embodiment of the present application obtains the log data of each API in the API gateway, determines the label of each API according to preset rules, trains a control model according to the label and the log data, obtains the real-time log data of each API in the API gateway, controls the API gateway according to the label, the control model and the real-time log data, and controls the API gateway based on the real-time log data analyzed by the control model. The solution has strong real-time performance, is conducive to avoiding human errors, and enhances management effects.

[0081] Reference Figure 2 In one implementation, the log of the API gateway is collected, and the log includes the log data of each API of the API gateway. Then, the log data can be pre-processed, such as cleaning, formatting, converting data types, etc.

[0082] In one embodiment, step S200 includes step S210 or S220:

[0083] S210. In response to the configuration instruction, configure a tag for the API when the API is registered in the API gateway.

[0084] Optionally, the user can generate configuration instructions by performing configuration operations in the system based on his or her actual needs. In response to the configuration instructions, the system configures tags for the API when the API is registered in the API gateway.

[0085] S220. Based on the preset list, when the API is registered in the API gateway, the corresponding tag is matched from the preset list.

[0086] Optionally, labels corresponding to each API in the API gateway may be set in advance to form a preset list. When the API is registered in the API gateway, the corresponding label is matched from the preset list.

[0087] It should be noted that when setting labels, factors such as business importance, frequency of use, and scope of impact can be considered; labels include but are not limited to priority (such as high priority, non-high priority such as medium priority and low priority), large traffic, high error rate, resource intensive, security sensitive, content type, user group, degradable, high response time, high error rate, etc.

[0088] In one implementation, step S300 includes steps S310-S340:

[0089] S310: Extract key performance indicators of log data of each API.

[0090] Reference Figure 2 Optionally, use Pandas to load the log data, and then perform feature extraction on the log data of each API, so as to extract the key performance indicators of the log data of each API. Optionally, the key performance indicators include but are not limited to the number of API requests, (average) response time, timeout rate, failure rate, number of concurrent connections, error codes (such as 502, 504 error codes), resource utilization, etc. It should be noted that log data can be obtained daily / weekly / monthly for feature extraction.

[0091] S320: Classify the key performance indicators according to the tags, and determine the target key performance indicators corresponding to the different tags.

[0092] Optionally, after determining the label of each API, the key performance indicators are classified to determine the target key performance indicators corresponding to different labels. For example, if one API is labeled as high priority and another API is labeled as low priority, the key performance indicators are classified, and the portion of the key performance indicators corresponding to the high-priority API is determined as the target key performance indicator corresponding to the high-priority label, and the portion of the key performance indicators corresponding to the low-priority API is determined as the target key performance indicator corresponding to the low-priority label.

[0093] S330, training the random forest model according to the labels and the corresponding target key performance indicators.

[0094] Optionally, the determined labels and corresponding target key performance indicators are input into a random forest model, and the random forest model is trained using Python's scikit-learn library.

[0095] S340. During the training process, the number of trees, the maximum depth of the trees, and the minimum number of sample segmentation leaf nodes are adjusted to train and obtain a control model.

[0096] Optionally, steps S3401-S3404 are included:

[0097] S3401. During the training process, the number of trees, the maximum depth of the trees, and the minimum number of sample segmentation leaf nodes are adjusted to obtain initial parameters and the corresponding initial model.

[0098] Optionally, during the training process, the number of trees, the maximum depth of the trees, and the minimum number of sample segmentation leaf nodes are adjusted to obtain initial parameters and a corresponding initial model determined based on the initial parameters.

[0099] S3402. Evaluate the initial model through cross-validation and performance indicators to obtain evaluation results.

[0100] Optionally, performance indicators can be set based on actual conditions, including but not limited to accuracy, error rate, etc., and the initial model obtained is evaluated by combining the performance indicators through a cross-validation method to obtain an evaluation result.

[0101] S3403. When the evaluation result meets the preset threshold, the initial model is used as the control model.

[0102] Optionally, when the evaluation result meets a preset threshold, the training is completed and the initial model at this time is used as the control model.

[0103] S3404. When the evaluation result does not meet the preset threshold, the initial parameters are tuned through grid search until the evaluation result corresponding to the tuned initial model meets the preset threshold, and the tuned initial model is used as the control model.

[0104] Optionally, when the evaluation result does not meet the preset threshold, indicating that further training is needed, the embodiment of the present application further tunes the initial parameters through grid search until the evaluation result corresponding to the tuned initial model meets the preset threshold, then the training is completed, and the final tuned initial parameters at this time are determined, and the tuned initial model corresponding to the final tuned initial parameters is determined and used as the control model. Finally, the control model is deployed to the API gateway.

[0105] In one implementation, in step S400, ELK (abbreviation of Elasticsearch, Logstash, and Kibana) Stack is used to collect the logs of the API gateway in real time, specifically including the log data of each API, and the collected log data can also be cleaned and formatted. It should be noted that the labels of each API can be determined in advance based on the principle of step S200.

[0106] In one embodiment, regulating the API gateway according to the tag, the regulation model, and the real-time log data includes steps S410-S430:

[0107] S410 , extracting real-time key performance indicators of real-time log data of each API.

[0108] Similarly, through feature extraction, real-time key performance indicators of real-time log data of each API are extracted.

[0109] S420, predicting performance thresholds of different real-time key performance indicators by regulating the model, labels of different APIs, and corresponding real-time key performance indicators.

[0110] Similarly, the corresponding real-time key performance indicators can be classified based on the labels of different APIs, and the target real-time key performance indicators of different labels can be determined. The labels of different APIs and the corresponding target real-time key performance indicators can be predicted by the control model, and the performance thresholds of the real-time key performance indicators of the labels of different APIs can be determined. It should be noted that the performance thresholds include but are not limited to the request number threshold, the first time threshold, the second time threshold, the first timeout threshold, the second timeout threshold, the failure threshold, the concurrency threshold, the frequency threshold, and the usage threshold.

[0111] S430: Regulate the API gateway according to the regulation model, performance threshold, and real-time key performance indicators.

[0112] Optionally, the real-time key performance indicators are analyzed through the control model to adjust the flow control strategy according to the analysis results, specifically:

[0113] 1. When the number of API requests is greater than the request threshold, if the API tag is high priority, increase the allocated server resources; if the API tag is not high priority, reduce the allocated server resources.

[0114] In an embodiment of the present application, when the number of API requests is greater than the request number threshold, for example, the request number threshold is 50% of the average value of the historical request number; if the API is labeled as high priority, due to the sudden increase in traffic, the allocated server resources are increased at this time, that is, the allocated resources of the API gateway are increased; if the API is labeled as non-high priority, that is, medium priority or low priority, the allocated server resources are reduced to prioritize the performance of high-priority APIs.

[0115] 2. If the API tag is high priority, the average response time of the control API is the first time threshold, otherwise the average response time of the control API is the second time threshold, and the first time threshold is less than the second time threshold.

[0116] In the embodiment of the present application, for example, the first time threshold can be 1s, and the second time threshold can be 60s (medium) / 150s (low). Prioritize high-priority APIs and optimize backend services to reduce response time; while for medium and low priority APIs, consider service degradation measures to increase the average response time. In addition, for APIs whose average response time is greater than the first time threshold, performance optimization or increase server resources are performed, and more active monitoring and automatic expansion strategies are implemented.

[0117] 3. If the API tag is high priority, when the timeout rate is greater than the first timeout threshold, a troubleshooting notification message is sent; if the API tag is non-high priority, when the timeout rate is greater than the second timeout threshold, a troubleshooting notification message is sent, and the first timeout threshold is less than the second timeout threshold.

[0118] In an embodiment of the present application, for example, the first timeout threshold is 1% and the second timeout threshold is 3% (intermediate) / 5% (low), so that the conditions for comparing and judging the first timeout threshold can respond more quickly and be adjusted with priority, and when sending troubleshooting notification information, the alarm operation and maintenance team can give priority to troubleshooting high-priority APIs.

[0119] 4. When the failure rate exceeds the failure threshold, locate the failure problem and implement current limiting on the corresponding API.

[0120] In an embodiment of the present application, when the failure rate exceeds a failure threshold, for example, exceeds 2%, errors are automatically reviewed, the source of the problem is quickly located, and current limiting measures are implemented for the affected API.

[0121] 5. When the number of concurrent connections exceeds the concurrent threshold, enhance the concurrent processing capability of the API gateway.

[0122] In an embodiment of the present application, when the number of concurrent connections exceeds the concurrency threshold, such as 5000 times / s, the concurrent processing capability of the API gateway is enhanced, and a connection limit can be implemented to prioritize the connection requirements of high-priority APIs.

[0123] 6. When the frequency of occurrence of the error code exceeds the frequency threshold, strengthen the alarm of the API where the error code occurs.

[0124] In the embodiment of the present application, when the frequency of occurrence of error codes (including 502 and 504 error codes) exceeds a frequency threshold, such as 5%, more active monitoring and alarming are implemented for the related APIs where the error codes occur.

[0125] 7. When resource usage exceeds the usage threshold, for example, the usage threshold is 90% of the CPU or memory usage. At this time, the server instance is expanded and the API labeled as resource-intensive is limited; at the same time, the resource allocation of high-priority DPI is prioritized.

[0126] 8. If the API is labeled as high traffic, distribute the API traffic to multiple servers through load balancing.

[0127] In an embodiment of the present application, a request rate limit is provided for the API volume that is larger than that of other APIs without large traffic labels to avoid overloading the system; in addition, load balancing technology is used to disperse API traffic requests to multiple servers.

[0128] 9. If the API is labeled as resource-intensive, allocate dedicated server resources to the API or reserve server resources for the API.

[0129] In an embodiment of the present application, if the API is labeled as resource-intensive, the resource usage of the resource-intensive API is limited based on a preset limit threshold, or it is run during low-load periods, and dedicated server resources are allocated to these APIs or server resources are reserved for the APIs.

[0130] 10. If the API is labeled as security-sensitive, encrypt communications and add access control for the API.

[0131] In the embodiments of the present application, encrypted communication and access control are added for the API to enhance security measures.

[0132] 11. If the API tag is content type, configure the serialization and deserialization mechanism for the API.

[0133] In an embodiment of the present application, if the label of the API is the content type, the content type returned by the analysis API is JSON or XML, etc., and based on the corresponding different content types analyzed, a serialization and deserialization mechanism is configured for the API to optimize data transmission and processing.

[0134] 12. If the API is labeled as a user group, provide corresponding differentiated services for the API based on different preset groups.

[0135] In an embodiment of the present application, if the label of the API is a user group, the user group of the API is specifically a preset user group, such as corporate users, ordinary consumers, etc., and then corresponding differentiated services are provided for the API based on different preset groups.

[0136] 13. If the API is labeled as degradable, when server resources are insufficient, reduce the response to the API and delay updates.

[0137] In an embodiment of the present application, when server resources are insufficient, that is, when the resources of the API gateway are insufficient, the response to the API is reduced and when an update is needed, the update is delayed to reduce its resource usage.

[0138] 14. For APIs with error rates higher than the error threshold, i.e., labeled as high error rates, implement stricter error tracking and rapid response mechanisms. For example, automatically trigger alarms and conduct problem troubleshooting.

[0139] In some embodiments, each API may have at least one label, and the label may also be newly registered, to be optimized, or third-party integrated; when the label is newly registered, the newly registered API is closely monitored to evaluate its performance and resource requirements, and adjusted according to actual usage; when the label is to be optimized, the performance of the API is optimized first, and the operation and maintenance team is notified to contact the service provider to optimize the API performance; when the label is third-party integrated, additional monitoring and fault-tolerance mechanisms are implemented for the API that integrates the third-party service to ensure that the availability and performance of the third-party service meet expectations.

[0140] In one implementation, the log-based API gateway control method of the embodiment of the present application may further include step S510 or S520:

[0141] S510: Detect abnormal fluctuations of real-time key performance indicators. When abnormal fluctuations occur, send notification information and provide solutions.

[0142] Optionally, the real-time key performance indicators can be analyzed in the control model while abnormal fluctuation detection can be performed on the real-time key performance indicators. Alternatively, after the control model is analyzed and the API gateway is regulated, the API gateway can be regulated in real time. If abnormal fluctuations still exist after the API gateway is regulated to achieve resource allocation optimization, the system sends a notification message to the user and provides the user with a corresponding solution strategy for exception handling. It should be noted that after the API gateway is regulated and the exception is handled, relevant data can be collected and fed back as training data for retraining the control model and optimizing the control model.

[0143] S520: Optimize the control model according to the real-time log data.

[0144] Optionally, the real-time log data obtained every day can be used as training data to regularly retrain the control model and perform adaptive learning, so as to continuously optimize the control model and optimize the flow control strategies of different APIs, such as more accurately implementing flow limiting and service degradation in specific situations.

[0145] In one embodiment, the regulatory model can also integrate gradient boosting trees and neural networks based on the random forest model to improve the prediction accuracy by combining multiple models into a final regulatory model.

[0146] Through the method of the embodiment of the present application, it is possible to overcome the static and response delay problems of the API gateway performance management solution in the related technology, the problem of being unable to adapt to the dynamically changing system load and performance requirements, the problem of uneven resource allocation and over-reliance on manual monitoring, and reduce manual intervention and improve performance management efficiency through automated key performance indicator monitoring and performance threshold prediction; real-time identification and response to performance bottlenecks, rapid response to performance changes, and enhanced API gateway stability and reliability; intelligent adjustment of resource allocation, optimization of resource utilization, guarantee of key API performance, and improvement of overall system performance and user experience, especially during high-load periods; use of the high accuracy and anti-overfitting ability of the random forest algorithm to improve the accuracy of performance bottleneck prediction; based on the analysis and prediction of real-time key performance indicators, dynamically adjust the flow control strategy to ensure that key APIs obtain sufficient resource support at critical moments; by preventing performance bottlenecks and timely adjusting the flow control strategy, service interruptions and delays are reduced, and user experience is improved. Therefore, through the method of the embodiment of the present application, the API gateway can manage performance more intelligently and automatically, adapt to the ever-changing network environment and business needs, and thus provide more stable and efficient services.

[0147] Reference Figure 3 , shows a structural block diagram of a log-based API gateway control device according to an embodiment of the present application, which may include:

[0148] The acquisition module is used to obtain the log data of each API in the API gateway;

[0149] A determination module, used to determine the label of each API according to preset rules;

[0150] The training module is used to train the control model based on the labels and log data;

[0151] The control module is used to obtain the real-time log data of each API in the API gateway, and control the API gateway according to the label, control model and real-time log data.

[0152] In one embodiment, the control module is further used to:

[0153] Deploy the control model to the API gateway;

[0154] By adjusting the model and the key performance indicators corresponding to different tags, the performance thresholds of different key performance indicators are predicted.

[0155] In one embodiment, the control module is further used to:

[0156] Detect abnormal fluctuations in real-time key performance indicators. When abnormal fluctuations occur, send notification information and provide solutions.

[0157] or,

[0158] Optimize the control model based on real-time log data.

[0159] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and will not be repeated here.

[0160] Reference Figure 4 , shows a structural block diagram of an electronic device according to an embodiment of the present application, the electronic device includes: a memory 310 and a processor 320, the memory 310 stores instructions that can be run on the processor 320, and the processor 320 loads and executes the instructions to implement the log-based API gateway control method in the above embodiment. The number of the memory 310 and the processor 320 can be one or more.

[0161] In one embodiment, the electronic device also includes a communication interface 330 for communicating with external devices and performing data exchange transmission. If the memory 310, the processor 320 and the communication interface 330 are implemented independently, the memory 310, the processor 320 and the communication interface 330 can be interconnected through a bus and complete mutual communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0162] Optionally, in a specific implementation, if the memory 310, the processor 320 and the communication interface 330 are integrated on a chip, the memory 310, the processor 320 and the communication interface 330 can communicate with each other through an internal interface.

[0163] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the log-based API gateway control method provided in the above embodiment.

[0164] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0165] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0166] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the advanced RISC machines (ARM) architecture.

[0167] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0168] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0169] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0170] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0171] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0172] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0173] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0174] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0175] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A log-based API gateway control method. It is characterized in that include: Get the log data of each API in the API gateway; Determine a label for each of the APIs according to a preset rule; According to the label and the log data, a control model is obtained by training; The real-time log data of each API in the API gateway is obtained, and the API gateway is regulated according to the label, the regulation model and the real-time log data.

2. According to the log-based API gateway control method of claim 1, Features: Determining the label of each of the APIs according to the preset rules includes: In response to a configuration instruction, configuring a tag for the API when the API is registered in the API gateway; or, Based on the preset list, when the API is registered in the API gateway, the corresponding tag is matched from the preset list.

3. According to the log-based API gateway control method of claim 1, Features: The training of the control model according to the label and the log data includes: Extract key performance indicators of log data of each of the APIs; Classifying the key performance indicators according to the tags, and determining target key performance indicators corresponding to different tags; Training a random forest model according to the labels and the corresponding target key performance indicators; During the training process, the number of trees, the maximum depth of the trees, and the minimum number of sample segmentation leaf nodes are adjusted to train and obtain the regulation model.

4. According to the log-based API gateway control method of claim 3, Features: During the training process, adjusting the number of trees, the maximum depth of the trees, and the minimum number of sample segmentation leaf nodes to train the control model includes: During the training process, the number of trees, the maximum depth of the trees, and the minimum number of sample segmentation leaf nodes are adjusted to obtain initial parameters and the corresponding initial model; The initial model is evaluated through cross-validation and performance indicators to obtain an evaluation result; When the evaluation result meets a preset threshold, the initial model is used as the control model; When the evaluation result does not meet the preset threshold, the initial parameters are tuned through grid search until the evaluation result corresponding to the tuned initial model meets the preset threshold, and the tuned initial model is used as the regulation model; the regulation model is deployed in the API gateway.

5. According to any one of claims 1 to 4, the log-based API gateway control method, Features: The regulating the API gateway according to the label, the regulation model and the real-time log data includes: Extract real-time key performance indicators of real-time log data of each API; Predicting different performance thresholds of the real-time key performance indicators through the control model, labels of different APIs, and corresponding real-time key performance indicators; The API gateway is regulated according to the regulation model, the performance threshold, and the real-time key performance indicators.

6. According to the log-based API gateway control method of claim 5, Features: The regulating the API gateway according to the regulation model, the performance threshold, and the real-time key performance indicator includes: The real-time key performance indicators are analyzed through the control model; wherein the performance thresholds include the number of requests threshold, the first time threshold, the second time threshold, the first timeout threshold, the second timeout threshold, the failure threshold, the concurrency threshold, the frequency threshold, and the usage threshold; the real-time key performance indicators include the number of API requests, the average response time, the timeout rate, the failure rate, the number of concurrent connections, the error code, and the resource utilization rate; When the number of API requests is greater than the request number threshold, if the tag of the API is high priority, increase the allocated server resources; if the tag of the API is not high priority, reduce the allocated server resources; If the label of the API is high priority, the average response time of the control API is the first time threshold, otherwise the average response time of the control API is the second time threshold, and the first time threshold is less than the second time threshold; If the tag of the API is high priority, when the timeout rate is greater than the first timeout threshold, a troubleshooting notification message is sent; if the tag of the API is not high priority, when the timeout rate is greater than the second timeout threshold, a troubleshooting notification message is sent, and the first timeout threshold is less than the second timeout threshold; When the failure rate exceeds the failure threshold, locate the failure problem and implement current limiting of the corresponding API; When the number of concurrent connections exceeds the concurrent threshold, enhancing the concurrent processing capability of the API gateway; When the occurrence frequency of the error code exceeds the frequency threshold, strengthen the alarm of the API where the error code occurs; When the resource usage rate exceeds the usage threshold, expanding the server instance; If the label of the API is high traffic, distribute the traffic of the API to multiple servers through load balancing; If the tag of the API is resource-intensive, allocating dedicated server resources to the API or reserving server resources for the API; If the label of the API is security-sensitive, encrypt communication and add access control for the API; If the tag of the API is a content type, configure serialization and deserialization mechanisms for the API; If the label of the API is a user group, provide corresponding differentiated services for the API based on different preset groups; If the label of the API is degradable, when server resources are insufficient, the response to the API is reduced and the update is delayed.

7. According to the log-based API gateway control method of claim 6, Features: The method further comprises: Detect abnormal fluctuations of the real-time key performance indicators, and when abnormal fluctuations occur, send notification information and provide solutions; or, The control model is optimized according to the real-time log data.

8. A log-based API gateway control device, It is characterized in that include: The acquisition module is used to obtain the log data of each API in the API gateway; A determination module, used to determine a label of each of the APIs according to a preset rule; A training module, used for training a control model according to the label and the log data; The control module is used to obtain the real-time log data of each API in the API gateway, and control the API gateway according to the label, the control model and the real-time log data.

9. An electronic device, It is characterized in that include: A processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images