API (Application Program Interface) security protection method, device and equipment based on HTTP (Hyper Text Transport Protocol) message streaming processing

Abstract

The invention discloses an API (Application Program Interface) security protection method, device and equipment based on HTTP (Hyper Text Transport Protocol) message streaming processing, and relates to the field of data processing. The method comprises the following steps: recombining a TCP message to obtain an HTTP message; a target HTTP message related to API calling is determined; extracting a plurality of parameters and values corresponding to the parameters from the target HTTP message to form parameter value pairs; performing determinant storage on the parameter value pairs to obtain structured storage data; judging whether the target HTTP message needs to be subjected to streaming computation or not based on the structured storage data; executing streaming computation on the target HTTP message to obtain a computation result; and according to the HTTP routing setting and the calculation result, determining a processing result for the target HTTP message, including releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response. By implementing the technical scheme, the security risk caused by delay processing is avoided, and the real-time performance of security protection is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to an API security protection method, device and equipment based on HTTP message streaming processing. Background Art

[0002] With the rapid development of the Internet, API (Application Programming Interface) has become an indispensable part of modern software architecture. API allows efficient data exchange and functional integration between various applications. However, the widespread use of API also introduces many security risks, especially security issues on the HTTP protocol have become a key challenge.

[0003] At present, traditional network security tools, such as WAF (Web Application Firewall) and big data risk control system, although providing a certain degree of protection for API security, still have some key deficiencies. WAF is a security tool based on HTTP protocol proxy and content detection engine, which is widely used in the market to protect Web applications from attacks. WAF mainly relies on static rules and signature matching. Although this method can intercept known attack patterns in real time, it cannot effectively deal with complex attacks that require real-time data analysis and decision-making. Big data risk control systems usually process data after data collection, relying on batch processing or intermittent streaming computing. This results in processing and response not being synchronized with data collection, and the processing results are often available a long time after the data is generated, and it is impossible to respond to the incident immediately. Therefore, traditional network security tools have the problem of insufficient real-time security protection processing.

[0004] Therefore, there is an urgent need for an API security protection method, device and equipment based on HTTP message streaming processing. Summary of the invention

[0005] The present application provides an API security protection method, device and equipment based on HTTP message streaming processing, which avoids the security risks caused by delayed processing and improves the real-time performance of security protection.

[0006] In a first aspect of the present application, a method for API security protection based on HTTP message streaming processing is provided, in which a TCP message is received by listening to a preset network port or a preset protocol, and the TCP message is reassembled to obtain an HTTP message; the HTTP message is parsed to obtain a message header and a message body, and a target HTTP message related to the API call is determined according to the message header and the message body; a plurality of parameters and values ​​corresponding to each of the parameters are extracted from the target HTTP message to form a parameter value pair; the parameter value pairs are stored in a columnar manner to obtain structured storage data; according to a preset streaming computing trigger condition, whether the target HTTP message needs to be stream-computed is determined based on the structured storage data; if it is determined that the target HTTP message needs to be stream-computed, the streaming computing is performed on the target HTTP message to obtain a computing result; according to the HTTP routing setting and the computing result, a processing result for the target HTTP message is determined, and the processing result includes releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

[0007] By adopting the above technical solution, by listening to the preset network port or the preset protocol to receive the TCP message, and reorganizing it to obtain the HTTP message, the complete message data of the application layer can be obtained to provide input for subsequent processing. Then the HTTP message is parsed, the message header and the message body are extracted, and the target HTTP message related to the API call is determined according to the information in the message header and the message body, so that the key message that needs to be protected can be accurately located to improve the pertinence of the processing. Then the parameters and parameter values ​​are extracted from the target HTTP message to form a parameter value pair, and the parameter value pair is converted into structured storage data in a column storage manner, which can facilitate subsequent streaming calculation and analysis. According to the preset streaming calculation trigger condition, it is judged whether the target HTTP message needs to be stream-calculated, and the calculation can be selectively performed on a specific message to improve the calculation efficiency. Performing streaming calculation on the target HTTP message that needs to be calculated can obtain statistical results or feature results in real time, and quickly discover anomalies or threats. Finally, according to the HTTP routing settings and calculation results, the processing result of the message is determined, and the message can be dynamically released, forwarded or blocked to achieve refined security protection measures. This method uses HTTP messages as processing objects and builds a complete API security protection process, which can effectively identify and intercept malicious requests and attacks against APIs, and improve the system's ability to resist risks. API security protection not only improves the recognition and response speed of complex attack patterns, but also improves the intelligence and accuracy of security policies. In addition, this solution effectively avoids security risks caused by delayed processing through real-time analysis and response, and improves the real-time nature of security processing.

[0008] Optionally, the parameter value pairs are stored in columnar format to obtain structured storage data, specifically including: determining a data type corresponding to each parameter value pair, the data type including text, number, Boolean value and single value; according to the data type, determining a corresponding data encoding rule in a preset encoding database, the preset encoding database including a correspondence between the data type and the data encoding rule; according to the data encoding rule, encoding each parameter value pair, and storing the encoded parameter value pairs in Arrow memory format to obtain the structured storage data.

[0009] By adopting the above technical solution, when storing parameter value pairs in column format, the data type of each parameter value pair is first determined, and different types of data, such as text, numbers, Boolean values, etc., can be identified to provide a basis for subsequent encoding processing. Then, the encoding rules corresponding to the data type are searched in the preset encoding database, and the appropriate encoding method can be selected according to the characteristics of the data to improve the efficiency of storage and calculation. Then, the parameter value pairs are encoded according to the determined encoding rules, converted into a format suitable for storage and calculation, and stored in the Arrow memory format to form structured storage data. By adopting the Arrow format, the reading, writing and transmission of data can be accelerated, the overhead of data conversion can be reduced, and the processing efficiency of the entire method can be improved. At the same time, column storage can easily aggregate and filter specific columns to meet the needs of streaming computing. Converting parameter value pairs into high-performance structured storage data provides efficient data support for subsequent streaming computing, and can quickly retrieve and analyze massive HTTP message data, and promptly determine suspicious behaviors and security incidents.

[0010] Optionally, the method of judging whether the target HTTP message needs to perform streaming computing based on the structured storage data according to the preset streaming computing trigger condition specifically includes: obtaining the preset streaming computing trigger condition, the preset streaming computing trigger condition including an API call condition and a computing configuration parameter; matching the target HTTP message with the API call condition; if it is determined that the target HTTP message matches the API call condition, determining the streaming computing logic for the target HTTP message according to the computing configuration parameters, and determining to perform streaming computing on the target HTTP message.

[0011] By adopting the above technical solution, when judging whether the target HTTP message needs to be stream-calculated, the preset stream-calculation trigger conditions are first obtained, including API call conditions and calculation configuration parameters, and the basis for triggering the stream-calculation is clarified. By matching the target HTTP message with the API call conditions, it is possible to quickly determine whether the target HTTP message needs to be stream-calculated, avoid unnecessary processing of irrelevant messages, and improve overall processing efficiency and accuracy. If the target HTTP message meets the API call conditions, the specific stream-calculation logic for the message is further determined according to the calculation configuration parameters to implement a customized calculation strategy. By setting flexible stream-calculation trigger conditions and calculation configuration parameters, API calls can be selectively stream-processed and analyzed, and abnormal behaviors and security threats can be discovered in a timely manner.

[0012] Optionally, if it is determined that the target HTTP message needs to perform streaming calculation, streaming calculation is performed on the target HTTP message to obtain a calculation result, which specifically includes: determining the type of streaming calculation that needs to be performed on the target HTTP message according to the calculation configuration parameters, the streaming calculation type including statistical streaming calculation and feature streaming calculation; if it is determined that the target HTTP message needs to perform statistical streaming calculation, then according to preset statistical rules, statistical streaming calculation is performed on the target HTTP message; if it is determined that the target HTTP message needs to perform feature streaming calculation, then according to preset feature extraction rules, feature streaming calculation is performed on the target HTTP message.

[0013] By adopting the above technical solution, when performing streaming computing on the target HTTP message, the type of streaming computing to be performed is first determined according to the computing configuration parameters, which is mainly divided into two categories: statistical streaming computing and feature streaming computing, so as to meet different analysis needs. Statistical streaming computing focuses on summarizing and aggregating multiple requests within a period of time, which can reflect the overall usage and performance of the API. Feature streaming computing focuses on in-depth analysis and extraction of a single request, which can discover abnormal characteristics and risks of a single request. According to the different types of calculations, the preset statistical rules or preset feature extraction rules are used for calculations, which can adapt to the analysis requirements of different types of data and improve the pertinence and accuracy of the calculations. By adopting different rules and algorithms for different types of streaming computing, multiple dimensions of HTTP messages can be comprehensively analyzed, and potential security issues can be identified from both the overall and individual levels, thereby providing more comprehensive and effective API security protection.

[0014] Optionally, the computing configuration parameters include state definitions and statistical rules, and the statistical streaming computing is performed on the target HTTP message according to the preset statistical rules, specifically including: obtaining the state definition in the computing configuration parameters, the state definition including a state identifier, a state initial value and a life cycle; calculating the statistical results based on the structured storage data of the target HTTP message according to the statistical rules in the computing configuration parameters; if it is determined that there is no state corresponding to the state identifier, creating a target state corresponding to the state identifier, and using the statistical result as the state initial value of the target state, and setting the life cycle of the target state to the survival time of the target state; if it is determined that there is already a target state corresponding to the state identifier, performing an aggregation operation on the statistical result and the current value of the target state to obtain a calculation result, and updating the calculation result to the current value of the target state; if it is determined that the survival time of the target state has expired, deleting the target state.

[0015] By adopting the above technical solution, the concept of state is introduced when performing statistical streaming calculations, which is used to store and maintain the cumulative calculation results over a period of time. The state definition includes attributes such as state identification, state initial value and life cycle, which can uniquely identify a state, set the initial value, and specify the validity period of the state, which is convenient for the creation, update and cleanup of the state. Through state-based statistical streaming calculations, data can be continuously summarized and aggregated in the HTTP message stream to obtain statistical results reflecting the cumulative indicators over a period of time, and abnormal situations and security threats of API calls can be discovered in a timely manner. At the same time, through the life cycle management of the state, the consumption of system resources can be controlled to improve the overall performance and stability.

[0016] Optionally, the calculation configuration parameters include feature extraction rules and feature calculation trigger conditions, and the feature-type streaming calculation is performed on the target HTTP message according to the preset feature extraction rules, specifically including: obtaining the feature extraction rules and feature calculation trigger conditions in the calculation configuration parameters; extracting feature parameters from the structured storage data of the target HTTP message according to the feature extraction rules; judging whether the feature calculation trigger conditions are met, and the feature calculation trigger conditions include time trigger, data volume trigger, event trigger and external trigger; if it is determined that the feature calculation trigger conditions are met, the feature parameters are input into a preset machine learning model, and the output result of the preset machine learning model is used as the feature calculation result; if it is determined that the feature calculation trigger conditions are not met, the feature parameters are temporarily stored until the feature calculation trigger conditions are met, and the temporarily stored feature parameters are input into the preset machine learning model.

[0017] By adopting the above technical solution, when performing feature-based streaming computing, key features reflecting anomalies or risks are extracted from the target HTTP message, and feature calculation and analysis are selectively performed according to the feature calculation trigger conditions. First, the feature extraction rules and feature calculation trigger conditions defined in the calculation configuration parameters are obtained to clearly identify which feature parameters to extract and when to trigger feature calculation. According to the feature extraction rules, relevant feature parameters are extracted from the structured storage data of the HTTP message. The extracted feature parameters can provide a data basis for subsequent anomaly detection and risk judgment. Then, it is determined whether the trigger conditions for feature calculation are met, and the timing and frequency of feature calculation are flexibly controlled to improve the timeliness and efficiency of the calculation. If the trigger conditions are met, the extracted feature parameters are input into the preset machine learning model for calculation. The preset machine learning model analyzes and detects anomalies on the feature parameters, and outputs calculation results reflecting the degree of request anomaly or risk level, providing a quantitative basis for security protection decisions.

[0018] Optionally, after determining the processing result for the target HTTP message according to the HTTP routing settings and the calculation result, the method further includes: if it is determined that the processing result is to release and forward the target HTTP message, forwarding the target HTTP message to the target server, and sending the response message returned by the target server to the original requester; if it is determined that the processing result is to block the target HTTP message, returning a preset blocking response message to the original requester.

[0019] By adopting the above technical solution, after determining the processing result of the target HTTP message, corresponding follow-up operations are taken according to different results to complete the entire API security protection process. If the processing result is to release and forward the message, the message is forwarded to the target server for normal business processing, and the response message returned by the server is sent to the original requester to complete the complete HTTP request-response process. By forwarding and responding to normal requests, the availability and continuity of API services can be guaranteed and normal business operations can be maintained. If the processing result is a blocked message, the message will no longer be forwarded, but a preset blocking response message will be directly returned to the original requester. By timely blocking and responding to abnormal requests, potential attacks and intrusions can be effectively prevented and the security of API services can be protected.

[0020] In a second aspect of the present application, an API security protection device based on HTTP message streaming processing is provided, the device comprising: a receiving module and a processing module, wherein: the receiving module is used to receive TCP messages by monitoring a preset network port or a preset protocol, and reassemble the TCP messages to obtain HTTP messages; the processing module is used to parse the HTTP messages to obtain message headers and message bodies, and determine a target HTTP message related to the API call according to the message headers and the message bodies; the processing module is also used to extract multiple parameters and values ​​corresponding to each of the parameters from the target HTTP message to form parameter value pairs; the processing module , and is also used to store the parameter value pairs in columnar form to obtain structured storage data; the processing module is also used to determine whether the target HTTP message needs to be stream-calculated based on the structured storage data according to a preset streaming computing trigger condition; the processing module is also used to perform streaming computing on the target HTTP message to obtain a computing result if it is determined that the target HTTP message needs to be stream-calculated; the processing module is also used to determine the processing result for the target HTTP message according to the HTTP routing setting and the computing result, and the processing result includes releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. This method uses HTTP messages as processing objects and builds a complete API security protection process, which can effectively identify and intercept malicious requests and attacks against APIs, and improve the system's ability to resist risks. API security protection not only improves the recognition and response speed of complex attack patterns, but also improves the intelligence and accuracy of security policies. In addition, this solution effectively avoids security risks caused by delayed processing through real-time analysis and response, and improves the real-time nature of security processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1It is a flowchart of an API security protection method based on HTTP message streaming processing disclosed in an embodiment of the present application; Figure 2 It is a module schematic diagram of an API security protection device based on HTTP message streaming processing disclosed in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0025] Explanation of the reference numerals: 201, receiving module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0026] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0027] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0028] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0029] This application provides an API security protection method based on HTTP message streaming processing. Figure 1 , Figure 1This is a flowchart of an API security protection method based on HTTP message streaming provided by an embodiment of the present application. The method is applied to a server, which is a server that executes a program for real-time processing of HTTP protocol network traffic. The server can be a single server, or a server cluster composed of multiple servers, or a cloud computing service center. The method includes steps S101 to S107, which are as follows: Step S101: receiving TCP messages by monitoring a preset network port or a preset protocol, and reassembling the TCP messages to obtain HTTP messages.

[0030] In step S101, the server receives TCP messages from the client or other servers by listening to a preset network port or a preset protocol, and reassembles these TCP messages to finally obtain a complete HTTP message. This process can be divided into the following key steps: the server first listens to a specific network port. Usually, the default port used by the HTTP protocol is 80. The server will start one or more listening processes, bind to the preset port, and wait for the client's connection request. When the client initiates an HTTP request to the server, a TCP connection will first be established. The client will send a SYN packet to the preset port of the server. After receiving it, the server will reply with a SYN-ACK packet, and the client will reply with an ACK packet to complete the three-way handshake and establish a TCP connection. The TCP connection provides a reliable data transmission channel for subsequent HTTP communications. After establishing the TCP connection, the client will split the HTTP request into multiple TCP segments according to the specifications of the TCP protocol and send them to the server. The server's listening process will continuously read these segments from the buffer of the TCP connection, and record and manage them according to information such as sequence number and length.

[0031] Since TCP messages may be out of order, repeated, or lost during network transmission, the TCP segments received by the server may be incomplete or discontinuous. The server reassembles these scattered segments into a complete and ordered data stream based on information such as the sequence number and length of the TCP segments. This process is called TCP message reassembly. After TCP message reassembly is completed, the server obtains a complete and continuous data stream. However, this data stream may contain multiple HTTP request or response messages, which are separated by specific delimiters (such as "\r\n\r\n"). The server identifies and extracts individual HTTP messages from this data stream according to the format specification of the HTTP protocol for subsequent processing.

[0032] For example, when a user visits a web page in a browser, the browser sends an HTTP GET request to the server. This request is split into multiple TCP segments and sent out. The server listens to port 80, and after receiving these TCP segments, it reassembles them according to the sequence number to obtain a complete data stream. Then, the server extracts the HTTP GET request message from this data stream.

[0033] Step S102: Parse the HTTP message to obtain a message header and a message body, and determine a target HTTP message related to the API call according to the message header and the message body.

[0034] In step S102, the server parses the received HTTP message and extracts the contents of the message header and the message body. Then, the server determines whether the HTTP message is related to the API call based on the specific information in the message header and the message body. If it is related, it is determined as the target HTTP message for subsequent processing. This process may include the following steps: the server first parses the received HTTP message. The HTTP message consists of three parts: the start line (request line or status line), the header field and the message body, which are separated by "\r\n". The server splits the HTTP message into various parts according to this format. The start line contains information such as the request method, request URL, HTTP version, etc.; the header field contains a series of key-value pairs that describe the metadata of the message, such as Host, User-Agent, Content-Type, etc.; the message body contains the actual content of the message, such as request parameters, response data, etc. After the HTTP message is parsed, the server extracts the contents of the message header and the message body. The message header includes the start line and the header field, which exists in the form of plain text. The server parses the message header into a dictionary or hash table of key-value pairs. The message body is in various formats such as plain text, JSON, XML, binary data, etc. The server determines the specific format of the message body based on the value of the Content-Type field and uses the corresponding parser to parse it. The server determines whether the HTTP message is related to the API call based on the specific information in the message header and message body. The server can make judgments based on the specific design and conventions of the application. For example, the server can check whether the requested URL matches the predefined API path pattern, such as " / api / v1 / users"; it can check whether the requested method complies with the API specifications, such as GET, POST, PUT, etc.; it can check whether the requested parameters contain the required and optional parameters required by the API; it can check whether the requested header fields contain the necessary authentication information, such as API Key, Token, etc. The server identifies HTTP messages related to API calls through this series of checks and matches.

[0035] For those HTTP messages identified as being related to API calls, the server identifies them as target HTTP messages for subsequent processing. These messages may come from the target client and contain normal API request content; they may also come from malicious attackers and contain illegal parameters or malicious payloads. The server further analyzes and processes these target HTTP messages to achieve API security protection.

[0036] Step S103: extract multiple parameters and the values ​​corresponding to the parameters from the target HTTP message to form parameter-value pairs.

[0037] In step S103, the server parses the target HTTP message and splits it into two parts: a message header and a message body. The message header contains the metadata of the request, such as the request method, URL, protocol version, header field, etc.; the message body contains the actual content of the request, such as parameters, files, etc. The server can use the HTTP protocol parsing library or a parser written by itself to complete this splitting process. The server extracts parameters and values ​​from the message header and message body according to the HTTP protocol specification. In the message header, the parameters usually appear in the form of header fields, such as Host, Cookie, Authorization, etc., and the server can directly read the values ​​of these fields; in the URL, the parameters usually appear in the form of a query string, and the server parses the query string into a key-value pair; in the message body, the format of the parameters depends on the design of the API, and there are currently two main forms: RESTful and SOAP. For the RESTful form, the message body is usually in JSON format, and the server can use a JSON parser to convert it into a key-value pair; for the SOAP form, the message body is usually in XML format, and the server can use an XML parser to convert it into a key-value pair.

[0038] In the process of extracting parameters and values, the server will also handle some special cases. For example, some HTTP messages may not contain parameters, such as the absence of a query string in the URL of a GET request. The server can identify these cases and not make unnecessary parsing attempts; some HTTP messages may contain unstructured content, such as pictures, videos, binary files, etc. The server can distinguish these contents and skip extracting their parameters.

[0039] After extracting all parameters and values, the server combines them into parameter-value pairs for easy processing and transmission. Parameter-value pairs are usually expressed in the form of key-value pairs, such as "key1=value1, key2=value2". The server uses the extracted parameters as keys and the corresponding values ​​as values ​​to construct a set of parameter-value pairs.

[0040] Step S104: storing the parameter value pairs in column format to obtain structured storage data.

[0041] In step S104, the parameter value pairs are stored in columnar form to obtain structured storage data, specifically including: determining the data type corresponding to each parameter value pair, the data type including text, number, Boolean value and single value; according to the data type, determining the corresponding data encoding rule in a preset encoding database, the preset encoding database including the correspondence between the data type and the data encoding rule; according to the data encoding rule, encoding each parameter value pair, and storing the encoded parameter value pairs in Arrow memory format to obtain structured storage data.

[0042] Specifically, the server first determines the data type of the value in each parameter value pair. Data types include text (string), number (integer, floating point number), Boolean value (true / false), and single value (enumeration). The server can automatically determine the data type of each value based on the format, length, range and other characteristics of the value using regular expressions. For example, "123" can be identified as a number type, "true" can be identified as a Boolean type, and "red" can be identified as a single value type. The server searches for the corresponding encoding rule in the preset encoding database based on the determined data type. The preset encoding database is a predefined mapping table that establishes a correspondence between data types and encoding rules. Different data types may use different encoding methods for more efficient storage and calculation. For example, the text type can use dictionary encoding to map repeated strings to integer numbers; the number type can use Delta encoding to store only the difference between two adjacent numbers; the Boolean type can use Bitmap encoding to represent a Boolean value with one bit; the single value type can use integer encoding to map enumeration values ​​to integers.

[0043] The server encodes the value in each parameter value pair according to the determined encoding rules. The encoding process converts the original text format value into a more compact binary format, reducing the data storage space and improving the data processing efficiency. At the same time, the encoded data is more suitable for columnar storage and query. Values ​​of the same data type can be stored together continuously, which is convenient for batch reading and calculation.

[0044] The server stores the encoded parameter value pairs in columns, that is, all values ​​of the same parameter are stored in a continuous memory area, and the values ​​of different parameters are stored in different columns. Arrow memory format is a columnar storage format that provides standardized memory layout and metadata definition. The server uses Arrow format to package and store the encoded parameter value pairs to obtain the final structured storage data.

[0045] Step S105: According to the preset stream computing trigger condition, it is determined whether the target HTTP message needs to be stream computed based on the structured storage data.

[0046] In step S105, according to the preset streaming computing trigger conditions, it is determined based on the structured storage data whether the target HTTP message needs to be stream-computed, specifically including: obtaining the preset streaming computing trigger conditions, the preset streaming computing trigger conditions include API call conditions and computing configuration parameters; matching the target HTTP message with the API call conditions; if it is determined that the target HTTP message matches the API call conditions, then determining the streaming computing logic for the target HTTP message according to the computing configuration parameters, and determining to perform streaming computing on the target HTTP message.

[0047] Specifically, the server obtains the preset stream computing trigger conditions. The preset stream computing trigger conditions define the circumstances under which stream computing is required for HTTP messages and how to perform the computing. The trigger conditions usually include two parts: API call conditions and computing configuration parameters. The API call conditions specify the characteristics of the API request to trigger the stream computing, such as the URL path, request method, parameter values, etc.; the computing configuration parameters specify how to perform the stream computing for the requests that meet the conditions, such as the computing window size, aggregation function, threshold conditions, etc. The server loads these trigger conditions from external sources such as configuration files and databases. The server matches the current target HTTP message with the preset API call conditions. The matching process can be based on various fields in the structured storage data, such as the request path, request method, parameter value, etc. The server can use string matching, regular expressions, wildcards, and other technologies to determine whether the HTTP message meets a certain API call condition. For example, an API call condition may require that the request path starts with " / api / v1 / ", the request method is POST, and contains a parameter named "action" with a value of "login". The server checks the corresponding fields in the HTTP message to determine whether it meets this condition.

[0048] If it is determined that the target HTTP message meets a certain API call condition, the server will determine how to perform streaming calculations on the target HTTP message based on the corresponding computing configuration parameters. Streaming calculations are usually performed on multiple requests within a period of time, so the size of the calculation window needs to be set, such as 10 seconds, 1 minute, etc. In each window, the server aggregates and analyzes the collected requests in real time, such as counting the number of requests, calculating the average response time, detecting abnormal parameters, etc. Streaming calculations can be implemented through streaming computing engines (such as Apache Flink, Spark Streaming, etc.). The server can select the corresponding computing logic and operators based on the computing configuration parameters.

[0049] The server will eventually determine whether to perform streaming calculation on the target HTTP message based on the matching results and streaming calculation logic. If the message meets a certain API call condition and the corresponding calculation configuration parameters are valid, the server will add the message to the input queue of the streaming calculation and start or update the corresponding calculation task. If the message does not meet any API call condition, or although it meets the condition but the calculation configuration parameters are invalid, the server will skip the message and not perform streaming calculation on it.

[0050] For example, assume that the following preset stream computing trigger conditions are set inside the server: API call condition 1: the request path is " / api / v1 / login" and the request method is POST; computing configuration parameter 1: the window size is 1 minute, the number of requests is counted, and an alarm is triggered if the number exceeds 100.

[0051] When the server receives an HTTP message with a request path of " / api / v1 / login" and a request method of POST, it first determines whether the message meets API call condition 1, and then determines the stream computing logic as "count the number of requests every minute, and issue an alarm if the number exceeds 100" based on computing configuration parameter 1, so it adds the message to the corresponding stream computing task.

[0052] Step S106: If it is determined that the target HTTP message needs to be stream-calculated, the stream-calculated is performed on the target HTTP message to obtain a calculation result.

[0053] In step S106, if it is determined that the target HTTP message needs to perform streaming calculation, streaming calculation is performed on the target HTTP message to obtain a calculation result, which specifically includes: determining the type of streaming calculation that needs to be performed on the target HTTP message according to the calculation configuration parameters, the streaming calculation types include statistical streaming calculation and feature streaming calculation; if it is determined that the target HTTP message needs to perform statistical streaming calculation, then according to the preset statistical rules, statistical streaming calculation is performed on the target HTTP message; if it is determined that the target HTTP message needs to perform feature streaming calculation, then according to the preset feature extraction rules, feature streaming calculation is performed on the target HTTP message.

[0054] Specifically, the server first determines the type of streaming computing that needs to be performed on the target HTTP message based on the computing configuration parameters. Streaming computing types are mainly divided into two categories: statistical streaming computing and feature streaming computing. Statistical streaming computing focuses on summarizing and aggregating multiple requests within a period of time, such as calculating the number of requests, average response time, etc.; feature streaming computing focuses on analyzing and extracting a single request to prepare data for subsequent machine learning predictions, such as extracting time series feature vectors. The server decides which type of streaming computing to perform by reading the calculation type field in the configuration parameters.

[0055] If it is determined that the target HTTP message needs to perform statistical streaming calculations, the server will calculate and update the message in real time according to the preset statistical rules. The preset statistical rules define how to summarize and analyze a group of requests, including trigger conditions, state definitions, calculation methods, expiration policies, etc. For example, a statistical HLL (HyperLogLog) rule can be described as: when the requested API is equal to x, perform HLL calculation on the password field in the request; if the corresponding HLL state does not exist, create the state and set the TTL; if the state already exists, merge the new HLL value into the state; when the TTL of the state expires, automatically delete the state. The server will calculate and merge the data of the current request with the state in real time according to the statistical rules, and return the calculation results.

[0056] If it is determined that the target HTTP message needs to perform feature-based streaming calculations, the server performs real-time feature calculations and storage on the target HTTP message according to the preset feature extraction rules. The preset feature extraction rules define how to obtain time series features from requests, including trigger conditions, feature definitions, calculation methods, output conditions, etc. For example, a time series feature calculation rule can be described as: when the requested API is equal to x, extract the timestamp and key parameters of the request according to the feature definition; if the corresponding time series feature state does not exist, create the state and set the TTL and output conditions; if the state already exists, update the new feature data to the state; when the state meets the output conditions (such as 10 times cumulatively) or the TTL expires, trigger the machine learning model prediction and clean up the state. The server will calculate and store the current request data in real time according to the feature rules, and trigger the model prediction at the appropriate time.

[0057] After completing the streaming calculation of the target HTTP message, the server will process and return the calculation results as needed. For statistical streaming calculations, the results are usually some aggregate indicators or Boolean conditions. The server can compare them with preset thresholds or rules to determine whether to trigger an alarm or block the request; for feature streaming calculations, the result is usually a time series feature vector, which the server will pass to the downstream machine learning model for real-time prediction and decide whether to keep or remove the request based on the prediction results (such as anomaly scores). The server performs subsequent processing and response based on the configuration and results of the streaming calculation.

[0058] For example, the server receives an HTTP request with the API " / login", which contains two parameters: username and password. According to the calculation configuration parameters, this request needs to perform two types of streaming calculations: statistical streaming calculations, which count whether the number of requests for the API in the last minute exceeds the threshold (such as 100 times); feature streaming calculations, which extract the timestamp, username, password and other parameters of the API request in the last hour to build a time series feature vector; the server first performs statistical streaming calculations, merges and updates the arrival time of the HTTP request with the corresponding HLL state, and determines whether the merged HLL estimate exceeds 100. If it exceeds, the request frequency is considered abnormal and the request may be blocked later; if it does not exceed, the request is considered normal and can continue to be processed. Then, the server performs feature streaming calculations, extracts the key parameters of the HTTP request, and appends them to the corresponding feature state in a time series manner. If the state has accumulated 1 hour of feature data, the entire feature vector is passed to the anomaly detection model, which determines whether the request is abnormal. The server decides whether to issue an alarm or intercept based on the anomaly score of the model. Finally, the server will make a comprehensive judgment based on multiple results of streaming computing, such as statistical indicators, anomaly scores, etc., according to the rule engine or HTTP routing configuration. If the request meets a certain combination of conditions, such as HLL exceeding 100 and anomaly score greater than 0.9, the request will be marked as abnormal and intercepted; otherwise, the server will forward the request to the backend service for processing according to the normal API routing rules.

[0059] In one possible implementation, according to preset statistical rules, statistical streaming calculations are performed on the target HTTP message, specifically including: obtaining the state definition in the calculation configuration parameters, the state definition including the state identifier, the state initial value and the life cycle; based on the structured storage data of the target HTTP message, according to the statistical rules in the calculation configuration parameters, calculating the statistical results; if it is determined that there is no state corresponding to the state identifier, creating a target state corresponding to the state identifier, and using the statistical result as the state initial value of the target state, and setting the life cycle of the target state to the survival time of the target state; if it is determined that there is already a target state corresponding to the state identifier, the statistical result and the current value of the target state are aggregated to obtain the calculation result, and the calculation result is updated to the current value of the target state; if it is determined that the survival time of the target state has expired, deleting the target state.

[0060] Specifically, the server first obtains the state definition from the calculation configuration parameters. The state definition includes three elements: state identifier, state initial value, and life cycle. The state identifier is used to uniquely identify a state; the state initial value represents the starting value when the state is created, such as 0; the life cycle represents the survival time of the state, such as 1 minute. The server calculates the statistical results based on the structured storage data of the target HTTP message and the statistical rules in the calculation configuration parameters. The statistical rules define how to aggregate the request data, such as accumulating the number of requests and taking the average of the request response time. The server extracts the fields required by the statistical rules from the structured storage data, and applies the corresponding aggregation function to obtain a statistical value as the statistical result.

[0061] Next, the server determines whether the corresponding state already exists based on the state identifier. If the state does not exist, the server creates a new target state and uses the statistical result as the initial value of the target state, and sets the life cycle of the target state to the survival time specified by the configuration parameters; if the target state already exists, the server aggregates the statistical result with the current value of the target state (such as accumulation) to obtain a new aggregate value, and updates it to the current value of the state. In this way, the server can maintain a continuously updated state value between multiple requests. At the same time, the server will periodically check the survival time of each state. If it is found that the survival time of a state has exceeded the life cycle set by the calculation configuration parameters, the state will be automatically deleted.

[0062] In one possible implementation, feature-based streaming computing is performed on a target HTTP message according to preset feature extraction rules, specifically including: obtaining feature extraction rules and feature computing trigger conditions in computing configuration parameters; extracting feature parameters from structured storage data of the target HTTP message according to the feature extraction rules; determining whether feature computing trigger conditions are met, the feature computing trigger conditions including time trigger, data volume trigger, event trigger, and external trigger; if it is determined that the feature computing trigger conditions are met, inputting the feature parameters into a preset machine learning model, and using the output result of the preset machine learning model as the feature computing result; if it is determined that the feature computing trigger conditions are not met, temporarily storing the feature parameters until the feature computing trigger conditions are met, and inputting the temporarily stored feature parameters into the preset machine learning model.

[0063] Specifically, the server first obtains the feature extraction rules and feature calculation trigger conditions from the calculation configuration parameters. The feature extraction rules define how to extract feature parameters from the structured storage data of the HTTP message, such as extracting the URL path, request body size, user agent, etc.; the feature calculation trigger conditions define when to trigger the machine learning model to calculate, and the triggering methods can include time triggering, data volume triggering, event triggering, and external triggering. The server obtains two types of rules at the same time to perform complete feature-based streaming calculations.

[0064] The server extracts a set of feature parameters from the structured storage data of the target HTTP message according to the feature extraction rules. For example, the feature extraction rules can define the extraction of the request URL path, request body size, user agent and other fields. The server will search for these fields in the structured storage data and extract their values ​​to form a feature vector. This feature vector represents the feature representation of the HTTP message in a specific dimension and is the input of the subsequent machine learning model.

[0065] The server determines whether the trigger conditions for feature calculation are currently met. The trigger conditions can be based on time, data volume, events, external factors, etc., such as time trigger: trigger calculation every 1 minute; data volume trigger: trigger calculation once every 100 requests; event trigger: trigger calculation when an abnormal request occurs; external trigger: trigger calculation when a trigger signal from an external system is received.

[0066] The server monitors these trigger conditions in real time. When any one of the conditions is met, it determines that the feature calculation is triggered. If the feature calculation trigger condition is met, the server inputs the extracted feature parameters into the preset machine learning model and obtains the output result of the preset machine learning model as the feature calculation result. The machine learning model can be of various types such as anomaly detection, user profiling, and behavior prediction. According to the specific business scenario and model type, the server passes the feature vector to the corresponding preset machine learning model for inference calculation to obtain a value or vector representing the calculation result, such as anomaly score, user label, behavior probability, etc.

[0067] If the feature calculation trigger condition is not met, the server will not immediately execute the machine learning model, but will temporarily store the extracted feature parameters and continue to process the next HTTP message. The temporarily stored feature parameters will continue to accumulate until the trigger condition is met.

[0068] Step S107: Determine the processing result for the target HTTP message according to the HTTP routing setting and the calculation result, the processing result includes releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

[0069] In step S107, the server makes a final processing decision on the target HTTP message based on the HTTP routing settings and the streaming calculation results, determining whether to release and forward the message, or to block the message and return a corresponding response. The decision-making process comprehensively considers the routing attributes and calculation indicators of the message, reflecting the intelligent protection concept of combining WAF and risk control engine. Specifically, it can be divided into the following key steps: The server obtains pre-configured HTTP routing settings, which define the mapping relationship between different URL paths and backend servers, as well as the security policies and forwarding rules of each path. By identifying the URL path of the target HTTP message, the server can quickly determine the routing destination and processing method of the message. HTTP routing settings are the basic configuration of WAF and determine the basic flow of messages. The server obtains the streaming calculation results for the target HTTP message from step S106. These results include statistical indicators (such as number of requests, abnormality ratio, etc.) and feature indicators (such as abnormality score, risk level, etc.). The server conducts a comprehensive analysis of the HTTP routing settings and streaming calculation results to determine the final processing method of the target HTTP message. The analysis process is based on a series of preset rules and thresholds, such as: If the URL path of the message matches a high-risk route and the anomaly score exceeds the preset score threshold, it is judged as a malicious request and needs to be blocked. If the URL path of the message matches a sensitive route and the number of requests exceeds the preset number threshold, it is judged as an abnormal request and an alarm is required. If the URL path of the message matches a common route and all indicators are within the normal range, it is judged as a normal request and needs to be released.

[0070] The server can flexibly combine HTTP routing settings and calculation indicators to form multi-dimensional judgment conditions, which can cope with various complex attack scenarios. This dual judgment based on routing and data greatly improves the detection accuracy and protection effect of WAF.

[0071] According to the results of the comprehensive analysis, the server performs corresponding processing operations on the target HTTP message: If the judgment result is release, the server forwards the message to the target server specified in the routing settings to complete the normal business request process. If the judgment result is blocking, the server will refuse to forward the message and return a custom blocking response to the requester, such as "403 Forbidden", and record the detailed information of the blocking event for subsequent security audits.

[0072] After step S107, the method further includes: if it is determined that the processing result is to release and forward the target HTTP message, then forward the target HTTP message to the target server, and send the response message returned by the target server to the original requester; if it is determined that the processing result is to block the target HTTP message, then return a preset blocking response message to the original requester.

[0073] Specifically, if the processing result is to release and forward the target HTTP message, it means that the message has passed the multiple checks of WAF and risk control engine, and is judged as a normal request, and needs to be forwarded to the backend target server for business processing. At this time, the server will forward the target HTTP message to the target server specified in the routing settings, which is usually the backend business server connected to WAF through the internal network or dedicated line. The forwarding process follows the standard process of the HTTP protocol. The server acts as a proxy to pass the client's request to the target server as is, and waits for the target server to return a response message. When the server receives the response message returned by the target server, it will perform necessary security checks on the response content again, such as filtering sensitive information, to ensure that possible risks are not passed to the client. After the check is completed, the server sends the response message to the requester through the original connection, completing a complete HTTP request-response process.

[0074] If the processing result is to block the target HTTP message, it means that the message has not passed the inspection of WAF and risk control engine, and is judged as a malicious request or abnormal request, which needs to be directly rejected and no longer forwarded to the backend server. At this time, the server will directly return a preset blocking response message to the original requester, informing the requester that the request has been rejected by the security policy.

[0075] Reference Figure 2 The present application also provides an API security protection device based on HTTP message streaming processing, the device is a server, the server includes a receiving module 201 and a processing module 202, wherein: the receiving module 201 is used to receive TCP messages by monitoring a preset network port or a preset protocol, and reassemble the TCP messages to obtain HTTP messages; the processing module 202 is used to parse the HTTP messages to obtain message headers and message bodies, and determine the target HTTP messages related to the API call according to the message headers and message bodies; the processing module 202 is also used to extract multiple parameters and values ​​corresponding to each parameter from the target HTTP message to form a parameter value The processing module 202 is also used to store the parameter value pairs in columnar form to obtain structured storage data; the processing module 202 is also used to determine whether the target HTTP message needs to be stream-calculated based on the structured storage data according to the preset streaming computing trigger condition; the processing module 202 is also used to perform streaming computing on the target HTTP message to obtain a calculation result if it is determined that the target HTTP message needs to be stream-calculated; the processing module 202 is also used to determine the processing result for the target HTTP message according to the HTTP routing setting and the calculation result, and the processing result includes releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

[0076] In a possible implementation, the processing module 202 stores the parameter value pairs in columnar form to obtain structured storage data, specifically including: the processing module 202 determines the data type corresponding to each parameter value pair, and the data type includes text, number, Boolean value and single value; the processing module 202 determines the corresponding data encoding rule in a preset encoding database according to the data type, and the preset encoding database includes the correspondence between the data type and the data encoding rule; the processing module 202 encodes each parameter value pair according to the data encoding rule, and stores the encoded parameter value pairs in Arrow memory format to obtain structured storage data.

[0077] In one possible implementation, the processing module 202 determines whether the target HTTP message needs to be streamed based on the preset streaming computing trigger condition and the structured storage data, specifically including: the processing module 202 obtains the preset streaming trigger condition, and the preset streaming trigger condition includes an API call condition and a computing configuration parameter; the processing module 202 matches the target HTTP message with the API call condition; if the processing module 202 determines that the target HTTP message matches the API call condition, it determines the streaming computing logic for the target HTTP message according to the computing configuration parameters, and determines to perform streaming computing on the target HTTP message.

[0078] In one possible implementation, if the processing module 202 determines that the target HTTP message needs to be streamed, then the streaming calculation is performed on the target HTTP message to obtain a calculation result, which specifically includes: the processing module 202 determines the type of streaming calculation that needs to be performed on the target HTTP message according to the calculation configuration parameters, and the streaming calculation types include statistical streaming calculation and feature streaming calculation; if the processing module 202 determines that the target HTTP message needs to perform statistical streaming calculation, then according to the preset statistical rules, the statistical streaming calculation is performed on the target HTTP message; if the processing module 202 determines that the target HTTP message needs to perform feature streaming calculation, then according to the preset feature extraction rules, the feature streaming calculation is performed on the target HTTP message.

[0079] In one possible implementation, the processing module 202 calculates configuration parameters including state definitions and statistical rules, and performs statistical streaming calculations on the target HTTP message according to preset statistical rules, specifically including: the processing module 202 obtains the state definition in the calculation configuration parameters, the state definition including the state identifier, the state initial value and the life cycle; based on the structured storage data of the target HTTP message, the statistical results are calculated according to the statistical rules in the calculation configuration parameters; if the processing module 202 determines that there is no state corresponding to the state identifier, it creates a target state corresponding to the state identifier, uses the statistical result as the state initial value of the target state, and sets the life cycle of the target state to the survival time of the target state; if the processing module 202 determines that there is already a target state corresponding to the state identifier, it aggregates the statistical result with the current value of the target state to obtain the calculation result, and updates the calculation result to the current value of the target state; if the processing module 202 determines that the survival time of the target state has expired, it deletes the target state.

[0080] In one possible implementation, the calculation configuration parameters include feature extraction rules and feature calculation trigger conditions, and the processing module 202 performs feature-type streaming calculations on the target HTTP message according to the preset feature extraction rules, specifically including: the processing module 202 obtains the feature extraction rules and feature calculation trigger conditions in the calculation configuration parameters; the processing module 202 extracts feature parameters from the structured storage data of the target HTTP message according to the feature extraction rules; the processing module 202 determines whether the feature calculation trigger conditions are met, and the feature calculation trigger conditions include time trigger, data volume trigger, event trigger and external trigger; if the processing module 202 determines that the feature calculation trigger conditions are met, the feature parameters are input into the preset machine learning model, and the output result of the preset machine learning model is used as the feature calculation result; if the processing module 202 determines that the feature calculation trigger conditions are not met, the feature parameters are temporarily stored until the feature calculation trigger conditions are met, and the temporarily stored feature parameters are input into the preset machine learning model.

[0081] In a possible implementation, after the processing module 202 determines the processing result for the target HTTP message based on the HTTP routing settings and calculation results, the method also includes: if the processing module 202 determines that the processing result is to release and forward the target HTTP message, the target HTTP message is forwarded to the target server, and the response message returned by the target server is sent to the original requester; if the processing module 202 determines that the processing result is to block the target HTTP message, a preset blocking response message is returned to the original requester.

[0082] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0083] The present application also provides an electronic device. Figure 3 , Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0084] The communication bus 302 is used to realize the connection and communication between these components.

[0085] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0086] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0087] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0088] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally also be at least one storage device located away from the aforementioned processor 301. Refer to Figure 3, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of an API security protection method based on HTTP message streaming processing.

[0089] exist Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program stored in the memory 305 that stores an API security protection method based on HTTP message streaming processing. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0090] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments.

[0091] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0093] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0094] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0095] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0096] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0097] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An API security protection method based on HTTP message streaming processing, characterized in that: The method comprises: By monitoring a preset network port or a preset protocol, a TCP message is received, and the TCP message is reassembled to obtain an HTTP message; Parsing the HTTP message to obtain a message header and a message body, and determining a target HTTP message related to the API call according to the message header and the message body; Extracting multiple parameters and values ​​corresponding to each of the parameters from the target HTTP message to form parameter-value pairs; Column-storing the parameter-value pairs to obtain structured storage data; According to a preset stream computing trigger condition, judging whether the target HTTP message needs to be stream computed based on the structured storage data; If it is determined that the target HTTP message needs to be stream-calculated, the stream-calculated is performed on the target HTTP message to obtain a calculation result; According to the HTTP routing setting and the calculation result, a processing result for the target HTTP message is determined, and the processing result includes releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

2. The method according to claim 1, characterized in that The column-type storage of the parameter-value pairs to obtain structured storage data specifically includes: Determine the data type corresponding to each of the parameter value pairs, the data type including text, number, Boolean value and single value; According to the data type, determining a corresponding data encoding rule in a preset encoding database, wherein the preset encoding database includes a correspondence between the data type and the data encoding rule; According to the data encoding rule, each of the parameter value pairs is encoded, and the encoded parameter value pairs are stored in Arrow memory format to obtain the structured storage data.

3. The method according to claim 1, characterized in that The step of determining whether the target HTTP message needs to be stream-computed based on the structured storage data according to the preset stream-compute trigger condition specifically includes: Acquire the preset stream computing trigger condition, where the preset stream computing trigger condition includes an API call condition and a computing configuration parameter; Matching the target HTTP message with the API call condition; If it is determined that the target HTTP message matches the API call condition, the stream computing logic for the target HTTP message is determined according to the computing configuration parameters, and it is determined to perform stream computing on the target HTTP message.

4. The method according to claim 3, characterized in that If it is determined that the target HTTP message needs to be stream-calculated, the stream-calculated is performed on the target HTTP message to obtain a calculation result, which specifically includes: Determine the stream computing type that needs to be executed for the target HTTP message according to the computing configuration parameters, where the stream computing type includes statistical stream computing and feature stream computing; If it is determined that the target HTTP message needs to perform statistical stream computing, then according to a preset statistical rule, perform statistical stream computing on the target HTTP message; If it is determined that the target HTTP message needs to perform feature-based streaming calculation, the feature-based streaming calculation is performed on the target HTTP message according to a preset feature extraction rule.

5. The method according to claim 4, characterized in that The calculation configuration parameters include state definitions and statistical rules. The statistical stream calculation is performed on the target HTTP message according to the preset statistical rules, specifically including: Obtaining a state definition in the computing configuration parameters, wherein the state definition includes a state identifier, a state initial value, and a life cycle; Based on the structured storage data of the target HTTP message, according to the statistical rules in the calculation configuration parameters, calculate the statistical results; If it is determined that there is no state corresponding to the state identifier, create a target state corresponding to the state identifier, use the statistical result as the state initial value of the target state, and set the life cycle of the target state to the survival time of the target state; If it is determined that a target state corresponding to the state identifier already exists, performing an aggregation operation on the statistical result and the current value of the target state to obtain an operation result, and updating the operation result as the current value of the target state; If it is determined that the survival time of the target state has expired, the target state is deleted.

6. The method according to claim 4, characterized in that The calculation configuration parameters include feature extraction rules and feature calculation trigger conditions. The feature-based streaming calculation is performed on the target HTTP message according to the preset feature extraction rules, specifically including: Acquire feature extraction rules and feature calculation triggering conditions in the calculation configuration parameters; Extracting characteristic parameters from the structured storage data of the target HTTP message according to the characteristic extraction rule; Determine whether the feature calculation trigger condition is met, the feature calculation trigger condition includes time trigger, data volume trigger, event trigger and external trigger; If it is determined that the feature calculation trigger condition is met, the feature parameters are input into a preset machine learning model, and the output result of the preset machine learning model is used as the feature calculation result; If it is determined that the feature calculation trigger condition is not met, the feature parameters are temporarily stored until the feature calculation trigger condition is met, and the temporarily stored feature parameters are input into the preset machine learning model.

7. The method according to claim 1, characterized in that After determining the processing result for the target HTTP message according to the HTTP routing setting and the calculation result, the method further includes: If it is determined that the processing result is to release and forward the target HTTP message, forward the target HTTP message to the target server, and send the response message returned by the target server to the original requester; If it is determined that the processing result is to block the target HTTP message, a preset blocking response message is returned to the original requester.

8. An API security protection device based on HTTP message streaming processing, characterized in that: The device comprises a receiving module (201) and a processing module (202), wherein: The receiving module (201) is used to receive TCP messages by monitoring a preset network port or a preset protocol, and to reassemble the TCP messages to obtain HTTP messages; The processing module (202) is used to parse the HTTP message to obtain a message header and a message body, and determine a target HTTP message related to the API call according to the message header and the message body; The processing module (202) is further used to extract multiple parameters and values ​​corresponding to each parameter from the target HTTP message to form parameter value pairs; The processing module (202) is further used to store the parameter value pairs in column format to obtain structured storage data; The processing module (202) is further used to determine whether the target HTTP message needs to be stream-computed based on the structured storage data according to a preset stream-compute trigger condition; The processing module (202) is further configured to, if it is determined that the target HTTP message needs to be stream-calculated, perform stream-calculation on the target HTTP message to obtain a calculation result; The processing module (202) is further used to determine a processing result for the target HTTP message according to the HTTP routing setting and the calculation result, wherein the processing result includes releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

9. An electronic device, characterized in that: The electronic device (300) comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images