An API security protection method, device and equipment based on HTTP message streaming processing

Abstract

An API security protection method, device and equipment based on HTTP message streaming processing, which relates to the field of data processing. In this method, TCP messages are reorganized to obtain HTTP messages; the target HTTP messages related to API calls are determined; multiple parameters and the values corresponding to each parameter are extracted from the target HTTP messages to form parameter-value pairs; the parameter-value pairs are stored in a columnar manner to obtain structured storage data; based on the structured storage data, it is judged whether the target HTTP messages need to perform streaming calculation; the target HTTP messages are subjected to streaming calculation to obtain calculation results; according to the HTTP routing setting and the calculation results, the processing results for the target HTTP messages are determined, including releasing and forwarding the target HTTP messages or blocking the target HTTP messages and returning responses. Implementing this technical solution avoids security risks caused by delayed processing and improves the real-time performance of security protection.

Description

Technical Field

[0001] The present application relates to the field of data processing, and particularly to an API security protection method, device, and equipment based on HTTP message streaming processing. Background Art

[0002] With the rapid development of the Internet, APIs (Application Programming Interfaces) have become an indispensable part of modern software architectures. APIs allow efficient data exchange and function integration between various applications. However, the widespread use of APIs has also introduced numerous security risks, especially security issues on the HTTP protocol have become a key challenge.

[0003] Currently, traditional network security tools, such as WAF (Web Application Firewall) and big data risk control systems, although providing a certain degree of protection for API security, still have some key deficiencies. WAF is a security tool based on HTTP protocol proxy and content detection engine, widely used in the market to protect Web applications from attacks. And WAF mainly relies on static rules and signature matching. Although this method can immediately intercept known attack patterns, in the face of complex attacks that require real-time data analysis and decision-making, this static and immediate processing method cannot effectively respond. And big data risk control systems usually process data after collection, relying on batch processing or intermittent streaming computing. This results in the processing and response not being synchronized with data collection, and the processing results are often available a long time after the data is generated, unable to make an immediate response to the events that occur. Therefore, traditional network security tools have the problem of insufficient real-time performance in security protection processing.

[0004] Therefore, there is an urgent need for an API security protection method, device, and equipment based on HTTP message streaming processing. Summary of the Invention

[0005] The present application provides an API security protection method, device, and equipment based on HTTP message streaming processing, avoiding security risks caused by delayed processing and improving the real-time performance of security protection.

[0006] In the first aspect of the present application, an API security protection method based on HTTP message streaming processing is provided. In this method, by listening to a preset network port or a preset protocol, TCP messages are received, and the TCP messages are reorganized to obtain HTTP messages; the HTTP messages are parsed to obtain a message header and a message body, and a target HTTP message related to API calls is determined according to the message header and the message body; multiple parameters and the values corresponding to each of the parameters are extracted from the target HTTP message to form parameter-value pairs; the parameter-value pairs are stored in a columnar manner to obtain structured storage data; according to a preset streaming calculation trigger condition, it is determined whether the target HTTP message needs to perform streaming calculation based on the structured storage data; if it is determined that the target HTTP message needs to perform streaming calculation, the target HTTP message is subjected to streaming calculation to obtain a calculation result; according to the HTTP routing setting and the calculation result, a processing result for the target HTTP message is determined, and the processing result includes releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

[0007] By adopting the above technical solution, by listening to a preset network port or a preset protocol to receive TCP messages and reorganizing them to obtain HTTP messages, complete message data at the application layer can be obtained, providing input for subsequent processing. Then, the HTTP messages are parsed to extract the message header and the message body, and a target HTTP message related to API calls is determined according to the information in the message header and the message body, which can accurately locate the key messages that need security protection and improve the pertinence of processing. Next, parameters and parameter values are extracted from the target HTTP message to form parameter-value pairs, and the parameter-value pairs are converted into structured storage data by using columnar storage, which is convenient for subsequent streaming calculation and analysis. According to the preset streaming calculation trigger condition, it is determined whether the target HTTP message needs to perform streaming calculation, which can selectively perform calculations on specific messages and improve the calculation efficiency. Performing streaming calculation on the target HTTP message that needs to be calculated can obtain statistical results or feature results in real time and quickly discover anomalies or threats. Finally, according to the HTTP routing setting and the calculation result, the processing result of the message is determined, which can dynamically release, forward or block the message, realizing refined security protection measures. This method takes HTTP messages as the processing object and constructs a complete API security protection process, which can effectively identify and intercept malicious requests and attacks against APIs and enhance the system's ability to resist risks. API security protection not only improves the recognition and response speed to complex attack patterns, but also improves the intelligence and accuracy of security policies. In addition, this solution effectively avoids security risks caused by delayed processing through real-time analysis and response, improving the real-time nature of security processing.

[0008] Optionally, the columnar storage of the parameter value pairs to obtain structured storage data specifically includes: determining the data types corresponding to each of the parameter value pairs, where the data types include text, number, boolean, and single value; determining the corresponding data encoding rules in a preset encoding database according to the data types, where the preset encoding database includes the correspondence between the data types and the data encoding rules; encoding each of the parameter value pairs according to the data encoding rules, and storing the encoded parameter value pairs in the Arrow memory format to obtain the structured storage data.

[0009] By adopting the above technical solution, when performing columnar storage on parameter value pairs, first determine the data types of each parameter value pair, which can identify different types of data, such as text, number, boolean, etc., providing a basis for subsequent encoding processing. Then, look up the encoding rules corresponding to the data types in the preset encoding database, and select an appropriate encoding method according to the characteristics of the data to improve the efficiency of storage and calculation. Next, encode the parameter value pairs according to the determined encoding rules, convert them into a format suitable for storage and calculation, and store them in the Arrow memory format to form structured storage data. By adopting the Arrow format, the reading, writing, and transmission of data can be accelerated, the overhead of data conversion can be reduced, and the processing efficiency of the entire method can be improved. At the same time, columnar storage can facilitate operations such as aggregation and filtering on specific columns to meet the requirements of streaming computing. Converting the parameter value pairs into high-performance structured storage data provides efficient data support for subsequent streaming computing, and can quickly retrieve and analyze massive HTTP message data to determine suspicious behaviors and security events in a timely manner.

[0010] Optionally, the step of determining whether the target HTTP message needs to be subjected to streaming calculation based on the structured storage data according to a preset streaming calculation trigger condition specifically includes: obtaining the preset streaming calculation trigger condition, where the preset streaming calculation trigger condition includes an API call condition and calculation configuration parameters; matching the target HTTP message with the API call condition; if it is determined that the target HTTP message matches the API call condition, then determine the streaming calculation logic for the target HTTP message according to the calculation configuration parameters, and determine to perform streaming calculation on the target HTTP message.

[0011] By adopting the above technical solutions, when determining whether a target HTTP message needs to be subjected to streaming calculation, the preset streaming calculation trigger conditions are first obtained, including API call conditions and calculation configuration parameters, which clarify the basis for triggering streaming calculation. Matching the target HTTP message with the API call conditions can quickly determine whether the target HTTP message needs to be subjected to streaming calculation, avoiding unnecessary processing of irrelevant messages and improving the overall processing efficiency and accuracy. If the target HTTP message meets the API call conditions, the specific streaming calculation logic for this message is further determined according to the calculation configuration parameters, implementing a customized calculation strategy. By setting flexible streaming calculation trigger conditions and calculation configuration parameters, selective streaming processing and analysis of API calls can be performed to timely detect abnormal behaviors and security threats.

[0012] Optionally, if it is determined that the target HTTP message needs to be subjected to streaming calculation, then the streaming calculation is performed on the target HTTP message to obtain a calculation result, which specifically includes: determining the type of streaming calculation that the target HTTP message needs to perform according to the calculation configuration parameters, and the type of streaming calculation includes statistical streaming calculation and feature streaming calculation; if it is determined that the target HTTP message needs to perform statistical streaming calculation, then according to the preset statistical rules, the statistical streaming calculation is performed on the target HTTP message; if it is determined that the target HTTP message needs to perform feature streaming calculation, then according to the preset feature extraction rules, the feature streaming calculation is performed on the target HTTP message.

[0013] By adopting the above technical solutions, when performing streaming calculation on a target HTTP message, the type of streaming calculation that needs to be performed is first determined according to the calculation configuration parameters, which are mainly divided into two categories: statistical streaming calculation and feature streaming calculation, so as to meet different analysis requirements. Statistical streaming calculation focuses on summarizing and aggregating multiple requests within a period of time, which can reflect the overall usage situation and performance of the API. Feature streaming calculation focuses on in-depth analysis and extraction of a single request, which can discover abnormal features and risks of a single request. According to the different calculation types, the preset statistical rules or preset feature extraction rules are respectively used for calculation, which can adapt to the analysis requirements of different types of data and improve the pertinence and accuracy of the calculation. By adopting different rules and algorithms for different types of streaming calculations, multiple dimensions of the HTTP message can be comprehensively analyzed, potential security problems can be identified from both the overall and individual levels, and thus more comprehensive and effective API security protection can be provided.

[0014] Optionally, the computing configuration parameters include a status definition and a statistical rule. Performing statistical streaming computing on the target HTTP message according to the preset statistical rule specifically includes: obtaining the status definition in the computing configuration parameters, where the status definition includes a status identifier, a status initial value, and a lifecycle; based on the structured storage data of the target HTTP message, calculating a statistical result according to the statistical rule in the computing configuration parameters; if it is determined that there is no status corresponding to the status identifier, creating a target status corresponding to the status identifier, using the statistical result as the status initial value of the target status, and setting the lifecycle of the target status to the survival time of the target status; if it is determined that there is already a target status corresponding to the status identifier, performing an aggregation operation on the statistical result and the current value of the target status to obtain an operation result, and updating the operation result to the current value of the target status; if it is determined that the survival time of the target status has expired, deleting the target status.

[0015] By adopting the above technical solution, when performing statistical streaming computing, the concept of status is introduced to store and maintain the cumulative computing results within a period of time. The status definition includes attributes such as a status identifier, a status initial value, and a lifecycle, which can uniquely identify a status, set the initial value, and specify the validity period of the status, facilitating the creation, update, and cleanup of the status. Through the status-based statistical streaming computing, data summarization and aggregation can be continuously performed in the HTTP message stream to obtain statistical results reflecting the cumulative metrics within a period of time, timely detect abnormal situations and security threats in API calls, and at the same time, through the lifecycle management of the status, the consumption of system resources can be controlled, improving the overall performance and stability.

[0016] Optionally, the computing configuration parameters include a feature extraction rule and a feature calculation trigger condition. Performing feature streaming computing on the target HTTP message according to the preset feature extraction rule specifically includes: obtaining the feature extraction rule and the feature calculation trigger condition in the computing configuration parameters; extracting feature parameters from the structured storage data of the target HTTP message according to the feature extraction rule; determining whether the feature calculation trigger condition is met, where the feature calculation trigger condition includes time trigger, data volume trigger, event trigger, and external trigger; if it is determined that the feature calculation trigger condition is met, inputting the feature parameters into a preset machine learning model and using the output result of the preset machine learning model as the feature calculation result; if it is determined that the feature calculation trigger condition is not met, temporarily storing the feature parameters until the feature calculation trigger condition is met, and inputting the temporarily stored feature parameters into the preset machine learning model.

[0017] By adopting the above technical solution, when performing feature-based streaming computing, key features reflecting anomalies or risks are extracted from the target HTTP message, and the trigger conditions for feature calculation are determined according to the features, and feature calculation and analysis are selectively performed. First, obtain the feature extraction rules and feature calculation trigger conditions defined in the calculation configuration parameters to clarify which feature parameters to extract and when to trigger feature calculation. According to the feature extraction rules, relevant feature parameters are extracted from the structured storage data of the HTTP message. The extracted feature parameters can provide a data basis for subsequent anomaly detection and risk judgment. Then, it is judged whether the trigger condition for feature calculation is satisfied, and the timing and frequency of feature calculation are flexibly controlled to improve the timeliness and efficiency of calculation. If the trigger condition is satisfied, the extracted feature parameters are input into a preset machine learning model for calculation, and the preset machine learning model analyzes the feature parameters and performs anomaly detection, and outputs a calculation result reflecting the abnormal degree or risk level of the request, providing a quantitative basis for security protection decisions.

[0018] Optionally, after determining the processing result for the target HTTP message according to the HTTP routing setting and the calculation result, the method further includes: if it is determined that the processing result is to release and forward the target HTTP message, then forward the target HTTP message to the target server, and send the response message returned by the target server to the original requestor; if it is determined that the processing result is to block the target HTTP message, then return a preset blocking response message to the original requestor.

[0019] By adopting the above technical solution, after determining the processing result of the target HTTP message, corresponding subsequent operations are taken according to different results to complete the entire API security protection process. If the processing result is to release and forward the message, the message is forwarded to the target server for normal business processing, and the response message returned by the server is sent to the original requestor to complete the complete HTTP request-response process. By forwarding and responding to normal requests, the availability and continuity of the API service can be ensured, and normal business operations can be maintained. If the processing result is to block the message, the message is no longer forwarded, but a preset blocking response message is directly returned to the original requestor. By timely blocking and responding to abnormal requests, potential attacks and intrusions can be effectively prevented, and the security of the API service can be protected.

[0020] In a second aspect of the present application, an API security protection device based on HTTP message streaming processing is provided. The device includes: a receiving module and a processing module, where: the receiving module is used to receive TCP messages by listening to a preset network port or a preset protocol, and reorganize the TCP messages to obtain HTTP messages; the processing module is used to parse the HTTP messages to obtain a message header and a message body, and determine a target HTTP message related to an API call according to the message header and the message body; the processing module is further used to extract a plurality of parameters and values corresponding to each of the parameters from the target HTTP message to form parameter-value pairs; the processing module is further used to store the parameter-value pairs in a columnar manner to obtain structured stored data; the processing module is further used to determine whether the target HTTP message needs to be subjected to streaming calculation based on the structured stored data according to a preset streaming calculation trigger condition; the processing module is further used to, if it is determined that the target HTTP message needs to be subjected to streaming calculation, perform streaming calculation on the target HTTP message to obtain a calculation result; the processing module is further used to determine a processing result for the target HTTP message according to an HTTP routing setting and the calculation result, and the processing result includes allowing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

[0021] In a third aspect of the present application, an electronic device is provided, which includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. Both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method described in any one of the above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are executed, the method described in any one of the above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0024] 1. This method takes HTTP messages as the processing object and constructs a complete set of API security protection processes, which can effectively identify and intercept malicious requests and attacks against APIs, and improve the system's ability to resist risks. API security protection not only improves the recognition and response speed to complex attack patterns, but also improves the intelligence and accuracy of security policies. In addition, this solution effectively avoids security risks caused by delayed processing through real-time analysis and response, and improves the real-time nature of security processing. Description of the Drawings

[0025] Figure 1 It is a schematic flowchart of an API security protection method based on HTTP message streaming processing disclosed in an embodiment of the present application;

[0026] Figure 2 It is a schematic module diagram of an API security protection device based on HTTP message streaming processing disclosed in an embodiment of the present application;

[0027] Figure 3 It is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.

[0028] Explanation of reference numerals: 201, receiving module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Detailed implementation manners

[0029] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0030] In the description of the embodiments of the present application, words such as "for example" or "for illustration" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for illustration" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of words such as "for example" or "for illustration" is intended to present relevant concepts in a specific manner.

[0031] In the description of the embodiments of the present application, the meaning of the term "a plurality" refers to two or more. For example, a plurality of systems refers to two or more systems, and a plurality of screen terminals refers to two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the technical features indicated. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0032] The present application provides an API security protection method based on HTTP message streaming processing, referring to Figure 1 , Figure 1It is a schematic flowchart of an API security protection method based on HTTP message streaming processing provided by an embodiment of the present application. This method is applied to a server, which is a server that executes a program for real-time processing of network traffic of the HTTP protocol. The server can be a single server, a server cluster composed of multiple servers, or a cloud computing service center. This method includes steps S101 to S107, and the above steps are as follows:

[0033] Step S101: Receive TCP messages by listening on a preset network port or a preset protocol, and reorganize the TCP messages to obtain HTTP messages.

[0034] In step S101, the server receives TCP messages from a client or other servers by listening on a preset network port or a preset protocol, and reorganizes these TCP messages to finally obtain a complete HTTP message. This process can be divided into the following key steps: The server first listens on a specific network port. Usually, the default port used by the HTTP protocol is 80. The server will start one or more listening processes, bind them to the preset port, and wait for connection requests from the client. When the client initiates an HTTP request to the server, a TCP connection will be established first. The client will send a SYN packet to the preset port of the server. After receiving it, the server will reply with a SYN-ACK packet, and the client will then reply with an ACK packet to complete the three-way handshake and establish a TCP connection. The TCP connection provides a reliable data transmission channel for subsequent HTTP communication. After establishing the TCP connection, the client will split the HTTP request into multiple TCP segments according to the TCP protocol specification and send them to the server. The listening process of the server will continuously read these segments from the buffer of the TCP connection and record and manage them according to information such as sequence numbers and lengths.

[0035] Since TCP messages may be out of order, repeated, lost, etc. during network transmission, the TCP segments received by the server may be incomplete or discontinuous. The server reassembles these scattered segments into a complete and ordered data stream according to information such as the sequence numbers and lengths of the TCP segments. This process is called TCP message reorganization. After the TCP message reorganization is completed, the server obtains a complete and continuous data stream. However, this data stream may contain multiple HTTP request or response messages, which are separated by a specific delimiter (such as "\r\n\r\n"). The server identifies and extracts individual HTTP messages from this data stream according to the format specification of the HTTP protocol for subsequent processing.

[0036] For example, when a user accesses a web page in a browser, the browser sends an HTTP GET request to the server. This request is split into multiple TCP segments and sent out. The server listens on port 80. After receiving these TCP segments, it reassembles them according to the sequence numbers to obtain the complete data stream. Then, the server extracts the HTTP GET request message from this data stream.

[0037] Step S102: Parse the HTTP message to obtain the message header and the message body, and determine the target HTTP message related to the API call based on the message header and the message body.

[0038] In step S102, the server parses the received HTTP message and extracts the content of two parts: the message header (Header) and the message body (Body). Then, based on the specific information in the message header and the message body, the server determines whether the HTTP message is related to an API call. If it is related, it is identified as the target HTTP message for subsequent processing. This process may include the following steps: First, the server parses the received HTTP message. An HTTP message consists of a start line (request line or status line), header fields, and a message body, which are separated by "\r\n". The server splits the HTTP message into its respective parts according to this format. The start line contains information such as the request method, request URL, and HTTP version; the header fields contain a series of key-value pairs that describe the metadata of the message, such as Host, User-Agent, Content-Type, etc.; and the message body contains the actual content of the message, such as request parameters, response data, etc. After the HTTP message is parsed, the server extracts the content of the message header and the message body. The message header includes the start line and the header fields and exists in plain text form. The server parses the message header into a dictionary or hash table of key-value pairs. The message body can be in various formats such as plain text, JSON, XML, binary data, etc. The server determines the specific format of the message body based on the value of the Content-Type field and uses the corresponding parser for parsing. The server determines whether the HTTP message is related to an API call based on the specific information in the message header and the message body. The server can make this determination according to the specific design and conventions of the application. For example, the server can check whether the requested URL matches a predefined API path pattern, such as " / api / v1 / users"; can check whether the request method conforms to the API specification, such as GET, POST, PUT, etc.; can check whether the request parameters contain the required and optional parameters for the API; can check whether the request header fields contain the necessary authentication information, such as API Key, Token, etc. Through this series of checks and matches, the server identifies the HTTP messages related to API calls.

[0039] For those HTTP messages identified as related to API calls, the server determines them as the target HTTP messages for subsequent processing. These messages may come from the target client and contain normal API request content; or they may come from malicious attackers and contain illegal parameters or malicious payloads. The server further analyzes and processes these target HTTP messages to achieve API security protection.

[0040] Step S103: Extract multiple parameters and the corresponding values of each parameter from the target HTTP message to form parameter-value pairs.

[0041] In step S103, the server parses the target HTTP message and splits it into two parts: a message header and a message body. The message header contains the metadata of the request, such as the request method, URL, protocol version, header fields, etc.; the message body contains the actual content of the request, such as parameters, files, etc. The server can use an HTTP protocol parsing library or a parser written by itself to complete this splitting process. According to the specifications of the HTTP protocol, the server extracts parameters and values from the message header and the message body. In the message header, parameters usually appear in the form of header fields, such as Host, Cookie, Authorization, etc., and the server can directly read the values of these fields; in the URL, parameters usually appear in the form of a query string, and the server parses the query string into key-value pairs; in the message body, the format of the parameters depends on the design of the API, and currently there are mainly two forms: Restful and SOAP. For the Restful form, the message body is usually in JSON format, and the server can use a JSON parser to convert it into key-value pairs; for the SOAP form, the message body is usually in XML format, and the server can use an XML parser to convert it into key-value pairs.

[0042] During the process of extracting parameters and values, the server also handles some special cases. For example, some HTTP messages may not contain parameters. For instance, there is no query string in the URL of a GET request, and the server can recognize such cases and avoid making unnecessary parsing attempts; some HTTP messages may contain unstructured content, such as pictures, videos, binary files, etc., and the server can distinguish this content and skip parameter extraction for them.

[0043] After the server extracts all the parameters and values, it combines them into the form of parameter-value pairs for subsequent processing and transmission. Parameter-value pairs are usually represented in the form of key-value pairs, such as "key1=value1, key2=value2". The server uses the extracted parameters as keys and the corresponding values as values to construct a set of parameter-value pairs.

[0044] Step S104: Store the parameter-value pairs in columns to obtain structured storage data.

[0045] In step S104, the parameter value pairs are stored in a columnar format to obtain structured storage data, which specifically includes: determining the data type corresponding to each parameter value pair, where the data types include text, number, boolean, and single value; according to the data type, determining the corresponding data encoding rule in a preset encoding database, where the preset encoding database includes the correspondence between the data type and the data encoding rule; according to the data encoding rule, encoding each parameter value pair, and storing the encoded parameter value pairs in the Arrow memory format to obtain structured storage data.

[0046] Specifically, the server first determines the data type of the value in each parameter value pair. The data types include text (string), number (integer, floating point), boolean (true / false), and single value (enumeration), etc. The server can automatically determine the data type of each value based on characteristics such as the format, length, and range of the value, using regular expressions. For example, "123" can be recognized as a number type, "true" can be recognized as a boolean type, and "red" can be recognized as a single value type. The server looks up the corresponding encoding rule in the preset encoding database according to the determined data type. The preset encoding database is a predefined mapping table that establishes the correspondence between the data type and the encoding rule. Different data types may adopt different encoding methods for more efficient storage and calculation. For example, the text type can adopt dictionary encoding, mapping repeated strings to integer numbers; the number type can adopt Delta encoding, only storing the difference between adjacent two numbers; the boolean type can adopt Bitmap encoding, using one bit to represent a boolean value; the single value type can adopt integer encoding, mapping the enumeration value to an integer.

[0047] The server encodes the value in each parameter value pair according to the determined encoding rule. The encoding process converts the original text-formatted value into a more compact binary format, reducing the data storage space and improving the data processing efficiency. At the same time, the encoded data is also more suitable for columnar storage and query, and values of the same data type can be stored continuously together, facilitating batch reading and calculation.

[0048] The server stores the encoded parameter value pairs in a columnar manner, that is, stores the values of all the same parameter in a continuous memory area, and the values of different parameters are stored in different columns. The Arrow memory format is a columnar storage format that provides a standardized memory layout and metadata definition. The server uses the Arrow format to package and store the encoded parameter value pairs to obtain the final structured storage data.

[0049] Step S105: Based on the preset streaming calculation trigger condition, determine whether the target HTTP message needs to be subjected to streaming calculation based on the structured stored data.

[0050] In step S105, based on the preset streaming calculation trigger condition, determine whether the target HTTP message needs to be subjected to streaming calculation based on the structured stored data, specifically including: obtaining the preset streaming calculation trigger condition, where the preset streaming calculation trigger condition includes an API call condition and calculation configuration parameters; matching the target HTTP message with the API call condition; if it is determined that the target HTTP message matches the API call condition, then determine the streaming calculation logic for the target HTTP message according to the calculation configuration parameters, and determine to perform streaming calculation on the target HTTP message.

[0051] Specifically, the server obtains the preset streaming calculation trigger condition. The preset streaming calculation trigger condition defines in which cases the HTTP message needs to be subjected to streaming calculation and how to perform the calculation. The trigger condition usually includes two parts: an API call condition and calculation configuration parameters. The API call condition specifies the characteristics of the API request that triggers the streaming calculation, such as the URL path, request method, parameter values, etc.; the calculation configuration parameters specify how to perform the streaming calculation on the requests that meet the conditions, such as the calculation window size, aggregation function, threshold condition, etc. The server loads these trigger conditions from external sources such as configuration files and databases. The server matches the current target HTTP message with the preset API call condition. The matching process can be carried out based on each field in the structured stored data, such as the request path, request method, parameter value, etc. The server can use techniques such as string matching, regular expressions, wildcards, etc. to determine whether the HTTP message meets a certain API call condition. For example, an API call condition may require that the request path starts with " / api / v1 / ", the request method is POST, and it contains a parameter named "action" with a value of "login". The server determines whether it meets this condition by checking the corresponding fields of the HTTP message.

[0052] If it is determined that the target HTTP message meets a certain API call condition, the server determines how to perform the streaming calculation on the target HHTP message according to the corresponding calculation configuration parameters. Streaming calculation is usually performed on multiple requests within a period of time, so it is necessary to set the size of the calculation window, such as 10 seconds, 1 minute, etc. Within each window, the server performs real-time aggregation and analysis on the collected requests, such as counting the number of requests, calculating the average response time, detecting abnormal parameters, etc. Streaming calculation can be implemented through a streaming calculation engine (such as Apache Flink, Spark Streaming, etc.), and the server can select the corresponding calculation logic and operators according to the calculation configuration parameters.

[0053] Based on the matching result and the streaming computing logic, the server finally determines whether to perform streaming computing on the target HTTP message. If the message meets a certain API call condition and the corresponding computing configuration parameters are valid, then the server will add the message to the input queue of the streaming computing and start or update the corresponding computing task. If the message does not meet any of the API call conditions, or although it meets the conditions but the computing configuration parameters are invalid, then the server will skip the message and not perform streaming computing on it.

[0054] For example, assume that the following preset streaming computing trigger conditions are set inside the server: API call condition 1: The request path is " / api / v1 / login" and the request method is POST; computing configuration parameter 1: The window size is 1 minute, count the number of requests, and trigger an alarm if the number exceeds 100.

[0055] When the server receives an HTTP message with a request path of " / api / v1 / login" and a request method of POST, it first determines that the message meets API call condition 1, and then determines the streaming computing logic as "count the number of requests every 1 minute and trigger an alarm if it exceeds 100" according to computing configuration parameter 1. Therefore, the message is added to the corresponding streaming computing task.

[0056] Step S106: If it is determined that the target HTTP message needs to perform streaming computing, then perform streaming computing on the target HTTP message to obtain a computing result.

[0057] In step S106, if it is determined that the target HTTP message needs to perform streaming computing, then perform streaming computing on the target HTTP message to obtain a computing result, which specifically includes: determining the type of streaming computing that the target HTTP message needs to perform according to the computing configuration parameters, and the types of streaming computing include statistical streaming computing and feature streaming computing; if it is determined that the target HTTP message needs to perform statistical streaming computing, then perform statistical streaming computing on the target HTTP message according to the preset statistical rules; if it is determined that the target HTTP message needs to perform feature streaming computing, then perform feature streaming computing on the target HTTP message according to the preset feature extraction rules.

[0058] Specifically, the server first determines the type of streaming calculation to be performed on the target HTTP message according to the calculation configuration parameters. The streaming calculation types are mainly divided into two categories: statistical streaming calculation and feature streaming calculation. Statistical streaming calculation focuses on summarizing and aggregating multiple requests over a period of time, such as calculating the number of requests, average response time, etc.; feature streaming calculation focuses on analyzing and extracting a single request to prepare data for subsequent machine learning prediction, such as extracting time series feature vectors, etc. The server decides which type of streaming calculation to perform by reading the calculation type field in the configuration parameters.

[0059] If it is determined that the target HTTP message needs to perform a statistical streaming calculation, the server will perform real-time calculation and update on the message according to the preset statistical rules. The preset statistical rules define how to summarize and analyze a group of requests, including trigger conditions, status definitions, calculation methods, expiration policies, etc. For example, a rule for statistical HyperLogLog (HLL) can be described as: when the API of the request is equal to x, perform HLL calculation on the password field in the request; if the corresponding HLL status does not exist, create the status and set the TTL; if the status already exists, merge the new HLL value into the status; when the TTL of the status expires, automatically delete the status. The server will calculate and merge the data of the current request with the status in real time according to the statistical rules and return the calculation result.

[0060] If it is determined that the target HTTP message needs to perform a feature streaming calculation, the server performs real-time feature calculation and storage on the target HTTP message according to the preset feature extraction rules. The preset feature extraction rules define how to obtain time series features from the request, including trigger conditions, feature definitions, calculation methods, output conditions, etc. For example, a rule for time series feature calculation can be described as: when the API of the request is equal to x, extract the timestamp and key parameters of the request according to the feature definition; if the corresponding time series feature status does not exist, create the status and set the TTL and output conditions; if the status already exists, update the new feature data to the status; when the status meets the output conditions (such as accumulating 10 times) or the TTL expires, trigger machine learning model prediction and clean up the status. The server will calculate and store the data of the current request in real time according to the feature rules and trigger model prediction at an appropriate time.

[0061] After the server finishes the streaming computation on the target HTTP message, it will process and return the computation results as needed. For statistical streaming computations, the results are usually some aggregated metrics or boolean conditions, and the server can compare them with preset thresholds or rules to determine whether to trigger an alarm or block the request; for feature-based streaming computations, the result is usually a time series feature vector, and the server will pass it to the downstream machine learning model for real-time prediction and decide whether to retain or discard the request based on the prediction results (such as anomaly scores). The server performs subsequent processing and responses based on the configuration and results of the streaming computation.

[0062] For example, the server receives an HTTP request with the API " / login" that contains two parameters: username and password. According to the computation configuration parameters, this request requires two types of streaming computations: a statistical streaming computation to count whether the number of requests to this API within the last 1 minute exceeds a threshold (such as 100); a feature-based streaming computation to extract parameters such as the timestamp, username, and password of requests to this API within the last 1 hour and construct a time series feature vector. The server first performs the statistical streaming computation, merges and updates the arrival time of this HTTP request with the corresponding HLL state, and determines whether the merged HLL estimated value exceeds 100. If it exceeds, the request frequency is considered abnormal, and the request may be blocked subsequently; if it does not exceed, the request is considered normal and can continue to be processed. Then, the server performs the feature-based streaming computation, extracts the key parameters of this HTTP request, and appends them to the corresponding feature state in a time series manner. If the state has accumulated feature data for 1 hour, the entire feature vector is passed to the anomaly detection model, and the anomaly detection model determines whether this request is abnormal. The server decides whether to alarm or intercept based on the anomaly score of the model. Finally, the server comprehensively judges multiple results of the streaming computation, such as statistical metrics and anomaly scores, according to the rule engine or HTTP routing configuration. If the request meets a certain combination of conditions, such as the HLL exceeding 100 and the anomaly score being greater than 0.9, the request is marked as abnormal and intercepted; otherwise, the server forwards the request to the backend service for processing according to the normal API routing rules.

[0063] In a possible implementation, according to a preset statistical rule, perform statistical streaming calculation on the target HTTP message, specifically including: obtaining the status definition in the calculation configuration parameters, where the status definition includes a status identifier, a status initial value, and a lifecycle; based on the structured storage data of the target HTTP message, calculate the statistical result according to the statistical rule in the calculation configuration parameters; if it is determined that there is no status corresponding to the status identifier, create a target status corresponding to the status identifier, use the statistical result as the status initial value of the target status, and set the lifecycle of the target status to the survival time of the target status; if it is determined that there is already a target status corresponding to the status identifier, perform an aggregation operation on the statistical result and the current value of the target status to obtain an operation result, and update the operation result to the current value of the target status; if it is determined that the survival time of the target status has expired, delete the target status.

[0064] Specifically, the server first obtains the status definition from the calculation configuration parameters. The status definition includes three elements: a status identifier, a status initial value, and a lifecycle. The status identifier is used to uniquely identify a status; the status initial value represents the starting value when the status is created, such as 0; the lifecycle represents the survival time of the status, such as 1 minute. The server calculates the statistical result based on the structured storage data of the target HTTP message and according to the statistical rule in the calculation configuration parameters. The statistical rule defines how to perform aggregation calculation on the request data, such as accumulating the number of requests, taking the average of the request response time, etc. The server extracts the fields required by the statistical rule from the structured storage data and applies the corresponding aggregation function to obtain a statistical value as the statistical result.

[0065] Next, the server determines whether the corresponding status already exists according to the status identifier. If the status does not exist, the server will create a new target status, use the statistical result as the initial value of the target status, and set the lifecycle of the target status to the survival time specified by the configuration parameters; if the target status already exists, the server will perform an aggregation operation (such as accumulation) on the statistical result and the current value of the target status to obtain a new aggregation value, and update it to the current value of the status. In this way, the server can maintain a continuously updated status value among multiple requests. At the same time, the server will regularly check the survival time of each status. If it is found that the survival time of a certain status has exceeded the lifecycle set by the calculation configuration parameters, the status will be automatically deleted.

[0066] In a possible implementation manner, according to a preset feature extraction rule, perform feature class streaming calculation on the target HTTP message, specifically including: obtaining the feature extraction rule and the feature calculation trigger condition in the calculation configuration parameters; extracting feature parameters from the structured storage data of the target HTTP message according to the feature extraction rule; determining whether the feature calculation trigger condition is satisfied, where the feature calculation trigger condition includes time trigger, data volume trigger, event trigger, and external trigger; if it is determined that the feature calculation trigger condition is satisfied, input the feature parameters into a preset machine learning model, and use the output result of the preset machine learning model as the feature calculation result; if it is determined that the feature calculation trigger condition is not satisfied, temporarily store the feature parameters until the feature calculation trigger condition is satisfied, and input the temporarily stored feature parameters into the preset machine learning model.

[0067] Specifically, the server first obtains the feature extraction rule and the feature calculation trigger condition from the calculation configuration parameters. The feature extraction rule defines how to extract feature parameters from the structured storage data of the HTTP message, such as extracting the URL path, request body size, user agent, etc.; the feature calculation trigger condition defines when to trigger the machine learning model for calculation, and the triggering methods can include time trigger, data volume trigger, event trigger, and external trigger. The server obtains both types of rules at the same time to perform a complete feature class streaming calculation.

[0068] The server extracts a set of feature parameters from the structured storage data of the target HTTP message according to the feature extraction rule. For example, the feature extraction rule can define fields such as the URL path of the request, the request body size, and the user agent to be extracted. The server will search for these fields in the structured storage data and extract their values to form a feature vector. This feature vector represents the feature representation of the HTTP message in a specific dimension and is the input to the subsequent machine learning model.

[0069] The server determines whether the current feature calculation trigger condition is satisfied. The trigger condition can be based on multiple factors such as time, data volume, event, and external. For example, time trigger: trigger the calculation once every 1 minute; data volume trigger: trigger the calculation when 100 requests are accumulated; event trigger: trigger the calculation when an abnormal request appears; external trigger: trigger the calculation when a trigger signal from an external system is received.

[0070] The server monitors these trigger conditions in real time. When any one of the conditions is met, it determines to trigger feature calculation. If the feature calculation trigger condition is satisfied, the server inputs the extracted feature parameters into a preset machine learning model and obtains the output result of the preset machine learning model as the feature calculation result. The machine learning model can be of various types such as anomaly detection, user profiling, behavior prediction, etc. The server passes the feature vector to the corresponding preset machine learning model for inference calculation according to the specific business scenario and model type, and obtains a value or vector representing the calculation result, such as an anomaly score, user label, behavior probability, etc.

[0071] If the feature calculation trigger condition is not satisfied, the server does not immediately execute the machine learning model, but temporarily stores the extracted feature parameters and continues to process the next HTTP message. The temporarily stored feature parameters will accumulate continuously until the trigger condition is met.

[0072] Step S107: Determine the processing result for the target HTTP message according to the HTTP routing setting and the calculation result. The processing result includes allowing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

[0073] In step S107, the server makes a final processing decision on the target HTTP message according to the HTTP routing setting and the streaming calculation result, determining whether to allow and forward the message or block the message and return the corresponding response. The decision-making process comprehensively considers the routing attributes and calculation metrics of the message, reflecting the intelligent protection concept of the combination of WAF and the risk control engine. Specifically, it can be divided into the following key steps:

[0074] The server obtains the pre-configured HTTP routing settings, which define the mapping relationship between different URL paths and the backend server, as well as the security policies, forwarding rules, etc. for each path. By identifying the URL path of the target HTTP message, the server can quickly judge the routing destination and processing method of the message. The HTTP routing setting is the basic configuration of WAF and determines the basic flow direction of the message. The server obtains the streaming calculation result for the target HTTP message from step S106, and these results include statistical metrics (such as the number of requests, anomaly ratio, etc.) and feature metrics (such as anomaly score, risk level, etc.). The server comprehensively analyzes the HTTP routing setting and the streaming calculation result to judge the final processing method of the target HTTP message. The analysis process is based on a series of preset rules and thresholds, such as:

[0075] If the URL path of the message matches a high-risk route and the anomaly score exceeds the preset score threshold, it is determined as a malicious request and needs to be blocked. If the URL path of the message matches a sensitive route and the number of requests exceeds the preset request threshold, it is determined as an abnormal request and needs to be alerted. If the URL path of the message matches a normal route and all indicators are within the normal range, it is determined as a normal request and needs to be allowed through.

[0076] The server can flexibly combine HTTP route settings and calculated metrics to form judgment conditions in multiple dimensions, which can handle various complex attack scenarios. This dual judgment based on routes and data greatly improves the detection accuracy and protection effect of the WAF.

[0077] According to the result of the comprehensive analysis, the server performs corresponding processing operations on the target HTTP message: If the judgment result is to allow through, the server forwards the message to the target server specified in the route setting to complete the normal business request process. If the judgment result is to block, the server refuses to forward the message and returns a custom blocking response, such as "403 Forbidden", etc., to the requester, and at the same time records the detailed information of this blocking event for subsequent security audits.

[0078] After step S107, the method further includes: If it is determined that the processing result is to allow through and forward the target HTTP message, the target HTTP message is forwarded to the target server, and the response message returned by the target server is sent to the original requester; If it is determined that the processing result is to block the target HTTP message, a preset blocking response message is returned to the original requester.

[0079] Specifically, if the processing result is to allow through and forward the target HTTP message, it means that the message has passed multiple checks by the WAF and the risk control engine and is determined as a normal request, and needs to be forwarded to the target server at the back end for business processing. At this time, the server forwards the target HTTP message to the target server specified in the route setting, usually through an internal network or a dedicated line to the back-end business server of the WAF. The forwarding process follows the standard process of the HTTP protocol, and the server acts as a proxy to pass the client's request as it is to the target server and waits for the target server to return a response message. When the server receives the response message returned by the target server, it will perform necessary security checks on the response content again, such as filtering sensitive information, to ensure that possible risks will not be passed on to the client. After the check is completed, the server sends the response message to the requester through the original connection to complete a complete HTTP request-response process.

[0080] If the processing result is to block the target HTTP message, it indicates that the message fails to pass the checks of the WAF and the risk control engine and is determined to be a malicious request or an abnormal request, and needs to be directly rejected without being forwarded to the backend server. At this time, the server will directly return a preset blocking response message to the original requester, informing the requester that the request has been rejected by the security policy.

[0081] Referring to Figure 2 , the present application further provides an API security protection device based on HTTP message streaming processing. The device is a server, and the server includes a receiving module 201 and a processing module 202, where: The receiving module 201 is configured to receive TCP messages by listening to a preset network port or a preset protocol, and reorganize the TCP messages to obtain HTTP messages; The processing module 202 is configured to parse the HTTP messages to obtain a message header and a message body, and determine a target HTTP message related to the API call according to the message header and the message body; The processing module 202 is further configured to extract multiple parameters and the values corresponding to each parameter from the target HTTP message to form parameter-value pairs; The processing module 202 is further configured to store the parameter-value pairs in a columnar manner to obtain structured storage data; The processing module 202 is further configured to determine whether the target HTTP message needs to perform streaming calculation based on the structured storage data according to a preset streaming calculation trigger condition; The processing module 202 is further configured to, if it is determined that the target HTTP message needs to perform streaming calculation, perform streaming calculation on the target HTTP message to obtain a calculation result; The processing module 202 is further configured to determine a processing result for the target HTTP message according to the HTTP routing setting and the calculation result, and the processing result includes allowing and forwarding the target HTTP message or blocking the target HTTP message and returning a response.

[0082] In a possible implementation manner, the processing module 202 stores the parameter-value pairs in a columnar manner to obtain structured storage data, specifically including: The processing module 202 determines the data types corresponding to the parameter-value pairs, and the data types include text, number, boolean value, and single value; The processing module 202 determines the corresponding data encoding rules in a preset encoding database according to the data types, and the preset encoding database includes the corresponding relationship between the data types and the data encoding rules; The processing module 202 encodes each parameter-value pair according to the data encoding rules, and stores the encoded parameter-value pairs in the Arrow memory format to obtain structured storage data.

[0083] In a possible implementation, the processing module 202 determines whether a target HTTP message needs to be subjected to streaming computation based on the structured stored data according to a preset streaming computation trigger condition, which specifically includes: the processing module 202 obtains the preset streaming computation trigger condition, and the preset streaming computation trigger condition includes an API call condition and computation configuration parameters; the processing module 202 matches the target HTTP message with the API call condition; if the processing module 202 determines that the target HTTP message matches the API call condition, it determines the streaming computation logic for the target HTTP message according to the computation configuration parameters and determines to perform streaming computation on the target HTTP message.

[0084] In a possible implementation, if the processing module 202 determines that the target HTTP message needs to be subjected to streaming computation, it performs streaming computation on the target HTTP message to obtain a computation result, which specifically includes: the processing module 202 determines the type of streaming computation that the target HTTP message needs to perform according to the computation configuration parameters, and the type of streaming computation includes statistical streaming computation and feature streaming computation; if the processing module 202 determines that the target HTTP message needs to perform statistical streaming computation, it performs statistical streaming computation on the target HTTP message according to a preset statistical rule; if the processing module 202 determines that the target HTTP message needs to perform feature streaming computation, it performs feature streaming computation on the target HTTP message according to a preset feature extraction rule.

[0085] In a possible implementation, the computation configuration parameters of the processing module 202 include a status definition and a statistical rule. Performing statistical streaming computation on the target HTTP message according to the preset statistical rule specifically includes: the processing module 202 obtains the status definition in the computation configuration parameters, and the status definition includes a status identifier, a status initial value, and a lifecycle; based on the structured stored data of the target HTTP message, it calculates a statistical result according to the statistical rule in the computation configuration parameters; if the processing module 202 determines that there is no target status corresponding to the status identifier, it creates a target status corresponding to the status identifier, uses the statistical result as the status initial value of the target status, and sets the lifecycle of the target status to the survival time of the target status; if the processing module 202 determines that there already exists a target status corresponding to the status identifier, it performs an aggregation operation on the statistical result and the current value of the target status to obtain an operation result, and updates the operation result to the current value of the target status; if the processing module 202 determines that the survival time of the target status has expired, it deletes the target status.

[0086] In a possible implementation, the calculation configuration parameters include a feature extraction rule and a feature calculation trigger condition. The processing module 202 performs feature-based streaming calculations on the target HTTP message according to the preset feature extraction rule, which specifically includes: the processing module 202 obtains the feature extraction rule and the feature calculation trigger condition in the calculation configuration parameters; the processing module 202 extracts feature parameters from the structured storage data of the target HTTP message according to the feature extraction rule; the processing module 202 determines whether the feature calculation trigger condition is satisfied, and the feature calculation trigger condition includes time trigger, data volume trigger, event trigger, and external trigger; if the processing module 202 determines that the feature calculation trigger condition is satisfied, it inputs the feature parameters into a preset machine learning model and uses the output result of the preset machine learning model as the feature calculation result; if the processing module 202 determines that the feature calculation trigger condition is not satisfied, it temporarily stores the feature parameters until the feature calculation trigger condition is satisfied, and then inputs the temporarily stored feature parameters into the preset machine learning model.

[0087] In a possible implementation, after the processing module 202 determines the processing result for the target HTTP message according to the HTTP routing setting and the calculation result, the method further includes: if the processing module 202 determines that the processing result is to release and forward the target HTTP message, it forwards the target HTTP message to the target server and sends the response message returned by the target server to the original requestor; if the processing module 202 determines that the processing result is to block the target HTTP message, it returns a preset blocking response message to the original requestor.

[0088] It should be noted that: when the device provided in the above embodiment realizes its functions, only the division of the above functional modules is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process can be seen in the method embodiment, which will not be repeated here.

[0089] This application also provides an electronic device. Refer to Figure 3 , Figure 3 is a schematic structural diagram of an electronic device provided in an embodiment of this application. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0090] Among them, the communication bus 302 is used to realize the connection and communication between these components.

[0091] Among them, the user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may further include standard wired interfaces and wireless interfaces.

[0092] Among them, the network interface 304 may optionally include standard wired interfaces and wireless interfaces (such as Wi-Fi interfaces).

[0093] Among them, the processor 301 may include one or more processing cores. The processor 301 connects various parts within the entire server through various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling the data stored in the memory 305, the processor 301 performs various functions of the server and processes data. Optionally, the processor 301 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 301 may integrate one or a combination of several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301 and may be implemented separately by a single chip.

[0094] Among them, the memory 305 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store the data involved in the above-mentioned various method embodiments. Optionally, the memory 305 may further be at least one storage device located far from the aforementioned processor 301. Refer to Figure 3In the memory 305, which is a computer storage medium, there may be included an operating system, a network communication module, a user interface module, and an application program for an API security protection method based on HTTP message streaming processing.

[0095] In Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an interface for the user to input data and obtain the data input by the user; while the processor 301 can be used to call the application program for an API security protection method based on HTTP message streaming processing stored in the memory 305. When executed by one or more processors 301, it causes the electronic device 300 to execute one or more of the methods described in the above embodiments. It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be in other sequences or performed simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0096] The present application also provides a computer-readable storage medium storing instructions. When executed by one or more processors 301, it causes the electronic device 300 to execute one or more of the methods described in the above embodiments.

[0097] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0098] In several implementation manners provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some service interfaces. The indirect couplings or communication connections of the devices or units can be in electrical or other forms.

[0099] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] In addition, in each embodiment of the present application, each functional unit can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned memory includes: various media such as USB flash drives, mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0102] The above are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited thereby. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and the practice of the present disclosure.

[0103] The present application aims to cover any variations, uses, or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An API security protection method based on HTTP message streaming processing, characterized in that: The method comprises: By monitoring a preset network port or a preset protocol, a TCP message is received, and the TCP message is reassembled to obtain an HTTP message; Parsing the HTTP message to obtain a message header and a message body, and determining a target HTTP message related to the API call according to the message header and the message body; Extracting multiple parameters and values ​​corresponding to each of the parameters from the target HTTP message to form parameter-value pairs; Column-storing the parameter-value pairs to obtain structured storage data; According to a preset stream computing trigger condition, judging whether the target HTTP message needs to be stream computed based on the structured storage data; If it is determined that the target HTTP message needs to be stream-calculated, the stream-calculated is performed on the target HTTP message to obtain a calculation result; Determine, according to the HTTP routing setting and the calculation result, a processing result for the target HTTP message, wherein the processing result includes releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response; The step of determining whether the target HTTP message needs to be stream-computed based on the structured storage data according to the preset stream-compute trigger condition specifically includes: Acquire the preset stream computing trigger condition, where the preset stream computing trigger condition includes an API call condition and a computing configuration parameter; Matching the target HTTP message with the API call condition; If it is determined that the target HTTP message matches the API call condition, determining the stream computing logic for the target HTTP message according to the computing configuration parameters, and determining to perform stream computing on the target HTTP message; If it is determined that the target HTTP message needs to be stream-computed, stream-computed is performed on the target HTTP message to obtain a calculation result, which specifically includes: Determine the stream computing type that needs to be executed for the target HTTP message according to the computing configuration parameters, where the stream computing type includes statistical stream computing and feature stream computing; If it is determined that the target HTTP message needs to perform statistical stream computing, then according to a preset statistical rule, perform statistical stream computing on the target HTTP message; If it is determined that the target HTTP message needs to perform feature-based streaming calculation, the feature-based streaming calculation is performed on the target HTTP message according to a preset feature extraction rule.

2. The method according to claim 1, characterized in that The column-type storage of the parameter value pairs to obtain structured storage data specifically includes: Determine the data type corresponding to each of the parameter value pairs, the data type including text, number, Boolean value and single value; According to the data type, determining a corresponding data encoding rule in a preset encoding database, wherein the preset encoding database includes a correspondence between the data type and the data encoding rule; According to the data encoding rule, each of the parameter value pairs is encoded, and the encoded parameter value pairs are stored in Arrow memory format to obtain the structured storage data.

3. The method according to claim 1, characterized in that The calculation configuration parameters include state definitions and statistical rules. The statistical stream calculation is performed on the target HTTP message according to the preset statistical rules, specifically including: Obtaining a state definition in the computing configuration parameters, wherein the state definition includes a state identifier, a state initial value, and a life cycle; Based on the structured storage data of the target HTTP message, according to the statistical rules in the calculation configuration parameters, calculate the statistical results; If it is determined that there is no state corresponding to the state identifier, create a target state corresponding to the state identifier, use the statistical result as the state initial value of the target state, and set the life cycle of the target state to the survival time of the target state; If it is determined that a target state corresponding to the state identifier already exists, performing an aggregation operation on the statistical result and the current value of the target state to obtain an operation result, and updating the operation result as the current value of the target state; If it is determined that the survival time of the target state has expired, the target state is deleted.

4. The method according to claim 1, characterized in that The calculation configuration parameters include feature extraction rules and feature calculation trigger conditions. The feature-based streaming calculation is performed on the target HTTP message according to the preset feature extraction rules, specifically including: Acquire feature extraction rules and feature calculation triggering conditions in the calculation configuration parameters; Extracting characteristic parameters from the structured storage data of the target HTTP message according to the characteristic extraction rule; Determine whether the feature calculation trigger condition is met, the feature calculation trigger condition includes time trigger, data volume trigger, event trigger and external trigger; If it is determined that the feature calculation trigger condition is met, the feature parameters are input into a preset machine learning model, and the output result of the preset machine learning model is used as the feature calculation result; If it is determined that the feature calculation trigger condition is not met, the feature parameters are temporarily stored until the feature calculation trigger condition is met, and the temporarily stored feature parameters are input into the preset machine learning model.

5. The method according to claim 1, characterized in that: After determining the processing result for the target HTTP message according to the HTTP routing setting and the calculation result, the method further includes: If it is determined that the processing result is to release and forward the target HTTP message, forward the target HTTP message to the target server, and send the response message returned by the target server to the original requester; If it is determined that the processing result is to block the target HTTP message, a preset blocking response message is returned to the original requester.

6. An API security protection device based on HTTP message streaming processing, characterized in that: The device is used to execute the method according to any one of claims 1 to 5, and the device comprises a receiving module (201) and a processing module (202), wherein: The receiving module (201) is used to receive TCP messages by monitoring a preset network port or a preset protocol, and to reassemble the TCP messages to obtain HTTP messages; The processing module (202) is used to parse the HTTP message to obtain a message header and a message body, and determine a target HTTP message related to the API call according to the message header and the message body; The processing module (202) is further used to extract multiple parameters and values ​​corresponding to each parameter from the target HTTP message to form parameter-value pairs; The processing module (202) is further used to store the parameter value pairs in column format to obtain structured storage data; The processing module (202) is further used to determine whether the target HTTP message needs to be stream-computed based on the structured storage data according to a preset stream-compute trigger condition; The processing module (202) is further configured to, if it is determined that the target HTTP message needs to be stream-calculated, perform stream-calculation on the target HTTP message to obtain a calculation result; The processing module (202) is further used to determine a processing result for the target HTTP message according to the HTTP routing setting and the calculation result, wherein the processing result includes releasing and forwarding the target HTTP message or blocking the target HTTP message and returning a response; The processing module (202) is further used to obtain the preset stream computing trigger condition, which includes an API call condition and a computing configuration parameter; match the target HTTP message with the API call condition; if it is determined that the target HTTP message matches the API call condition, determine the stream computing logic for the target HTTP message according to the computing configuration parameter, and determine to perform stream computing on the target HTTP message; The processing module (202) is also used to determine the type of streaming calculation that needs to be performed on the target HTTP message based on the calculation configuration parameters, and the streaming calculation type includes statistical streaming calculation and feature streaming calculation; if it is determined that the target HTTP message needs to perform statistical streaming calculation, then according to the preset statistical rules, the statistical streaming calculation is performed on the target HTTP message; if it is determined that the target HTTP message needs to perform feature streaming calculation, then according to the preset feature extraction rules, the feature streaming calculation is performed on the target HTTP message.

7. An electronic device, characterized in that: The electronic device (300) comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 5 is performed.

Images