An API security detection method and device
Patent Information
- Application Number
- CN202611454511.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-18
- Publication Date
- 2026-10-23
AI Technical Summary
第一，过度泛化导致漏报
（1）抑制过度泛化漏报
Smart Images
Figure CN122969907A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer network security technology, and in particular to an API security detection method and apparatus. Background Technology
[0002] Traditional API security management and Web application firewall whitelist construction generally rely on API auto-discovery and pattern learning technologies. These typically employ clustering algorithms, text alignment, and regular expression derivation to attempt to identify the location of dynamic parameters from massive amounts of raw Uniform Resource Locator (URL) traffic and abstract it into a whitelist in regular expression form. However, this learning-then-generalization approach has the following drawbacks: First, overgeneralization leads to false negatives. Once the system abstracts dynamic API paths containing numeric identifiers into a regular expression whitelist, when an attacker sends a request carrying an excessively long, malformed number or a specific boundary value, it will be allowed through directly because it still matches the regular expression, resulting in a false negative.
[0003] Second, the limitations of high computing power and cold start. Running clustering, alignment, probabilistic statistics, or deep learning models consumes a lot of central processing unit and memory resources, and a long audit window is required to accumulate enough samples when the system goes online or the API changes, during which the protection capabilities are weak.
[0004] Third, the regular expression engine is vulnerable to parameter obfuscation and regular expression denial-of-service attacks. When HTTP query parameters are out of order or random interference key-value pairs are inserted, the regular expression engine can be easily bypassed, and backtracking may trigger a regular expression denial-of-service attack.
[0005] On the other hand, existing precise path whitelist schemes based on deterministic automata or trie avoid the problem of regular expression generalization. However, when faced with high cardinality dynamic parameters (such as a large number of distinct numeric identifiers or globally unique identifiers), the number of states and transition edges will increase approximately linearly with the number of different parameter values, resulting in uncontrollable whitelist size. If high cardinality branches are simply generalized back to regular expressions to compress the size, it degenerates into the aforementioned problem of overgeneralization and underreporting.
[0006] Therefore, how to strike a balance between the accuracy of API security detection and memory controllability, and how to avoid direct false blocking or direct access when the coverage of the behavior whitelist is insufficient, are technical problems that have not yet been effectively solved by existing technologies. Summary of the Invention
[0007] In view of the above-mentioned defects or deficiencies in the prior art, the present invention provides an API security detection method and apparatus, which uses a finite state converter topology graph structure to carry API request paths, and replaces state nodes whose out-degree exceeds a first threshold and whose access frequency is lower than a second threshold with parameter verification nodes that carry triple constraints of character type, length range and observed legal value set; during online detection, after byte-by-byte state flow and triple verification, the flow is divided into allow, review or block, thus achieving a balance between compressed storage and accurate verification.
[0008] One aspect of the present invention provides an API security detection method, comprising the following steps: The query parameter key-value pairs in the pre-collected valid API requests are rearranged lexicographically by parameter key name, and a finite state converter topology graph structure is constructed using the rearranged API request string. The finite state converter topology graph structure includes state nodes and transition edges connecting two state nodes. Each state node represents the current position when the API request string is compared with the state transition path stored in the finite state converter topology graph structure. Each state node includes a starting node for initiating a state transition and a final state node for indicating successful completion of the state transition. Each transition edge has a character marker, used to guide the transition from the current state node to the next state node when the current character of the API request string matches the character marker of the transition edge. In the finite state converter topology graph structure, state nodes with an out-degree exceeding the first threshold and an access frequency lower than the second threshold are replaced with parameter verification nodes; wherein, the parameter verification node carries triple verification information for the character type, length range, and whether the parameter value belongs to the set of observed legal values; All API requests acquired online during the security monitoring process are transferred byte by byte in the finite state converter topology graph structure, and when the transfer reaches the parameter verification node, the parameter value corresponding to the parameter verification node in each API request is verified using the triple verification information. Based on whether the state transition was successfully completed and the verification result of the triple verification information, each obtained API request is subject to a release operation, a security review operation, or an interception operation.
[0009] Another aspect of the present invention provides an API security detection device, comprising: A construction module is used to rearrange the query parameter key-value pairs in pre-collected valid API requests according to the lexicographical order of the parameter keys, and construct a finite state converter topology graph structure using the rearranged API request string. The finite state converter topology graph structure includes state nodes and transition edges connecting two state nodes. Each state node represents the current position when the API request string is compared with the state transition paths stored in the finite state converter topology graph structure. Each state node includes a starting node for initiating a state transition and a final state node for indicating successful completion of the state transition. Each transition edge has a character marker, used to guide the transition from the current state node to the next state node when the current character of the API request string matches the character marker of the transition edge. The replacement module is used to replace state nodes in the finite state converter topology graph structure whose out-degree exceeds a first threshold and whose access frequency is lower than a second threshold with parameter verification nodes; wherein, the parameter verification node carries triple verification information for the character type, length range, and whether the parameter value belongs to the set of observed legal values; The verification module is used to perform byte-by-byte state transition of all API requests obtained online during the security monitoring process in the finite state converter topology graph structure, and to verify the parameter value of each API request corresponding to the parameter verification node when the transition reaches the parameter verification node using the triple verification information. The processing module is used to perform a release operation, a security review operation, or an interception operation on each obtained API request based on whether the status transition is successfully completed and the verification result of the triple verification information.
[0010] The API security detection method and apparatus provided by this invention have the following beneficial effects: (1) Suppressing overgeneralization of underreporting The dynamic parameter position is carried by a parameter validation node that carries character type constraints and length range constraints. If the character composition does not conform to the expected type or the length exceeds the reasonable range, the parameter value will be directed to the verification channel, thereby avoiding the boundary value bypass caused by the generalization matching of traditional regular expressions.
[0011] (2) Memory usage is controllable After the dynamic parameter location is replaced by the parameter verification node, a single parameter verification node replaces the original large number of exact matching branch paths, so that the memory usage of the whitelist does not expand indefinitely with the increase of the number of observed parameter values.
[0012] (3) Reduce false alarms For API requests with valid structures but whose parameter values do not exist in the observed set, neither allow them directly nor block them directly. Instead, guide them to a review operation for further analysis, thereby avoiding false alarms when coverage is insufficient.
[0013] (4) Online incremental update The whitelist topology is updated online through an incremental double buffering mechanism and pointer switching. Newly added legitimate API request paths are first written into the incremental layer, and the background periodically merges and generates a new version of the base layer and replaces it through pointer switching. The update process does not affect the continuous processing of online detection requests. Attached Figure Description
[0014] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating an API security detection method provided in one embodiment of this application; Figure 2 This is a schematic diagram of the structure of an API security detection device provided in one embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation
[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] In this invention, unless otherwise expressly stated, the following terms have the following meanings: API security testing refers to the process of performing security analysis on the request traffic of application programming interfaces in order to identify and handle malicious or abnormal requests.
[0017] Finite State Transducers (FST) topology: A deterministic finite state automaton that maps input strings to output strings. When minimized, it provides a compact representation of the string set. Its topology includes state nodes and transition edges. In this invention, the FST topology is used to store a set of valid API request paths and provides byte-by-byte state transition matching.
[0018] State node: refers to the basic unit in the finite state converter topology graph structure, representing the current position when the API request string is compared with the state transition path stored in the topology graph structure. The state node includes the start node, the final state node, and the intermediate state nodes.
[0019] Transition edge: refers to a directed edge connecting two state nodes in the topology of the finite state converter. It has a character label and is used to guide the transition from the current state node to the next state node when the current character of the API request string matches the character label of the transition edge.
[0020] Out-degree: refers to the number of transition edges that start from a state node in the finite state converter topology graph and point to the next state node.
[0021] Parameter verification node: This is one of the innovations of the present invention. In the present invention, it refers to a special state node that replaces the high cardinality state node in the topology graph structure of the finite state converter. It carries triple verification information for the character type, length range and whether the parameter value belongs to the set of observed legal values.
[0022] Triple verification information: refers to the three types of verification constraints carried by the parameter verification node, including character type constraints, length range constraints, and constraints on whether the parameter value belongs to the observed set of legal values.
[0023] Approximate membership filters: refer to a type of probabilistic data structure that supports set membership queries in a sublinear space, including but not limited to Bloom filters, Cuckoo filters, or XOR filters.
[0024] Observed valid value set: In this invention, it refers to the set of all parameter values extracted from pre-collected valid API request samples that correspond to the position of a specific parameter verification node.
[0025] Count-Min Sketch: A probabilistic data structure for estimating the frequency of elements in a data stream using a sublinear space. Count-Min Sketch maps elements to a two-dimensional counting array using multiple hash functions, enabling the estimation of element frequency with controllable precision.
[0026] Incremental double-buffering mechanism: This is one of the innovative aspects of this invention. It refers to the mechanism used in this invention to achieve online updates of the finite state converter topology graph structure. By dividing the topology graph structure into a variable incremental layer and an immutable base layer, newly emerging valid paths are temporarily stored in the incremental layer and periodically merged and reconstructed in the base layer, achieving zero-downtime online updates.
[0027] Incremental Minimum Acyclic Finite Automaton Algorithm: This refers to an incremental construction algorithm that minimizes the finite automaton in real time while adding new strings one by one. For example, the Daciuk incremental construction algorithm can directly and incrementally incorporate new strings into the existing automaton structure without rebuilding the entire automaton.
[0028] The API security detection method and apparatus provided by the present invention will be described in detail below with reference to the accompanying drawings.
[0029] like Figure 1 As shown, the method provided by the present invention includes the following steps.
[0030] Step S101: Collect legitimate API request samples and preprocess them.
[0031] This step is used to collect samples of legitimate API requests and preprocess the collected samples to eliminate string differences caused by parameter order, providing a unified input format for the subsequent construction of the finite state converter topology graph.
[0032] Specifically, traffic from legitimate API requests on the business side is collected via bypass mirroring. This traffic includes request methods, paths, and query parameter key-value pairs. Bypass mirroring involves configuring port mirroring on a network switch or router, copying the business traffic, and sending a copy to the collection system. The collection system only receives the copy of the traffic for analysis, without interfering with the normal forwarding of business traffic. Legitimate API requests refer to normal business requests that have been confirmed by the business party and do not contain any malicious intent or illegal information.
[0033] The query parameter key-value pairs in the collected legitimate API request samples are rearranged lexicographically based on the parameter key names. Lexicographical order is a common concept in this field, referring to the rule of comparing and arranging characters according to their encoded values (e.g., ASCII or Unicode) from smallest to largest. Specifically, for each API request, all parameter key-value pairs are extracted from its query string, the parameter key names are sorted lexicographically, and then the key-value pairs are reassembled into a unified query string format according to the sorted order.
[0034] For example, in the API request GET: / api / users?city=Shanghai&age=25&name=Zhang San, the key-value pairs of the query parameters are city=Shanghai, age=25, and name=Zhang San. After sorting the parameter keys in lexicographical order, age (starting with 'a') comes before city (starting with 'c'), and city comes before name (starting with 'n'). The rearranged query string is age=25&city=Shanghai&name=Zhang San.
[0035] By rearranging the parameters in lexicographical order as described above, API requests with the same set of query parameter key-value pairs but different ordering are converted into the same string representation, eliminating the string diversity caused by the parameter ordering.
[0036] It is important to note that the above-described lexicographical rearrangement of parameter keys is a deterministic structured process. Its transformation rules are purely logical rules independent of statistical results from data samples. That is, for any given API request, regardless of the number of times it is processed or in which processing session, as long as the input is the same, the output will be the same and unique. This operation does not involve any clustering, text alignment, probability statistics, or machine learning inference, thus maintaining the non-pre-learning characteristic of this invention.
[0037] This step normalizes semantically identical API requests with different parameter order into the same string representation by rearranging the parameter key names lexicographically. This fundamentally eliminates the path diversity caused by parameter disorder, enabling the subsequent finite state converter topology to identify semantically identical API requests as the same state transition path, effectively resisting parameter disorder bypass attacks.
[0038] Step S102: Construct the topology graph structure of the finite state converter.
[0039] This step is used to construct a compact finite state converter topology graph structure from the set of valid API request strings processed in step S101.
[0040] Specifically, the API request strings rearranged in step S101 are used as input and streamed into the finite state converter builder. This streaming writing refers to inserting each API request string into the finite state converter topology graph being built one by one, rather than collecting all the strings and building it all at once.
[0041] During the construction process, the finite state converter topology graph structure achieves a compact representation of the API request string set through prefix sharing and suffix sharing.
[0042] Prefix sharing refers to the sharing of the same set of state nodes and transition edges in the topology graph structure by multiple API request strings containing the same starting character sequence. For example, for the API request strings GET: / api / v1 / users / 123 / profile and GET: / api / v1 / users / 456 / profile, their common prefix GET: / api / v1 / users / is stored only once in the topology graph structure, and the two request strings share the same set of state nodes and transition edges in this common prefix portion.
[0043] Suffix sharing refers to the sharing of the same set of state nodes and transition edges in the topology graph structure among multiple API request strings that share the same ending character sequence. For example, for the API request strings GET: / api / v1 / users / 123 / profile and GET: / api / v1 / users / 456 / profile, their common suffix / profile is stored only once in the topology graph structure. The two request strings share the same set of state nodes and transition edges in this common suffix part through state merging.
[0044] Through the synergistic effect of prefix sharing and suffix sharing, the finite state converter topology graph structure can compress and store the common parts of a large number of API request strings into a single structure, thereby achieving bounded compressed storage of the API path set. Here, bounded compression means that the storage space of the finite state converter topology graph structure is related to the complexity of the static structure part in the API path set, but not to the total number of API requests.
[0045] The finite state converter topology graph structure includes state nodes and transition edges. A state node represents the current position when the API request string is compared with the state transition path stored in the finite state converter topology graph structure. Each state node includes a starting node for initiating a state transition and a final state node for indicating successful completion of the transition. The starting node is a state node with an in-degree of zero in the topology graph structure; all state transitions for API request strings begin from this starting node. The final state node is a state node in the topology graph structure that indicates successful completion of a state transition. When the last character of the API request string is successfully matched and reaches the final state node, it indicates that the API request has a corresponding complete state transition path in the topology graph structure.
[0046] The transition edge connects two state nodes, and each transition edge carries a character marker. The transition edge guides the transition from the current state node to the next state node when the current character of the API request string matches the character marker of the transition edge. Specifically, during the state transition process, starting from the current state node, it checks if there exists a transition edge whose character marker matches the currently read API request string; if so, the transition proceeds along that transition edge to the next state node; otherwise, the state transition terminates.
[0047] This step significantly reduces storage overhead by using prefix and suffix sharing in the finite state converter topology graph to perform bounded compression storage of the static structure portion of the API path. Furthermore, since the finite state converter topology graph is a deterministic automaton, its state transition process does not involve backtracking, thus preventing regular expression denial-of-service attacks.
[0048] Step S103: Identify and replace high cardinality state nodes with parameter verification nodes.
[0049] This step is used to identify high cardinality state nodes in the finite state converter topology and replace them with parameter check nodes carrying triple check information.
[0050] Specifically, each state node in the finite state converter topology graph is traversed, and the out-degree and access frequency of each state node are counted.
[0051] The out-degree refers to the number of transition edges originating from the current state node and pointing to the next state node. When the out-degree of a state node is large, it indicates that the state node has a large number of different outgoing edge branches, meaning that the state node corresponds to a high cardinality dynamic parameter position in the API request path.
[0052] The access frequency refers to the number of times the state node is accessed in the legitimate API request samples collected in step S101. The access frequency can be estimated using a Count-Min Sketch. A Count-Min Sketch is a probabilistic data structure that estimates the frequency of elements in a data stream using sublinear space. Count-Min Sketch maps elements to a two-dimensional counting array using multiple hash functions, enabling the estimation of element occurrence frequency with controllable precision, and its space complexity is far lower than the storage space required for precise counting. The estimation result of the Count-Min Sketch is used to determine whether the state node meets the condition that the access frequency is below a second threshold.
[0053] When the out-degree of a state node exceeds the first threshold and the access frequency is lower than the second threshold, the state node is determined to be a high cardinality state node and meets the conditions to be replaced as a parameter verification node.
[0054] The first threshold is used to determine whether a state node is a high cardinality node. That is, if the out-degree exceeds the first threshold, it indicates that the node has a large number of different outgoing edge branches, corresponding to a high cardinality dynamic parameter position in the API request path. The first threshold can be adaptively determined according to the out-degree distribution of each state node in the finite state converter topology graph structure. For example, it can be taken as the high quantile of the out-degree distribution of each node, or it can be preset as a fixed constant by those skilled in the art based on factors such as system scale and memory resources.
[0055] The second threshold is used to determine whether a status node is a low-frequency node. A frequency lower than the second threshold indicates that all outgoing edges of the node are low-frequency paths, meaning no specific parameter value path significantly dominates the sample. Therefore, the node and all its outgoing edges can be safely replaced with a parameter verification node without losing the ability to identify observed legitimate requests. The second threshold can be preset by those skilled in the art as a relative proportion or a fixed frequency value based on the total traffic volume.
[0056] Furthermore, after determining that a node is in a high cardinality state, it is replaced with a parameter verification node. The parameter verification node carries triple verification information: the character type of the parameter value, the length range, and whether the parameter value belongs to the set of observed valid values.
[0057] The character type constraint is derived from the character composition of parameter values corresponding to the current state node in pre-collected legitimate API requests. Specifically, parameter value samples from all API requests passing through the current state node are collected, and the character categories appearing in these parameter values are statistically analyzed. Examples include numeric characters 0-9, hexadecimal characters 0-9A-F, and Base64 URL characters. These statistically analyzed character categories are used as the character type constraint for this parameter validation node. This constraint requires that the character composition of parameter values at corresponding positions in subsequent online requests must belong to the aforementioned statistically analyzed character categories.
[0058] The length range constraint is obtained by statistically analyzing the lengths of parameter values corresponding to the state node in pre-collected valid API requests, and a preset margin can be added. Specifically, parameter value samples from all API requests passing through the state node are collected, the length distribution of these parameter values is statistically analyzed, and the minimum and maximum values are taken as the lower and upper limits of the length range. A preset margin can be added on this basis, such as subtracting 1 from the lower limit and adding 1 to the upper limit, to accommodate boundary cases. This constraint is used to ensure that the length of the parameter value at the corresponding position in subsequent online requests must be within the above-mentioned length range.
[0059] The constraint on whether the parameter value belongs to the observed valid value set is implemented through an approximate member filter. The observed valid value set refers to the set of all parameter values corresponding to the state node extracted from the valid API request samples collected in step S101. This set uses an approximate member filter as its storage structure. The approximate member filter is used to perform a member query on the parameter value to determine whether the parameter value belongs to the observed valid value set. The approximate member filter includes, but is not limited to, Bloom filters, Cuckoo filters, or XOR filters. The approximate member filter is configured with a constant upper bound on the single-slot capacity and a configurable false positive rate. For example, a Cuckoo filter that supports deletion operations can be used to eliminate expired values. The target false positive rate can be set empirically, and a constant upper bound is set on the filter capacity per single slot.
[0060] This step is one of the innovative aspects of this invention and has the following beneficial effects: First, by replacing high-cardinality state nodes with parameter verification nodes, a large number of precisely matched transition edges and state nodes are replaced by a single parameter verification node, fundamentally solving the memory bloat problem in scenarios with high-cardinality dynamic parameters. The space complexity of the replaced finite state converter topology is no longer related to the number of different values of the dynamic parameters, but rather to the number of parameter verification nodes.
[0061] Second, by using the triple verification information carried by the parameter verification node, the ability to constrain dynamic parameter values is preserved while compressing storage. That is, for requests with valid structures but parameter values that fail verification, they are directed to the review channel, so that the dynamic parameter at that position does not need to be generalized to an arbitrary number, thus avoiding the release of malformed parameter values due to excessively broad generalization.
[0062] Step S104: Online status transition and parameter verification.
[0063] This step is used to perform byte-by-byte state transfer of API requests obtained online during security monitoring in the finite state converter topology graph structure, and to perform triple verification when the transfer reaches the parameter verification node.
[0064] Specifically, for each API request acquired online during security monitoring, it is converted into a string representation, including the request method, path, and rearranged query parameters. Then, starting from the starting node of the finite state converter topology graph, a byte-by-byte state transition is performed.
[0065] The specific process of the byte-by-byte state transition is as follows: each character of the API request string is read sequentially, and it is checked under the current state node whether there is a transition edge whose character marker is the same as the currently read character; if there is, the transition edge is used to move to the next state node and continue processing the next character; if there is no such transition edge, the state transition is terminated and the API request fails to complete the state transition successfully.
[0066] Furthermore, when the state transition reaches a parameter verification node, the state transition is paused, and the triple verification information carried by the parameter verification node is used to verify the parameter value in the API request corresponding to the position of the parameter verification node.
[0067] The verification process specifically includes the following three sub-steps: Sub-step S1041: Character type verification.
[0068] Extract the parameter value from the API request corresponding to the current parameter validation node position, and check whether each character of the parameter value belongs to the character category limited by the character type constraint of this parameter validation node. If all characters belong to the allowed character categories, the character type validation passes; otherwise, the character type validation fails.
[0069] Sub-step S1042: Length range verification.
[0070] Calculate the length (i.e., number of characters) of the above parameter value, and check whether the length is between the minimum and maximum lengths defined by the length range constraint of the parameter validation node. If the length is within the allowed range, the length range validation passes; otherwise, the length range validation fails.
[0071] Sub-step S1043: Query and verify the members of the observed valid value set.
[0072] The approximate member filter is used to query whether the above parameter value belongs to the set of observed valid values of the parameter verification node. The query result of the approximate member filter has two possibilities: if the approximate member filter determines that the parameter value definitely does not exist, it means that the parameter value does not belong to the set of observed valid values and fails the member query verification; if the approximate member filter determines that the parameter value may exist, it means that the parameter value may belong to the set of observed valid values and passes the member query verification.
[0073] It is important to note that the approximate member filter has a one-sided error characteristic, meaning that the determination of "non-existent" is deterministic, while only the determination of "possibly existing" carries a controllable false positive rate. Therefore, when the approximate member filter determines that a parameter value definitely does not exist, this determination is 100% certain; the parameter value indeed does not belong to the set of observed valid values. When the approximate member filter determines that a parameter value may exist, this determination has a certain probability of being a false positive; that is, the parameter value does not actually belong to the set of observed valid values but is mistakenly judged as possibly existing.
[0074] In summary, step S104 divides online detection into two stages: state transition and parameter verification. In the state transition stage, deterministic byte-by-byte matching is maintained, while in the parameter verification stage, efficient constraint checks are performed using triple verification information. This allows online detection to maintain the linear matching complexity of a deterministic automaton while also achieving effective verification of dynamic parameters.
[0075] Step S104 achieves the following beneficial effects: First, the state transition matching time of a single API request is linearly related to the length of the request string, and is independent of the number of different values of dynamic parameters. Furthermore, the state transition process does not involve backtracking and will not trigger a regular expression denial-of-service attack.
[0076] Second, through the triple verification of parameter verification nodes, malformed parameter values that exceed the length range constraint or whose character composition does not conform to the expected type can be identified and intercepted, avoiding the problem of malformed parameter values being misjudged as legal and bypassing detection caused by the generalized matching of traditional regular expressions.
[0077] Third, by leveraging the one-sided error characteristics of the approximate member filter, the direction of uncertainty error is converged towards the review process. Specifically, when the approximate member filter determines that the parameter value does not exist, the determination is considered definitive, with no possibility of misjudgment, and therefore the corresponding API request is directed to the review channel. When the parameter value may exist, the determination has a controllable probability of misjudgment, and therefore the corresponding API request is allowed. By configuring the probability of misjudgment to a sufficiently small value, most legitimate requests can be allowed while ensuring security.
[0078] Step S105: Diversion and processing.
[0079] This step is used to perform a release operation, a review operation, or an interception operation on each API request based on the status transition result and triple verification result of step S104.
[0080] Specifically, the following three situations will be handled accordingly: (1) Release operation.
[0081] If the API request successfully completes its state transition within the finite state converter topology (i.e., starting from the starting node, sequentially matching each character of the API request string until reaching the final state node), and the parameter value passes all checks in the triple verification information (i.e., character type verification passes, length range verification passes, and the approximate member filter determines that the parameter value may exist), then the API request is allowed. This scenario corresponds to the API request's structure perfectly matching a legitimate state transition path in the behavior whitelist, and its dynamic parameter value meeting the requirements in terms of character type, length range, and observed legitimacy; in this case, the request is deemed definitively legitimate.
[0082] (2) Review operation.
[0083] If the API request successfully completes the state transition in the finite state converter topology, but the parameter value fails at least one of the three verification checks, including: the character composition of the parameter value does not conform to the character type constraint, the length of the parameter value exceeds the length range constraint, or the approximate member filter determines that the parameter value definitely does not exist, then a review operation is performed on the API request. This situation corresponds to the API request structure matching a legal state transition path in the behavior whitelist, but its dynamic parameter value failing to completely pass the three verifications. In this case, the present invention neither directly allows nor directly blocks the request, but instead directs it to the deep detection engine for further analysis and review to reduce the false positive rate.
[0084] The review process involves directing the API request to a deep detection engine for further security analysis. The deep detection engine employs one or more detection methods, including semantic analysis, behavioral analysis, or rule matching, to perform security checks on the API request and outputs a final judgment result. If the deep detection engine determines the API request is legitimate, it allows it; if it determines the API request is malicious, it blocks it.
[0085] It is important to note that when the approximate member filter determines that a parameter value does not exist, this determination is definitive, indicating that the parameter value indeed does not belong to the set of observed valid values. In such cases, traditional solutions either allow the pass directly or block it outright. This invention guides this situation to a review process; if the review passes, the pass is allowed; if the review fails, the pass is blocked, thus avoiding both blind trust and blind rejection.
[0086] (3) Interception operation.
[0087] If the API request fails to complete the state transition in the finite state converter topology graph, i.e., during the state transition process, a character cannot find a matching transition edge under the current state node, causing the state transition to terminate prematurely and fail to reach the final state node, then the API request will be intercepted. This situation corresponds to the path corresponding to the API request not being in the behavior whitelist, belonging to a shadow API request or directory scanning request, and should be intercepted.
[0088] This step introduces a review operation as a third handling method, directing uncertain traffic to a secure review operation and using secure review as a fallback, thereby avoiding the security risks that may be introduced by directly allowing the traffic to pass through, as well as avoiding false alarms that may be caused by directly blocking the traffic.
[0089] More preferably, it may also include step S106: online incremental update.
[0090] This step is used to implement online incremental updates to the finite state converter topology graph structure, which can include newly emerging legitimate API request paths into the behavior whitelist without downtime or full reconstruction.
[0091] Since the minimized finite state converter topology is usually an immutable structure, it is difficult to modify the topology in place at runtime. Therefore, this invention provides two parallel online update implementation methods.
[0092] Implementation method 1: Incremental double buffering mechanism.
[0093] First, a variable incremental layer and an immutable base layer are partitioned into the finite state converter topology. The incremental layer, as part of the finite state converter topology, is used to temporarily store newly emerging valid API request paths and their corresponding state nodes and transition edges. The base layer, as the other part of the finite state converter topology, is used to store fixed valid API request paths and their corresponding state nodes and transition edges, and provides online state transition queries.
[0094] Then, the newly emerging valid API request paths and their corresponding state nodes and transition edges stored in the incremental layer are periodically merged into the base layer to rebuild a new base layer. Specifically, the background process periodically (e.g., at regular intervals, or when the number of paths accumulated in the incremental layer reaches a certain threshold) initiates the merge and rebuild operation: reading all new paths stored in the incremental layer, merging these new paths with the existing paths in the base layer, and reconstructing a new immutable base layer containing all paths.
[0095] Next, the base layer pointer used by the online query is switched from the old base layer to the new base layer. During the switch, the base layer pointer used by the online query is pointed from the old base layer to the new base layer, and this switch operation is completed instantly. Therefore, the threads processing the online query will not be aware of the switch process, and there will be no mixing of old and new data or partial updates.
[0096] Subsequent online queries prioritize the incremental layer; if no match is found, the query continues in the base layer. Specifically, when an online API request arrives, the system first checks the incremental layer for a corresponding state transition path. If a path exists, it is used directly for state transition; otherwise, the search continues in the base layer. This query order—first the incremental layer, then the base layer—ensures that newly added paths take effect immediately. Furthermore, the immutable base layer eliminates the need for locking during simultaneous multi-threaded queries, preventing data from being mixed with old data or waiting for locks to be released.
[0097] Implementation Method 2: Incremental Minimum Acyclic Finite Automaton Algorithm.
[0098] The incremental minimum acyclic finite state automaton algorithm is used to incrementally incorporate newly emerging valid API request paths directly into the topology of the finite state converter. This incremental minimum acyclic finite state automaton algorithm (e.g., Daciuk's incremental construction algorithm) is an incremental construction algorithm that minimizes the finite state automaton in real time while adding new strings one by one. This algorithm can directly insert new strings into the appropriate positions in the existing automaton without rebuilding the entire automaton, and maintains the minimization property of the automaton in real time during the insertion process.
[0099] It should be noted that although both implementation methods can achieve online incremental updates of the topology graph, the first implementation method (incremental double buffering mechanism) has the following advantages over the second implementation method (incremental minimum acyclic finite automaton algorithm): First, the query process is unaffected by the update process. The base layer is immutable, allowing multiple threads to perform lock-free queries simultaneously. Updates are completed independently in the background and take effect via pointer switching, thus unaffected query performance.
[0100] Second, the system is more robust. The switching process is completed instantly, and the old base layer can still be retained as a rollback backup after the switch. If there are problems with the new base layer, it can be quickly rolled back.
[0101] Third, the implementation path is clear, and the engineering reliability is higher. The responsibilities of the incremental layer and the base layer are clearly defined. The incremental layer is responsible for temporarily accommodating new paths, while the base layer is responsible for stable service. The process of periodic merging and pointer switching is highly standardized and does not depend on the implementation details of specific automata algorithms.
[0102] This step, as a preferred embodiment of the present invention, realizes online incremental updates of the topology graph structure of the finite state converter through an incremental double buffering mechanism or an incremental minimum acyclic finite state automaton algorithm. The entire process is lock-free and has zero pauses, ensuring the consistency and availability of online queries.
[0103] The method provided by this invention will be fully explained below with a specific API path example.
[0104] A certain business system has the following API endpoint: GET: / api / v1 / users / {id} / profile, where {id} is the user identifier, which is a variable parameter. During system operation, the following valid API request samples were collected through a bypass mirroring method: GET: / api / v1 / users / 1 / profile GET: / api / v1 / users / 2 / profile GET: / api / v1 / users / 100000 / profile (1) Pre-processing stage.
[0105] The query parameter key-value pairs in the three collected valid API request samples are rearranged lexicographically by parameter key name. In this example, none of the three requests contain query parameters, so the rearranged string is the same as the original string: GET: / api / v1 / users / 1 / profile GET: / api / v1 / users / 2 / profile GET: / api / v1 / users / 100000 / profile (2) Finite state converter topology construction stage.
[0106] Input the three rearranged API request strings into the finite state converter builder. During the construction process: The prefix GET: / api / v1 / users / is identical in all three requests, and is stored only once in the topology structure through prefix sharing.
[0107] The numbers 1, 2, and 100000 are all different, forming three parallel branches in the topology graph structure. Starting from the first character after GET: / api / v1 / users / , they lead to different state nodes corresponding to 1, 2, and 1 (where the first character of 100000 is also 1, sharing part of the path with 1).
[0108] The suffix / profile is identical in all three requests, and by sharing the suffix at the end of the topology graph structure, the three branches are merged into the same group of state nodes.
[0109] After the above construction process, the finite state converter topology graph structure stores three API request paths in a compact form.
[0110] (3) High base number state node identification and parameter verification node replacement stage.
[0111] Traverse each state node in the finite state converter topology graph and count the out-degree and access frequency of each state node.
[0112] At the state node corresponding to the starting position of the numeric parameter after GET: / api / v1 / users / , there are multiple different outgoing edge branches (corresponding to paths starting with different numbers such as 1, 2, etc.). The outgoing degree of this state node is statistically greater than the first threshold of 256. In a real production environment, for the user information API path of an e-commerce platform, since the number of registered users usually reaches millions or even tens of millions, the outgoing degree of this state node will be much greater than 256, for example, reaching tens of thousands or even hundreds of thousands. The access frequency of this state node is estimated by Count-Min Sketch to be less than the second threshold of 10 times, that is, each specific parameter value path is a low-frequency path.
[0113] Because the out-degree of this state node exceeds the first threshold of 256 and the access frequency is lower than the second threshold of 10, this state node is determined to be a high-cardinality state node and is replaced with a parameter verification node. The triple verification information carried by this parameter verification node is as follows: Character type constraint: This constraint is derived from the character composition of the numeric parameter portion in the three samples. The parameter values for the three samples are 1, 2, and 100000, and their character composition is all numeric characters 0-9. Therefore, the character type constraint is limited to the numeric character class.
[0114] Length range constraint: obtained by statistically analyzing the lengths of the numerical parameter portions in the three samples. The parameter values of the three samples are 1, 1, and 6 respectively. The minimum and maximum values are taken to obtain the length range [1, 6], and a preset margin (adding 1 above and below) can be added to obtain [0, 7].
[0115] Observed valid value set constraint: The parameter values of three samples {1, 2, 100000} are stored using a Cuckoo filter, and the target misclassification rate is set to 1%.
[0116] After the replacement is completed, the original three parallel digital branches in the finite state converter topology are compressed into a single parameter validation node, and the structure becomes: GET: / api / v1 / users / → [parameter validation node] → / profile.
[0117] (4) Online detection stage.
[0118] Scenario 1: Release.
[0119] When the online request GET: / api / v1 / users / 999 / profile arrives: State transition: The request string is matched byte by byte starting from the starting node. GET: / api / v1 / users / partially matches successfully and reaches the parameter validation node.
[0120] Character type validation: Each character in parameter value 999 is a numeric character, thus passing character type validation.
[0121] Length range verification: The length of parameter value 999 is 3, which is within the range of [0,7] after adding margin, and passes the length range verification.
[0122] Observed valid value set member query: The approximate member filter queries whether 999 belongs to {1, 2, 100000}. Since 999 is not in the set, the approximate member filter may return "definitely not present" or "possibly present". If it returns "possibly present", the member query validation passes. If all three validations pass, the request is allowed.
[0123] Scenario 2: Security review.
[0124] When the online request GET: / api / v1 / users / 888 / profile arrives: Status transition: Successfully reached the parameter verification node.
[0125] Character type validation: The parameter value 888 is a numeric character, so it passes.
[0126] Length range verification: The length of parameter value 888 is 3, which is within the length range, so it passes.
[0127] Membership query of the observed valid value set: The approximate membership filter queries whether 888 belongs to {1, 2, 100000}. Since 888 is not in the set, the approximate membership filter returns "definitely does not exist". Failing the membership query validation, the request is subjected to a review operation, i.e., redirected to the deep detection engine for further analysis.
[0128] Scenario 3: Interception (structural anomaly).
[0129] When the online request GET: / api / v1 / users / 999 / admin_test arrives: State transition: GET: / api / v1 / users / 999 / partially matched, but at the subsequent character 'a' (the first character of admin_test), no transition edge marked with the character 'a' can be found starting from the current state node (because the suffix at this position in the valid path is / profile, which starts with the character 'p'), the state transition terminates prematurely and fails to reach the final state node.
[0130] The request failed to complete the state transition in the finite state converter topology and was intercepted, i.e., it was determined to be a shadow API request or a directory scan request and was blocked.
[0131] Scenario 4: Security check (parameter distortion).
[0132] When the online request GET: / api / v1 / users / 99999999999 / profile arrives: Status transition: Successfully reached the parameter verification node.
[0133] Character type validation: Each character in the parameter value 99999999999 is a numeric character, so it passes.
[0134] Length range check: The length of parameter value 99999999999 is 11, which exceeds the length range [0,7] after adding margin, and fails the length range check.
[0135] The request failed at least one of the three verification messages and was subject to a security review, meaning it was directed to the deep detection engine for further analysis.
[0136] (5) Online incremental update phase.
[0137] During system operation, a new legitimate user identifier 123456 was discovered, and the corresponding API request was GET: / api / v1 / users / 123456 / profile. This request was directed to security review during the online detection in step (4). After manual or automatic confirmation, it was confirmed as a legitimate request and needs to be added to the behavior whitelist.
[0138] The new path GET: / api / v1 / users / 123456 / profile is stored in the variable incremental layer. Subsequent online queries prioritize the incremental layer, thus the path takes effect immediately without requiring system downtime or a full rebuild. The background process periodically merges the accumulated new paths from the incremental layer into the immutable base layer and rebuilds a new base layer, replacing it via pointer switching. The entire process is lock-free and interrupts-free for online queries.
[0139] The above embodiments fully demonstrate the entire process of the API security detection method provided by this invention, from offline construction to online detection and then to online incremental updates. The entire embodiment verifies the technical effectiveness of this invention in suppressing overgeneralization and false negatives, controlling memory usage, reducing false positives, and supporting online updates.
[0140] See Figure 2 Another embodiment of the present invention provides an API security detection device 200, including a construction module 201, a replacement module 202, a verification module 203, and a processing module 204. The API security detection device 200 is capable of executing the API security detection method in the method embodiment.
[0141] Specifically, the API security testing device 200 includes: The construction module 201 is used to rearrange the query parameter key-value pairs in the pre-collected valid API requests according to the lexicographical order of the parameter key names, and construct a finite state converter topology graph structure using the rearranged API request string; wherein, the finite state converter topology graph structure includes state nodes and transition edges connecting two state nodes; the state node represents the current position when the API request string is compared with the state transition path stored in the finite state converter topology graph structure, and the state node includes a starting node for initiating state transition and a final state node for identifying successful completion of state transition; the transition edge has a character marker, which is used to guide the transition from the current state node to the next state node when the current character of the API request string matches the character marker of the transition edge; The replacement module 202 is used to replace state nodes in the finite state converter topology graph structure whose out-degree exceeds a first threshold and whose access frequency is lower than a second threshold with parameter verification nodes; wherein, the parameter verification node carries triple verification information for the character type, length range, and whether the parameter value belongs to the set of observed legal values; The verification module 203 is used to perform byte-by-byte state transition of all API requests obtained online during the security monitoring process in the finite state converter topology graph structure, and to verify the parameter value of each API request corresponding to the parameter verification node when the transition reaches the parameter verification node using the triple verification information. The processing module 204 is used to perform a release operation, a security review operation, or an interception operation on each obtained API request based on whether the status transition is successfully completed and the verification result of the triple verification information.
[0142] It should be noted that the API security detection device 200 provided in this embodiment corresponds to the technical solutions that can be used to execute various method embodiments. Its implementation principle and technical effect are similar to the methods, and will not be repeated here.
[0143] See Figure 3 This embodiment also provides a structural schematic diagram of an electronic device 300, which is used to implement the API security detection method in the method embodiment. The electronic device 300 in this embodiment can be a network security device deployed on the business traffic path, including but not limited to: a standalone Web application firewall device, a security detection module integrated into an API gateway or load balancer, a lightweight proxy node deployed in a microservice architecture as a sidecar container, a traffic processing node built on a general-purpose server equipped with a high-performance network card and a network data plane development kit, and a detection component as part of a cloud-native security service mesh. This invention does not impose special restrictions on the specific type of electronic device 300, as long as it has the ability to load the behavior whitelist binary graph into memory in a read-only manner through memory mapping, and to perform byte-by-byte state transitions and parameter verification for online API requests. Figure 3 The electronic device 300 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.
[0144] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing device 301, ROM 302, and RAM 303 are interconnected via a bus 305. An input / output (I / O) interface 304 is also connected to the bus 305.
[0145] Typically, the following devices can be connected to I / O interface 304: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0146] The above description is merely a preferred embodiment of the present invention. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to the specific combination of the above-described technical features, but should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.
Claims
1. An API security detection method, characterized in that, Includes the following steps: The query parameter key-value pairs in the pre-collected valid API requests are rearranged lexicographically by parameter key name, and a finite state converter topology graph structure is constructed using the rearranged API request string. The finite state converter topology graph structure includes state nodes and transition edges connecting two state nodes. Each state node represents the current position when the API request string is compared with the state transition path stored in the finite state converter topology graph structure. Each state node includes a starting node for initiating a state transition and a final state node for indicating successful completion of the state transition. Each transition edge has a character marker, used to guide the transition from the current state node to the next state node when the current character of the API request string matches the character marker of the transition edge. In the finite state converter topology graph structure, state nodes with an out-degree exceeding the first threshold and an access frequency lower than the second threshold are replaced with parameter verification nodes; wherein, the parameter verification node carries triple verification information for the character type, length range, and whether the parameter value belongs to the set of observed legal values; All API requests acquired online during the security monitoring process are transferred byte by byte in the finite state converter topology graph structure, and when the transfer reaches the parameter verification node, the parameter value corresponding to the parameter verification node in each API request is verified using the triple verification information. Based on whether the state transition was successfully completed and the verification result of the triple verification information, each obtained API request is subject to a release operation, a security review operation, or an interception operation.
2. The API security detection method according to claim 1, characterized in that: If the API request successfully completes the state transition in the finite state converter topology and the parameter value passes all the checks in the triple verification information, then the API request is allowed. If the API request successfully completes the state transition in the finite state converter topology, but the parameter value fails at least one of the triple verification information, then a security review operation is performed on the API request. If the API request fails to complete the state transition in the finite state converter topology, an interception operation is performed on the API request.
3. The API security detection method according to claim 1, characterized in that, The finite state converter topology graph structure is updated online using the following steps: In the finite state converter topology, a variable incremental layer and an immutable base layer are constructed. The incremental layer is used to temporarily store newly emerging legal API request paths and their corresponding state nodes and transition edges. The base layer is used to store fixed legal API request paths and their corresponding state nodes and transition edges, and provides online state transition query. The newly emerging valid API request paths and their corresponding state nodes and transition edges stored in the incremental layer are periodically merged into the base layer to reconstruct and generate a new base layer. Switch the base layer pointer used for online queries from the old base layer to the new base layer; When querying online, the system first checks if a corresponding state transition path exists in the incremental layer. If it exists, the path in the incremental layer is used directly for state transition. If it does not exist, the system continues to search in the base layer.
4. The API security detection method according to claim 1, characterized in that, An approximate member filter is used to verify whether a parameter value belongs to the set of observed valid values. The approximate member filter is used to perform a member query on the parameter value to determine whether the parameter value belongs to the set of observed valid values.
5. The API security detection method according to claim 1, characterized in that, The finite state converter topology is deployed in read-only mode in the memory of a web application firewall or proxy node via memory mapping, and does not trigger disk I / O during online state transitions.
6. The API security detection method according to claim 1, characterized in that, The same start and end character sequences in multiple API request strings share the same set of state nodes and transition edges in the finite state converter topology.
7. An API safety testing device, characterized in that, include: A construction module is used to rearrange the query parameter key-value pairs in pre-collected valid API requests according to the lexicographical order of the parameter keys, and construct a finite state converter topology graph structure using the rearranged API request string. The finite state converter topology graph structure includes state nodes and transition edges connecting two state nodes. Each state node represents the current position when the API request string is compared with the state transition paths stored in the finite state converter topology graph structure. Each state node includes a starting node for initiating a state transition and a final state node for indicating successful completion of the state transition. Each transition edge has a character marker, used to guide the transition from the current state node to the next state node when the current character of the API request string matches the character marker of the transition edge. The replacement module is used to replace state nodes in the finite state converter topology graph structure whose out-degree exceeds a first threshold and whose access frequency is lower than a second threshold with parameter verification nodes; wherein, the parameter verification node carries triple verification information for the character type, length range, and whether the parameter value belongs to the set of observed legal values; The verification module is used to perform byte-by-byte state transition of all API requests obtained online during the security monitoring process in the finite state converter topology graph structure, and to verify the parameter value of each API request corresponding to the parameter verification node when the transition reaches the parameter verification node using the triple verification information. The processing module is used to perform a release operation, a security review operation, or an interception operation on each obtained API request based on whether the status transition is successfully completed and the verification result of the triple verification information.
8. The API security detection device according to claim 7, characterized in that, The processing module is further used for: If the API request successfully completes the state transition in the finite state converter topology and the parameter value passes all the checks in the triple verification information, then the API request is allowed. If the API request successfully completes the state transition in the finite state converter topology, but the parameter value fails at least one of the triple verification information, then a security review operation is performed on the API request. If the API request fails to complete the state transition in the finite state converter topology, an interception operation is performed on the API request.
9. An API security detection device according to claim 7, characterized in that, It also includes an online update module for: In the finite state converter topology, a variable incremental layer and an immutable base layer are constructed. The incremental layer is used to temporarily store newly emerging legal API request paths and their corresponding state nodes and transition edges. The base layer is used to store fixed legal API request paths and their corresponding state nodes and transition edges, and provides online state transition query. The newly emerging valid API request paths and their corresponding state nodes and transition edges stored in the incremental layer are periodically merged into the base layer to reconstruct and generate a new base layer. Switch the base layer pointer used for online queries from the old base layer to the new base layer; When querying online, the system first checks if a corresponding state transition path exists in the incremental layer. If it exists, the path in the incremental layer is used directly for state transition. If it does not exist, the system continues to search in the base layer.
10. An API security detection device according to claim 7, characterized in that: Multiple API request strings sharing the same start and end character sequences in the finite state converter topology graph structure share the same set of state nodes and transition edges. The finite state converter topology is deployed in the memory of a Web application firewall or proxy node in read-only mode via memory mapping, and does not trigger disk I / O during online state transitions. An approximate member filter is used to verify whether a parameter value belongs to the set of observed valid values. The approximate member filter is used to perform a member query on the parameter value to determine whether the parameter value belongs to the set of observed valid values.