An API asset processing method, system, device and medium based on sample entropy and load entropy

Abstract

The application provides an API asset processing method, system, device and medium based on sample entropy and load entropy, and belongs to the technical field of API asset processing. The method first collects HTTP interface information, and calculates the randomness of load entropy quantization content. If the load entropy is less than a threshold value, it is determined as a static interface and is directly discovered or collected. Otherwise, the difference between the load entropy of the interface and the average load entropy of interfaces with the same URL prefix is compared, and if the difference is small, the interfaces are merged. For the remaining dynamic interfaces, sample entropy is used to quantify the time regularity of the calling sequence, and the interfaces are classified and processed according to the entropy value: less than 1.2 is forced to merge, 1.2-2.5 is parameterized, and greater than 2.5 is isolated and alarmed. The load entropy is used to quickly filter static interfaces, reducing the subsequent calculation amount. The sample entropy is combined with time regularity analysis to avoid the limitation of path matching, improve the accuracy of dynamic interface merging, and reduce asset duplicate records.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of API asset processing, and particularly relates to an API asset processing method, system, device and medium based on sample entropy and load entropy. BACKGROUND

[0002] In the field of network security, API has become an important entry for attack surface expansion. The API asset list established by general enterprises supports security capabilities such as vulnerability scanning, access control, and abnormal behavior detection. For large enterprises, usually hundreds to thousands of microservices are operated, and each service exposes multiple API endpoints. Unified registration, version management, call monitoring and permission control of these interfaces are the premise of realizing efficient service governance.

[0003] At present, Shannon entropy is introduced to realize network security and traffic analysis, which is used to measure the randomness and uncertainty of data. However, it depends on URL complete matching. If the static interface URL contains dynamic interfaces similar to time stamp, it will be misjudged as multiple independent interfaces, resulting in asset library redundancy. For example, the fixed picture interface of CDN service will be recorded as 3 different interfaces due to the addition of random parameters in the front end.

[0004] There are also interface upgrades, but the URL prefix changes, and the core function does not change. URL matching cannot identify the same logical interface, resulting in scattered asset library. From the version iteration of the user information interface of an e-commerce system, version 1 and version 2 have different URL prefixes, and the traditional method will regard them as two independent interfaces. Moreover, malicious scanning requests and normal dynamic interfaces are difficult to distinguish, which may misjudge abnormal traffic as normal interface, or delete normal interface due to random content, causing interface misjudgment and affecting normal use. SUMMARY

[0005] The application provides an API asset processing method based on sample entropy and load entropy, which optimizes resource use efficiency while ensuring identification accuracy, and improves the standardization and security of API asset management.

[0006] The method comprises the following steps:

[0007] S101, acquiring HTTP interface information;

[0008] S102, analyzing the load entropy value of the HTTP interface information, the load entropy value being used to quantify the randomness of interface call content;

[0009] S103, filtering and deciding according to the load entropy value, if the load entropy value is greater than or equal to the dynamic critical threshold, executing S104;

[0010] S104, difference comparison is made between the load entropy value of the interface to be processed and the average load entropy value of the interfaces with the same URL prefix in the existing assets, and if the merging condition is not met, S105 is executed;

[0011] S105, sample entropy value of the traffic message of the interface to be processed is calculated, and the sample entropy value is used to quantify the time regularity of the interface call sequence;

[0012] S106, filtering decision rules are made according to the sample entropy value to realize the discovery and merging of the API interface;

[0013] The decision rules are: the sample entropy value less than the lower limit value of the first interval is determined as a fixed interface and forced to be merged, the sample entropy value between the upper and lower limit values of the first interval is determined as a dynamic parameter interface and subjected to path parameterization processing, and the sample entropy value greater than the upper limit value of the second interval is determined as abnormal traffic and subjected to alarm isolation.

[0014] Further, in step S102, the calculation formula of the HTTP interface information load entropy value is:

[0015]

[0016] Wherein, p k is the frequency of byte value, and k is all byte values appearing in the HTTP interface load data.

[0017] Further, in step S105, the step of calculating the sample entropy value of the traffic message is:

[0018] S1051, initial parameters for sample entropy calculation are defined, and embedding dimension m and similarity tolerance r are set;

[0019] S1052, the length value of the request body or response body is extracted from the traffic message of the interface to be processed, and is arranged in time sequence to form a time sequence T={t1, t2,..., tn}, where n is the length of the time sequence; n

[0020] S1053, m-dimensional message length subsequence is constructed, and all continuous m-dimensional length subsequences , 1≤i≤n−m are generated based on the time sequence T, wherein each subsequence Xm i corresponds to the interface call length in the time window ;

[0021] S1054, the number of similar vectors of the m-dimensional subsequence is calculated, and for each m-dimensional subsequence Xm i , all other m-dimensional subsequences Xm j , j≠i are traversed, and Xm i and Xm j ​Chebyshev distance; the number of sub-sequences satisfying the Chebyshev distance less than or equal to the similarity tolerance r;

[0022] S1055, for each sub-sequence Xm i , calculate the similarity vector proportion = × the number of similarity vectors, where the denominator is the total number of comparisons after excluding itself;

[0023] S1056, calculate the m-dimensional overall similarity proportion, and average the similarity vector proportions of all m-dimensional sub-sequences to obtain , , which represents the average self-similarity of all m-dimensional sub-sequences, reflecting the overall regularity of the length sequence in m dimensions;

[0024] S1057, extend to m+1-dimensional sub-sequences and repeat steps S1054 to S1056 to calculate the m+1-dimensional overall similarity proportion B m +1 (r);

[0025] S1058, based on the m-dimensional and m+1-dimensional overall similarity proportions, calculate the sample entropy value by the formula SampEn(m, r) = -ln(B m+1 (r) / B m (r)), where the sample entropy value is used to quantify the temporal regularity of the interface call length sequence.

[0026] It should be further explained that step S104 specifically includes:

[0027] retrieve a set of historical interfaces from the API asset library that match the current to-be-processed interface URL prefix, where the URL prefix match means that the first M characters of the current interface URL are the same as the first M characters of the historical interface URL, and M is a preset matching length;

[0028] calculate the average load entropy of all interfaces in the set of historical interfaces;

[0029] calculate the absolute difference between the load entropy value of the current to-be-processed interface and the average value to obtain the entropy difference degree;

[0030] compare the entropy difference degree with a preset difference degree threshold value, if the difference degree is less than the threshold value, determine that the current interface and the historical interface are the same logical interface, and merge the current interface into the API asset to which the historical interface belongs; if the difference degree is greater than or equal to the threshold value, end step S104 and enter step S105.

[0031] It should be further explained that step S103 specifically includes:

[0032] acquire a dynamic critical threshold preset by the system, which is generated by statistical load entropy values of historical static interfaces;

[0033] verify the validity of the load data of the current interface to be processed, and exclude abnormal short messages or non-HTTP protocol messages caused by network packet loss or truncation;

[0034] numerically compare the verified valid load entropy value with the dynamic critical threshold to determine whether the load entropy value is less than the dynamic critical threshold;

[0035] If the load entropy value is less than the dynamic critical threshold, secondary verification is performed in combination with the structural characteristics of the URL, and after confirming that the static interface characteristics are met, new API asset discovery or matching of interfaces with the same URL and content in the asset library is performed. If the condition of being less than is not met or the secondary verification fails, step S103 is ended and step S104 is entered.

[0036] Further, step S106 specifically includes:

[0037] acquire sample entropy interval parameters preset by the system, including a first interval lower limit value, a first interval upper limit value, and a second interval upper limit value, wherein the first interval upper limit value and the second interval upper limit value coincide, and are used to distinguish dynamic parameter interfaces and abnormal traffic;

[0038] extract the sample entropy value of the current interface to be processed from the output result of step S105;

[0039] perform multi-level comparison of the sample entropy value with the preset interval parameters: first determine whether it is less than the first interval lower limit value, and if so, mark it as a fixed interface; if not, continue to determine whether it is less than or equal to the first interval upper limit value, and if so, mark it as a dynamic parameter interface; if neither condition is met, mark it as abnormal traffic;

[0040] perform classification processing operations according to the marking results: for a fixed interface, verify the consistency of the request body and response body content with the same URL interface in the existing asset, and after passing, forcibly merge it into the corresponding API asset; for a dynamic parameter interface, extract the position of the length sequence change in the traffic message, parameterize the corresponding position in the URL path, and update the path template in the API asset; for abnormal traffic, record its URL, timestamp, and sample entropy value, trigger an alarm, and store the related information in an exception log.

[0041] Further, in S103, if the load entropy value is less than the dynamic critical threshold, it is determined that the static interface characteristics are met, and accordingly, new API assets are directly discovered or collected into existing API assets, and the process ends;

[0042] In S104, if the entropy difference degree is less than the difference degree threshold, it is determined that the same logical interface, and the interfaces are merged and collected into the API asset, and the process ends.

[0043] The application further provides an API asset processing system based on sample entropy and load entropy, which comprises:

[0044] An interface information acquisition module is configured to acquire HTTP interface information.

[0045] A load entropy calculation module is configured to analyze the load entropy value of the HTTP interface information, and the load entropy value is used to quantify the randomness of the interface call content.

[0046] An interface filtering module is configured to make a filtering decision according to the load entropy value, and if the load entropy value is greater than or equal to the dynamic critical threshold, the entropy difference comparison and merging module is executed.

[0047] An entropy difference comparison and merging module is configured to compare the load entropy value of the interface to be processed with the average load entropy value of the interface with the same URL prefix in the existing asset, and if the merging condition is not met, the sample entropy calculation module is executed.

[0048] A sample entropy calculation module is configured to calculate the sample entropy value of the interface to be processed based on the traffic message of the interface, and the sample entropy value is used to quantify the time regularity of the interface call sequence.

[0049] A sample entropy decision module is configured to make a filtering decision according to the sample entropy value, so as to realize the discovery and merging of the API interface.

[0050] According to another embodiment of the application, an electronic device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the API asset processing method based on sample entropy and load entropy when executing the program.

[0051] According to another embodiment of the application, an electronic device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the API asset processing method based on sample entropy and load entropy when executing the program.

[0052] From the above technical solutions, it can be seen that the application has the following advantages:

[0053] The application provides an API asset processing method based on sample entropy and load entropy.

[0054] The application filters static interfaces quickly through load entropy, improves processing efficiency, accurately merges similar dynamic interfaces by comparing the load entropy mean values of interfaces with the same URL prefix, reduces asset redundancy, enhances dynamic parameter interface identification accuracy by combining sample entropy analysis from the time dimension, and effectively distinguishes abnormal traffic. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the application, the drawings needed in the description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0056] Figure 1 The API asset processing method based on sample entropy and load entropy is a flowchart.

[0057] Figure 2 The API asset processing method based on sample entropy and load entropy is a flowchart.

[0058] Figure 3 The API asset processing method based on sample entropy and load entropy is a flowchart.

[0059] Figure 4 The API asset processing system based on sample entropy and load entropy is a schematic diagram.

[0060] Figure 5 The electronic device is a schematic diagram. DETAILED DESCRIPTION

[0061] The API asset processing method based on sample entropy and load entropy provided by the application is established on the mathematical framework of Shannon information entropy, which quantifies the uncertainty in HTTP communication, and then realizes API asset discovery and merging.

[0062] Among them, sample entropy (Sample Entropy, SampEn) is a nonlinear dynamics index for quantifying the complexity and randomness of time series. It evaluates the disorder of the sequence by measuring the probability of new patterns in the signal. The higher the value, the higher the complexity of the sequence and the lower the self-similarity; otherwise, it means that the regularity is stronger. In terms of API interface assets, the fixed URL interface has strong regularity and presents low entropy value, while the dynamic parameter interface has strong randomness and presents high entropy value.

[0063] Payload entropy is a specific application of Shannon entropy theory in the field of network security, which is used to quantify the randomness of the payload part of network data packets. Its essence is a measure of data disorder, and the higher the entropy value, the closer the payload content to random distribution, and the lower the entropy value, the more regularity or predictable pattern. In API asset discovery, the payload entropy value of normal API requests presents a stable distribution, and the dynamic parameter part significantly increases the local entropy value. The entropy value curve of the same interface has similar waveform characteristics.

[0064] Based on the sample entropy and payload entropy, both of which can improve the asset identification accuracy based on the traditional URL path matching scheme, the two entropy values are different. Sample entropy can quantify the time regularity of interface call sequence and is suitable for identifying call frequency / interval pattern. Payload entropy can measure the randomness of request content and effectively distinguish parameter dynamics. From the perspective of resources, payload entropy is a single interface traffic content analysis and calculation, and single-step calculation, with small performance consumption. Sample entropy is based on time series, which needs to extract multiple interface traffic within a certain time range, construct a multi-dimensional matrix, and the calculation process is complex, with larger performance consumption than payload entropy.

[0065] The present application proposes a fusion processing idea of the two entropy values. Through the rapid filtering decision of payload entropy and the comparison of payload entropy difference, about 80% of the interfaces can be identified and merged. Then, a small number of undecided interfaces are identified in depth through sample entropy, and the combination of content dimension and time dimension cross verification can significantly improve the API interface identification accuracy.

[0066] The API asset processing method based on sample entropy and payload entropy according to the present application will be described in detail below. In order to illustrate but not to limit, specific details such as specific system structure, technology are proposed to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details.

[0067] Statements in this application that refer to "one embodiment" or "an embodiment" or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in this application are not necessarily all referring to the same embodiment, but can refer to different embodiments.

[0068] In an embodiment of the application, computer program code to carry out operations of the present disclosure can be written in one or more programming languages or combinations of languages including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages such as "C" or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0069] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without any creative effort fall within the scope of protection of the present application.

[0070] Please refer to Figure 1 As shown in the flowchart of the API asset processing method based on sample entropy and load entropy in an embodiment, the method comprises:

[0071] S101: Collect HTTP interface information from the network or query the HTTP interface information from a data source, wherein the HTTP interface information comprises a URL.

[0072] In some embodiments, the basic information of the HTTP interface is obtained by active collection or passive query. The main sources include HTTP request / response packets captured through switch port mirroring, API gateway logs, Nginx, Apache access logs, etc. The obtained information mainly includes URL, request method GET / POST, User-Agent in request header, response status code 200 / 404, etc.

[0073] S102, parse the load entropy value of the HTTP interface information, and the load entropy value is used to quantify the randomness of the interface call content.

[0074] In some embodiments, the load entropy is used to quantify the randomness of the interface call content, and the specific calculation is based on the byte stream of the HTTP request body or response body. First, the load content is extracted, and the content is converted into a byte sequence, such as each character corresponding to a byte value of 0-255, and the frequency p of each byte value is counted k .

[0075] The specific application of the Shannon entropy formula is calculated as follows:

[0076]

[0077] where p k is the frequency of the byte value, and k is all byte values in the HTTP interface load data.

[0078] Specifically, each byte is composed of 8 binary bits, and its value range is 0 to 255. Here, k is sequentially valued from 0 to 255, which corresponds to the 256 possible byte values. For example, k=0 corresponds to the case where the byte value is 0, k=1 corresponds to the case where the byte value is 1, and so on, until k=255 corresponds to the case where the byte value is 255. By traversing all values of k from 0 to 255, all possible bytes in the load can be covered, and the entropy value of the entire load data can be calculated to quantify the randomness of the interface call content.

[0079] For example, if the load content is repeated abcabcabc, the frequencies of the byte values 'a', 'b', and 'c' are 3 / 9, 3 / 9, and 3 / 9, respectively, and the rest are 0. The calculation gives Hpayload≈1.58; if the load content is random binary data, the frequency of each byte is close to 1 / 256, and Hpayload≈8.

[0080] It can be seen that for the API interface, the return of the fixed picture or the load content of the configuration file has high repetition and concentrated byte distribution, and thus has low entropy value. The JSON return of the user real-time data has large change in load content, and the byte contains random fields such as user ID and timestamp, and thus has high entropy value. By calculating the load entropy, the interface can be divided into two categories of low-entropy with highly fixed content and high-entropy with dynamically changed content. The time regularity can be further analyzed to avoid misjudgment caused by content randomness.

[0081] S103: making a rapid filtering decision according to the calculated load entropy value, if the load entropy value is less than a dynamic critical threshold, it is determined to meet the static interface characteristics, and a new API asset is directly discovered or collected into an existing API asset according to this, and the process ends; if the load entropy value is greater than or equal to the dynamic critical threshold, S104 is executed.

[0082] In some embodiments, the preset dynamic critical threshold can be optionally set to 3.0, 4.0, etc., and the load entropy value calculated in S102 is compared with the threshold. If Hpayload<3.0, it is determined to be a static interface, and asset discovery or collection is directly performed; if Hpayload≥3.0, subsequent steps are entered.

[0083] The threshold setting is based on historical data statistics. For example, if the load entropy of an interface is 2.5, which is less than 3.0, it is considered that the content is highly fixed, the URL is matched with the existing interface in the asset library, and if the matching is successful, the collection is performed, and if the matching fails, the asset is newly added. The statistical setting of the threshold ensures the universality of the classification and avoids the deviation caused by subjective setting.

[0084] For the present application, step S103 also involves a specific implementation mode as shown in Figure 2 , which specifically further includes the following steps:

[0085] S1031: obtaining a system preset dynamic critical threshold, and the dynamic critical threshold is generated by statistical generation of load entropy values of historical static interfaces.

[0086] Optionally, the calculation of the dynamic critical threshold uses the 95th percentile of the load entropy values of the historical static interfaces, and the calculation method is to arrange the load entropy values of the historical static interfaces in ascending order, and take the value corresponding to the 95th percentile. The calculation is based on statistical distribution, and ensures that the threshold can cover 95% of the static interfaces, and only 5% of the static interfaces can exceed this value, balancing the strictness and coverage of the threshold.

[0087] S1032: verifying the validity of the load data of the current interface to be processed, and excluding abnormal short messages or non-HTTP protocol messages caused by network packet loss or truncation.

[0088] It should be noted that setting the message length threshold and the protocol identifier is to filter the payload data through logical judgment (length ≥ 100 bytes and protocol header matching). The logical judgment can be set as length ≥ 100 bytes and protocol header matching. Based on the basic characteristics of the HTTP protocol, invalid messages caused by network failure or attack are excluded.

[0089] S1033, the checked payload entropy value is compared with the dynamic critical threshold value, and it is judged whether the payload entropy value is strictly less than the dynamic critical threshold value;

[0090] S1034, if the condition that the payload entropy value is less than the dynamic critical threshold value is met, secondary verification is performed in combination with the structural characteristics of the URL, and after it is confirmed that the static interface characteristics are met, new API asset discovery or matching of the interfaces with the same URL and content in the asset library is performed; if the condition of being less than is not met or the secondary verification fails, the step S103 is ended, and the step S104 is entered.

[0091] As can be seen, the step S103 realizes rapid and accurate identification of the static interface through threshold setting driven by statistics, data validity verification and multi-dimensional feature verification. Abnormal messages are excluded through data validity verification, so as to avoid invalid data interference with the judgment of the static interface; after the entropy value comparison passes, the structural characteristics of the URL are further combined, so as to ensure rapid filtering of the dynamic interface and reduce the probability of misjudgment of the static interface.

[0092] S104: the payload entropy value of the interface to be processed is compared with the average value of the payload entropy of the interfaces with the same URL prefix in the existing asset, if the entropy difference is less than the difference threshold value, it is determined that it is the same logical interface, and the interfaces are merged and collected into the API asset, and the process is ended; if the merging condition is not met, S105 is executed.

[0093] In some embodiments, for the interface to be processed determined as a dynamic interface in S103, all historical interfaces with the same URL prefix are first retrieved from the asset library.

[0094] In this embodiment, the average value of the payload entropy of the historical interfaces is first calculated, and then the absolute difference between the entropy value of the interface to be processed and the average value is calculated, such as the entropy value 3.4 of the interface to be processed, and the difference is |3.4-3.35|=0.05.

[0095] If the difference is less than the difference threshold value, it is determined that it is the same logical interface, and is merged into the asset; otherwise, S105 is executed.

[0096] It can be seen that the URL prefix matching is based on the assumption that "different versions or variants of the same logical interface usually share the same prefix". The average of the load entropy reflects the average level of randomness of the content of the historical interface, and the difference degree measures the deviation of the current interface from the historical group. If the difference degree is small, it means that the randomness of the content of the current interface is consistent with that of the historical interface, which belongs to the variant of the same logical interface, and therefore can be combined, thereby improving the accuracy of the combination and avoiding the repeated recording of assets caused by the slight difference of the URL.

[0097] S105, calculating the sample entropy value of the to-be-processed interface based on the flow message of the to-be-processed interface, wherein the sample entropy value is used to quantify the time regularity of the interface call sequence.

[0098] In some embodiments, the sample entropy is used to quantify the time regularity of the interface call sequence, and the calculation is based on the time sequence of the interface call.

[0099] Specifically, the sample entropy quantifies the complexity of the sequence by comparing the self-similarity of different dimension sub-sequences. The m-dimensional sub-sequence reflects the pattern in a short time window, and the m+1-dimensional sub-sequence reflects the pattern in a longer window. If the similarity ratio does not decrease significantly after increasing the dimension, it means that the sequence still maintains regularity in a longer window, that is, the entropy value is small; if the similarity ratio decreases significantly, it means that the sequence pattern quickly collapses with the increase of the dimension, that is, the entropy value is large. In this way, the calculation of the sample entropy only depends on the request / response length sequence, without the need to parse the specific content, thereby reducing the calculation complexity.

[0100] S106, filtering decision rule according to the sample entropy value, so as to realize the discovery and combination of the API interface;

[0101] The decision rule is: if the sample entropy value is less than the lower limit value of the first interval, it is determined as a fixed interface and is forced to be combined; if the sample entropy value is between the upper and lower limit values of the first interval, it is determined as a dynamic parameter interface and is subjected to path parameterization processing; and if the sample entropy value is greater than the upper limit value of the second interval, it is determined as abnormal flow and is subjected to alarm isolation.

[0102] In some embodiments, a preset sample entropy interval is provided, and optionally, the lower limit of the first interval is 1.2 and the upper limit is 2.5; the upper limit of the second interval is 2.5, and the sample entropy value calculated in S105 is classified and processed:

[0103] If SampEn<1.2: it is determined as a fixed interface and is combined into the interface with the same URL in the asset library;

[0104] If 1.2≤SampEn≤2.5: it is determined as a dynamic parameter interface, the position of the length change is extracted, and the corresponding position in the URL path is parameterized;

[0105] If SampEn>2.5: it is determined as abnormal flow, the URL, timestamp and length sequence are recorded, an alarm is triggered and the storage is isolated.

[0106] The embodiment is based on sample entropy for filtering decision, i.e. discovery and merging of API interfaces. The merging decision rules are shown in Table 1.

[0107] Table 1: Merging decision rules based on sample entropy

[0108]

[0109] It can be seen that the interval division of sample entropy is based on the distribution statistics of historical normal interfaces. The entropy value of fixed interfaces is extremely low, indicating that the calling sequence is highly regular, and it can be directly merged to avoid duplication; the entropy value of dynamic parameter interfaces is moderate, indicating that the calling sequence has predictable changes, and the path template can be unified by parameterization extraction; the entropy value of abnormal traffic is extremely high, indicating that the calling sequence is irregular and beyond the normal business range, and needs to be isolated for detection to prevent attacks. In this way, the introduction of sample entropy avoids repeated recording of interfaces due to parameter changes, and at the same time improves the security of the system by isolating abnormal traffic.

[0110] It needs to be further explained that step S106 further involves the following specific steps, specifically including:

[0111] S1061, obtaining sample entropy interval parameters preset by the system, including a first interval lower limit value, a first interval upper limit value and a second interval upper limit value, wherein the first interval upper limit value and the second interval upper limit value coincide, and are used to distinguish dynamic parameter interfaces and abnormal traffic;

[0112] S1062, extracting the sample entropy value of the current interface to be processed from the output result of step S105;

[0113] S1063, comparing the sample entropy value with the preset interval parameters in multiple levels: first, judging whether it is less than the first interval lower limit value, if yes, marking it as a fixed interface; if not, continue to judge whether it is less than or equal to the first interval upper limit value, if yes, mark it as a dynamic parameter interface; if both are not satisfied, mark it as abnormal traffic.

[0114] Optionally, the sample entropy value comparison method can adopt a multi-level threshold judgment method, which first compares with the lower limit value to determine the fixed interface, then compares with the upper limit value to determine the dynamic parameter interface, and the remaining cases are determined as abnormal traffic, avoiding misjudgment of a single threshold.

[0115] S1064. Perform classification processing based on the marking results: For fixed interfaces, verify the consistency of their request and response body content with existing assets of the same URL interface, and force them to be merged into the corresponding API asset after passing the verification; For dynamic parameter interfaces, extract the positions of length sequence changes in traffic packets, parameterize the corresponding positions in the URL path, and update the path template in the API asset; For abnormal traffic, record its URL, timestamp, and sample entropy value, trigger alarms, and isolate and store the relevant information in the abnormal log.

[0116] Optionally, the fixed interface processing method can be: when verifying the consistency of the request body and response body content, extract key fields for hash matching. If the hash values ​​are the same, it is determined to be the same interface, triggering a forced merge.

[0117] The dynamic parameter interface can be processed as follows: when extracting the position of length sequence change, compare the difference between adjacent length sequences through a sliding window, count the positions of high frequency change, and replace the fixed value of the corresponding position in the URL path with parameter placeholders.

[0118] For abnormal traffic handling, the URL, timestamp, sample entropy value, and length sequence of the last 10 calls of the interface are recorded during isolated storage. This information is used for subsequent manual analysis or machine learning model training to optimize threshold parameters.

[0119] It is evident that by using sample entropy interval classification and multi-dimensional processing logic, the API interface achieves refined discovery, merging, and anomaly detection, improving asset identification accuracy while also ensuring processing efficiency and business interpretability.

[0120] In one embodiment of the present invention, based on step S105, the following is a possible embodiment and its specific implementation is described in a non-limiting manner. In step S105, the step of calculating the entropy value of the traffic packet sample is as follows:

[0121] S1051. Define the initial parameters for calculating sample entropy, set the embedding dimension m, and set the similarity tolerance r.

[0122] Optionally, the embedding dimension m is set to 2 to capture the second-order time dependency of the API call length sequence. The similarity tolerance r is set to 0.1 to 0.25 times the standard deviation of the API time series dataset to be processed, in order to balance the strictness and robustness of the similarity judgment.

[0123] S1052. Extract the length values ​​of the request body or response body from the traffic packets of the interface to be processed, and arrange them in chronological order to form a time series T={t1,t2,...,t...} n}, where n is the length of the time series.

[0124] S1053, construct m-dimensional message length subsequences, generate all continuous m-dimensional length subsequences based on time series T , where each subsequence Xm i corresponds to a time window .

[0125] Optionally, m-dimensional subsequence Xm m represents a vector composed of continuous m length values cut from time series T, used to analyze length patterns within a short time window.

[0126] S1054, calculate the number of similar vectors of m-dimensional subsequence, for each m-dimensional subsequence Xm i , traverse all other m-dimensional subsequence Xm j , j≠i, calculate the Chebyshev distance of Xm i and Xm j ; count the number of subsequence that satisfies the Chebyshev distance less than or equal to the similarity tolerance r, denoted as Ci.

[0127] It should be noted that the fluctuation of API interface call length may contain local random noise, such as slight changes caused by network delay, but does not affect the overall pattern. Chebyshev distance focuses on the most significant difference dimension by taking the maximum value, and is more suitable for capturing the core pattern of interface call, which is a targeted improvement on similarity measurement.

[0128] S1055, for each subsequence Xm i , calculate its similar vector proportion = × similar vector number, where the denominator is the total number of comparisons after excluding itself.

[0129] S1056, calculate the m-dimensional overall similarity proportion, average the similar vector proportions of all m-dimensional subsequences , to obtain , , which represents the average self-similarity of all m-dimensional subsequences, reflecting the overall regularity of length sequence under m-dimensional.

[0130] S1057, extend to m+1-dimensional subsequence and repeat the calculation.

[0131] This embodiment increases the embedding dimension to m+1, constructs m+1-dimensional message length subsequences , repeats steps S1054 to S1056, and calculates the m+1-dimensional overall similarity proportion B m+1 (r).

[0132] S1058, based on the m-dimensional and m+1-dimensional overall similarity proportions, calculate the SampEn(m,r)=-ln(Bm+1 (r) / B m (r))calculating a sample entropy value, wherein the sample entropy value is used to quantify the temporal regularity of the interface call length sequence. The smaller the entropy value, the stronger the sequence regularity; the larger the entropy value, the stronger the sequence randomness.

[0133] It should be noted that the embedding dimension m represents the number of consecutive time points used when constructing the subsequence, which is used to capture the second-order temporal dependence of the interface call length sequence. The similarity tolerance r represents the threshold for judging the similarity of two subsequences, which is used to balance the identification of noise interference and real regularity. The time sequence T represents the sequence of request / response lengths of the interface to be processed in a period of time arranged in chronological order, reflecting the dynamic behavior trajectory of the interface call.

[0134] Step S105 deeply integrates sample entropy calculation and the temporal regularity of API interface call sequence by parameter definition, multi-dimensional subsequence construction, similarity measurement and progressive dimension analysis, solving the problem that path matching cannot quantify temporal patterns.

[0135] In an embodiment of the present application, based on step S104, a possible embodiment will be given below to illustrate the specific implementation thereof. As shown in Figure 3 S104 specifically includes:

[0136] S1041, retrieving a set of historical interfaces matching the URL prefix of the current interface to be processed from the API asset library, wherein the URL prefix matching means that the first M characters of the current interface URL are the same as the first M characters of the historical interface URL, and M is a preset matching length;

[0137] S1042, calculating the mean value of the load entropy of all interfaces in the set of historical interfaces, wherein the mean value is the arithmetic mean of the historical interface load entropy values, i.e. the sum of all historical interface load entropy values divided by the number of historical interfaces;

[0138] S1043, calculating the absolute difference between the load entropy value of the current interface to be processed and the mean value to obtain the entropy difference degree, wherein the absolute difference is the absolute value of the difference between the current load entropy value and the mean value;

[0139] S1044, comparing the entropy difference degree with a preset difference degree threshold, if the difference degree is less than the threshold, determining that the current interface and the historical interface are the same logical interface, and merging the current interface into the API asset to which the historical interface belongs; if the difference degree is greater than or equal to the threshold, ending step S104 and entering step S105.

[0140] It should be noted that the entropy difference degree calculation uses the absolute difference method, specifically Sc=|H c -μ|, wherein μ is the mean value of the load entropy. H cThe absolute difference value avoids the inapplicability of the relative difference value when the mean value is zero, and directly reflects the deviation degree of the current interface from the group mean value.

[0141] Optionally, the difference threshold value can be set to 0.2, and the difference threshold value is used to distinguish the normal fluctuation of the same logical interface and the significant difference of different logical interfaces. Generally, the difference is greater than or equal to 0.2.

[0142] The step S104 of the embodiment filters the historical interface through the semantically associated URL prefix, and realizes the accurate merging of the dynamic interface variants by combining the statistical mean value and quantitative difference analysis of the load entropy. By calculating the absolute difference value of the current interface and the mean value, the deviation degree of the current interface from the historical group is quantified; whether the current interface belongs to the normal dynamic change of the same logical interface is judged by the difference threshold value, which reduces the invalid comparison and improves the merging accuracy.

[0143] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0144] The following is an embodiment of an API asset processing system based on sample entropy and load entropy provided by the embodiment of the present disclosure. The system and the above-mentioned API asset processing method based on sample entropy and load entropy belong to the same inventive concept. Details not described in the embodiment of the API asset processing system based on sample entropy and load entropy can be referred to the above-mentioned embodiment of the API asset processing method based on sample entropy and load entropy.

[0145] As shown in Figure 4 , the system comprises:

[0146] The interface information acquisition module 201 is configured to acquire HTTP interface information.

[0147] The load entropy calculation module 202 is configured to analyze the load entropy value of the HTTP interface information, and the load entropy value is used to quantify the randomness of the interface call content.

[0148] The interface filtering module 203 is configured to make a filtering decision according to the load entropy value. If the load entropy value is greater than or equal to the dynamic critical threshold value, the entropy difference comparison and merging module is executed.

[0149] The entropy difference comparison and merging module 204 is configured to compare the load entropy value of the interface to be processed with the load entropy mean value of the interface with the same URL prefix in the existing asset. If the merging condition is not met, the sample entropy calculation module is executed.

[0150] The sample entropy calculation module 205 calculates a sample entropy value of the traffic message of the interface to be processed, and the sample entropy value is used to quantify the time regularity of the interface calling sequence.

[0151] The sample entropy decision module 206 is used to make filtering decision rules according to the sample entropy value, so as to realize the discovery and merging of the API interface.

[0152] As shown in the method for processing API assets based on sample entropy and load entropy, the method comprises the following steps: Figure 5 As shown in the method for processing API assets based on sample entropy and load entropy, the method comprises the following steps:

[0153] In the embodiments of the present application, the electronic device includes but is not limited to a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or claimed herein.

[0154] In the embodiments of the present application, the processor 101 can be implemented by using at least one of an application-specific integrated circuit, a programmable logic device, a field programmable gate array, a processor, a controller, a microcontroller, a microprocessor, an electronic unit designed to perform the functions described herein, and in some cases, such implementation can be implemented in a controller. For software implementation, the implementation of such processes or functions can be implemented with a separate software module allowing at least one function or operation to be performed. The software code can be implemented by a software application (or program) written in any appropriate programming language, which can be stored in the memory and executed by the controller.

[0155] The display module 103 is used to display information input by a user or information provided to the user. The display module 103 can include a display panel, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc.

[0156] The memory 102 can be used to store software programs and various data. The memory 102 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0157] The application further provides a storage medium, which stores a computer program, and the computer program is executed by a processor to implement steps of the API asset processing method based on sample entropy and load entropy.

[0158] The storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0159] In the storage medium, the readable signal medium can include a data signal borne in a baseband or as part of a carrier wave, in which readable program codes are borne. Such a propagated data signal can adopt various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The readable signal medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit programs for use by or in connection with an instruction execution system, device or apparatus.

[0160] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An API asset processing method based on sample entropy and load entropy, characterized in that, The methods include: S101. Obtain HTTP interface information; S102. Parse the load entropy value of the HTTP interface information, wherein the load entropy value is used to quantify the randomness of the interface call content; S103. Make a filtering decision based on the load entropy value. If the load entropy value is greater than or equal to the dynamic critical threshold, then execute S104. S104. Compare the load entropy value of the interface to be processed with the average load entropy value of interfaces with the same URL prefix in the existing assets. If the merging conditions are not met, proceed to S105. S105. Calculate the sample entropy value of the traffic packets of the interface to be processed. The sample entropy value is used to quantify the temporal regularity of the interface call sequence. S106. Filtering decision rules are made based on sample entropy values ​​to achieve API interface discovery and merging; The decision rules are as follows: if the sample entropy value is less than the lower limit of the first interval, it is determined to be a fixed interface and forced to be merged; if the sample entropy value is between the upper and lower limits of the first interval, it is determined to be a dynamic parameter interface and path parameterization is performed; if the sample entropy value is greater than the upper limit of the second interval, it is determined to be abnormal traffic and alarm isolation is performed.

2. The API asset processing method based on sample entropy and load entropy according to claim 1, characterized in that, In step S102, the formula for calculating the HTTP interface information load entropy is: Where, p k represents the frequency of occurrence of byte values, and k represents all byte values ​​that appear in the HTTP interface payload data.

3. The API asset processing method based on sample entropy and load entropy according to claim 1, characterized in that, In step S105, the step of calculating the entropy value of the traffic packet sample is as follows: S1051. Define the initial parameters for calculating sample entropy, set the embedding dimension m, and set the similarity tolerance r; S1052. Extract the length values ​​of the request body or response body from the traffic packets of the interface to be processed, and arrange them in chronological order to form a time series T={t1,t2,...,t...} n }, where n is the length of the time series; S1053. Construct m-dimensional message length subsequences. Based on the time series T, generate all consecutive m-dimensional length subsequences. , where each subsequence Xm i Corresponding time window The length of the interface call within; S1054. Calculate the number of similarity vectors for each m-dimensional subsequence Xm. i Iterate through all other m-dimensional subsequences Xm j , j≠i, calculate Xm i With Xm j Calculate the Chebyshev distance; count the number of subsequences whose Chebyshev distance is less than or equal to the similarity tolerance r; S1055, For each subsequence Xm i Calculate the proportion of their similarity vectors = × Number of similar vectors, where the denominator is the number of similar vectors. The total number of comparisons excluding itself; S1056. Calculate the overall m-dimensional similarity ratio, which is the ratio of similarity vectors for all m-dimensional subsequences. Calculate the average and get , It represents the average self-similarity of all m-dimensional subsequences, reflecting the overall regularity of the length sequence in m dimensions; S1057. Extend to the m+1 dimensional subsequence and repeat steps S1054 to S1056 to calculate the overall similarity ratio B in the m+1 dimensional dimension. m+1 (r); S1058. Based on the overall similarity ratio of m-dimensional and m+1-dimensional components, the formula SampEn(m,r)=-ln(B) is used. m+1 (r) / B m (r) calculates the sample entropy value, where the sample entropy value is used to quantify the temporal regularity of the interface call length sequence.

4. The API asset processing method based on sample entropy and load entropy according to claim 1, characterized in that, Step S104 specifically includes: Retrieve a set of historical interfaces from the API asset library that match the URL prefix of the current interface to be processed. The URL prefix matching means that the first M characters of the current interface URL are the same as the first M characters of the historical interface URL, where M is a preset matching length. Calculate the average load entropy of all interfaces in the historical interface set; Calculate the absolute difference between the current load entropy value of the interface to be processed and the mean value to obtain the entropy difference degree; The entropy difference is compared with a preset difference threshold. If the difference is less than the threshold, the current interface and the historical interface are determined to be the same logical interface, and the current interface is merged into the API asset to which the historical interface belongs. If the difference is greater than or equal to the threshold, step S104 ends and step S105 begins.

5. The API asset processing method based on sample entropy and load entropy according to claim 1, characterized in that, Step S103 specifically includes: Obtain the system's preset dynamic critical threshold, which is generated by statistically analyzing the load entropy values ​​of historical static interfaces; Verify the validity of the load data of the interface to be processed, and exclude abnormal short messages or non-HTTP protocol messages caused by network packet loss or truncation; The verified effective load entropy value is compared with the dynamic critical threshold to determine whether the load entropy value is less than the dynamic critical threshold. If the load entropy value is less than the dynamic critical threshold, a secondary verification is performed based on the structural characteristics of the URL. After confirming that it conforms to the static interface characteristics, the new API asset discovery or matching of the same URL and content in the asset library is executed. If the condition of being less than is not met or the secondary verification fails, step S103 ends and step S104 is entered.

6. The API asset processing method based on sample entropy and load entropy according to claim 1, characterized in that, Step S106 specifically includes: Obtain the system's preset sample entropy interval parameters, including the lower limit of the first interval, the upper limit of the first interval, and the upper limit of the second interval. The upper limit of the first interval and the upper limit of the second interval coincide, which are used to distinguish between dynamic parameter interfaces and abnormal traffic. Extract the sample entropy value of the current interface to be processed from the output of step S105; The sample entropy value is compared with the preset interval parameters in multiple levels: First, it is determined whether it is less than the lower limit of the first interval. If it is, it is marked as a fixed interface. If it is not satisfied, it is further determined whether it is less than or equal to the upper limit of the first interval. If it is, it is marked as a dynamic parameter interface. If neither is satisfied, it is marked as abnormal traffic. Based on the marking results, perform classification processing operations: For fixed interfaces, verify the consistency of their request and response body content with existing assets of the same URL interface, and force them to be merged into the corresponding API asset after passing the verification; for dynamic parameter interfaces, extract the positions of length sequence changes in traffic packets, parameterize the corresponding positions in the URL path, and update the path template in the API asset; for abnormal traffic, record its URL, timestamp, and sample entropy value, trigger alarms, and isolate and store the relevant information in the abnormal log.

7. The API asset processing method based on sample entropy and load entropy according to claim 1, characterized in that, In S103, if the load entropy value is less than the dynamic critical threshold, it is determined to meet the static interface characteristics, and is directly identified as a new API asset or aggregated into an existing API asset, and the process ends. In S104, if the entropy difference is less than the difference threshold, it is determined to be the same logical interface, and the interface is merged and aggregated into the API asset, and the process ends.

8. An API asset processing system based on sample entropy and load entropy, characterized in that, The system is used to implement the API asset processing method based on sample entropy and load entropy as described in any one of claims 1 to 7; The system includes: The interface information acquisition module is used to obtain HTTP interface information; The load entropy calculation module is used to parse the load entropy value of HTTP interface information. The load entropy value is used to quantify the randomness of the interface call content. The interface filtering module is used to make filtering decisions based on the load entropy value. If the load entropy value is greater than or equal to the dynamic critical threshold, the entropy difference comparison and merging module is executed. The entropy difference comparison and merging module is used to compare the load entropy value of the interface to be processed with the average load entropy value of interfaces with the same URL prefix in the existing assets. If the merging conditions are not met, the sample entropy calculation module is executed. The sample entropy calculation module calculates the sample entropy value based on the traffic packets of the interface to be processed. The sample entropy value is used to quantify the temporal regularity of the interface call sequence. The sample entropy decision module is used to make filtering decision rules based on sample entropy values ​​in order to realize the discovery and merging of API interfaces.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the API asset processing method based on sample entropy and load entropy as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the API asset processing method based on sample entropy and load entropy as described in any one of claims 1 to 7.

Images