A method, apparatus, device, and medium for constructing API topology graphs

Abstract

This invention provides a method, apparatus, device, and medium for constructing an API topology graph, relating to the field of data classification technology. It involves acquiring traffic probe data from key network nodes; determining various API data based on the traffic probe data; grouping the API data into multiple API groups based on target IPs and target ports; determining a first API template corresponding to each API data within each group based on a template generation model; determining a second API template corresponding to each API group based on confidence thresholds and the confidence levels of each first API template; and constructing an API data topology graph based on each API group and its corresponding second API template. This achieves automated construction of a three-level API data topology graph, namely, a target IP—target port—normalized API three-level topology graph, reducing manual intervention and lowering the operational costs of API data management.

Description

Technical Field

[0001] This invention relates to the field of data classification technology, and more specifically, to a method, apparatus, device, and medium for constructing an API topology graph. Background Technology

[0002] In cloud-native architectures, the surge in the volume and dynamic changes of API (Application Programming Interface) data makes it difficult to efficiently generalize API data into semantically consistent templates according to specific paths, and existing technologies such as regular expression rule matching based on log analysis tools, clustering based on string similarity, and single machine learning classifiers cannot automatically construct topological relationship graphs of API data. This leads to difficulties in managing API data and security risks. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus, device and medium for constructing API topology graphs, so as to realize the automated construction of API data topology graphs.

[0004] Firstly, this application provides a method for constructing an API topology graph, including:

[0005] Obtain traffic probe data at key network nodes;

[0006] Based on traffic probe data, determine the data for each API; the API data includes the target IP and the target port.

[0007] Based on the target IP and target port, the API data is grouped to obtain multiple API groups;

[0008] Based on the template generation model, the first API template corresponding to each API data in each group is determined; wherein, the template generation model is to determine the first candidate API template corresponding to the API data through the semantic feature sub-model, determine the second candidate API template corresponding to the API data through the similarity sub-model, and then determine the first API template corresponding to the API data through the candidate template sub-model based on the first candidate API template and the second candidate API template.

[0009] Based on the confidence threshold and the confidence of each first API template, determine the second API template corresponding to each API group;

[0010] Based on each API group and the corresponding second API template, construct an API data topology diagram.

[0011] Optionally, based on the template generation model, determine the first API template corresponding to each API data within each group, including:

[0012] Semantic features are extracted from API data using a semantic feature sub-model, and the first candidate API template corresponding to the API data is determined based on the semantic features.

[0013] A similarity sub-model is used to extract mixed features from the API data, and based on the mixed features and weights, the second candidate API template corresponding to the API data is determined; wherein, the mixed features include static segment features and dynamic segment features;

[0014] Based on the first candidate API template, the second candidate API template, and the weight, the first API template corresponding to the API data is determined.

[0015] Optionally, a similarity sub-model is used to extract mixed features from the API data, and based on the mixed features and weights, a second candidate API template corresponding to the API data is determined.

[0016] Static segment features and dynamic segment features are extracted using a similarity sub-model.

[0017] Based on static segment features, cosine similarity is used to determine the first similarity between the static segment features and the static segments of the candidate template;

[0018] Based on dynamic segment features, regularization rules are used to determine the second similarity between dynamic segment features and dynamic segments of candidate templates;

[0019] Based on the first similarity and static segment weight, as well as the second similarity and dynamic segment weight, the third similarity between the API data and the candidate template is determined, and the second candidate API template corresponding to the API data is determined based on the third similarity.

[0020] Optionally, based on the first candidate API template, the second candidate API template, and the weights, the first API template corresponding to the API data is determined, including:

[0021] The semantic features of the API data are converted into a first-category probability distribution; where the first candidate API template belongs to all candidate templates corresponding to the first-category probability distribution.

[0022] The mixed features of the API data are transformed into a second-class probability distribution; wherein, the second candidate API template belongs to all candidate templates corresponding to the second-class probability distribution;

[0023] Based on the probability distribution and weight of the first category, and the probability distribution and weight of the second category, the first API template corresponding to the API data is determined.

[0024] Optionally, based on the confidence threshold and the confidence level of each first API template, a second API template corresponding to each API group is determined, including:

[0025] When the confidence of multiple first API templates is determined to be greater than the confidence threshold, the first API template corresponding to each API group is determined as the second API template, and the regular expressions corresponding to the second API template and the second API template are stored in the template library and the first weight and the second weight are adjusted.

[0026] Optionally, the method for constructing an API topology graph provided in this application further includes:

[0027] If the confidence level of the first API template is less than the confidence level threshold, then manual annotation is performed to determine the new annotation data corresponding to the API data, and a new second API template corresponding to the API group is determined based on the new annotation data. The template library is then updated and the first and second weights are adjusted based on the new second API template.

[0028] Optionally, the method for constructing an API topology graph provided in this application further includes:

[0029] Obtain the training dataset; wherein the training dataset includes multiple training sample data; each training sample data includes API data and the standard first API template corresponding to the API data;

[0030] Based on the training dataset, iterative training operations are performed on the initial template generation model until the termination condition of the iterative training is met. Then, based on the weights of the initial template generation model updated during the last iteration, a template generation model is obtained. The iterative training operations include:

[0031] Select target training sample data from the training dataset;

[0032] The API data in the target training sample data is input into the initial template generation model, so that the initial template generation model determines the first candidate API template corresponding to the API data through the semantic feature sub-model, determines the second candidate API template corresponding to the API data through the similarity sub-model, and then determines the predicted first API template corresponding to the API data based on the first candidate API template and the second candidate API template through the candidate template sub-model.

[0033] Based on the prediction error between the predicted first API template and the standard first API template in the target training sample data, the weights of the initial template generation model are updated.

[0034] Secondly, this application provides an apparatus for constructing an API topology graph, comprising:

[0035] The traffic probe module is used to acquire traffic probe data at the locations of key network nodes;

[0036] The data preprocessing module is used to determine the API data based on traffic probe data; the API data includes the target IP and the target port.

[0037] The data grouping module is used to group API data based on the target IP and target port to obtain multiple API groups;

[0038] The data classification module is used to determine the first API template corresponding to each API data in each group based on the template generation model. The template generation model is used to determine the first candidate API template corresponding to the API data through the semantic feature sub-model, and the second candidate API template corresponding to the API data through the similarity sub-model. Then, based on the first candidate API template and the second candidate API template, the first API template corresponding to the API data is determined through the candidate template sub-model.

[0039] The template verification module is used to determine the second API template corresponding to each API group based on the confidence threshold and the confidence of each first API template.

[0040] The relationship graph construction module is used to construct an API data topology relationship graph based on each API group and the corresponding second API template.

[0041] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for constructing an API topology graph.

[0042] Fourthly, this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the above-described method for constructing an API topology diagram.

[0043] This invention provides a method, apparatus, device, and medium for constructing an API topology diagram. The method involves: acquiring traffic probe data from key network nodes; determining various API data based on the traffic probe data; grouping the API data into multiple API groups based on target IPs and target ports; determining a first API template corresponding to each API data within each group based on a template generation model; determining a second API template corresponding to each API group based on a confidence threshold and the confidence level of each first API template; and constructing an API data topology diagram based on each API group and its corresponding second API template. This achieves automated construction of a three-level API data topology diagram, namely, a target IP-target port-normalized API three-level topology diagram, reducing manual intervention and lowering the operational costs of API data management.

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating a method for constructing an API topology graph provided by an embodiment of the present invention is shown.

[0047] Figure 2 A flowchart illustrating the process of determining a normalized template provided in an embodiment of the present invention is shown;

[0048] Figure 3 A flowchart illustrating the training process of the template generation model provided in an embodiment of the present invention is shown.

[0049] Figure 4 A schematic diagram of the structure of an apparatus for constructing an API topology graph provided in an embodiment of the present invention is shown;

[0050] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0052] In a cloud-native architecture, the following shortcomings exist when dealing with multiple services per IP address and multiple interfaces per service: For example, regular expression rule matching based on log analysis tools heavily relies on manually predefined rules, resulting in high maintenance costs and poor flexibility. When faced with new or changing APIs, processing latency can reach hours, failing to meet real-time requirements. Clustering based on string similarity (such as using Levenstein distance) only focuses on character-level similarity of path strings and cannot identify and distinguish semantic parameters in the path (e.g., it cannot correctly identify and normalize ORD-48949 in / orders / ORD-48949 / items to {ord...). The `er_id` parameter leads to inaccurate clustering results and a lack of semantic information in the generated normalized templates. Furthermore, models based on a single machine learning classifier typically rely on only a single feature source (such as lexical features or simple semantic features), lacking the ability to effectively integrate multi-dimensional features (such as structural similarity and semantic parameters), resulting in insufficient classification accuracy (such as normalization accuracy) and difficulty in handling complex and ever-changing API paths. Additionally, the reliance on system metadata cleaning, requiring complete API documentation or log metadata, limits applicability to APIs not connected to monitoring systems or lacking complete metadata (such as legacy systems, third-party services, and shadow APIs). Therefore, this application provides a method for constructing an API topology graph. This method involves: acquiring traffic probe data from key network nodes; determining various API data based on the traffic probe data; grouping the API data into multiple API groups based on target IP and target port; determining the first API template corresponding to each API data within each group based on a template generation model; determining the second API template corresponding to each API group based on a confidence threshold and the confidence level of each first API template; and constructing an API data topology graph based on each API group and its corresponding second API template. This achieves automated construction of a three-level API data topology graph, namely, a target IP—target port—normalized API three-level topology graph, thereby reducing manual intervention and lowering the operational costs of API data management.

[0053] This application provides a method for constructing an API topology graph, see below. Figure 1 As shown in the embodiments of this application, the general flow of the method for constructing an API topology graph is as follows:

[0054] Step 110: Obtain traffic probe data at the locations of key network nodes.

[0055] In this embodiment of the application, traffic probes are set at key network nodes in the core path nodes and core business nodes of the network traffic transmission to capture traffic probe data transmitted at key network nodes in real time. The traffic probe data includes raw HTTP traffic and / or raw HTTPS traffic.

[0056] Step 120: Based on the traffic probe data, determine the data for each API; the API data includes the target IP and the target port.

[0057] In this embodiment of the application, the API data is determined based on traffic probe data, including but not limited to the following methods:

[0058] Based on preset API identification rules, traffic probe data is filtered and extracted to identify each API data and verify that each API data corresponds to an API interaction process in the network.

[0059] Furthermore, each API data is processed through protocol parsing. Information related to the target address is extracted from the API data to obtain the target IP address; port information used to establish the connection is extracted to obtain the target port; content representing the request operation type in the API data is identified to obtain the request method; and path information used to locate the specific API interface is parsed from the API data to obtain the API path information. The target IP address, target port, request method, and API path information corresponding to each API data are associated and stored to transform the raw data into structured key information data, which facilitates subsequent comprehensive analysis of the structured key information data.

[0060] Step 130: Based on the target IP and target port, group the API data to obtain multiple API groups.

[0061] In this embodiment, the API data can be grouped and processed using the stream technology in Java programming, with the combination of target IP and target port as the grouping basis.

[0062] Furthermore, all API data with the same target IP and the same target port are grouped together to form an API group, resulting in multiple API groups. Each API group is then uniquely identified to facilitate differentiation and processing of different API groups in the future.

[0063] This application ensures low processing latency by using real-time traffic probe acquisition, group preprocessing, and an efficient classification model (i.e., grouping API data based on the combination of target IP and target port) to achieve processing speeds of seconds or even sub-seconds. By grouping API data based on the combination of target IP and target port, APIs with the same path but different request methods within a group are considered as different API data, which reduces the dispersion of data processing and the computational complexity of subsequent classification, and provides basic grouping units for constructing a topology graph.

[0064] Step 140: Based on the template generation model, determine the first API template corresponding to each API data in each group; wherein, the template generation model is to determine the first candidate API template corresponding to the API data through the semantic feature sub-model, determine the second candidate API template corresponding to the API data through the similarity sub-model, and then determine the first API template corresponding to the API data based on the first candidate API template and the second candidate API template through the candidate template sub-model.

[0065] In this embodiment of the application, a template generation model is used to perform template matching on each API data in each API group to obtain the first API template corresponding to each API data. The template generation model includes a semantic feature sub-model and a similarity sub-model that are respectively connected to the candidate template sub-model. The first candidate API template corresponding to the API data is determined by the semantic feature sub-model. After the second candidate API template corresponding to the API data is determined by the similarity sub-model, the first API template corresponding to the API data is determined by the candidate template sub-model based on the first candidate API template and the second candidate API template.

[0066] Furthermore, the first API template corresponding to each API data within each group can be determined in the following way:

[0067] A semantic feature sub-model is used to extract semantic features from the API data, and based on the semantic features, the first candidate API template corresponding to the API data is determined. A similarity sub-model is used to extract mixed features from the API data, and based on the mixed features and weights, the second candidate API template corresponding to the API data is determined. The mixed features include static segment features and dynamic segment features. Based on the first candidate API template, the second candidate API template, and the weights, the first API template corresponding to the API data is determined.

[0068] Furthermore, the second candidate API template corresponding to the API data can be determined in the following way: Static segment features and dynamic segment features are extracted using a similarity sub-model; based on the static segment features, cosine similarity is used to determine the first similarity between the static segment features and the static segments of the candidate template; based on the dynamic segment features, regularization rules are used to determine the second similarity between the dynamic segment features and the dynamic segments of the candidate template; based on the first similarity and static segment weights, and the second similarity and dynamic segment weights, the third similarity between the API data and the candidate template is determined, and the second candidate API template corresponding to the API data is determined based on the third similarity.

[0069] Furthermore, the first API template corresponding to the API data can be determined in the following way: convert the semantic features of the API data into a first category probability distribution; wherein, the first candidate API template belongs to all candidate templates corresponding to the first category probability distribution; convert the mixed features of the API data into a second category probability distribution; wherein, the second candidate API template belongs to all candidate templates corresponding to the second category probability distribution; and determine the first API template corresponding to the API data based on the first category probability distribution and the first weight, as well as the second category probability distribution and the second weight.

[0070] In this embodiment, the template generation model can be an AdaBoost ensemble model based on a dual-base classifier, used to determine the first API template corresponding to each API data; the semantic feature sub-model can be an LLM semantic base classifier, used to determine the first candidate API template corresponding to the API data; wherein, the first candidate API template includes a parameterized template and a classification confidence score; the similarity sub-model can be a vector similarity base classifier, used to calculate the similarity between the path of the API data and the template library, and determine the second candidate API template corresponding to the API data; wherein, the second candidate API template includes a candidate template set and the similarity of each candidate template, and the candidate template set includes at least three candidate templates; the candidate template sub-model can be an AdaBoost strong classifier, used to determine the first API template corresponding to the API data and the confidence score corresponding to the first API template based on the first candidate API template, the second candidate API template, and the weights.

[0071] Specifically, the LLM semantic base classifier uses prompt word engineering, such as "Please normalize the following API path to a parameterized template: {API path}", to identify and extract semantic parameters (such as UUID_123 -> {user_id}) from the original path and generate semantic feature codes (such as parameter type code 1001 representing user ID) and determine the first API template corresponding to the API data based on the semantic feature codes.

[0072] Vector similarity base classifiers can use hybrid feature extraction to extract similarity. This involves extracting hybrid features for each API path based on a template library, and then calculating the TF-IDF feature vectors of static segments (e.g., / user / queryById, / system / profile) in the hybrid features using TF-IDF to obtain the first similarity. Finally, a predefined regularization rule library (e.g., / [a-f0-9]{8} / ) is used to match short IDs. / [0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12} / Matching UUID) Matches the dynamic segment (such as UUID_123) in the mixed features with the template in the template library to obtain the second similarity. Finally, by calculating the first similarity and the static segment weight, as well as the second similarity and the dynamic segment weight, the third similarity between the API data and the candidate template is determined, and the second candidate API template corresponding to the API data is determined based on the third similarity; where the sum of the static segment weight and the dynamic segment weight is 1.

[0073] It should be noted that the templates in the template library are built based on historical API data.

[0074] The AdaBoost strong classifier normalizes and assigns confidence to the outputs of the LLM base classifier and the vector similarity base classifier. Specifically, it converts the semantic feature encoding (e.g., 1001) output by the LLM base classifier into a first-class probability distribution and the similarity output by the vector similarity base classifier into a second-class probability distribution. Then, the SAMME.R algorithm is used to determine the target class probability distribution of the AdaBoost strong classifier based on the respective weights of the LLM base classifier and the vector similarity base classifier. Based on the target class probability distribution, the first API template corresponding to the API data and the confidence level corresponding to the first API template are determined.

[0075] Furthermore, the similarity output by the vector similarity base classifier can be converted into a second-class probability distribution using the following formula:

[0076]

[0077] In the formula, To determine whether an API path belongs to a template in the template library for a vector similarity base classifier The probability, For API paths and templates in the template library similarity, This is a temperature parameter, typically 0.8, which controls the smoothness of the probability distribution. This represents the total number of candidate templates in the template library. For the API path and template library Similarity between templates.

[0078] The AdaBoost strong classifier determines the probability distribution of the target class using the following formula:

[0079]

[0080] In the formula, For strong classifiers to template The probability of the target category, As the first weight, Output template for LLM base classifier The probability, Set as To prevent numerical overflow caused by probability P = 0, As the second weight, To determine whether an API path belongs to a template in the template library for a vector similarity base classifier The probability of;

[0081] Furthermore, the confidence level corresponding to the first API template can be determined using the following formula:

[0082]

[0083] In the formula, template The final confidence level, For strong classifiers to template The probability of the target category.

[0084] This application utilizes a dual-base classifier ensemble model (i.e., template generation model) that combines semantic and structural features. Compared to single methods such as simple clustering or a single classifier, this significantly improves the accuracy of API normalization, particularly in terms of semantic parameter recognition. Specifically, the LLM classifier in the dual-base classifier ensemble model handles semantic parameter recognition, such as identifying timestamps, order numbers, and user IDs; the vector similarity classifier in the dual-base classifier ensemble model handles structural similarity calculation, such as by mixing TF-IDF cosine and regularization rules; and the AdaBoost in the dual-base classifier ensemble model significantly improves normalization accuracy by optimizing weights, such as minimizing the classification error rate, and adaptively fusing the outputs of both methods.

[0085] Step 150: Based on the confidence threshold and the confidence of each first API template, determine the second API template corresponding to each API group.

[0086] In this embodiment of the application, the second API template corresponding to each API is determined in ways including but not limited to the following:

[0087] When the confidence of multiple first API templates is determined to be greater than the confidence threshold, the first API template corresponding to each API group is determined as the second API template, and the regular expressions corresponding to the second API template and the second API template are stored in the template library and the first weight and the second weight are adjusted.

[0088] If the confidence level of the first API template is less than the confidence level threshold, then manual annotation is performed to determine the new annotation data corresponding to the API data, and a new second API template corresponding to the API group is determined based on the new annotation data. The template library is then updated and the first and second weights are adjusted based on the new second API template.

[0089] Specifically, based on a preset confidence threshold, it is determined whether the confidence of the first API template corresponding to each API data in each API group is greater than the confidence threshold. If it is greater, the first API template is determined as the second API template corresponding to the API group. If it is less, a manual annotation operation is performed to determine the new annotated data corresponding to the API data. Based on the new annotated data, a new second API template corresponding to the API group is determined, and the templates in the template library are updated based on the new second API template. The new annotated data is the new first API template corresponding to the API data determined by the manual annotation operation. The template library is also updated and the first and second weights are adjusted based on the new second API template.

[0090] This application designs a confidence-driven, two-layer closed-loop optimization mechanism. Specifically, when the confidence level of the first API template output by the AdaBoost strong classifier is lower than the confidence threshold and during the cold start phase, a manual annotation operation is triggered to obtain new labeled data. This new labeled data is then used for full-scale iterative training of the model and updating the template library. Similarly, when the confidence level of the first API template output by the AdaBoost strong classifier is lower than the confidence threshold and during the runtime phase, a manual annotation operation is triggered to obtain new labeled data. Based on this new labeled data, the model weights and template library are incrementally updated. This two-layer closed-loop optimization mechanism ensures that the method can continuously learn and adapt to API changes without requiring frequent manual maintenance of the rule base, thus solving the maintenance delay problem of static rule bases and reducing dependence on initial data.

[0091] Step 160: Construct an API data topology diagram based on each API group and the corresponding second API template.

[0092] In this embodiment of the application, a three-level API data topology diagram is constructed based on each API group and the corresponding second API template. The first level of the three-level API data topology diagram is the target IP, such as 128.2.4.76; the second level is the target port, such as 443; and the third level is the normalized API interface (i.e., the second API template), such as / orders / {id} / items.

[0093] This application automatically constructs a tree-like service topology structure of target IP - target port - normalized API interface based on the raw data and normalization results collected by traffic probes, and records the API access frequency in the leaf nodes. This solves the coarse-grained problem of traditional port-based governance, provides a structural foundation for precise interface-level monitoring, anomaly detection and governance, and improves the efficiency of fault location, performance analysis and security policy implementation.

[0094] The process of determining the normalization template (i.e., the second API template) provided in the embodiments of this application will be described in detail below. (See attached document for details.) Figure 2 As shown in the embodiments of this application, the process for determining the second API template is as follows:

[0095] Step 210: Obtain the data for each API within the API group;

[0096] Step 220: Based on the templates in the template library, determine whether each API data matches each template in the template library. If they match, determine the normalized template corresponding to each API data. If they do not match, input each API data into the template generation model.

[0097] Step 230: Based on the template generation model, semantic features are extracted using an LLM base classifier to determine the first candidate API template. After determining the second candidate API template by calculating structural features using a vector similarity base classifier, the first API template and the confidence level corresponding to the first API template are determined using an AdaBoost classifier based on the first candidate API template, the second candidate API template, and weighted decision.

[0098] Step 240: Determine whether the confidence level of the first API template is greater than the preset confidence threshold. If yes, determine that the first API template is a normalized template and update the weights of the template library and the template generation model. If not, determine whether the current stage is a cold start. If yes, trigger manual labeling to determine the new labeled data corresponding to the API data, and determine the new second API template corresponding to the API group based on the new labeled data. Retrain the template generation model based on the new second API template and adjust the first and second weights. If not, trigger manual labeling to determine the new labeled data corresponding to the API data, and determine the new second API template corresponding to the API group based on the new labeled data. Incrementally update the first and second weights of the template generation model based on the new second API template.

[0099] The training process of the template generation model training method provided in the embodiments of this application will be described in detail below. (See attached document for details.) Figure 3 As shown in the embodiments of this application, the template generation model training method process is as follows:

[0100] Step 310: Obtain the training data set; wherein, the training data set includes multiple training sample data; each training sample data includes API data and the standard first API template corresponding to the API data;

[0101] Step 320: Select target training sample data from the training dataset;

[0102] Step 330: Input the API data in the target training sample data into the initial template generation model so that the initial template generation model determines the first candidate API template corresponding to the API data through the semantic feature sub-model, determines the second candidate API template corresponding to the API data through the similarity sub-model, and then determines the predicted first API template corresponding to the API data based on the first candidate API template and the second candidate API template through the candidate template sub-model.

[0103] Step 340: Based on the prediction error between the predicted first API template and the standard first API template in the target training sample data, update the weights of the initial template generation model;

[0104] Furthermore, the weights of the initial template generation model can be updated using the following formula:

[0105]

[0106] In the formula, Let be the weights of the template generation model in the t-th iteration. Let be the weighted error rate of the t-th iteration. The number of candidate template categories. for To prevent the denominator from being 0;

[0107] Among them, the weighted error rate is determined by calculating the matching error between the predicted first API template and the standard first API template.

[0108] Step 350: Determine whether the iterative training termination condition is met. If yes, proceed to step 360; otherwise, return to step 320. The iterative training termination condition is that the number of iterations is not less than the number threshold, or the prediction error is not higher than the error threshold.

[0109] Step 360: Based on the weights of the initial template generation model updated during the last iteration of training, obtain the template generation model.

[0110] This application provides an apparatus for constructing an API topology graph, see below. Figure 4 As shown in the embodiments of this application, the apparatus for constructing an API topology graph includes:

[0111] Traffic probe module 410 is used to acquire traffic probe data at the location of key network nodes;

[0112] The data preprocessing module 420 is used to determine each API data based on the traffic probe data; wherein, the API data includes the target IP and the target port;

[0113] The data grouping module 430 is used to group the API data based on the target IP and target port to obtain multiple API groups;

[0114] The data classification module 440 is used to determine the first API template corresponding to each API data in each group based on the template generation model. The template generation model is used to determine the first candidate API template corresponding to the API data through the semantic feature sub-model, and the second candidate API template corresponding to the API data through the similarity sub-model. Then, based on the first candidate API template and the second candidate API template, the first API template corresponding to the API data is determined through the candidate template sub-model.

[0115] Furthermore, the template generation model can be an AdaBoost ensemble model based on a dual-base classifier, used to determine the first API template corresponding to each API data; the semantic feature sub-model can be an LLM semantic base classifier, used to determine the first candidate API template corresponding to the API data; wherein, the first candidate API template includes a parameterized template and a classification confidence score; the similarity sub-model can be a vector similarity base classifier, used to calculate the similarity between the path of the API data and the template library, and determine the second candidate API template corresponding to the API data; wherein, the second candidate API template includes a candidate template set and the similarity of each candidate template, and the candidate template set includes at least three candidate templates; the candidate template sub-model can be an AdaBoost strong classifier, used to determine the first API template corresponding to the API data and the confidence score corresponding to the first API template based on the first candidate API template, the second candidate API template, and the weights.

[0116] The template verification module 450 is used to determine the second API template corresponding to each API group based on the confidence threshold and the confidence of each first API template.

[0117] Furthermore, when it is determined that the confidence level of multiple first API templates is greater than the confidence level threshold, the first API template corresponding to each API group is determined as the second API template, and the regular expressions corresponding to the second API template and the second API template are stored in the template library, and the first weight and the second weight are adjusted; when it is determined that the confidence level of the first API template is less than the confidence level threshold, manual annotation is performed to determine the new annotation data corresponding to the API data, and a new second API template corresponding to the API group is determined based on the new annotation data, and the template library is updated and the first weight and the second weight are adjusted based on the new second API template.

[0118] The relationship graph construction module 460 is used to construct an API data topology relationship graph based on each API group and the corresponding second API template.

[0119] It should be noted that the principle of the device for constructing API topology graphs provided in this application embodiment to solve the technical problem is similar to the method for constructing API topology graphs provided in this application embodiment. Therefore, the implementation of the device for constructing API topology graphs provided in this application embodiment can refer to the implementation of the method for constructing API topology graphs provided in this application embodiment, and the repeated parts will not be described again.

[0120] After introducing the method and apparatus for constructing API topology graphs provided in the embodiments of this application, the electronic device provided in the embodiments of this application will be briefly introduced next.

[0121] See Figure 5As shown, the electronic device 500 provided in this application embodiment includes at least a processor 501, a memory 502, and a computer program stored on the memory 502 and executable on the processor 501. When the processor 501 executes the computer program, it implements the method for constructing an API topology graph provided in this application embodiment.

[0122] The electronic device 500 provided in this application embodiment may further include a bus 503 connecting different components (including processor 501 and memory 502). The bus 503 represents one or more types of bus structures, including memory bus, peripheral bus, local area bus, etc.

[0123] Memory 502 may include a readable storage medium in the form of volatile memory, such as random access memory (RAM) 5021 and / or cache memory 5022, and may further include read-only memory (ROM) 5023. Memory 502 may also include a program tool 5025 having a set (at least one) of program modules 5024, including but not limited to an operating subsystem, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0124] Processor 501 can be a single processing element or a collective term for multiple processing elements. For example, processor 501 can be a central processing unit (CPU) or one or more integrated circuits configured to implement the method for constructing API topology graphs provided in the embodiments of this application. Specifically, processor 501 can be a general-purpose processor, including but not limited to CPUs, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0125] Electronic device 500 can communicate with one or more external devices 504 (e.g., keyboard, remote control, etc.), and also with one or more devices that enable a user to interact with electronic device 500 (e.g., mobile phone, computer, etc.), and / or with devices that enable electronic device 500 to communicate with one or more other electronic devices 500 (e.g., router, modem, etc.). This communication can be performed through input / output (I / O) interface 505. Furthermore, electronic device 500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 506. Figure 5 As shown, network adapter 506 communicates with other modules of electronic device 500 via bus 503. It should be understood that, although... Figure 5 As not shown, other hardware and / or software modules may be used in conjunction with the electronic device 500, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) subsystems, tape drives, and data backup storage subsystems.

[0126] It should be noted that, Figure 5 The electronic device 500 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0127] The following describes the computer-readable storage medium provided in the embodiments of this application. The computer-readable storage medium provided in the embodiments of this application stores computer instructions, which, when executed by a processor, implement the method for constructing an API topology graph provided in the embodiments of this application. Specifically, the computer instructions can be built into or installed in the processor, so that the processor can implement the method for constructing an API topology graph provided in the embodiments of this application by executing the built-in or installed computer instructions.

[0128] In addition, the method for constructing an API topology graph provided in this application embodiment can also be implemented as a computer program product, which includes program code that implements the method for constructing an API topology graph provided in this application embodiment when the program code is run on a processor.

[0129] The computer program product provided in this application embodiment may employ one or more computer-readable storage media, which may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. Specifically, more specific examples (a non-exhaustive list) of computer-readable storage media include electrical connections with one or more wires, portable disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0130] The computer program product provided in this application embodiment can be a CD-ROM and include program code, and can also run on electronic devices such as computers. However, the computer program product provided in this application embodiment is not limited thereto. In this application embodiment, the computer-readable storage medium can be any tangible medium that contains or stores program code, which can be used by or in conjunction with an instruction execution system, device, or apparatus.

[0131] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0132] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0133] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0134] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A method for constructing an API topology graph, characterized in that, include: Obtain traffic probe data at key network nodes; Based on the traffic probe data, each API data is determined; wherein, the API data includes the target IP and the target port; Based on the target IP and target port, the API data is grouped to obtain multiple API groups; Based on a template generation model, a first API template corresponding to each API data in each group is determined. The template generation model involves extracting semantic features from the API data using a semantic feature sub-model, determining a first candidate API template based on these semantic features, extracting mixed features from the API data using a similarity sub-model, and determining a second candidate API template based on these mixed features and weights. The mixed features include static segment features and dynamic segment features. The first API template corresponding to the API data is then determined based on the first candidate API template, the second candidate API template, and weights. The first candidate API template includes a parameterized template and a first classification confidence score. The second candidate API template includes a candidate template set and the similarity between each candidate template. The candidate template set includes at least three candidate templates. Based on the confidence threshold and the confidence of each of the first API templates, a second API template corresponding to each API group is determined; wherein, determining the second API template corresponding to each API group includes: when the confidence of multiple first API templates is greater than the confidence threshold, then the first API template corresponding to each API group is determined as the second API template, and the second API template and the regular expression corresponding to the second API template are stored in the template library, and the first weight and the second weight are adjusted; An API data topology diagram is constructed based on each API group and the corresponding second API template. The construction of the API data topology diagram includes: constructing a three-level API data topology diagram based on the target IP, target port, and determined second API template corresponding to each API group. The first level of the three-level API data topology diagram is the target IP, the second level is the target port, and the third level is the second API template.

2. The method for constructing an API topology graph according to claim 1, characterized in that, A similarity sub-model is used to extract mixed features from the API data, and based on the mixed features and weights, a second candidate API template corresponding to the API data is determined, including: The static segment features and the dynamic segment features are extracted using a similarity sub-model, respectively. Based on the static segment features, cosine similarity is used to determine the first similarity between the static segment features and the static segments of the candidate template; Based on the dynamic segment features, regularity rules are used to determine the second similarity between the dynamic segment features and the dynamic segments of the candidate template; Based on the first similarity and static segment weight, and the second similarity and dynamic segment weight, a third similarity is determined between the API data and the candidate template, and a second candidate API template corresponding to the API data is determined based on the third similarity.

3. The method for constructing an API topology graph according to claim 1, characterized in that, Based on the first candidate API template, the second candidate API template, and the weight, the first API template corresponding to the API data is determined, including: The semantic features of the API data are converted into a first category probability distribution; wherein, the first candidate API template belongs to all candidate templates corresponding to the first category probability distribution; The mixed features of the API data are converted into a second category probability distribution; wherein, the second candidate API template belongs to all candidate templates corresponding to the second category probability distribution; Based on the first category probability distribution and the first weight, and the second category probability distribution and the second weight, the first API template corresponding to the API data and the confidence level corresponding to the first API template are determined.

4. The method for constructing an API topology graph according to claim 1, characterized in that, Also includes: If the confidence level of the first API template is less than the confidence level threshold, then a manual annotation operation is performed to determine the new annotation data corresponding to the API data, and a new second API template corresponding to the API group is determined based on the new annotation data. Based on the new second API template, the template library is updated and the first weight and the second weight are adjusted.

5. The method for constructing an API topology graph according to any one of claims 1 to 4, characterized in that, Also includes: Obtain a training data set; wherein the training data set includes multiple training sample data; each training sample data includes API data and a standard first API template corresponding to the API data; Based on the training dataset, an iterative training operation is performed on the initial template generation model until the iterative training termination condition is met. Then, based on the weights of the initial template generation model updated during the last execution of the iterative training operation, the template generation model is obtained; wherein, the iterative training operation includes: Select target training sample data from the training dataset; The API data in the target training sample data is input into the initial template generation model, so that the initial template generation model determines the first candidate API template corresponding to the API data through the semantic feature sub-model, determines the second candidate API template corresponding to the API data through the similarity sub-model, and then determines the predicted first API template corresponding to the API data based on the first candidate API template and the second candidate API template through the candidate template sub-model. Based on the prediction error between the predicted first API template and the standard first API template in the target training sample data, the weights of the initial template generation model are updated.

6. An apparatus for constructing an API topology graph, characterized in that, include: The traffic probe module is used to acquire traffic probe data at the locations of key network nodes; A data preprocessing module is used to determine various API data based on the traffic probe data; wherein the API data includes the target IP and the target port; The data grouping module is used to group the API data based on the target IP and target port to obtain multiple API groups; A data classification module is used to determine a first API template corresponding to each API data in each group based on a template generation model. The template generation model extracts semantic features from the API data using a semantic feature sub-model, determines a first candidate API template based on these semantic features, extracts mixed features from the API data using a similarity sub-model, and determines a second candidate API template based on these mixed features and weights. The mixed features include static segment features and dynamic segment features. Based on the first candidate API template, the second candidate API template, and the weights, the first API template corresponding to the API data is then determined. The template verification module is used to determine the second API template corresponding to each API group based on a confidence threshold and the confidence of each first API template; wherein, determining the second API template corresponding to each API group includes: when the confidence of multiple first API templates is greater than the confidence threshold, then determining the first API template corresponding to each API group as the second API template, storing the second API template and the regular expression corresponding to the second API template in the template library, and adjusting the first weight and the second weight; A relationship graph construction module is used to construct an API data topology relationship graph based on each API group and the corresponding second API template. The construction of the API data topology relationship graph includes: constructing a three-level API data topology relationship graph based on the target IP, target port, and the determined second API template corresponding to each API group; wherein the first level of the three-level API data topology relationship graph is the target IP, the second level is the target port, and the third level is the second API template.

7. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method for constructing an API topology graph as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for constructing an API topology graph as described in any one of claims 1 to 5.

Images