Detection method, device, equipment, storage medium and computer program product

By building the call relationship and feature information of the API gateway, the training model performs exception detection, which solves the problem of poor intelligence in the existing technology of API gateway detection, and realizes efficient abnormal detection of real-time data, improving system security.

CN119094383AActive Publication Date: 2024-12-06CHINA MOBILE COMM GRP CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411081843.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-12-06
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

The existing API gateway abnormality detection method is poorly intelligent and relies on manual preset rules or real-time data model training, resulting in poor detection results and inability to effectively deal with network pressure caused by hacker attacks and overload traffic.

Method used

By obtaining the preset time log data of the API gateway, building a call relationship, determining node feature information, and using this information to train the model for abnormal detection, we realize intelligent abnormal detection of real-time log data.

Benefits of technology

It improves the detection intelligence of API gateways, improves the accuracy of abnormal detection of real-time data, reduces manual intervention, and enhances system security.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094383A_ABST
    Figure CN119094383A_ABST
Patent Text Reader

Abstract

The invention provides a detection method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining first log data of an API gateway within a preset time, and constructing a first call relation based on the first log data; acquiring a second node related to the first node based on the first calling relationship; determining first node data corresponding to the first node and second node data corresponding to the second node; determining first feature information corresponding to the first node according to the first node data and the second node data; training by using the first feature information and the first log data to obtain a first model; and performing anomaly detection on second log data acquired by the API gateway in real time based on the first model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer network technology, and in particular to a detection method, apparatus, device, storage medium and computer program product. Background Art

[0002] With the rise of cloud services and the widespread use of Application Programming Interface (API) technology, many network capabilities of operators are usually exported and enabled in the form of APIs. API gateways have become an important way to interconnect north-south and east-west systems and share data. As the number of APIs and the frequency of calls continue to rise, there are many problems in actual calls, such as hackers using API vulnerabilities to attack enterprise systems and cause data leakage, and overloaded traffic access will put pressure on the network and consume system resources, directly affecting the satisfaction of operators' customers and customer retention. However, existing API gateway anomaly detection methods usually rely on manually preset rules or require real-time collection of API gateway data for model training before detection, resulting in poor detection intelligence. Summary of the invention

[0003] The present application provides a detection method, apparatus, device, storage medium and computer program product that can improve detection intelligence.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, the present application proposes a detection method, the method comprising:

[0006] Obtain first log data of the API gateway within a preset time, and build a first call relationship based on the first log data;

[0007] Acquire a second node related to the first node based on the first call relationship; and determine first node data corresponding to the first node and second node data corresponding to the second node;

[0008] Determine first feature information corresponding to the first node according to the first node data and the second node data;

[0009] Using the first feature information and the first log data for training to obtain a first model;

[0010] Based on the first model, anomaly detection is performed on the second log data collected in real time by the API gateway.

[0011] In the above solution, the step of constructing a first call relationship based on the first log data includes:

[0012] Determining parameter information corresponding to the first log data;

[0013] Constructing a second call relationship based on the parameter information;

[0014] Obtaining registration information of the API gateway, and establishing a third call relationship according to the registration information;

[0015] The second calling relationship and the third calling relationship are merged to obtain the first calling relationship.

[0016] In the above solution, the determining the first characteristic information corresponding to the first node according to the first node data and the second node data includes:

[0017] Inputting the first node data and the second node data into a second model to obtain second feature information corresponding to the first node;

[0018] The second feature information is input into the third model to obtain the first feature information.

[0019] In the above solution, the first feature information and the first log data are used for training to obtain a first model, including:

[0020] Using the first feature information to train at least one fourth model to obtain at least one training data;

[0021] Training is performed based on the at least one training data and the first log data to obtain the first model.

[0022] In the above solution, the training based on the at least one training data and the first log data to obtain the first model includes:

[0023] Determining abnormal data from the first log data;

[0024] Training the initial first model based on the abnormal data and the at least one training data to obtain a loss function;

[0025] The model parameters are updated according to the loss function to obtain the first model.

[0026] In the above solution, the training of the initial first model based on the abnormal data and the at least one training data to obtain a loss function includes:

[0027] Training the initial first model according to the at least one training data to obtain a trained first model;

[0028] The trained first model is trained based on the abnormal data to obtain the loss function.

[0029] In a second aspect, the present application proposes a detection device, the device comprising:

[0030] A construction unit, configured to obtain first log data of the API gateway within a preset time, and to construct a first call relationship based on the first log data;

[0031] A first determining unit, configured to acquire a second node related to the first node based on the first calling relationship; and determine first node data corresponding to the first node and second node data corresponding to the second node;

[0032] Determine first feature information corresponding to the first node according to the first node data and the second node data;

[0033] A training unit, configured to perform training using the first feature information and the first log data to obtain a first model;

[0034] A detection unit is used to perform anomaly detection on the second log data collected in real time by the API gateway based on the first model.

[0035] In a third aspect, the present application proposes a detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned methods when executing the program.

[0036] In a fourth aspect, the present application proposes a storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0037] In a fifth aspect, the present application proposes a computer program product, comprising a computer program, which implements the steps of any of the above methods when executed by a processor.

[0038] The present application proposes a detection method, device, equipment, storage medium and computer program product, the method comprising: obtaining the first log data of the API gateway within a preset time, constructing a first call relationship based on the first log data; obtaining a second node related to the first node based on the first call relationship; and determining the first node data corresponding to the first node and the second node data corresponding to the second node; determining the first feature information corresponding to the first node according to the first node data and the second node data; training with the first feature information and the first log data to obtain a first model; and performing anomaly detection on the second log data collected in real time by the API gateway based on the first model. With the above implementation scheme, the first call relationship constructed by the first log data within the preset time characterizes the changes in the first log data within the preset time, and the first model obtained by training with the first feature information determined based on the first call relationship can realize anomaly detection of log data that changes in real time, thereby improving the detection intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of a detection method provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of an exemplary API gateway anomaly detection system module provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of an exemplary API gateway network process anomaly detection process provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the structure of a detection device provided in an embodiment of the present application;

[0043] Figure 5 This is a schematic diagram of the structure of a detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0046] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first\second\third", etc., involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second\third", etc., may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein may be implemented in an order other than that illustrated or described herein.

[0047] The present application embodiment provides a detection method, Figure 1 A flow chart of a detection method provided in an embodiment of the present application is as follows: Figure 1 As shown, the method may include:

[0048] S101. Obtain first log data of an API gateway within a preset time, and construct a first call relationship based on the first log data.

[0049] In the embodiment of the present application, the first log data may be monitoring log data of the API gateway, that is, the monitoring log data of the API gateway within a certain time period (preset time) is obtained.

[0050] In an embodiment of the present application, constructing a first call relationship based on the first log data specifically includes: determining parameter information corresponding to the first log data; constructing a second call relationship based on the parameter information; obtaining registration information of the API gateway, and constructing a third call relationship based on the registration information; merging the second call relationship and the third call relationship to obtain the first call relationship.

[0051] Among them, determining the parameter information corresponding to the first log data can be understood as preprocessing the first log data to determine the parameter information. The preprocessing process can be determined according to the actual situation, and is not limited here. As an example, the preprocessing process specifically includes: first eliminating the invalid information in the first log data, including duplicate information and useless information, and then parsing the first log data, and extracting the parameter information in the first log data according to the event template. The parameter information can be any parameter information related to the API gateway, and the specific content can be determined according to the actual situation, and is not limited here. As an example, the parameter information specifically includes the calling application name, the calling party Internet Protocol (IP), the called service name, the called API name, the API description, the API type, the request method, the request time, the request parameters, the request traffic size, the response traffic size, the gateway forwarding time, the response time, the response result, the response data, whether to trigger the API gateway configuration policy, and the triggering policy name.

[0052] In an embodiment of the present application, the second call relationship can be understood as a network relationship diagram constructed based on the caller's data and the callee's data in the first log data. Among them, the caller and the callee can be regarded as nodes in the constructed network relationship diagram, and the node type can be determined according to the actual situation, and is not limited here. As an example, the caller is a southbound node and the callee is a northbound node; or the caller can be a northbound node and the callee can be a southbound node, or the caller can be a southbound node and the callee can also be a southbound node, etc. It should be noted that the same node can be both a caller and a callee, which can be determined specifically according to the actual situation.

[0053] Among them, building the second calling relationship based on the parameter information can be understood as determining the first attribute information of the caller node and the second attribute information of the called party according to the parameter information, and building the second calling relationship based on the first attribute information and the second attribute information; the specific construction process can be illustrated as follows: if a calling behavior is sent between node A and node B through the API gateway, then nodes A and B are connected, and the calling relationship includes northbound calls to southbound calls and southbound calls to each other; if there is a subscription relationship between node A and node B, then nodes A and B are connected, and the frequency of calls per unit time represents the weight of the edge.

[0054] It should be noted that the first attribute information and the second attribute information contain the same content. The specific content can be determined based on actual conditions and is not limited here. As an example, the attribute information may include: application type, application name, application description, average daily response traffic, average daily request traffic, average daily number of calls, gateway forwarding frequency, number of registered gateway APIs, and registered gateway API information.

[0055] For ease of understanding in practical applications, the network relationship diagram at a certain moment can be represented by G t =(V,E,W,A), where V represents the node set, E represents the edge set, W is the edge weight matrix, and A is the attribute set of each node. A network relationship diagram on the time flow is constructed for data in different time periods, that is, the second call relationship can be represented by G={G1,G2,…,G T}, where T is the total number of time snapshot steps.

[0056] In the embodiment of the present application, the registration information of the API gateway can be the information obtained when registering any API network. The specific content can be determined according to the actual situation and is not limited here. As an example, the registration information of the API gateway can include the name information and system information when registering the API gateway. The third call relationship can be understood as a network relationship diagram constructed based on the registration information, which can also be called a feature similarity graph G' in practical applications.

[0057] Constructing the third call relationship based on the registration information can be understood as performing text processing on the registration information to obtain text information corresponding to the nodes in the registration information, calculating the similarity between node features based on the text information, and constructing the third call relationship based on the similarity.

[0058] Among them, the process of text processing of registration information can be specifically described as merging all text information such as north-south system descriptions, API descriptions, etc. into a corpus, and using a bidirectional encoder representation from Transformers (Bidirectional Encoder Representation from Transformers, BERT) language model to obtain the text features of the nodes.

[0059] The similarity between node features can be calculated using formulas (1) and (2), which are as follows:

[0060]

[0061] L gsr =||A l || (2)

[0062] Among them, ε′ i,j represents the similarity, ε∈[0,1] represents the threshold value for controlling the density of the generated feature similarity graph, E(X i ',X' j ) represents the similarity calculation between two nodes, and the commonly used calculation method is cosine similarity; L gsr represents the structured sparse regularizer, which is introduced through the Manhattan (L1) norm to ensure the sparsity of the feature similarity graph G'. A is the adjacency matrix of G'.

[0063] The second call relationship and the third call relationship are merged to obtain the first call relationship; it can be understood that the original network graph G and the generated feature similarity graph G' are merged to obtain a new dynamic network graph. The specific merging process can be: if there is a connection relationship between two nodes in G', but the same two nodes in G do not have a connection relationship, then the two nodes in the merged graph have a connection relationship, and the edge weight is the edge weight in G'; if G' and the same two nodes in G have a connection relationship, then the edge weight of the node in the merged graph is the edge weight in G.

[0064] The solution of the embodiment of the present application generates a new refined network relationship graph based on metric learning, which can solve the problem of inactivity of some southbound APIs.

[0065] S102: Acquire a second node related to the first node based on the first calling relationship; and determine first node data corresponding to the first node and second node data corresponding to the second node.

[0066] In the embodiment of the present application, the first node can be any node in the first call relationship, which can be determined according to actual conditions and is not limited here. The second node can be understood as an adjacent node in the first node; it should be noted that the number of the second nodes can be one or more.

[0067] Acquiring the second node related to the first node based on the first calling relationship can be understood as acquiring all the second nodes adjacent to the first node in the network relationship graph.

[0068] The first node data and the second node data can be understood as the attribute data of the node; the process of determining the node data can be obtained from the first log data. The first node data and the second node data can specifically include: application type, application name, application description, average daily response flow, average daily request flow, average daily call times, gateway forwarding frequency, number of registered gateway APIs, and registered gateway API information.

[0069] S103: Determine first characteristic information corresponding to the first node according to the first node data and the second node data.

[0070] In an embodiment of the present application, the specific process of determining the first characteristic information corresponding to the first node based on the first node data and the second node data includes: inputting the first node data and the second node data into the second model to obtain the second characteristic information corresponding to the first node; inputting the second characteristic information into the third model to obtain the first characteristic information.

[0071] The second model may be any neural network model, which may be determined according to actual conditions. As an example, the second model may be a relational graph convolutional neural network model (R-GCN). The third model may be any time-related network model. As an example, the third model may be an attention mechanism model.

[0072] It should be noted that the process of obtaining the second feature information can also be called node embedding learning based on the structural dimension in practical applications; the process of obtaining the first feature information based on the second feature information can also be called node embedding learning based on the time dimension in practical applications.

[0073] The second feature information can be understood as the structural information of the network relationship graph obtained by the first node. In practical applications, the second feature information can be the embedded identifier of the first node at different time snapshots, which can be used It is represented by, where v represents the specified node and T represents the total number of time snapshot steps. The first feature information can be understood as the structure and time information of the network relationship graph obtained by the first node. In practical applications, the second feature information can be used To indicate that, F' represents the dimension of the output, and T represents the total number of time snapshot steps.

[0074] For ease of understanding, the specific process of obtaining the first feature information is illustrated as follows:

[0075] Node embedding learning based on structural dimension: Use the relationship-based R-GCN to learn node embedding, Z = R-GCN (X, A, R) to represent the R-GCN model, where X is the attribute feature matrix and A is the adjacency matrix of R. R-GCN considers different types of relationships between two nodes when aggregating neighborhood features. The hierarchical propagation rule is shown in formula (3). The embedding vectors of neighboring nodes are normalized and weighted summed, and then weighted and added to the embedding vector of the central node itself, and then nonlinearly transformed. Then, the central node is updated. Formula (3) is as follows:

[0076]

[0077] in, represents the representation of the node at layer l+1, W l represents the learning weight, N r represents the neighbor set of node i under relationship r, σ(·) is a nonlinear activation function, c i,r is a normalization constant, generally defined as (Number of neighbors with edge type r).

[0078] The node embedding representation of the specified node v at different time snapshots is obtained through the R-GCN model:

[0079] Node embedding learning based on time dimension: Use the scaled dot product attention mechanism to capture the changes in the time level of the graph structure. The input is the representation sequence of the specified node v at different time snapshot structural dimensions, that is, Where T is the total number of time snapshot steps, and D' is the dimension of the input representation. The output is the new representation sequence of the node at each time snapshot, that is, F' represents the dimension of the output. Set the three attention vectors as queries, keys, and values, and transform the queries, keys, and values ​​through the linear projection matrix W q ∈R D'×F' , W k ∈R D'×F , Wv ∈R D'×F' They are transformed into different spaces respectively, where the temporal attention formulas (4), (5) and (6) are as follows:

[0080] Z v =β v (X v W v ) (4)

[0081]

[0082]

[0083] Among them, Z v is the new representation sequence of the node at each time snapshot, X v is the representation sequence of nodes in different time snapshot structural dimensions, i and j are different nodes, W q is the projection matrix corresponding to Queries, W k is the projection matrix corresponding to Keys, W v is the projection matrix corresponding to Values, β v ∈R T×T is the attention weight matrix obtained by the multiplicative attention function, M∈R T×T is the mask matrix, M ij ∈{-∞,0} is used to enhance the autoregressive property. In order to encode the time order, M is defined as shown in formula (7). Formula (7) is as follows:

[0084]

[0085] The solution of the embodiment of the present application comprehensively considers graph structure information and time dimension information to embed heterogeneous graph nodes and learn node features, which can improve the accuracy of the detection model.

[0086] S104: Perform training using the first feature information and the first log data to obtain a first model.

[0087] In an embodiment of the present application, the first model can be understood as an anomaly detection model; the specific process of obtaining the first model includes: training at least one fourth model using the first feature information to obtain at least one training data; training based on at least one training data and the first log data to obtain the first model.

[0088] Among them, the fourth model can be any neural network model, which can be determined according to actual conditions and is not limited here; the number of the fourth models can also be determined according to actual conditions and is not limited here. As an example, the number of the fourth models can be 4. When the number of the fourth models is 4, the fourth model can include eXtreme Gradient Boosting (XGB), Light Gradient Boosting Trees (LGBT), Random Forest (RF), and Deep Neural Networks (DNN).

[0089] The at least one fourth model is trained using the first feature information to obtain at least one training data; it can be understood that the first feature information is feature divided to obtain at least one third feature information, and each fourth model in the at least one fourth model is trained based on the at least one third feature information to obtain at least one training data.

[0090] It is understandable that the feature division process can be determined according to the actual situation, and is not limited here. As an example, the first feature information can be divided into five equal parts, and the first four equal parts of feature information are used to train each fourth model to obtain at least one training data, and the last equal part of feature information is used to verify the training data. It should be noted that at least one training data can be understood as the prediction result of at least one fourth model in actual application.

[0091] In an embodiment of the present application, the process of obtaining a first model based on training of at least one training data and first log data specifically includes: determining abnormal data from the first log data; training the initial first model based on the abnormal data and at least one training data to obtain a loss function; and updating the model parameters according to the loss function to obtain the first model.

[0092] It is understandable that the process of determining abnormal data from the first log data can be determined according to actual conditions, and is not limited here. As an example, the abnormal data can be determined by manual labeling.

[0093] The specific process of training the initial first model based on the abnormal data and at least one training data to obtain the loss function includes: training the initial first model according to the at least one training data to obtain the trained first model; training the trained first model based on the abnormal data to obtain the loss function.

[0094] The initial first model may be any anomaly detection model, which is not limited here. As an example, the initial first model may be a Logistic classification model.

[0095] The trained first model is trained according to the abnormal data to obtain the loss function; it can be understood that the abnormal data is sampled to obtain the support set and the query set respectively, the trained first model is trained based on the support set, and the loss function is calculated based on the query set as the verification set.

[0096] The model parameters are updated according to the loss function to obtain the first model; it can be understood that the model parameters of the trained first model are updated based on the loss function until the model converges to obtain the first model.

[0097] For ease of understanding, in practical applications, the specific process of obtaining the first model can be illustrated as follows:

[0098] Anomaly prediction module based on stacked generalization (stacking): Use supervised learning for anomaly detection, and use the node embedding features obtained in step 2 as input for model training. Use the stacking model fusion method to fuse the prediction results of multiple single models to improve the accuracy of anomaly prediction. In the early stage of model construction, it is necessary to manually annotate the attack behaviors in the collected data. The specific steps are as follows:

[0099] (1) The first T-1 time snapshots are selected as the training set, and the node embedding features obtained in step 2 are used as feature input to build and predict the first-layer model. The selected models include XGB, LGBT, RF, and DNN. The training set data is divided into five equal parts using a five-fold cross-validation method. The model is trained using the first four equal parts and validated on the fifth equal part.

[0100] (2) The prediction results of each model are used as the feature input of the second-layer integrated model to build a new model. The second-layer model can use the Logistic classification model to finally obtain the abnormal nodes in the network relationship diagram.

[0101] Optimize the classification model module based on meta-learning: In the actual network environment, the occurrence of abnormal events belongs to the small sample scenario. This module optimizes the prediction model based on the model-agnostic meta-learning (MAML) algorithm. It is mainly used in the second layer model in the stacking model structure to improve the generalization ability of the integrated classification model on small sample tasks. Meta-learning is divided into two processes: meta-training and meta-testing. The specific steps are as follows:

[0102] (1) Given a set of tasks T defined as different types of abnormal event detection problems, sample the data of each task τ∈T to obtain the support set S τ and the query set Q τ .

[0103] (2) For each training task τ i , based on the support set S τ As training data, the model is trained and gradient descent is performed based on the query set Q τ As a validation set, calculate the loss function L τ , the parameters are updated by the loss of the model on the query set. The parameter update of a gradient step can be expressed by formula (8), which is as follows:

[0104]

[0105] in, is the learning rate, θ is the initial parameter set of the model, and f θ is the base learner, θ i ' is the updated model parameter set, L τ is the loss function.

[0106] (3) Optimize and update metamodel parameters θ i ', so that the average verification loss on all tasks is minimized, the updated formula (9) is as follows:

[0107]

[0108] in, is the learning rate, θ i ' is the model parameter obtained after formula (8), L is the loss function, the θ on the left side of the arrow is the model parameter updated by formula (9), and the θ on the right side of the arrow is the initial model parameter set.

[0109] (4) Repeat steps (2)-(3) until convergence and obtain the final model parameter θ.

[0110] (5) Use the parameter θ as the initialization parameter of the meta-test model, input the support set into the model, continue to optimize the parameters by calculating the loss function, input the query set into the model as the test set, and predict the query set sample labels.

[0111] The solution of the embodiment of the present application adopts a meta-learning algorithm to optimize the anomaly detection model, which can improve the generalization ability of the model in small sample scenarios.

[0112] S105: Perform anomaly detection on the second log data collected in real time by the API gateway based on the first model.

[0113] It should be noted that, without directly intervening in the operation of the gateway, the first model is independently deployed to the API gateway forwarding server, and the model can be trained, established, and optimized using offline logs.

[0114] Figure 2 A schematic diagram of an exemplary API gateway anomaly detection system module provided in an embodiment of the present application is shown as follows: Figure 2 As shown in the figure, the model includes API gateway, Kafka, search engine library (Elasticsearch, ES) log library, anomaly detection artificial intelligence (Artificial Intelligence, AI) model, real-time analysis distributed streaming data engine (flink) service, gateway (Apisix) rule configuration, API caller and access routing; Kafka can collect logs for API gateway, and the anomaly detection AI model is independently deployed to the API gateway forwarding server. The data in the Elasticsearch log library can be used for training and offline optimization. The real-time analysis flink service calls the anomaly detection AI model to detect the collected real-time logs, and the obtained detection structure is fed back to the API gateway. Among them, the Apisix rule configuration can collaboratively realize the intelligent management and control of the gateway.

[0115] The solution of the embodiment of the present application utilizes the first log data and the first feature information to train a first model, and realizes the use of offline log data to train the detection model. The first feature information integrates the first node data and the second node data, which can improve the accuracy of the detection model, and directly uses the first model to perform anomaly detection on the API gateway data collected in real time, avoiding errors caused by relying on manual detection, thereby improving the security of the API gateway system.

[0116] Based on the above embodiments, the present application embodiment proposes a method for detecting anomalies in an API gateway network process. Figure 3 A flowchart of an exemplary API gateway network process anomaly detection provided in an embodiment of the present application; Figure 3 As shown:

[0117] 1. Network dynamic relationship diagram construction module: Based on API gateway monitoring logs and metric learning, the network dynamic relationship diagram between application systems is refined.

[0118] 2. Learning heterogeneous graph node embedding module based on graph neural network: Use the relationship-based graph convolutional neural network to learn graph structure features, and use the attention neural network to learn time dimension features.

[0119] 3. Stacking-based anomaly prediction module: integrates the prediction results of multiple single models to improve the accuracy of anomaly prediction.

[0120] 4. Optimize the classification model module based on meta-learning: Optimize the second-layer prediction model of the integrated model based on the MAML meta-learning algorithm to improve the generalization ability of the model in small sample tasks.

[0121] 5. API gateway anomaly intelligent detection system module: independently deployed to the API gateway forwarding server, thereby indirectly realizing intelligent management and control of the network.

[0122] The present application embodiment provides a detection device, Figure 4 A schematic diagram of the structure of a detection device provided in an embodiment of the present application; Figure 4 As shown, the detection device 400 includes:

[0123] A construction unit 401 is used to obtain first log data of the API gateway within a preset time, and to construct a first call relationship based on the first log data;

[0124] A determining unit 402 is configured to obtain a second node related to the first node based on the first calling relationship; and determine first node data corresponding to the first node and second node data corresponding to the second node;

[0125] Determine first feature information corresponding to the first node according to the first node data and the second node data;

[0126] A training unit 403 is used to perform training using the first feature information and the first log data to obtain a first model;

[0127] The detection unit 404 is used to perform anomaly detection on the second log data collected in real time by the API gateway based on the first model.

[0128] Optionally, the construction unit 401 is also used to determine parameter information corresponding to the first log data; construct a second call relationship based on the parameter information; obtain the registration information of the API gateway, and construct a third call relationship based on the registration information; merge the second call relationship and the third call relationship to obtain the first call relationship.

[0129] Optionally, the determination unit 402 is further used to input the first node data and the second node data into a second model to obtain second feature information corresponding to the first node; and input the second feature information into a third model to obtain the first feature information.

[0130] Optionally, the training unit 403 is further used to train at least one fourth model using the first feature information to obtain at least one training data; and to perform training based on the at least one training data and the first log data to obtain the first model.

[0131] Optionally, the training unit 403 is further used to determine abnormal data from the first log data; train the initial first model based on the abnormal data and the at least one training data to obtain a loss function; and update the model parameters according to the loss function to obtain the first model.

[0132] Optionally, the training unit 403 is further used to train the initial first model according to the at least one training data to obtain a trained first model; and train the trained first model based on the abnormal data to obtain the loss function.

[0133] The present application also provides a detection device, Figure 5 Schematic diagram of the structure of a detection device provided in an embodiment of the present application; Figure 5 As shown, the detection device 500 includes: a processor 501 and a memory 503 . Optionally, the detection device 500 may also include a communication bus 502 .

[0134] In the process of a specific embodiment, the processor 501 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing image processing device (DSPD), a programmable logic image processing device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the function of the processor may also be other, which is not specifically limited in this embodiment.

[0135] In the embodiment of the present application, the communication bus 502 is used to realize the connection and communication between the processor 501 and the memory 503; when the processor 501 executes the running program stored in the memory 503, the following detection method is implemented:

[0136] Obtain first log data of an API gateway within a preset time, and construct a first call relationship based on the first log data; obtain a second node related to the first node based on the first call relationship; and determine first node data corresponding to the first node and second node data corresponding to the second node; determine first feature information corresponding to the first node based on the first node data and the second node data; use the first feature information and the first log data for training to obtain a first model; and perform anomaly detection on the second log data collected in real time by the API gateway based on the first model.

[0137] Furthermore, the above-mentioned processor 501 is also used to determine parameter information corresponding to the first log data; construct a second call relationship based on the parameter information; obtain the registration information of the API gateway, and construct a third call relationship based on the registration information; merge the second call relationship and the third call relationship to obtain the first call relationship.

[0138] The processor 501 is further configured to input the first node data and the second node data into a second model to obtain second feature information corresponding to the first node; and input the second feature information into a third model to obtain the first feature information.

[0139] The processor 501 is further configured to train at least one fourth model using the first feature information to obtain at least one training data; and to perform training based on the at least one training data and the first log data to obtain the first model.

[0140] The processor 501 is further configured to determine abnormal data from the first log data; train the initial first model based on the abnormal data and the at least one training data to obtain a loss function; and update the model parameters according to the loss function to obtain the first model.

[0141] An embodiment of the present application provides a storage medium having a computer program stored thereon. The computer-readable storage medium stores one or more programs. The one or more programs can be executed by one or more processors. The computer program implements the detection method as described above.

[0142] Based on the above embodiments, an embodiment of the present application provides a computer program product, including a computer program, which can be executed by one or more processors, and the computer program implements the detection method as described above.

[0143] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for enabling an image display device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0145] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A detection method, It is characterized in that The method comprises: Obtaining first log data of an application programming interface API gateway within a preset time, and building a first call relationship based on the first log data; Acquire a second node related to the first node based on the first call relationship; and determine first node data corresponding to the first node and second node data corresponding to the second node; Determine first feature information corresponding to the first node according to the first node data and the second node data; Using the first feature information and the first log data for training to obtain a first model; Based on the first model, anomaly detection is performed on the second log data collected in real time by the API gateway.

2. The method according to claim 1, It is characterized in that The step of constructing a first call relationship based on the first log data includes: Determining parameter information corresponding to the first log data; Constructing a second call relationship based on the parameter information; Obtaining registration information of the API gateway, and establishing a third call relationship according to the registration information; The second calling relationship and the third calling relationship are merged to obtain the first calling relationship.

3. The method according to claim 1, It is characterized in that The determining, according to the first node data and the second node data, first feature information corresponding to the first node includes: Inputting the first node data and the second node data into a second model to obtain second feature information corresponding to the first node; The second feature information is input into the third model to obtain the first feature information.

4. The method according to claim 1, It is characterized in that The using the first feature information and the first log data to perform training to obtain a first model includes: Using the first feature information to train at least one fourth model to obtain at least one training data; Training is performed based on the at least one training data and the first log data to obtain the first model.

5. The method according to claim 4, It is characterized in that The performing training based on the at least one training data and the first log data to obtain the first model includes: Determining abnormal data from the first log data; Training the initial first model based on the abnormal data and the at least one training data to obtain a loss function; The model parameters are updated according to the loss function to obtain the first model.

6. The method according to claim 5, It is characterized in that The training of the initial first model based on the abnormal data and the at least one training data to obtain a loss function includes: Training the initial first model according to the at least one training data to obtain a trained first model; The trained first model is trained based on the abnormal data to obtain the loss function.

7. A detection device, It is characterized in that The device comprises: A construction unit, configured to obtain first log data of the API gateway within a preset time, and to construct a first call relationship based on the first log data; a determining unit, configured to acquire a second node related to the first node based on the first calling relationship; and determine first node data corresponding to the first node and second node data corresponding to the second node; and determine first feature information corresponding to the first node according to the first node data and the second node data; A training unit, configured to perform training using the first feature information and the first log data to obtain a first model; A detection unit is used to perform anomaly detection on the second log data collected in real time by the API gateway based on the first model.

8. A detection device, It is characterized in that The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 when executing the program.

9. A storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Citation Information

Patent Citations

  • Gateway test method and device, electronic equipment and storage medium

    CN112994976A

  • Data detection method, device and equipment for cloud auditing, medium and program product

    CN116980162A

  • Power monitoring system main body anomaly detection method, device, equipment and medium

    CN117640342A

  • Packaging and deploying algorithms for flexible machine learning

    US20190155633A1