Abnormal access object determination method and apparatus, storage medium, and electronic device
By constructing knowledge graph data and utilizing node clustering and graph matching algorithms, abnormal access objects of target applications can be accurately identified, solving the problems of detection accuracy and efficiency in existing technologies that are difficult to detect web crawlers, and achieving efficient identification of web crawlers.
Patent Information
- Application Number
- CN202310608339.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-24
- Publication Date
- 2025-12-30
- Estimated Expiration
- 2043-05-24
AI Technical Summary
Existing technologies struggle to accurately detect groups of people making unusual access to websites, especially since detection methods for web crawlers are easily tampered with and difficult to identify attacks from groups of devices.
By constructing knowledge graph data for the target application, multiple graphs are generated. Node clustering and graph matching algorithms are used to determine the first type of nodes carrying abnormal access tags, forming a set of risk nodes, and then identifying abnormal access objects.
It enables accurate identification of abnormal access groups to target applications, improves the accuracy and efficiency of crawler detection, and avoids the omission of noisy data.
Smart Images

Figure CN116614282B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of network security technology, and more specifically, to a method, apparatus, storage medium, and electronic device for identifying abnormal access objects. Background Technology
[0002] Machine traffic refers to the traffic generated by programs that automatically execute access (also known as web crawlers). On the Internet, web crawlers bring a lot of machine traffic, especially in industries such as travel ticketing, where popular data interfaces can have more than 95% machine traffic.
[0003] A large number of access requests from web crawlers often bring huge traffic and processing burdens to the target website's server, and may also lead to the leakage of private data, causing data security problems. To solve these problems, crawler detection is needed. Related technologies calculate the relevant access behavior characteristics of the current access session and then use rules to determine whether the access originates from a web crawler, or use machine learning models to determine whether the access originates from a web crawler. However, the relevant access behavior characteristics of access sessions are mostly statistical features in their construction, lacking good adversarial capabilities and easily tampered with. Furthermore, these detection schemes all treat individual IPs as detection targets, making it difficult to accurately detect attacks from increasingly prevalent groups of devices.
[0004] There is currently no effective solution to the above problems. Summary of the Invention
[0005] This application provides a method, apparatus, storage medium, and electronic device for determining abnormal access targets, so as to at least solve the technical problem in the related art of it being difficult to accurately detect groups of objects that abnormally access a website.
[0006] According to one aspect of the embodiments of this application, a method for determining abnormal access objects is provided, comprising: obtaining multiple graphs based on knowledge graph data of a target application, wherein the knowledge graph data is determined by traffic data of the target application, the graphs are composed of multiple nodes and edges, and the multiple nodes in the graphs include at least a first type of node, the first type of node representing an object carrying an abnormal access tag, and the probability of abnormal access being determined by the traffic data; determining a risk node set based on the distribution of the first type of nodes in the multiple graphs, wherein the risk node set includes multiple first type of nodes; and determining abnormal access objects of the target application based on the risk node set.
[0007] Furthermore, determining the risk node set based on the distribution of the first type of nodes in multiple graphs includes: using a target algorithm to cluster the nodes in multiple graphs to obtain multiple node sets; determining whether the proportion of the number of first type nodes in the node set is greater than a first preset proportion; and determining the node set whose proportion of the number of first type nodes is greater than the first preset proportion as the risk node set.
[0008] Furthermore, after clustering nodes in multiple graphs using the target algorithm to obtain multiple node sets, before determining whether the proportion of the number of nodes of the first type in the node set is greater than the first preset proportion, the method further includes: determining nodes outside the node set that have a connection relationship with nodes in the node set, obtaining at least one target node; adding at least one target node to the node set, obtaining an updated node set.
[0009] Furthermore, if the graph contains multiple nodes of the second type, where the probability of abnormal access to the target application is unknown, after clustering the nodes in the multiple graphs using the target algorithm to obtain multiple node sets, before determining whether the proportion of the number of first-type nodes in the node sets is greater than a first preset proportion, the method further includes: calculating the probability of abnormal access to the target application by the second-type nodes when they exist in multiple graphs; if the probability of abnormal access to the target application by the second-type nodes is greater than a preset probability, the second-type nodes are determined as first-type nodes.
[0010] Furthermore, when there are second-type nodes in multiple graphs, after calculating the probability of the second-type nodes abnormally accessing the target application, the method further includes: forming an updated first-type node by combining the first-type nodes in the multiple graphs and the second-type nodes whose probability of abnormal accessing the target application is greater than a preset probability; updating the knowledge graph data of the target application based on the updated first-type nodes; obtaining updated multiple graphs based on the updated knowledge graph data; determining an updated risk node set based on the distribution of the first-type nodes in the updated multiple graphs; and determining the abnormal access objects of the target application based on the nodes in the updated risk node set.
[0011] Furthermore, determining the risk node set based on the distribution of the first type of nodes in multiple graphs includes: obtaining the first type of graphs from multiple graphs, and identifying graphs other than the first type of graphs in the multiple graphs as graphs to be tested, wherein the proportion of the number of first type nodes in the first type of graphs is greater than a second preset proportion; calculating the similarity between the graph to be tested and the first type of graphs; identifying the graph to be tested with a similarity greater than a preset similarity as the first type of graph, and determining the risk node set based on the nodes in the first type of graphs.
[0012] Furthermore, if the graph includes a second type of node, where the second type of node represents an object whose probability of abnormal access to the target application is unknown, determining the risk node set based on the nodes in the first type of graph includes: calculating the probability of abnormal access to the target application by the second type of node when the second type of node exists in the first type of graph; and determining the risk node set by the second type of node whose probability of abnormal access to the target application is greater than a preset probability and the first type of node in the first type of graph.
[0013] Furthermore, before obtaining multiple graphs based on the knowledge graph data of the target application, the method further includes: acquiring traffic data of accessing the target application within a preset time period; extracting entity data, relation data, and attribute data from the traffic data, wherein the entity data includes at least one of the following: data of the accessed object and / or data of the accessed object; the relation data refers to the relationship between the accessed object and / or the accessed object; and the attribute data includes at least one of the following: attribute data of the accessed object, attribute data of the accessed object, and attribute data of the relationship between the accessed object and / or the accessed object; and generating knowledge graph data of the target application based on the entity data, relation data, and attribute data.
[0014] According to another aspect of the embodiments of this application, a method for determining abnormal access objects is also provided, comprising: a cloud server receiving knowledge graph data of a target application, wherein the knowledge graph data is determined by traffic data of the target application; the cloud server obtaining multiple graphs based on the knowledge graph data of the target application, determining a risk node set based on the distribution of first-type nodes in the multiple graphs, and determining abnormal access objects of the target application based on the risk node set, wherein the graph is composed of nodes and edges, and the multiple nodes in the graph include at least first-type nodes, the first-type nodes representing objects carrying abnormal access tags, the probability of abnormal access being determined by traffic data, and the risk node set including multiple first-type nodes; and the cloud server feeding back the abnormal access objects of the target application to the client.
[0015] According to another aspect of the embodiments of this application, an apparatus for determining abnormal access objects is also provided, comprising: a first determining unit, configured to obtain multiple graphs based on knowledge graph data of a target application, wherein the knowledge graph data is determined by traffic data of the target application, the graphs are composed of multiple nodes and edges, and the multiple nodes in the graphs include at least a first type of node, the first type of node representing an object carrying an abnormal access tag, and the probability of abnormal access being determined by the traffic data; a second determining unit, configured to determine a risk node set based on the distribution of the first type of nodes in the multiple graphs, wherein the risk node set includes multiple first type of nodes; and a third determining unit, configured to determine the abnormal access objects of the target application based on the risk node set.
[0016] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein, when the program is running, it controls the device where the computer-readable storage medium is located to execute a method for determining abnormal access objects.
[0017] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes a method for determining an abnormal access object during runtime.
[0018] In this embodiment, multiple graphs are obtained based on the knowledge graph data of the target application, and a set of risk nodes is determined based on the distribution of the first type of nodes in the graphs. Since the first type of nodes represent objects carrying abnormal access tags, the set of risk nodes obtained from the first type of nodes represents the set of nodes with abnormal access risks. Then, the abnormal access objects of the target application are determined based on the set of risk nodes, thus achieving the purpose of determining the abnormal access object group of the target application based on the knowledge graph data of the target application. This achieves the technical effect of accurately determining the object group that performs abnormal access to the target application, thereby solving the technical problem in related technologies that it is difficult to accurately detect the object group that performs abnormal access to the website. Attached Figure Description
[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0020] Figure 1 This is a hardware structure block diagram of a computer terminal according to an embodiment of this application;
[0021] Figure 2 This is a flowchart of the method for determining abnormal access objects according to Embodiment 1 of this application;
[0022] Figure 3 This is a schematic diagram of knowledge graph data construction according to Embodiment 1 of this application;
[0023] Figure 4 This is the process for determining the set of risk nodes based on the distribution of the first type of nodes, as provided in Embodiment 1 of this application. Figure 1 ;
[0024] Figure 5 This is the process for determining the set of risk nodes based on the distribution of the first type of nodes, as provided in Embodiment 1 of this application. Figure 2 ;
[0025] Figure 6This is a flowchart of the method for determining abnormal access objects according to Embodiment 2 of this application;
[0026] Figure 7 This is a schematic diagram of the device for determining abnormal access objects according to Embodiment 3 of this application;
[0027] Figure 8 This is a structural block diagram of a computer terminal provided according to Embodiment 4 of this application. Detailed Implementation
[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0030] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0031] The following are explanations of some of the nouns or terms that appear in the description of the embodiments of this application:
[0032] Knowledge graph: A structured semantic knowledge base used to process and integrate unstructured data, transforming it into a "entity-relationship-entity" triple structure to describe the concepts of various things and their interrelationships.
[0033] Web crawler: A program or script that automatically completes a series of network requests according to certain rules to retrieve information from a website.
[0034] Example 1
[0035] According to an embodiment of this application, a method for determining abnormal access objects is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0036] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 This is a hardware structure block diagram of a computer terminal according to an embodiment of this application. For example... Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.
[0037] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).
[0038] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the abnormal access object determination method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned abnormal access object determination method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0039] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.
[0040] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).
[0041] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.
[0042] Under the aforementioned operating environment, this application provides the following: Figure 2 The method for identifying abnormal access objects is shown. Figure 2 This is a flowchart of a method for determining abnormal access objects according to Embodiment 1 of this application, the method including:
[0043] Step S202: Obtain multiple graphs based on the knowledge graph data of the target application. The knowledge graph data is determined by the traffic data of the target application. The graph consists of multiple nodes and edges. Among the multiple nodes in the graph, there are at least first-type nodes. The first-type nodes represent objects carrying abnormal access tags. The probability of abnormal access is determined by the traffic data.
[0044] In one alternative implementation, the target application is an application that requires crawler detection. It can be a website or an application installed on a client. For example, the target application is a travel website, which has a large amount of machine traffic and therefore requires detection of crawlers that generate machine traffic.
[0045] It should be noted that, since knowledge graph data can reflect the implicit relationships inherent in things, the first embodiment of this application can generate knowledge graph data of the target application through the traffic data of the target application.
[0046] Optionally, in the method for determining abnormal access objects provided in Embodiment 1 of this application, before obtaining multiple graphs based on the knowledge graph data of the target application, the method further includes: acquiring traffic data of accessing the target application within a preset time period; extracting entity data, relation data, and attribute data from the traffic data, wherein the entity data includes at least one of the following: data of the accessed object and / or data of the accessed object; the relation data refers to the relationship between the accessed object and / or the accessed object; and the attribute data includes at least one of the following: attribute data of the accessed object, attribute data of the accessed object, and attribute data of the relationship between the accessed object and / or the accessed object; and generating knowledge graph data of the target application based on the entity data, relation data, and attribute data.
[0047] In one optional implementation, the preset time period refers to the time period during which crawler detection is required. This period can be set according to needs; for example, the preset time period could be the most recent quarter in which the target application has been running. The sources of traffic data accessing the target application can include multiple sources, such as data from web browsers, application-side data, and supplementary data collected from external network environments.
[0048] Traffic data can be recorded in tabular form, including information such as source IP, destination IP, domain name, browser fingerprint, and device fingerprint. After obtaining multi-source traffic data of the target application, the required entity data, relational data, and attribute data are extracted from the tabular traffic data according to rules. Based on the entity data, relational data, and attribute data, a knowledge graph data of the target application is generated. The knowledge graph data of the target application is heterogeneous data.
[0049] Entity data includes data of the accessed object and / or the accessed object, such as IP (Internet Protocol Address), device fingerprint (UMID, Unique Material Identifier), domain name, and browser fingerprint.
[0050] Relationship data refers to the relationship between the accessed object and / or the accessed object. This can be an access relationship or other types of relationships. For example, device fingerprints UMID1 and UMID2 can establish a relationship if they are in the same Wi-Fi (Wireless Fidelity) segment; a relationship can be established between UMID1 and UMID2 if they access the same IP address; a relationship can also be established between UMID1 and the IP address; a relationship can be established between UMID2 and the IP address; a relationship can be established between the IP address and the domain name if a website's IP address is used to access its domain name; a relationship can be established between IP1 and IP2 if both IP1 and IP2 access the same domain name; and a relationship can also be established between the IP address and the device fingerprint UMID when accessing a website's IP address.
[0051] Attribute data includes attribute data associated with entity data and attribute data associated with relational data. For example, an IP address is an entity, and its attributes can include IP attributes, IP geographic location, etc. Relationships are access relationships, and their attributes can include access direction and access weight. Access weight can be determined by the number of accesses. It should be noted that for access objects that have historically accessed multiple applications automatically, a "crawler" tag can be assigned. Access objects carrying this tag have a higher probability of being crawlers when accessing the target application. In the knowledge graph data, access objects with the "crawler" tag are represented as first-type nodes, and objects indicated by first-type nodes have a higher probability of abnormal access to the target application. For access objects that have historically genuinely accessed multiple applications, a "non-crawler" tag can be assigned. Access objects carrying this tag have a higher probability of not being crawlers when accessing the target application. In the knowledge graph data, access objects without the "crawler" tag are represented as third-type nodes, and objects indicated by third-type nodes have a higher probability of normal access to the target application. Furthermore, the probability that an access object without a crawler tag or without a crawler tag is a crawler when accessing the target application is unknown. In the knowledge graph data, access objects without a crawler tag are represented as second-type nodes, and the probability that the objects indicated by the second-type nodes perform abnormal access to the target application is unknown.
[0052] Optionally, after obtaining the knowledge graph data of the target application, the entity data and relation data in the knowledge graph data are converted into a graph representation of nodes and edges, the attribute data in the knowledge graph data are associated with the nodes and edges in the graph to obtain multiple graph structures, and the features of the nodes and edges of the graph structures are converted into vector representations to obtain multiple graphs.
[0053] Figure 3 This is a schematic diagram of knowledge graph data construction according to Embodiment 1 of this application, such as... Figure 3 As shown, accessing a domain name via a website's IP address establishes an access relationship between the IP address and the domain name. A connection exists between the website's IP address and the device's UMID (device fingerprint). During the process of accessing a domain name via a website's IP address, a browser fingerprint is generated, establishing a generation / being-generated relationship between the IP address and the domain name. Similarly, a generation / being-generated relationship exists between the device fingerprint (UMID) and the browser fingerprint. Furthermore, an attribution relationship exists between the browser fingerprint and the domain name. Therefore, the entity data in the target application's knowledge graph data includes IP addresses, domain names, device fingerprints, and browser fingerprints, while the relational data includes access relationships, connection relationships, attribution relationships, and generation / being-generated relationships. When constructing the knowledge graph based on the knowledge graph data, nodes are generated for each entity such as IP address, domain name, device fingerprint, and browser fingerprint. Edges are then established between these nodes based on the access relationships, connection relationships, attribution relationships, and generation / being-generated relationships, thus generating the graph.
[0054] It should be noted that nodes in the diagram representing access objects with the "crawler" tag are classified as Category 1 nodes, which can be black. Nodes representing access objects without the "crawler" tag or neither are classified as Category 2 nodes, which can be gray. Nodes representing access objects without the "crawler" tag are classified as Category 3 nodes, which can be white.
[0055] By fusing and processing multi-source data, knowledge graph data is constructed and transformed into graph form, providing a comprehensive and accurate data foundation for crawler detection, thereby improving the accuracy and efficiency of detection.
[0056] Step S204: Determine the risk node set based on the distribution of the first type of nodes in multiple graphs, wherein the risk node set includes multiple first type nodes.
[0057] It should be noted that the first type of node is a node generated by an object carrying a crawler tag, which can be represented as a black node in the diagram. Risk nodes are nodes that pose a risk to the operation of the target application. Nodes in a densely distributed area of black nodes are more likely to be risk nodes. The risk node set consists of nodes in a densely distributed area of black nodes. The risk node set contains multiple black nodes and may also include nodes that are connected to black nodes, such as gray nodes and white nodes. Although gray nodes are nodes generated by objects without crawler tags and white nodes are nodes generated by objects without crawler tags, white nodes and gray nodes connected to black nodes pose an indirect risk to the operation of the target application. To avoid missing abnormal access objects, other types of nodes that are connected to black nodes are also classified into the risk node set.
[0058] Step S206: Determine the abnormal access objects of the target application based on the risk node set.
[0059] In one optional implementation, the abnormal access object refers to the object that performs abnormal access to the target application, i.e., the crawler of the target application. Since there are risk nodes generated by the object that performs access to the target application and other risk nodes in the risk node set, the risk nodes generated by the object that performs access to the target application are obtained from them, and the object indicated by such nodes is identified as the abnormal access object of the target application.
[0060] In Embodiment 1 of this application, multiple graphs are obtained based on the knowledge graph data of the target application. The knowledge graph data is determined by the traffic data of the target application. Each graph consists of multiple nodes and edges. At least one type of node is included among the multiple nodes in the graph. The first type of node represents an object carrying an abnormal access tag, and the probability of abnormal access is determined by the traffic data. A risk node set is determined based on the distribution of the first type of nodes in the multiple graphs. This risk node set includes multiple first type nodes. The abnormal access objects of the target application are then determined based on the risk node set. It should be noted that this embodiment obtains multiple graphs based on the knowledge graph data of the target application. The knowledge graph data contains implicit relationships between things. The graph obtained from the knowledge graph data contains nodes and edges. Nodes indicate objects, and edges represent relationships between objects. This graph can intuitively reflect the relationships between objects and the distribution of different types of nodes. The risk node set is determined based on the distribution of nodes carrying abnormal access tags, and the abnormal access objects of the target application are determined based on the risk node set in the graph. This achieves the technical effect of accurately identifying the group of objects abnormally accessing the target application, thereby solving the technical problem in related technologies where it is difficult to accurately detect the group of objects abnormally accessing a website.
[0061] When determining the risk node set, one can first determine the densely distributed area of nodes, and then determine the densely distributed area of the first type of nodes from it. The nodes in the densely distributed area of nodes constitute the node set, and the nodes in the densely distributed area of the first type of nodes constitute the risk node set. Optionally, in the method for determining abnormal access objects provided in Embodiment 1 of this application, determining the risk node set based on the distribution of the first type of nodes in multiple graphs includes: using a target algorithm to cluster the nodes in multiple graphs to obtain multiple node sets; determining whether the proportion of the number of the first type of nodes in the node set is greater than a first preset proportion; and determining the node set whose proportion of the number of the first type of nodes is greater than the first preset proportion as the risk node set.
[0062] In one alternative implementation, the nodes in a densely distributed area are determined by a target algorithm, which can be a community discovery algorithm or other clustering algorithms. The target algorithm aggregates or divides related nodes in the graph into different node clusters.
[0063] It should be noted that since a cluster can contain other types of nodes besides the first type, such as second-type nodes without crawler tags and third-type nodes without crawler tags, in the diagram, first-type nodes can be black, second-type nodes can be gray, and third-type nodes can be white. The proportion of first-type nodes to the total number of nodes in the cluster can be used to determine whether a node cluster is a risky cluster. This proportion can be called the black node concentration. The black node concentration is used as the criterion for determining risk. Clusters with a black node concentration exceeding a threshold are identified as risky clusters. For example, if the threshold is 80%, and a cluster has a black node concentration of 90%, exceeding 80%, then the cluster can be identified as a risky cluster.
[0064] After generating a graph using the knowledge graph data of the target application, the nodes in the graph are first clustered to obtain a node set. Then, the risk node set is determined from the node set based on the proportion of the first type of nodes. Since the first type of nodes represent objects carrying abnormal access tags, the nodes in the risk node set represent objects that have risky access to the target application. This achieves the purpose of determining the risk node set using the implicit relationships in the knowledge graph data of the target application.
[0065] To avoid missing risk nodes during the detection process, optionally, in the method for determining abnormal access objects provided in Embodiment 1 of this application, after clustering nodes in multiple graphs using a target algorithm to obtain multiple node sets, before determining whether the proportion of the number of first-type nodes in the node sets is greater than a first preset proportion, the method further includes: determining nodes outside the node set that have a connection relationship with nodes in the node set, obtaining at least one target node; adding at least one target node to the node set, obtaining an updated node set.
[0066] In one optional implementation, the node that has a connection relationship with the node can be a node within the preset jump connection edge of the node. After the related nodes in the graph are aggregated or divided into different node clusters by the target algorithm, the target node within the preset jump connection edge of the node in the node set can be recalled, wherein the target node is a node outside the node set.
[0067] For example, the preset hop can be 3 hops. The nodes that have 1 connecting edge, 2 connected edges, or 3 connected edges in the node set constitute the nodes within the 3-hop connecting edges of a certain node. That is, the target node recalled through this node.
[0068] It's important to note that if node A exists and has been assigned to node set 1, but is connected to nodes in node set 2, then during node recall in node set 2, node A will not be recalled to node set 2 and will remain in node set 1. Conversely, if node B exists and is not assigned to any node set, but is connected to nodes in at least two node sets, then during node recall in these node sets, node B can be recalled by one of these node sets. In other words, during node recall in node sets, a node will not exist in two node sets simultaneously.
[0069] In this first embodiment, the node set is updated according to the target node, which increases the number of nodes in the node set. After the node set is determined to be a risk node set, the omission of risk nodes can be reduced.
[0070] It should be noted that, in addition to the first type of nodes carrying the crawler tag, the graph may also contain a second type of nodes without the crawler tag or non-crawler tag. Optionally, in the method for determining abnormal access objects provided in Embodiment 1 of this application, if the graph includes a second type of node among multiple nodes, the second type of node represents an object whose probability of abnormal access to the target application is unknown. After clustering the nodes in multiple graphs using the target algorithm to obtain multiple node sets, before determining whether the proportion of the number of first type nodes in the node set is greater than a first preset proportion, the method further includes: calculating the probability of abnormal access to the target application by the second type of node when there are second type of nodes in multiple graphs; if the probability of abnormal access to the target application by the second type of node is greater than the preset probability, the second type of node is determined as a first type of node.
[0071] In this context, since the second type of node refers to nodes that do not carry either a crawler tag or a non-crawler tag, the probability of the object indicated by the second type of node abnormally accessing the target application is unknown. If the set of nodes with a proportion greater than a first preset proportion is defined as the risk node set when multiple nodes in the graph include the second type, the resulting risk node set may still contain some noisy data, leading to inaccurate risk node set judgments. It should be noted that the noisy data in the risk node set refers to nodes that do not indicate an object abnormally accessing the target application. Specifically, for the object represented by the second type of node, since it does not carry either a crawler tag or a non-crawler tag, the probability of abnormal access to the target application is unknown; therefore, the second type of node has a certain probability of being noisy data.
[0072] To improve the accuracy of determining the risk node set, a graph matching algorithm based on random walk and a label walk algorithm can be used to search for second-class nodes similar to the first-class nodes in the graph. The obtained second-class nodes are then sorted, and the second-class nodes with a higher probability of abnormal access to the target application than a preset probability are determined from the sorting results. This results in the newly added first-class nodes. The proportion of the number of first-class nodes in the node set is then used as an indicator to redetermine the risk node set.
[0073] In this first embodiment, before determining whether a node set belongs to a risk point set based on the proportion of the number of first-type nodes, the second-type nodes with unknown abnormal access probability are reclassified, some second-type nodes are updated to first-type nodes, and the risk node set is re-determined based on the classification results. If there is a node set containing a large number of second-type nodes among multiple node sets, the accuracy of risk node set judgment can be improved, and omissions in risk node set judgment can be avoided.
[0074] To further improve the accuracy of determining the risk node set and thus the accuracy of determining abnormal access objects, the knowledge graph data of the target application can be updated by updating the first type of nodes. Based on the updated knowledge graph data, the abnormal access objects of the target application can be re-determined. Optionally, in the method for determining abnormal access objects provided in Embodiment 1 of this application, after calculating the probability of abnormal access to the target application by the second type of nodes in multiple graphs where second type of nodes exist, the method further includes: constructing an updated first type of nodes from the first type of nodes in multiple graphs and second type of nodes whose probability of abnormal access to the target application is greater than a preset probability; updating the knowledge graph data of the target application based on the updated first type of nodes; obtaining updated multiple graphs based on the updated knowledge graph data; determining an updated risk node set based on the distribution of the first type of nodes in the updated multiple graphs; and determining the abnormal access objects of the target application based on the nodes in the updated risk node set.
[0075] In one optional implementation, nodes of the second type with a probability greater than a preset probability of abnormal access to the target application are identified as newly added nodes of the first type. The original nodes of the first type and the newly added nodes of the first type together constitute the updated nodes of the first type. Since the knowledge graph data of the target application will also change after the node attributes change, the knowledge graph data of the target application is updated and reconstructed. The target algorithm aggregates or divides the related nodes in the graph into different node clusters. The node clusters will change. The proportion of the number of nodes of the first type in the node cluster is recalculated, and the updated risk node set is determined according to the proportion of the number of nodes of the first type in the node cluster. The objects in the risk node set that indicate access to the target application are identified as abnormal access objects.
[0076] Figure 4 This is the process for determining the set of risk nodes based on the distribution of the first type of nodes, as provided in Embodiment 1 of this application. Figure 1 ,like Figure 4 As shown, the method includes: after constructing a knowledge graph based on the target application's knowledge graph data, on the one hand, using a community detection algorithm to aggregate or divide related nodes in the graph into different node clusters, and recalling nodes outside the node clusters that are associated with nodes in the node clusters, obtaining an updated node set, and determining whether a node cluster is a risk node cluster based on the concentration of the first type of nodes in the node cluster. On the other hand, searching for second type of nodes similar to the first type of nodes in the graph, and sorting the obtained second type of nodes, determining the top-ranked second type of nodes from the sorting results, and identifying these second type of nodes as the purified first type of nodes, and purifying the risk node set based on the purified first type of nodes, that is, recalculating the concentration of the first type of nodes in the node cluster, and then updating the risk node cluster based on the concentration of the first type of nodes in the node cluster. The above steps can be iterated a preset number of times, and the risk node set obtained after the last iteration is output. Through the first embodiment of this application, second type of nodes with unknown abnormal access probability are reclassified, and the knowledge graph data of the target application is updated based on the classification results. The risk node set is then re-determined based on the updated knowledge graph data, improving the accuracy of determining the risk node set.
[0077] There are various ways to determine the risk node set based on the distribution of the first type of nodes. Optionally, in the method for determining abnormal access objects provided in Embodiment 1 of this application, determining the risk node set based on the distribution of the first type of nodes in multiple graphs includes: obtaining the first type of graphs from multiple graphs, and determining the graphs other than the first type of graphs in the multiple graphs as the graphs to be tested, wherein the proportion of the number of the first type of nodes in the first type of graphs is greater than a second preset proportion; calculating the similarity between the graphs to be tested and the first type of graphs; determining the graphs to be tested with a similarity greater than a preset similarity as the first type of graphs, and determining the risk node set based on the nodes in the first type of graphs.
[0078] Figure 5 This is the process for determining the set of risk nodes based on the distribution of the first type of nodes, as provided in Embodiment 1 of this application. Figure 2 ,like Figure 5 As shown, after constructing the graph based on the knowledge graph data of the target application, if a graph is known to belong to the first type of graph (the first type of graph can be a graph in which the proportion of first type nodes is greater than a preset proportion), it can be used as a reference. Other graphs to be tested are matched with the reference graph for similarity to determine whether the two graphs belong to the same type of graph or have similar features. If the similarity between the graph to be tested and the reference graph exceeds a certain threshold, it means that the two are similar and the nodes of the graph to be tested are also risky nodes. The graph to be tested is determined as a newly added first type of graph. The risk node set is determined based on the nodes in the original first type of graph and the newly added first type of graph. In an optional implementation, the nodes in both the original first type of graph and the newly added first type of graph can be used as nodes in the risk node set.
[0079] In one alternative implementation, graph matching algorithms, subgraph isomorphism algorithms, etc., can be used to calculate the similarity between two graphs. For example, if the threshold is 80%, and the calculated similarity between the graph to be tested and the reference graph exceeds 80%, then the graph to be tested and the reference graph are similar and also belong to the first type of graph.
[0080] Through Embodiment 1 of this application, when the first type of graph is known, other graphs are determined to be of the first type by judging whether two graphs belong to the same type of graph or have similar features, thereby improving the efficiency of determining the first type of graph and thus improving the efficiency of determining risk node clusters.
[0081] It should be noted that, in addition to the first type of nodes carrying the crawler tag, the diagram may also include a second type of nodes that do not carry the crawler tag or are not crawler-tagged. Optionally, in the method for determining abnormal access objects provided in Embodiment 1 of this application, if the second type of nodes are included among the multiple nodes in the diagram, the second type of nodes represent objects whose probability of abnormal access to the target application is unknown. Determining the risk node set based on the nodes in the first type of diagram includes: if there are second type of nodes in the first type of diagram, calculating the probability of abnormal access to the target application by the second type of nodes; and determining the risk node set by the second type of nodes whose probability of abnormal access to the target application is greater than a preset probability and the first type of nodes in the first type of diagram.
[0082] Since the second type of node refers to a node that does not carry either a crawler tag or a non-crawler tag, the probability of the object indicated by the second type of node being accessed abnormally by the target application is unknown. Therefore, in order to accurately determine the risk node set when the first type of node exists, we can search for second type of nodes similar to the first type of node in the graph, sort the obtained second type of node, and determine the second type of node whose probability of abnormal access to the target application is greater than the preset probability from the sorting results. This results in the newly added first type of node. The original first type of node in the first type of graph and the newly added first type of node constitute the risk node set, thereby improving the accuracy of determining the risk node set.
[0083] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0085] Example 2
[0086] Figure 6 This is a flowchart of a method for determining abnormal access objects according to Embodiment 2 of this application. Figure 6 As shown, the method includes:
[0087] In step S602, the cloud server receives the knowledge graph data of the target application, wherein the knowledge graph data is determined by the traffic data of the target application.
[0088] In one alternative implementation, the target application is an application that requires crawler detection. It can be a website or an application installed on a client. For example, the target application is a travel website, which has a large amount of machine traffic and therefore requires detection of crawlers that generate machine traffic.
[0089] It should be noted that, since knowledge graph data can reflect the implicit relationships inherent in things, this embodiment can generate knowledge graph data for the target application using its traffic data. The sources of traffic data for the target application can be multiple, including data from web browsers, application-side data, and supplementary data collected from external network environments. Traffic data can be recorded in tabular form, recording information such as source IP, destination IP, domain name, browser fingerprint, and device fingerprint. After obtaining the multi-source traffic data of the target application, the required entity data, relationship data, and attribute data are extracted from the tabular traffic data according to rules. Based on the entity data, relationship data, and attribute data, knowledge graph data for the target application is generated. This knowledge graph data for the target application is heterogeneous.
[0090] Step S604: The cloud server obtains multiple graphs based on the knowledge graph data of the target application, determines the risk node set based on the distribution of the first type of nodes in the multiple graphs, and determines the abnormal access objects of the target application based on the risk node set. The graph consists of nodes and edges, and the multiple nodes in the graph include at least the first type of nodes. The first type of nodes represent objects carrying abnormal access tags. The probability of abnormal access is determined by traffic data, and the risk node set includes multiple first type of nodes.
[0091] In one alternative implementation, after obtaining the knowledge graph data of the target application, the entity data and relation data in the knowledge graph data are converted into a graph representation of nodes and edges. The attribute data in the knowledge graph data are associated with the nodes and edges in the graph to obtain multiple graph structures. The features of the nodes and edges of the graph structures are converted into vector representations to obtain multiple graphs, thus laying a data foundation for the detection of abnormal access objects.
[0092] It should be noted that the diagram can contain multiple types of nodes. The first type of node is a node generated by an object carrying a crawler tag, which can be represented as a black node in the diagram. Risk nodes are nodes that pose a risk to the operation of the target application. Nodes in a densely distributed area of black nodes are more likely to be risk nodes. The risk node set consists of nodes in a densely distributed area of black nodes. The risk node set contains multiple black nodes and can also include nodes that are connected to black nodes, such as gray nodes and white nodes. Although gray nodes are nodes generated by objects without crawler tags and white nodes are nodes generated by objects without crawler tags, since white nodes and gray nodes connected to black nodes pose an indirect risk to the operation of the target application, in order to avoid missing abnormal access objects, other types of nodes that are connected to black nodes are also classified into the risk node set.
[0093] An abnormal access object refers to an object that performs abnormal access to the target application, i.e., a crawler of the target application. Since there are risk nodes generated by objects that perform abnormal access to the target application in the risk node set, as well as other risk nodes, the risk nodes generated by objects that perform abnormal access to the target application are obtained from them, and the objects indicated by such nodes are identified as abnormal access objects of the target application.
[0094] In step S606, the cloud server sends the abnormal access object of the target application back to the client.
[0095] In one alternative implementation, the cloud server feeds back the abnormal access objects of the target application to the client, so that the client can be aware of the abnormal access objects of the target application.
[0096] It should be noted that the method for identifying abnormal access objects in the cloud server is the same as the method for identifying abnormal access objects in Example 1, and will not be repeated here.
[0097] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0099] Example 3
[0100] According to an embodiment of this application, an apparatus for implementing the above-described method for determining abnormal access objects is also provided. Figure 7 This is a schematic diagram of the device for determining abnormal access objects according to Embodiment 3 of this application, as shown below. Figure 7 As shown, the device includes:
[0101] The first determining unit 702 is used to obtain multiple graphs based on the knowledge graph data of the target application. The knowledge graph data is determined by the traffic data of the target application. The graph consists of multiple nodes and edges. The multiple nodes in the graph include at least a first type of node. The first type of node represents an object carrying an abnormal access tag. The probability of abnormal access is determined by the traffic data.
[0102] The second determining unit 704 is used to determine the risk node set based on the distribution of the first type of nodes in multiple graphs, wherein the risk node set includes multiple first type nodes.
[0103] The third determining unit 706 is used to determine the abnormal access objects of the target application based on the risk node set.
[0104] In the device for determining abnormal access objects provided in this embodiment three, the first determining unit 702 obtains multiple graphs based on the knowledge graph data of the target application. The knowledge graph data is determined by the traffic data of the target application. The graphs consist of multiple nodes and edges, and at least one type of node is included among the multiple nodes in the graph. The first type of node represents an object carrying an abnormal access tag, and the probability of abnormal access is determined by the traffic data. The second determining unit 704 determines a risk node set based on the distribution of the first type of nodes in the multiple graphs. The risk node set includes multiple first type nodes. The third determining unit 706 determines the abnormal access objects of the target application based on the risk node set. By obtaining multiple graphs based on the knowledge graph data of the target application and determining the abnormal access objects of the target application based on the risk node set in the graphs, the technical effect of accurately determining the group of objects abnormally accessing the target application is achieved, thereby solving the technical problem in related technologies that it is difficult to accurately detect the group of objects abnormally accessing a website.
[0105] Optionally, in the device for determining abnormal access objects provided in Embodiment 3 of this application, the second determining unit 704 includes: a clustering module, used to cluster nodes in multiple graphs using a target algorithm to obtain multiple node sets; a judging module, used to judge whether the proportion of the number of first-type nodes in the node set is greater than a first preset proportion; and a first determining module, used to determine the node set whose proportion of the number of first-type nodes is greater than the first preset proportion as a risk node set.
[0106] Optionally, in the device for determining abnormal access objects provided in Embodiment 3 of this application, the device further includes: a second determining module, used to determine, after clustering nodes in multiple graphs using a target algorithm to obtain multiple node sets, before determining whether the proportion of the number of first-type nodes in the node sets is greater than a first preset proportion, a node outside the node set that has a connection relationship with a node in the node set, thereby obtaining at least one target node; and an adding module, used to add at least one target node to the node set, thereby obtaining an updated node set.
[0107] Optionally, in the device for determining abnormal access objects provided in Embodiment 3 of this application, if the multiple nodes in the graph include a second type of node, and the second type of node represents an object whose probability of abnormal access to the target application is unknown, the device further includes: a first calculation module, used to calculate the probability of abnormal access to the target application by the second type of node when there are second type of nodes in the multiple graphs, after clustering the nodes in the multiple graphs using the target algorithm to obtain multiple node sets, before determining whether the proportion of the number of first type of nodes in the node sets is greater than a first preset proportion; and a third determination module, used to determine the second type of node as a first type of node if the probability of abnormal access to the target application by the second type of node is greater than a preset probability.
[0108] Optionally, in the device for determining abnormal access objects provided in Embodiment 3 of this application, the device further includes: a first update module, configured to, when there are second type nodes in multiple graphs, calculate the probability of the second type nodes abnormally accessing the target application, and then construct an updated first type node based on the first type nodes in the multiple graphs and the second type nodes whose probability of abnormal access to the target application is greater than a preset probability, and update the knowledge graph data of the target application based on the updated first type nodes; and a second update module, configured to, obtain updated multiple graphs based on the updated knowledge graph data, determine an updated risk node set based on the distribution of the first type nodes in the updated multiple graphs, and determine the abnormal access objects of the target application based on the nodes in the updated risk node set.
[0109] Optionally, in the device for determining abnormal access objects provided in Embodiment 3 of this application, the second determining unit 704 includes: an acquisition module, used to acquire a first type of graph from multiple graphs and determine graphs other than the first type of graph in the multiple graphs as graphs to be tested, wherein the proportion of the number of first type nodes in the first type of graph is greater than a second preset proportion; a second calculation module, used to calculate the similarity between the graph to be tested and the first type of graph; and a fourth determining module, used to determine the graph to be tested with a similarity greater than a preset similarity as the first type of graph and determine a set of risk nodes based on the nodes in the first type of graph.
[0110] Optionally, in the device for determining abnormal access objects provided in Embodiment 3 of this application, if the multiple nodes in the figure include a second type of node, and the second type of node represents an object whose probability of abnormal access to the target application is unknown, the fourth determining module includes: a first calculation submodule, used to calculate the probability of abnormal access to the target application by the second type of node when there is a second type of node in the first type of figure; and a determining submodule, used to determine a set of risk nodes by the second type of node whose probability of abnormal access to the target application is greater than a preset probability and the first type of node in the first type of figure.
[0111] Optionally, in the device for determining abnormal access objects provided in Embodiment 3 of this application, before obtaining multiple graphs based on the knowledge graph data of the target application, the device further includes: an acquisition unit, used to acquire traffic data of accessing the target application within a preset time period; an extraction unit, used to extract entity data, relation data, and attribute data from the traffic data, wherein the entity data includes at least one of the following: data of the accessed object and / or data of the accessed object, the relation data refers to the relationship between the accessed object and / or the accessed object, and the attribute data includes at least one of the following: attribute data of the accessed object, attribute data of the accessed object, and attribute data of the relationship between the accessed object and / or the accessed object; and a generation unit, used to generate knowledge graph data of the target application based on the entity data, relation data, and attribute data.
[0112] It should be noted that the above-mentioned units correspond to the steps in Embodiment 1. The instances and application scenarios implemented by the units and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above-mentioned modules or units may be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above-mentioned modules may also be part of a device and can run in the computer terminal 10 provided in Embodiment 1.
[0113] It should be noted that the preferred implementation schemes involved in the above embodiments of this application are the same as the schemes, application scenarios and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.
[0114] Example 4
[0115] Embodiments of this application may provide a computer terminal, which may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the aforementioned computer terminal may also be replaced by a mobile terminal or other terminal device.
[0116] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.
[0117] In this embodiment, the computer terminal described above can execute the program code for the following steps in the method for determining abnormal access objects: obtaining multiple graphs based on the knowledge graph data of the target application, wherein the knowledge graph data is determined by the traffic data of the target application, the graph is composed of multiple nodes and edges, and the multiple nodes in the graph include at least a first type of node, the first type of node representing an object carrying an abnormal access tag, and the probability of abnormal access being determined by the traffic data; determining a risk node set based on the distribution of the first type of nodes in the multiple graphs, wherein the risk node set includes multiple first type of nodes; and determining the abnormal access objects of the target application based on the risk node set.
[0118] Optionally, Figure 8 This is a structural block diagram of a computer terminal provided according to Embodiment 4 of this application. Figure 8 As shown, the computer terminal 10 may include: one or more ( Figure 8 (Only one is shown) processor 102, memory 104, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.
[0119] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and apparatus for determining abnormal access objects in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the aforementioned method for determining abnormal access objects. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0120] The processor can invoke information and applications stored in memory via a transmission device to perform the following steps: obtaining multiple graphs based on the knowledge graph data of the target application, wherein the knowledge graph data is determined by the traffic data of the target application, the graphs consist of multiple nodes and edges, and the multiple nodes in the graphs include at least a first type of node, the first type of node representing an object carrying an abnormal access label, the probability of abnormal access being determined by the traffic data; determining a risk node set based on the distribution of the first type of nodes in the multiple graphs, wherein the risk node set includes multiple first type of nodes; and determining the abnormal access objects of the target application based on the risk node set.
[0121] This application provides a method for determining abnormal access targets. Multiple graphs are obtained based on the knowledge graph data of the target application. A set of risk nodes is determined based on the distribution of first-type nodes in the graphs. Since the first-type nodes represent objects carrying abnormal access tags, the set of risk nodes obtained from the first-type nodes represents the set of nodes with abnormal access risks. The abnormal access targets of the target application are then determined based on the risk node set. This achieves the goal of determining the group of abnormal access targets of the target application based on the knowledge graph data of the target application, thereby achieving the technical effect of accurately determining the group of objects abnormally accessing the target application. This solves the technical problem in related technologies where it is difficult to accurately detect the group of objects abnormally accessing a website.
[0122] Those skilled in the art will understand that the structure shown in the figure is for illustrative purposes only, and the computer terminal may also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, and other terminal devices. Figure 8 This does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include components that are more advanced than those described above. Figure 8 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 8 The different configurations shown.
[0123] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0124] Example 5
[0125] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the method for determining abnormal access objects provided in Embodiment 1.
[0126] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.
[0127] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining multiple graphs based on the knowledge graph data of the target application, wherein the knowledge graph data is determined by the traffic data of the target application, the graphs consist of multiple nodes and edges, and the multiple nodes in the graphs include at least a first type of node, the first type of node representing an object carrying an abnormal access tag, the probability of abnormal access being determined by the traffic data; determining a risk node set based on the distribution of the first type of nodes in the multiple graphs, wherein the risk node set includes multiple first type of nodes; and determining the abnormal access object of the target application based on the risk node set.
[0128] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0129] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0130] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0131] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0132] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0133] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0134] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method of determining an abnormal access object, characterized by, The method comprises the following steps: obtaining a plurality of graphs according to knowledge graph data of a target application, wherein the knowledge graph data is determined by traffic data of the target application, the traffic data of the target application is traffic data of accessing the target application, the graph is composed of a plurality of nodes and edges, the node represents data of an access object or data of an accessed object, the edge represents relationship data between the access object and / or the accessed object, at least a first type of node is included in the plurality of nodes in the graph, the first type of node represents an object carrying an abnormal access label, and a probability of abnormal access is determined by the traffic data; determining a risk node set according to a distribution of the first type of node in the plurality of graphs, wherein the risk node is a node that exists risk to the running of the target application, and the risk node set includes a plurality of first type of nodes and nodes having a connection relationship with the first type of nodes; determining an abnormal access object of the target application according to the risk node set, wherein the abnormal access object refers to an object that performs abnormal access to the target application.
2. The method of claim 1, wherein, Determining a risk node set according to a distribution of the first type of node in the plurality of graphs comprises: performing clustering on the nodes in the plurality of graphs by using a target algorithm to obtain a plurality of node sets; determining whether the number proportion of the first type of node in the node set is greater than a first preset proportion; determining the node set in which the number proportion of the first type of node is greater than the first preset proportion as the risk node set.
3. The method of claim 2, wherein, After performing clustering on the nodes in the plurality of graphs by using a target algorithm to obtain a plurality of node sets, before determining whether the number proportion of the first type of node in the node set is greater than a first preset proportion, the method further comprises: determining nodes having a connection relationship with the nodes in the node set outside the node set to obtain at least one target node; adding the at least one target node to the node set to obtain an updated node set.
4. The method of claim 2, wherein, If the plurality of nodes in the graph include a second type of node, the second type of node represents an object whose probability of abnormal access to the target application is unknown, after performing clustering on the nodes in the plurality of graphs by using a target algorithm to obtain a plurality of node sets, before determining whether the number proportion of the first type of node in the node set is greater than a first preset proportion, the method further comprises: in the case that the second type of node exists in the plurality of graphs, calculating a probability of abnormal access to the target application of the second type of node; if the probability of abnormal access to the target application of the second type of node is greater than a preset probability, determining the second type of node as a first type of node.
5. The method of claim 4, wherein, After the probability of abnormal access to the target application of the second type of node is calculated in the case that the second type of node exists in the plurality of graphs, the method further comprises: composing an updated first type of node from the first type of node in the plurality of graphs and the second type of node whose probability of abnormal access to the target application is greater than the preset probability, and updating the knowledge graph data of the target application according to the updated first type of node; According to the updated knowledge graph data, a plurality of updated graphs are obtained, a set of updated risk nodes is determined according to the distribution of the first type of nodes in the plurality of updated graphs, and the abnormal access object of the target application is determined according to the nodes in the set of updated risk nodes.
6. The method of claim 1, wherein, Determining the set of risk nodes according to the distribution of the first type of nodes in the plurality of graphs comprises: Obtaining a first type of graph from the plurality of graphs, and determining the graphs other than the first type of graph in the plurality of graphs as test graphs, wherein the number of first type of nodes in the first type of graph accounts for more than a second preset proportion; Calculating the similarity between the test graph and the first type of graph; Determining the test graph with a similarity greater than a preset similarity as a first type of graph, and determining the set of risk nodes according to the nodes in the first type of graph.
7. The method of claim 6, wherein, If the plurality of nodes in the graph include a second type of node, the second type of node represents an object with unknown probability of abnormal access to the target application, and determining the set of risk nodes according to the nodes in the first type of graph comprises: In the case that the second type of node exists in the first type of graph, calculating the probability of abnormal access to the target application by the second type of node; Determining the set of risk nodes from the second type of node with a probability of abnormal access to the target application greater than a preset probability and the first type of node in the first type of graph.
8. The method of claim 1, wherein, Before obtaining the plurality of graphs according to the knowledge graph data of the target application, the method further comprises: Obtaining traffic data accessing the target application within a preset time period; Extracting entity data, relationship data and attribute data from the traffic data, wherein the entity data includes at least one of the following: data of an access object and / or data of an accessed object, the relationship data refers to the relationship between the access object and / or the accessed object, and the attribute data includes at least one of the following: attribute data of the access object, attribute data of the accessed object, and attribute data of the relationship between the access object and / or the accessed object; Generating the knowledge graph data of the target application according to the entity data, relationship data and attribute data.
9. A method of determining an abnormal access object, characterized by, Comprise: The cloud server receives the knowledge graph data of the target application, wherein the knowledge graph data is determined by the traffic data of the target application, and the traffic data of the target application is the traffic data accessing the target application; The cloud server obtains a plurality of graphs according to knowledge graph data of a target application, determines a risk node set according to a distribution of first-type nodes in the plurality of graphs, and determines an abnormal access object of the target application according to the risk node set, wherein the graph is composed of nodes and edges, the nodes represent data of an access object or data of an accessed object, the edges represent relationship data between the access object and / or the accessed object, a plurality of nodes in the graph at least include first-type nodes, the first-type nodes represent objects carrying an abnormal access label, a probability of the abnormal access is determined by the traffic data, the risk node is a node that has a risk to the running of the target application, and the risk node set includes a plurality of first-type nodes and nodes having a connection relationship with the first-type nodes; The cloud server feeds back the abnormal access object of the target application to a client, wherein the abnormal access object refers to an object that performs abnormal access on the target application.
10. An apparatus for determining an abnormal access object, the apparatus comprising: Comprise: A first determination unit configured to obtain a plurality of graphs according to knowledge graph data of a target application, wherein the knowledge graph data is determined by traffic data of the target application, the traffic data of the target application is traffic data of accessing the target application, the graph is composed of a plurality of nodes and edges, the nodes represent data of an access object or data of an accessed object, and the edges represent relationship data between the access object and / or the accessed object, a plurality of nodes in the graph at least include first-type nodes, the first-type nodes represent objects carrying an abnormal access label, and a probability of the abnormal access is determined by the traffic data; A second determination unit configured to determine a risk node set according to a distribution of first-type nodes in the plurality of graphs, wherein the risk node is a node that has a risk to the running of the target application, and the risk node set includes a plurality of first-type nodes and nodes having a connection relationship with the first-type nodes; A third determination unit configured to determine an abnormal access object of the target application according to the risk node set, wherein the abnormal access object refers to an object that performs abnormal access on the target application.
11. A computer readable storage medium characterized by, The computer readable storage medium comprises a stored program, wherein the program controls a device where the storage medium is located to execute the method of any one of claims 1 to 9 when the program is running.
12. An electronic device, comprising: Comprise: A memory storing an executable program; A processor configured to run the program, wherein the program executes the method of any one of claims 1 to 9 when the program is running.
Citation Information
Patent Citations
Method and device for determining abnormal network protocol address
CN113259402A
Transaction relation graph processing method and device, computer equipment and storage medium
CN114218397A
Flow detection method, device, equipment and medium
CN114422267A