A graph structure data anomaly assessment method for big data scenarios

Abstract

The present invention relates to the field of data processing technology, and in particular to a graph structure data anomaly assessment method for a big data scenario, the method comprising: acquiring graph structure data; determining a starting vertex in the graph structure data, and determining the shortest first distance between a target vertex and the starting vertex according to connection information; acquiring an associated vertex directly connected to the target vertex, and determining a candidate path for the target vertex to reach the starting vertex through the associated vertex, and determining a second distance of the candidate path according to the connection information; acquiring a density sequence of each candidate path, and determining a fitting coefficient of the candidate path according to the density sequence; determining a density change factor of the target vertex according to the first distance, the second distance and the fitting coefficient; determining a target neighborhood radius of the target vertex according to the density change factor of the target vertex, and performing anomaly assessment based on the target neighborhood radius to improve the accuracy and effect of the anomaly assessment.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a graph structure data anomaly assessment method for big data scenarios. Background Art

[0002] Currently, the commonly used method for anomaly assessment of graph structured data networks is usually to divide the data patterns in the graph structured data based on DBSCAN density clustering, and complete the detection of abnormal data based on the noise data points or abnormal data points in the clustering results.

[0003] When the amount of data is large, using traditional DBSCAN density clustering to analyze graph structure data is prone to ignore the different association patterns in the graph structure data in sparse areas, and judge the graph structure vertices in the sparse areas as abnormal data. Therefore, the same parameter settings will lead to large errors in the abnormal evaluation results. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a graph structure data anomaly assessment method for big data scenarios to solve the problem of low anomaly assessment accuracy in big data scenarios.

[0005] An embodiment of the present invention provides a graph structure data anomaly assessment method for a big data scenario, and the graph structure data anomaly assessment method for a big data scenario includes the following steps:

[0006] Acquire graph structure data, wherein the graph structure data includes vertices and connection information between vertices;

[0007] Determine a starting vertex in the graph structure data, and determine the shortest first distance between a target vertex and the starting vertex according to the connection information, wherein the target vertex is a vertex in the graph structure data other than the starting vertex;

[0008] Acquire an associated vertex directly connected to the target vertex, determine a candidate path for the target vertex to reach the start vertex through the associated vertex, and determine a second distance of the candidate path according to the connection information;

[0009] Acquire a density sequence of each candidate path, and determine a fitting coefficient of the candidate path according to the density sequence;

[0010] Determining a density change factor of the target vertex according to the first distance, the second distance and the fitting coefficient;

[0011] A target neighborhood radius of the target vertex is determined according to a density variation factor of the target vertex, and an anomaly assessment is performed based on the target neighborhood radius.

[0012] Preferably, determining the shortest first distance between the target vertex and the starting vertex according to the connection information includes:

[0013] Determine the length of the connecting edge between vertices in the graph structure data based on the connection information;

[0014] Determine a candidate connection path between the target vertex and the starting vertex, and determine a candidate distance of the candidate connection path according to the sum of the lengths of the connecting edges between every two vertices in the candidate connection path;

[0015] The minimum candidate distance among the candidate connection paths is selected as the shortest first distance between the target vertex and the starting vertex.

[0016] Preferably, determining the second distance of the candidate path according to the connection information includes:

[0017] Determine the length of the connecting edge between vertices in the graph structure data based on the connection information;

[0018] The sum of the lengths of the connecting edges between every two vertices in the candidate path is obtained as the second distance of the candidate path.

[0019] Preferably, the obtaining of the density sequence of each candidate path includes:

[0020] Obtaining a target neighborhood radius of each vertex in the candidate path;

[0021] Counting the number of vertices within the target neighborhood radius corresponding to each vertex as the neighborhood density corresponding to the vertex;

[0022] The density sequence is composed based on the neighborhood densities of all vertices in the candidate path.

[0023] Preferably, determining the fitting coefficient of the candidate path according to the density sequence includes:

[0024] A linear fit is performed according to the density sequence, and the fitting slope is obtained as the fitting coefficient.

[0025] Preferably, determining the density change factor of the target vertex according to the first distance, the second distance and the fitting coefficient includes:

[0026] The calculation formula of the density change factor is:

[0027]

[0028] in, Represents the target vertex Density variation factor; Indicates the starting vertex With the target vertex The shortest first distance between Represents candidate paths m The second distance; Represents candidate paths m ; Represents candidate paths m The fitting coefficient of Indicates the number of associated vertices; is the normalized exponential function.

[0029] Preferably, determining the target neighborhood radius of the target vertex according to the density change factor of the target vertex includes:

[0030] Any associated vertex of the target vertex , get the associated vertex The number of associated vertices of the target vertex and the number of associated vertices of the target vertex The number of associated vertices of , determine the first absolute difference;

[0031] Determine the first neighborhood distance mean, the first distance variance and the associated vertex of the target vertex according to the connection information of the graph structure data The second neighborhood distance mean and second distance variance;

[0032] Determine a second absolute difference according to the first neighborhood distance mean, the first distance variance, the second neighborhood distance mean, and the second distance variance;

[0033] Determine an adjustment index according to the number of associated vertices of the target vertex, the first absolute difference, and the second absolute difference;

[0034] A neighborhood radius optimization factor is determined based on the adjustment index and the density change factor, and a preset initial neighborhood radius is optimized according to the neighborhood radius optimization factor to obtain a target neighborhood radius of the target vertex.

[0035] Preferably, optimizing a preset initial neighborhood radius according to the neighborhood radius optimization factor to obtain a target neighborhood radius of the target vertex includes:

[0036] The product of the neighborhood radius optimization factor and the initial neighborhood radius is calculated as the target neighborhood radius.

[0037] Preferably, determining the neighborhood radius optimization factor based on the adjustment index and the density change factor includes:

[0038] The calculation formula of the neighborhood radius optimization factor is:

[0039] ;

[0040] in, Represents the target vertex Neighborhood radius optimization factor; Represents the target vertex Density variation factor; represents the adjustment index; Represents the target vertex The number of associated vertices; Represents the associated vertex The number of associated vertices; represents the first absolute difference; Represents the target vertex With associated vertex The distance between Represents the associated vertex Associated with this vertex Other associated vertices of The distance between represents the mean of the first neighborhood distance; represents the mean of the second neighborhood distance; represents the first distance variance; represents the second distance variance; Indicates the second absolute difference.

[0041] Preferably, performing anomaly assessment based on the target neighborhood radius includes:

[0042] Clustering is performed based on the target neighborhood radius of each vertex in the graph structure data to obtain multiple clusters;

[0043] Obtaining the maximum density reachable distance of each of the clusters, and determining the abnormality degree of the cluster according to the maximum density reachable distance;

[0044] In response to the abnormality level being greater than a preset abnormality level threshold, it is determined that the graph structure data in the cluster is abnormal.

[0045] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: determining graph structure data to be analyzed, determining the shortest first distance between a target vertex and a starting vertex based on connection information between vertices in the graph structure data, further determining a candidate path for the target vertex to reach the starting vertex through each associated vertex, and a second distance of each candidate path, performing analysis based on the shortest first distance and the second distance of the candidate path, comprehensively considering factors affecting the distance between vertices, further determining a density sequence of the candidate paths, and determining a fitting coefficient of the candidate paths based on the density sequence, using the fitting coefficient to reflect the density characteristics of the candidate paths, determining a density variation factor of the target vertex based on the second distance of the candidate paths and the shortest first distance, further acquiring a target neighborhood radius based on the density variation factor containing richer information, improving the accuracy of acquiring the target neighborhood radius, thereby performing anomaly assessment based on the accurate target neighborhood radius, reducing the error in anomaly assessment of graph structure data in big data scenarios, and improving the effect of anomaly assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 It is a method flow chart of a method for evaluating graph structure data anomaly for a big data scenario provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0048] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, but should not be understood as limiting the present disclosure.

[0049] It should be noted that the terms "first", "second", etc. in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0050] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.

[0051] See also Figure 1 , is a method flow chart of a method for evaluating graph structure data anomaly in a big data scenario provided by the first embodiment of the present invention, such as Figure 1 As shown, the graph structure data anomaly assessment method for big data scenarios may include:

[0052] S101, obtaining graph structure data, where the graph structure data includes vertices and connection information between vertices.

[0053] Each data object in large-scale data is taken as a vertex, and the vertices are connected through structural edges according to the actual relationship between the vertices to obtain a graph structure network; for example, for social network big data, each individual in the social network is taken as a vertex, and the vertices are connected through the social associations between individuals to obtain a social graph structure network, wherein each vertex attribute may include attribute information corresponding to the individual, such as interest tags and other information, and the structural edges may be used to characterize the social association relationship between individuals. For example, when the social association relationship is relatively close, the distance between individuals is relatively low and the structural edges are relatively short. Therefore, when constructing graph structure data, the length of each structural edge in the graph structure data may be determined and stored in the connection information of the graph structure data.

[0054] S102, determining a starting vertex in the graph structure data, and determining the shortest first distance between a target vertex and the starting vertex according to the connection information, wherein the target vertex is a vertex in the graph structure data other than the starting vertex.

[0055] Optionally, the starting vertex may be a randomly selected point, for example, the vertex with the most connections in the graph structure network may be used as the starting vertex, or the vertex at the most central position in the graph structure data may be used as the starting vertex, which will not be described in detail.

[0056] In some implementations, the length of the connecting edge between vertices in the graph structure data can be determined based on the connection information; that is, the length of the connecting edge between each vertex is determined from the connection information; the candidate connection path between the target vertex and the starting vertex is further determined, and the candidate distance of the candidate connection path is determined based on the sum of the lengths of the connecting edges between every two vertices in the candidate connection path; it can be understood that there may be multiple candidate connection paths between the target vertex and the starting vertex, and after determining the length of the connecting edge between every two vertices, the sum of the lengths of the connecting edges between all vertices in each candidate connection path is used as the candidate distance of the candidate connection path.

[0057] Furthermore, the minimum candidate distance in the candidate connection path is selected as the shortest first distance between the target vertex and the starting vertex; that is, the distance of the minimum connection path between the target vertex and the starting vertex is selected as the first distance, so as to better reflect the distribution of vertices in the graph structure data by the shortest path.

[0058] S103, acquiring an associated vertex directly connected to the target vertex, and determining a candidate path from the target vertex to the start vertex via the associated vertex, and determining a second distance of the candidate path according to the connection information.

[0059] In this embodiment, the associated vertex is a vertex that has a direct connection edge with the target vertex; in some implementations, each associated vertex can be used as an extension direction, and each extension direction has one or more candidate paths, that is, there may be one or more candidate paths for the target vertex to reach the starting vertex position through each associated vertex.

[0060] Based on the connection information of the graph structure data, the length of the connecting edge between every two vertices in the graph structure data can be determined, that is, the length of the connecting edge between each vertices can be determined from the connection information; the sum of the lengths of the connecting edges between every two vertices in the candidate path is obtained as the second distance of the candidate path, that is, after determining the length of the connecting edge between every two vertices, the sum of the lengths of the connecting edges between all vertices in each candidate path is taken as the second distance of the candidate path.

[0061] S104, obtaining a density sequence of each candidate path, and determining a fitting coefficient of the candidate path according to the density sequence.

[0062] Optionally, the target neighborhood radius of each vertex in the candidate path can be obtained; the number of vertices within the target neighborhood radius corresponding to each vertex is counted as the neighborhood density corresponding to the vertex; and the density sequence is composed based on the neighborhood densities of all vertices in the candidate path.

[0063] It can be understood that, in this embodiment, the target path is the path from the starting vertex to the target vertex. In this embodiment, the analysis order of the vertices is set to analyze one by one starting from the starting vertex. Therefore, the target neighborhood radius of the vertices before the target vertex is a calculated known value, and the target neighborhood radius of the starting vertex is a preset initial neighborhood radius. According to the target neighborhood radius of each vertex, the number of vertices within the target neighborhood radius is determined as the neighborhood density corresponding to the vertex, thereby forming a density sequence according to the neighborhood densities of all vertices on the candidate path.

[0064] Furthermore, linear fitting can be performed based on the density sequence to obtain a fitting slope as a fitting coefficient. For example, linear fitting can be performed based on the least squares method to obtain a linear fitting function and a fitting slope of the linear fitting function. The fitting slope is used as a fitting coefficient to reflect the density change trend of the candidate path. When the fitting coefficient of the candidate path shows an increase in density, it is necessary to increase the density change factor to increase the neighborhood radius of the target vertex. When the fitting coefficient of the candidate path shows a decrease in density, it is necessary to reduce the density change factor to reduce the neighborhood radius of the target vertex.

[0065] S105, determining a density change factor of the target vertex according to the first distance, the second distance and the fitting coefficient.

[0066] It can be understood that the process of density clustering is to perform a density-based extension process through randomly selected starting points to expand the clusters. For example, for the selected starting points When there are other data points that meet the neighborhood density threshold in the initial neighborhood radius of the starting point, it will be marked as a cluster, and the neighborhood density threshold in the neighborhood radius will be judged from the data points in its neighborhood radius, so as to start analyzing one by one to obtain all cluster clusters; in some embodiments, the neighborhood radius can be adjusted according to the topological relationship corresponding to the vertex, but if the judgment is only made through the topological relationship corresponding to each vertex, then in a process of density decreasing in a certain direction, if the topological relationship corresponding to the vertex causes the neighborhood radius to increase, the problem of density truncation cannot be performed, and the optimization effect of the neighborhood radius is not good.

[0067] This embodiment determines the density change factor of the target vertex based on the first distance, the second distance and the fitting coefficient. The calculation formula of the density change factor is:

[0068]

[0069] in, Represents the target vertex Density variation factor; Indicates the starting vertex With the target vertex The shortest first distance between Represents candidate paths m The second distance; Represents candidate paths m ; Represents candidate paths m The fitting coefficient of Indicates the number of associated vertices; is the normalized exponential function.

[0070] It can be understood that there are multiple associated vertices and multiple candidate paths for the target vertex. Each candidate path is analyzed to determine the density change factor of the target vertex. In this embodiment, the difference between the second distance of each candidate path and the first distance of the shortest path is determined, and the normalized difference is used as the evaluation weight of the corresponding candidate path, that is, As a candidate path m The evaluation weight of the target vertex in each extension direction, that is, the evaluation weight of different candidate paths, is used to weight the density change trend of the candidate path; the density change trend of each candidate path is calculated by the fitting coefficient of the candidate path It means that the greater the density change trend is, the greater the impact on the neighborhood radius of the target vertex is. The difference between the second distance of each candidate path and the nearest first distance is used as the evaluation weight of the density change trend analysis to improve the accuracy of obtaining the density change factor.

[0071] S106, determining a target neighborhood radius of the target vertex according to the density change factor of the target vertex, and performing anomaly assessment based on the target neighborhood radius.

[0072] It is understandable that when using the traditional DBSCAN clustering algorithm to perform density clustering on graph structure data, the neighborhood radius is the maximum distance from a vertex to other vertices that can be reached through the structural edges, such as the neighborhood radius represents the direct neighbors of the vertex, It represents the second-level neighbors that can be reached through another vertex. For structural edges with weights, the structural edges are weighted by the weights and the number of connectable edges is obtained according to the neighborhood radius value, such as the neighborhood radius This means that the weights of the structural edges are accumulated until the sum of the weights is greater than All structural edge-reachable neighbors of .

[0073] However, in the big data scenario of the present embodiment, only using a single neighborhood radius for the density clustering process will result in the clustering process being unable to adapt to the different association patterns presented in the graph structure. That is, when the dense areas and sparse areas are measured using the same neighborhood radius, there will always be an area where evaluation anomalies will occur, resulting in inaccurate clustering. For example, using a smaller neighborhood radius for the density clustering process will result in vertices with fixed patterns in the sparse areas being evaluated as abnormal data points. If a higher neighborhood radius is used for the density clustering process, the clustering accuracy will be reduced during the density clustering process, thereby classifying vertices that do not pass the pattern into the same cluster, resulting in poor anomaly recognition.

[0074] After determining the density change factor of the target vertex, this embodiment can dynamically adjust and optimize the neighborhood radius of the vertex according to the density change factor, so that during the DBSCAN clustering process, density clustering can be performed with a dynamic target neighborhood radius for vertices of different topological patterns; since vertices are associated through structural edges, for the target vertex, the neighborhood radius optimization factor can be evaluated based on the topological similarity between the vertices associated with the target vertex.

[0075] In some implementations, for any associated vertex of the target vertex , get the associated vertex The number of associated vertices of the target vertex and the number of associated vertices of the target vertex The number of associated vertices is determined to determine the first absolute difference; it can be understood that the associated vertices The number of associated vertices is also the number of associated vertices The number of directly connected vertices, the first absolute difference is the number of associated vertices of the target vertex and the number of associated vertices The absolute value of the difference in the number of incident vertices.

[0076] Determine the first neighborhood distance mean, first distance variance and associated vertex of the target vertex based on the connection information of the graph structure data The second neighborhood distance mean and the second distance variance of the target vertex are respectively the average value of the distance between the associated vertex corresponding to the target vertex and the target vertex, and the first distance variance refers to the variance of the distance between the associated vertex corresponding to the target vertex and the target vertex. Similarly, the associated vertex The mean of the second neighbor distances refers to the associated vertex The corresponding associated vertices and associated vertices The second distance variance refers to the average value of the distance between the associated vertices. The corresponding associated vertices and associated vertices The variance of the distance between .

[0077] Furthermore, the second absolute difference is determined according to the first neighborhood distance mean, the first distance variance, the second neighborhood distance mean and the second distance variance; the adjustment index is determined according to the number of associated vertices of the target vertex, the first absolute difference and the second absolute difference; the neighborhood radius optimization factor is determined based on the adjustment index and the density change factor, and the preset initial neighborhood radius is optimized according to the neighborhood radius optimization factor to obtain the target neighborhood radius of the target vertex.

[0078] The calculation formula of the neighborhood radius optimization factor is:

[0079]

[0080] in, Represents the target vertex Neighborhood radius optimization factor; Represents the target vertex Density variation factor; represents the adjustment index; Represents the target vertex The number of associated vertices; Represents the associated vertex The number of associated vertices; represents the first absolute difference; Represents the target vertex With associated vertex The distance between Represents the associated vertex Associated with this vertex Other associated vertices of The distance between represents the mean of the first neighborhood distance; represents the mean of the second neighborhood distance; represents the first distance variance; represents the second distance variance; Indicates the second absolute difference.

[0081] For the target vertex and its associated vertices , between the target vertex and its associated vertex When the corresponding number of associated nodes and connection distances are similar, it indicates that the target vertex and its associated vertex The similarity is high, so the target vertex and its associated vertex can be determined based on the first absolute difference and the second absolute difference. The smaller the first absolute difference and the second absolute difference, the smaller the similarity between the target vertex and its associated vertex. The more similar, and the second absolute difference is obtained based on the distance mean and variance, to improve the accuracy of the index calculation, the adjustment coefficient is determined by the similarity between the target vertex and all its associated vertices. When the similarity between the target vertex and its associated vertices is higher, the adjustment coefficient is larger, so the degree of adjustment of the density change factor is greater and the accuracy is higher.

[0082] In this embodiment, the product of the neighborhood radius optimization factor and the initial neighborhood radius is calculated as the target neighborhood radius, that is, the target vertex The target neighborhood radius is ,in, is the preset initial neighborhood radius; based on the above method for obtaining the target neighborhood radius, the target neighborhood radius corresponding to each vertex in the graph structure data is determined.

[0083] Furthermore, clustering can be performed based on the target neighborhood radius of each vertex in the graph structure data to obtain multiple clusters; during the clustering process, density judgment is performed based on the target neighborhood radius of each vertex, thereby completing the density clustering process and obtaining the density clustering result in the big data scenario, that is, obtaining multiple clusters after clustering.

[0084] Optionally, the maximum density reachable distance of each cluster cluster, that is, the maximum distance in each cluster cluster, can be obtained, and the abnormality degree of the cluster cluster can be determined based on the maximum density reachable distance. For example, after obtaining the maximum density reachable distance of all cluster clusters, the maximum density reachable distance of all cluster clusters are normalized, and the normalized result is used as the abnormality degree of the corresponding cluster cluster; in response to the abnormality degree being greater than a preset abnormality degree threshold, the graph structure data in the cluster cluster is determined to be abnormal, that is, when the abnormality degree is greater than the abnormality degree threshold, the cluster cluster is considered to be an abnormal cluster cluster, and the vertices in the abnormal cluster cluster are used as abnormal data points, where the abnormality degree threshold can be 0.3, and can be other values ​​in other embodiments, which will not be described in detail.

[0085] In summary, this embodiment obtains graph structure data, determines the starting vertex in the graph structure data, determines the first distance of the shortest path between the target vertex and the starting vertex according to the connection information, obtains the associated vertex directly connected to the target vertex, and determines the candidate path from the target vertex to the starting vertex through the associated vertex, determines the second distance of the candidate path according to the connection information, takes the minimum first distance as the comparison basis, calculates the difference between the second distance of each candidate path and the first distance as the evaluation weight of the candidate path, further obtains the density sequence of each candidate path, and determines the fitting coefficient of the candidate path according to the density sequence, adjusts the fitting coefficient according to the evaluation weight obtained by the first distance and the second distance, thereby determining the density change factor of the target vertex and improving the accuracy of obtaining the density change factor; further determines the similarity according to the number of associated vertices and the connection distance between the target vertex and its associated vertices, and then determines the adjustment coefficient according to the similarity to optimize and adjust the density change factor, determines the target neighborhood radius of the target vertex, and obtains a more accurate and reasonable target neighborhood radius of the vertex, and uses the accurate target neighborhood radius to perform density clustering and anomaly evaluation, improves the accuracy and rationality of anomaly evaluation, avoids the problem of large errors when clustering with a single neighborhood radius, and has a better anomaly evaluation effect.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A graph structure data anomaly assessment method for big data scenarios, characterized in that: The graph structure data anomaly assessment method for social network big data scenarios includes: Acquire graph structure data, wherein the graph structure data includes vertices and connection information between vertices; the vertices are each individual in the social network, each vertex attribute includes interest tag information corresponding to the individual, and the connection information between vertices is the social association relationship between individuals; Determine a starting vertex in the graph structure data, and determine the shortest first distance between a target vertex and the starting vertex according to the connection information, wherein the target vertex is a vertex in the graph structure data other than the starting vertex; Acquire an associated vertex directly connected to the target vertex, determine a candidate path for the target vertex to reach the start vertex through the associated vertex, and determine a second distance of the candidate path according to the connection information; Acquire a density sequence of each candidate path, and determine a fitting coefficient of the candidate path according to the density sequence; Determining a density change factor of the target vertex according to the first distance, the second distance and the fitting coefficient; A target neighborhood radius of the target vertex is determined according to a density variation factor of the target vertex, and an anomaly assessment is performed based on the target neighborhood radius.

2. The graph structure data anomaly assessment method for big data scenarios according to claim 1 is characterized in that: The determining the shortest first distance between the target vertex and the starting vertex according to the connection information includes: Determine the length of the connecting edge between vertices in the graph structure data based on the connection information; Determine a candidate connection path between the target vertex and the starting vertex, and determine a candidate distance of the candidate connection path according to the sum of the lengths of the connecting edges between every two vertices in the candidate connection path; The minimum candidate distance among the candidate connection paths is selected as the shortest first distance between the target vertex and the starting vertex.

3. The graph structure data anomaly assessment method for big data scenarios according to claim 1 is characterized in that: The determining the second distance of the candidate path according to the connection information includes: Determine the length of the connecting edge between vertices in the graph structure data based on the connection information; The sum of the lengths of the connecting edges between every two vertices in the candidate path is obtained as the second distance of the candidate path.

4. The graph structure data anomaly assessment method for big data scenarios according to claim 1, characterized in that: The obtaining of the density sequence of each candidate path includes: Obtaining a target neighborhood radius of each vertex in the candidate path; Counting the number of vertices within the target neighborhood radius corresponding to each vertex as the neighborhood density corresponding to the vertex; The density sequence is composed based on the neighborhood densities of all vertices in the candidate path.

5. The graph structure data anomaly assessment method for big data scenarios according to claim 4 is characterized in that: The step of determining the fitting coefficient of the candidate path according to the density sequence includes: A linear fit is performed according to the density sequence, and the fitting slope is obtained as the fitting coefficient.

6. The graph structure data anomaly assessment method for big data scenarios according to any one of claims 1 to 4, characterized in that: The step of determining the density change factor of the target vertex according to the first distance, the second distance and the fitting coefficient includes: The calculation formula of the density change factor is: ； in, Represents the target vertex Density variation factor; Indicates the starting vertex With the target vertex The shortest first distance between Represents candidate paths m The second distance; Represents candidate paths m ; Represents candidate paths m The fitting coefficient of Indicates the number of associated vertices; is the normalized exponential function.

7. The graph structure data anomaly assessment method for big data scenarios according to claim 6, characterized in that: The step of determining a target neighborhood radius of the target vertex according to the density change factor of the target vertex includes: Any associated vertex of the target vertex , get the associated vertex The number of associated vertices of the target vertex and the number of associated vertices of the target vertex The number of associated vertices of , determine the first absolute difference; Determine the first neighborhood distance mean, the first distance variance and the associated vertex of the target vertex according to the connection information of the graph structure data The second neighborhood distance mean and second distance variance; Determine a second absolute difference according to the first neighborhood distance mean, the first distance variance, the second neighborhood distance mean, and the second distance variance; Determine an adjustment index according to the number of associated vertices of the target vertex, the first absolute difference, and the second absolute difference; A neighborhood radius optimization factor is determined based on the adjustment index and the density change factor, and a preset initial neighborhood radius is optimized according to the neighborhood radius optimization factor to obtain a target neighborhood radius of the target vertex.

8. The graph structure data anomaly assessment method for big data scenarios according to claim 7, characterized in that: The step of optimizing a preset initial neighborhood radius according to the neighborhood radius optimization factor to obtain a target neighborhood radius of the target vertex includes: The product of the neighborhood radius optimization factor and the initial neighborhood radius is calculated as the target neighborhood radius.

9. The graph structure data anomaly assessment method for big data scenarios according to claim 8, characterized in that: The determining of the neighborhood radius optimization factor based on the adjustment index and the density change factor includes: The calculation formula of the neighborhood radius optimization factor is: ； in, Represents the target vertex Neighborhood radius optimization factor; Represents the target vertex Density variation factor; represents the adjustment index; Represents the target vertex The number of associated vertices; Represents the associated vertex The number of associated vertices; represents the first absolute difference; Represents the target vertex With associated vertex The distance between Represents the associated vertex Associated with this vertex Other associated vertices of The distance between represents the mean of the first neighborhood distance; represents the mean of the second neighborhood distance; represents the first distance variance; represents the second distance variance; Indicates the second absolute difference.

10. The graph structure data anomaly assessment method for big data scenarios according to claim 9, characterized in that: The performing anomaly assessment based on the target neighborhood radius includes: Clustering is performed based on the target neighborhood radius of each vertex in the graph structure data to obtain multiple clusters; Obtaining the maximum density reachable distance of each of the clusters, and determining the abnormality degree of the cluster according to the maximum density reachable distance; In response to the abnormality level being greater than a preset abnormality level threshold, it is determined that the graph structure data in the cluster is abnormal.

Images