Distributed Parallel PageRank Algorithm for Large Scale Graphs

Abstract

The present invention provides a distributed parallel PageRank algorithm for large-scale graphs. The algorithm evenly distributes the calculation of vertex PageRank values ​​of target graph data to a number of computing threads, and controls the number of computing threads through independent control threads to achieve parallel computing. In order to prevent data competition in a multi-threaded environment, the algorithm adds a vertex mutex lock to ensure that the cumulative information PR calculation of different vertex objects does not interfere with each other, thereby ensuring the accuracy of the vertex PageRank value. The number of computing threads is set specifically depending on the scale of the target graph data and the distributed hardware environment conditions. Users can set the number of computing threads according to actual needs to achieve rapid calculation of the PageRank values ​​of vertices of large-scale graph data. The present invention can be widely applied to various Internet importance ranking problems and fully supports application in distributed hardware environments.

Description

Technical Field

[0001] The present invention relates to the field of Internet importance ranking, and in particular to a distributed parallel PageRank algorithm for large-scale graphs. Background Art

[0002] The PageRank algorithm originated in the internet world, initially used to solve the web page ranking problem of the Google search engine. When a search engine returns a large number of relevant web pages based on a search term, users naturally want to rank "important" pages higher. The first problem faced in ranking web pages is how to define their importance. The entire network is abstracted into a directed graph with web pages as vertices and hyperlinks as edges. The PageRank value is a measure of the importance of the vertex, and the ranking of web pages can be obtained by calculating the PageRank value of the web page (vertex). Graphs are a very broad abstraction, and PageRank calculations are only related to the graph's topological structure. Today, the theory of PageRank calculations has been continuously improved, and its application has expanded to chemistry, biology, sports, and the humanities and social sciences.

[0003] The traditional power method is often used to calculate the PageRank value. The power method is to iterate Calculating PageRank values ​​is simple and stable, but it cannot eliminate dependencies between iterations and is not fully parallelizable. With the development of the internet and the expansion of its applications, graphs are becoming increasingly large, and power methods often face performance bottlenecks when working with large-scale graphs.

[0004] The prior art discloses a patent for a method and device for optimizing the communication process of a distributed PageRank algorithm, the method comprising: dividing the vertices and edges in the entire graph to corresponding computing nodes based on partition calculation; wherein the vertex that has completed the division is called the main point of the corresponding computing node, and one vertex of the edge that has completed the division belongs to the same computing node as the edge. If the other vertex and the edge belong to different computing nodes, a virtual point corresponding to the other vertex is created in the same computing node; traversing the main points in each computing node and calculating the PageRank value of the main point; sending a preset proportion of the PageRank value of the main point to the main point or virtual point in other computing nodes that have a corresponding relationship with the main point through a round of network communication; iterating the calculation and network communication until the preset conditions for stopping the iteration are met; the patent optimizes the storage consumption and communication times of the existing algorithm, shortens the time required for the overall calculation, and improves the algorithm operation efficiency, but the patent method realizes distributed parallel computing through main points and virtual points, and has the problems of relatively high requirements for memory allocation and large amount of calculation. Summary of the Invention

[0005] The present invention provides a distributed parallel PageRank algorithm for large-scale graphs, which solves the problem of quickly performing distributed parallel calculation of PageRank values ​​of vertices in large-scale graph data.

[0006] In order to achieve the above technical effects, the technical solutions of the present invention are as follows:

[0007] The distributed parallel PageRank algorithm for large-scale graphs includes the following steps:

[0008] S1: Determine the parallelism of computing threads based on the target graph data scale and distributed hardware environment conditions , and initialize the target graph data to obtain the initialized variables and initialized list;

[0009] S2: The parallelism of the calculation threads obtained in step S1 , started The computing thread is then started, and the control thread is used to The calculation thread is controlled, and the vertex cumulative information of all vertex objects in the target graph data is calculated by combining the initialization variables and initialization list obtained in step S1;

[0010] S3: Calculate the PageRank values ​​of all vertex objects based on the vertex cumulative information of all vertex objects obtained in step S2, and output a PageRank vector containing the PageRank values ​​of all vertex objects.

[0011] The algorithm of the present invention distributes the calculation of the PageRank values ​​of the vertices of the target graph data evenly to a number of computing threads, and controls the number of computing threads through independent control threads to achieve parallel computing. The number of computing threads is set depending on the scale of the target graph data and the distributed hardware environment conditions. The user can set the number of computing threads according to actual needs to achieve rapid calculation of the PageRank values ​​of vertices of large-scale graph data. The present invention can be widely applied to various Internet importance ranking problems and fully supports application in distributed hardware environments.

[0012] Furthermore, in step S1, the vertex objects in the target graph data ,in Represents vertex object encoding, vertex object There are two states: one is the current source vertex object ,in , and another as the target vertex object ,in ; All vertex objects All through directed edges Connect, when a vertex object As the current source vertex object When it passes the directed edge All other vertex objects pointed to Both are called target vertex objects , its corresponding relationship combined with the directed edge can be recorded as , that is, the current source vertex object Points to the target vertex object ; When the vertex object As the current source vertex object When it points to the target vertex object The total number of vertex objects The out-degree of the vertex object As the target vertex object When it is pointed to, the current source vertex object The total number of vertex objects The in-degree.

[0013] The target graph data can correspond to a variety of practical application scenarios, such as web pages, social networks, and academic literature citation networks. For web pages, the vertex objects of the algorithm of the present invention correspond to web pages, and the directed edges correspond to the link-pointing and linked-to relationships of web pages. When a web page is the current source web page, all other web pages it links to are called target web pages. The total number of target web pages it links to is the out-degree of the web page, and the total number of current source web pages it links to is the in-degree of the web page. For social networks, the vertex objects of the algorithm of the present invention correspond to social users, and the directed edges correspond to the user's follow-up and followed relationships. When a user is the current source user, all other users it follows are called target users, the total number of target users it follows is the user's out-degree, and the total number of current source users it follows is the user's in-degree; for the academic literature citation network, the vertex objects of the algorithm of the present invention correspond to academic literature, and the directed edges correspond to the citation and citation relationships of academic literature. When a certain academic literature is the current source academic literature, all other academic literature it cites are called target academic literature, the total number of target academic literature it cites is the out-degree of the academic literature, and the total number of current source academic literature it cites is the in-degree of the academic literature;

[0014] Furthermore, in step S1, the initialization process includes the following steps:

[0015] S11: Determine the required initialization variables and initialization list, where the initialization variables include the damping coefficient , the lower limit of vertex information , vertex object related variables, computing thread related variables and control object related variables, where vertex object related variables include vertex cumulative information , the amount of vertex information to be processed and ; Calculation thread related variables include calculation thread code ; The control object related variables include atomic Boolean thread state identifiers and atomic Boolean thread operation status flags ;

[0016] S12: Create initialized variables and initializer lists.

[0017] Since the calculation of the PageRank value of the vertex object will generate information value transmission, in order to facilitate the information value transmission of the calculation process, the vertex cumulative information amount is set and the amount of vertex information to be processed ; To prevent data competition in a multi-threaded environment, a vertex mutex lock is set ; In order to ensure the convergence of the final calculation results, the damping coefficient is set ; In order to determine whether all vertex objects have been processed, a lower limit on the amount of vertex information is set ; In order to facilitate the control of the computing thread by the control thread, the thread status flag is set and thread operation status flags .

[0018] Furthermore, the initialization list in step S11 includes a vertex object list , Vertex object memory address list List of directed edge weights , and computation threads Vertex Object Mapping Table .

[0019] Vertex Object List The purpose is to convert the vertex objects of the target data set, which originally only have number information, into vertex objects corresponding to numbers and memory addresses; the vertex object memory address list Is the vertex object list The degree of exit is not The vertex objects are collected and the memory addresses of the vertex objects are stored; the directed edge weight list When each vertex object is the current source vertex, it corresponds to all target vertex objects and the weight of the corresponding edges; calculation thread Vertex Object Mapping Table It is a list that stores each thread and its corresponding set of vertex objects.

[0020] Furthermore, in step S12, the creation of initialization variables and initialization lists specifically includes the creation of damping coefficients , determined according to the specific application scenario; create a lower limit for vertex information , according to the required result accuracy; create vertex cumulative information , the initial value is ; Create vertex information to be processed , the initial value is ;create , each vertex object There is only one corresponding ; Create calculation thread code , each computing thread has a unique corresponding computing thread code ; Create thread status identifier , initially in false state; create thread operation state identifier , initially in false state; create a list of vertex objects , where each element contains information encoded by the vertex and vertex objects Composition; create a list of vertex object memory addresses , where each element corresponds to a vertex object The corresponding memory address , element information is provided by the vertex object list Get; create a list of directed edge weights , where each element contains information about the target vertex object and directed edge weights , where the directed edge weights By vertex object The out-degree is determined by taking the vertex object One out-degree of Vertex Object Mapping Table , where it is assumed that the vertex object memory address list The number of elements of , list the vertex object memory addresses All elements are evenly distributed computing threads, each computing thread corresponds to Vertex objects.

[0021] Damping coefficient It is used to specify the ratio of vertex information sent, which affects the convergence rate of the algorithm and needs to be determined according to the specific application scenario. ; Lower limit of vertex information It is used to specify the end condition of the algorithm, which affects the accuracy of the result. ; In addition, create initialization variables and initialization lists for subsequent control threads Prepare for control of a computing thread.

[0022] Furthermore, in step S2, the startup The computation thread consists of Thread state identifier of each computing thread Enable is true, The computing threads are all in the runnable state. Controlling a computing thread includes the following steps:

[0023] S21: When Thread state identifier of each computing thread When all are in true state, the control thread marks the thread operation status of all computing threads Enable is true, at which point all computing threads begin to call the core computing function to calculate the vertex cumulative information of all vertex objects. ;

[0024] S22: The control thread loops through the vertex object memory address list , check the amount of vertex information to be processed for all vertex objects , if both are less than the lower limit of vertex information , then the vertex cumulative information of all vertex objects is After the calculation is completed, the control thread ends the loop and marks the thread status of all calculation threads Set to false to shut down all computation threads.

[0025] The control thread can independently control the start and stop of each computing thread by simply using the thread status identifier of the corresponding thread. By enabling, you can control the start and stop of the calculation thread; you can also control whether each calculation thread calls the core calculation function to perform calculations separately, just by using the thread calculation status flag of the corresponding thread Enable to control whether the calculation thread calls the core calculation function. The control thread can independently process the vertex information of all vertex objects during the calculation process. Monitor and once the vertex information to be processed is met Both are less than the lower limit of vertex information , you can close all computing threads.

[0026] Furthermore, in step S21, the core calculation function also includes temporary storage variables , the core calculation function needs to traverse the mapping table corresponding to the calculation thread , that is, calculate all vertex objects corresponding to the calculation thread The amount of vertex cumulative information ; Traverse each vertex object First determine the vertex object The amount of vertex information to be processed Is it less than or equal to the lower limit of vertex information? , if it is less than or equal to the lower limit of vertex information , then the vertex object No calculation is performed and the next vertex object is determined. ; If it is greater than the lower limit of vertex information , then calculate the vertex object The amount of vertex cumulative information ; The entire traversal process is completed until all vertex objects are The amount of vertex cumulative information calculation till now.

[0027] Each computing thread calls the core computing function to calculate the vertex cumulative information of all vertex objects corresponding to the computing thread. , the amount of information to be processed at the vertex It is related to the out-degree of the vertex object and is used to transmit the connection between different vertex objects to determine the vertex cumulative information of the vertex object. The final result is used to calculate the PageRank value of the vertex object.

[0028] Furthermore, the calculation of the vertex object The amount of vertex cumulative information The following steps are involved:

[0029] Ⅰ: Vertex Object As the current source vertex object , get the current source vertex object of , the current source vertex object The amount of vertex information to be processed Increase the amount of accumulated information at its vertex And make temporary storage variables , clear the vertex pending information , release the current source vertex object of ;

[0030] Ⅱ: Traverse the current source vertex object Corresponding list , for the list Each target vertex object in Do the same process: Get the target vertex object of , the damping coefficient The result increases to the amount of information to be processed at the vertex , release the target vertex object of .

[0031] Vertex Object The amount of vertex cumulative information The calculation process is done by taking the current source vertex object As the center, the amount of information to be processed To the target vertex object with which it is connected Pass to update the accumulated information of the vertex object ; Added vertex mutex lock during calculation , ensuring that different vertices will not be interfered with when updating their own vertex cumulative information volume PR,

[0032] Furthermore, in step S3, the formula for calculating the PageRank values ​​of all vertex objects is: Assuming that the vertex objects The corresponding PageRank value is ,

[0033] is the total number of non-sunken vertices in the target graph data.

[0034] Vertex Object The corresponding PageRank value is the amount of information accumulated by the vertex The amount of information accumulated at the vertices of all vertex objects The percentage of the total is determined and is also a reference value for measuring the importance of vertex objects.

[0035] Furthermore, in step S3, the output includes the PageRank vector of all vertex objects PageRank values: , Corresponding vertex object PageRank value.

[0036] According to the output PageRank vector , according to the needs, you can sort the vertex objects from high to low in terms of PageRank value, where the larger the PageRank value, the higher the importance, and then filter the vertices of different importance according to user needs.

[0037] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0038] The present invention provides a distributed parallel PageRank algorithm for large-scale graphs. The algorithm evenly distributes the PageRank value calculation of the vertices of the target graph data to a number of computing threads. The independent control threads control the number of computing threads to achieve parallel computing. In order to prevent data competition in a multi-threaded environment, the algorithm adds a vertex mutex lock to ensure the cumulative information of different vertex objects. The calculations will not interfere with each other, ensuring the accuracy of the vertex PageRank value; the number of calculation threads is set depending on the target graph data scale and the distributed hardware environment conditions. Users can set the number of calculation threads according to actual needs to achieve rapid calculation of the PageRank values ​​of vertices of large-scale graph data. The present invention can be widely used in various Internet importance ranking problems and fully supports application in distributed hardware environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of the distributed parallel PageRank algorithm for large-scale graphs disclosed in the present invention;

[0040] Figure 2 Schematic diagram of the control thread controlling the computing thread in the algorithm of the present invention;

[0041] Figure 3 This is a schematic diagram of allocating vertex objects in the core calculation function of the algorithm of the present invention to each calculation thread. DETAILED DESCRIPTION

[0042] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0043] It is understandable to those skilled in the art that some well-known descriptions may be omitted in the drawings.

[0044] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0045] Example 1: The target graph data used in this example is a web page network. Vertex objects correspond to web pages, and directed edges correspond to the link-to-link relationship of web pages. When a web page is used as the current source web page, all other web pages it links to are called target web pages. The total number of target web pages it links to is the out-degree of the web page, and the total number of current source web pages it links to is the in-degree of the web page. The parameters of the web page network are shown in Table 1 below.

[0046] Table 1 Web page network parameters

[0047]

[0048] Among them, the number of sunken vertices is 0, so the number of non-sunken vertices is .

[0049] This embodiment uses a distributed parallel PageRank algorithm for large-scale graphs, such as Figure 1 As shown in the figure, select the top ten web pages in terms of importance. The specific steps are as follows:

[0050] S1: Determine the parallelism of the computing threads. , that is, need Computing threads are created and the target graph data is initialized to obtain the initialized variables and initialized lists.

[0051] S11: Determine the required initialization variables and initialization list, where the initialization variables include the damping coefficient , the lower limit of vertex information , vertex object related variables, computing thread related variables and control object related variables, where vertex object related variables include vertex cumulative information , the amount of vertex information to be processed and ; Calculation thread related variables include calculation thread code , damping coefficient , the lower limit of vertex information ; The control object related variables include atomic Boolean thread state identifiers and atomic Boolean thread operation status flags ; The initialization list includes the vertex object list , Vertex object memory address list List of directed edge weights , and computation threads Vertex Object Mapping Table .

[0052] S12: Create initialization variables and initialization lists, including creating damping coefficients ,Pick ; Create a lower limit on vertex information ,Pick ; Create vertex cumulative information , the initial value is ; Create vertex information to be processed , the initial value is ;create , each vertex object There is only one corresponding ; Create calculation thread code , each computing thread has a unique corresponding computing thread code ; Create thread status identifier , initially in false state; create thread operation state identifier , initially in false state; create a list of vertex objects , where each element contains information encoded by the vertex and vertex objects Composition; create a list of vertex object memory addresses , where each element corresponds to a vertex object The corresponding memory address , element information is provided by the vertex object list Get; create a list of directed edge weights , where each element contains information about the target vertex object and directed edge weights , where the directed edge weights By vertex object The out-degree is determined by taking the vertex object One out-degree of Vertex Object Mapping Table , where it is assumed that the vertex object memory address list The number of elements of , list the vertex object memory addresses All elements are evenly distributed computing threads, each computing thread corresponds to Vertex objects.

[0053] S2: Thread state identifier of each computing thread Enable is true, All computing threads are in the runnable state.

[0054] S21: When Thread state identifier of each computing thread When all are in true state, the control thread marks the thread operation status of all computing threads Enable is true, at which point all computing threads begin to call the core computing function to calculate the vertex cumulative information of all vertex objects. , each computing thread traverses the mapping table corresponding to the computing thread , traverse each vertex object First determine the vertex object The amount of vertex information to be processed Is it less than or equal to the lower limit of vertex information? , if it is less than or equal to the lower limit of vertex information , then the vertex object No calculation is performed and the next vertex object is determined. ; If it is greater than the lower limit of vertex information , then calculate the vertex object The amount of vertex cumulative information , where the calculation process steps are as follows:

[0055] Ⅰ: Vertex Object As the current source vertex object : Get the current source vertex object of , the current source vertex object The amount of vertex information to be processed Increase the amount of accumulated information at its vertex And make temporary storage variables , clear the vertex pending information , release the current source vertex object of ;

[0056] Ⅱ: Traverse the current source vertex object Corresponding list , for the list Each target vertex object in Do the same process: Get the target vertex object of , the damping coefficient The result increases to the amount of information to be processed at the vertex , release the target vertex object of .

[0057] The entire traversal process is completed until all vertex objects are The amount of vertex cumulative information calculation till now.

[0058] S22: The control thread loops through the vertex object memory address list , check the amount of vertex information to be processed for all vertex objects , if both are less than the lower limit of vertex information , then the vertex cumulative information of all vertex objects is After the calculation is completed, the control thread ends the loop and marks the thread status of all calculation threads Set to false to shut down all computation threads.

[0059] S3: Calculate the PageRank values ​​of all vertex objects based on the vertex cumulative information of all vertex objects described in step S2 , output the PageRank vector containing the PageRank values ​​of all vertex objects.

[0060] The total number of non-sunken vertices of the target graph data .

[0061] The output PageRank vector containing the PageRank values ​​of all vertex objects is , Corresponding vertex object The PageRank value of the web page is larger. The larger the PageRank value, the more links the web page has and the greater its importance. The top ten web pages in terms of importance can be screened out, as shown in Table 2 below:

[0062] Table 2 Top ten web pages ranked by importance

[0063]

[0064] Example 2:

[0065] The target graph data used in this embodiment is a social network. Vertex objects correspond to social users, and directed edges correspond to the following and followed relationships of users. When a user is the current source user, all other users it follows are called target users. The total number of target users it follows is the user's out-degree, and the total number of current source users it follows is the user's in-degree. The parameters of the social network are shown in Table 3 below.

[0066] Table 3 Social network parameters

[0067]

[0068] Among them, the number of sunken vertices is 0, so the number of non-sunken vertices is .

[0069] This embodiment uses a distributed parallel PageRank algorithm for large-scale graphs, such as Figure 1 As shown in the figure, select the top ten users in terms of importance. The specific steps are as follows:

[0070] S1: Determine the parallelism of the computing threads. , that is, need Computing threads are created and the target graph data is initialized to obtain the initialized variables and initialized lists.

[0071] S11: Determine the required initialization variables and initialization list, where the initialization variables include the damping coefficient , the lower limit of vertex information , vertex object related variables, computing thread related variables and control object related variables, where vertex object related variables include vertex cumulative information , the amount of vertex information to be processed and ; Calculation thread related variables include calculation thread code , damping coefficient , the lower limit of vertex information ; The control object related variables include atomic Boolean thread state identifiers and atomic Boolean thread operation status flags ; The initialization list includes the vertex object list , Vertex object memory address list List of directed edge weights , and computation threads Vertex Object Mapping Table .

[0072] S12: Create initialization variables and initialization lists, including creating damping coefficients ,Pick ; Create a lower limit on vertex information ,Pick ; Create vertex cumulative information , the initial value is ; Create vertex information to be processed , the initial value is ;create , each vertex object There is only one corresponding ; Create calculation thread code , each computing thread has a unique corresponding computing thread code ; Create thread status identifier , initially in false state; create thread operation state identifier , initially in false state; create a list of vertex objects , where each element contains information encoded by the vertex and vertex objects Composition; create a list of vertex object memory addresses , where each element corresponds to a vertex object The corresponding memory address , element information is provided by the vertex object list Get; create a list of directed edge weights , where each element contains information about the target vertex object and directed edge weights , where the directed edge weights By vertex object The out-degree is determined by taking the vertex object One out-degree of Vertex Object Mapping Table , where it is assumed that the vertex object memory address list The number of elements of , list the vertex object memory addresses All elements are evenly distributed computing threads, each computing thread corresponds to Vertex objects.

[0073] S2: Thread state identifier of each computing thread Enable is true, All computing threads are in the runnable state.

[0074] S21: When Thread state identifier of each computing thread When all are in true state, the control thread marks the thread operation status of all computing threads Enable is true, at which point all computing threads begin to call the core computing function to calculate the vertex cumulative information of all vertex objects. , each computing thread traverses the mapping table corresponding to the computing thread , traverse each vertex object First determine the vertex object The amount of vertex information to be processed Is it less than or equal to the lower limit of vertex information? , if it is less than or equal to the lower limit of vertex information , then the vertex object No calculation is performed and the next vertex object is determined. ; If it is greater than the lower limit of vertex information , then calculate the vertex object The amount of vertex cumulative information , where the calculation process steps are as follows:

[0075] Ⅰ: Vertex Object As the current source vertex object : Get the current source vertex object of , the current source vertex object The amount of vertex information to be processed Increase the amount of accumulated information at its vertex And make temporary storage variables , clear the vertex pending information , release the current source vertex object of ;

[0076] Ⅱ: Traverse the current source vertex object Corresponding list , for the list Each target vertex object in Do the same process: Get the target vertex object of , the damping coefficient The result increases to the amount of information to be processed at the vertex , release the target vertex object of .

[0077] The entire traversal process is completed until all vertex objects are The amount of vertex cumulative information calculation till now.

[0078] S22: The control thread loops through the vertex object memory address list , check the amount of vertex information to be processed for all vertex objects , if both are less than the lower limit of vertex information , then the vertex cumulative information of all vertex objects is After the calculation is completed, the control thread ends the loop and marks the thread status of all calculation threads Set to false to shut down all computation threads.

[0079] S3: Calculate the PageRank values ​​of all vertex objects based on the vertex cumulative information of all vertex objects described in step S2 , output the PageRank vector containing the PageRank values ​​of all vertex objects.

[0080] The total number of non-sunken vertices of the target graph data .

[0081] The output PageRank vector containing the PageRank values ​​of all vertex objects is , Corresponding vertex object The PageRank value of the user is higher. The larger the PageRank value, the more attention the user receives and the greater the importance. The top ten users in terms of importance can be screened out, as shown in Table 4 below:

[0082] Table 4 Top ten users ranked by importance

[0083]

[0084] Example 3:

[0085] The target graph data used in this embodiment is an academic literature citation network. Vertex objects correspond to academic literature, and directed edges correspond to the citation and citation relationships of academic literature. When a certain academic literature is used as the current source academic literature, all other academic literatures cited by it are called target academic literature. The total number of target academic literatures cited by it is the out-degree of the academic literature, and the total number of current source academic literatures cited by it is the in-degree of the academic literature. The parameters of the academic literature citation network are shown in Table 1 below:

[0086] Table 5 Academic literature citation network parameters

[0087]

[0088] Among them, the number of sunken vertices is 0, so the number of non-sunken vertices is .

[0089] This embodiment uses a distributed parallel PageRank algorithm for large-scale graphs, such as Figure 1 As shown in the figure, the top ten academic papers ranked by importance are selected by the following steps:

[0090] S1: Determine the parallelism of the computing threads. , that is, need Computing threads are created and the target graph data is initialized to obtain the initialized variables and initialized lists.

[0091] S11: Determine the required initialization variables and initialization list, where the initialization variables include the damping coefficient , the lower limit of vertex information , vertex object related variables, computing thread related variables and control object related variables, where vertex object related variables include vertex cumulative information , the amount of vertex information to be processed and ; Calculation thread related variables include calculation thread code , damping coefficient , the lower limit of vertex information ; The control object related variables include atomic Boolean thread state identifiers and atomic Boolean thread operation status flags ; The initialization list includes the vertex object list , Vertex object memory address list List of directed edge weights , and computation threads Vertex Object Mapping Table .

[0092] S12: Create initialization variables and initialization lists, including creating damping coefficients ,Pick ; Create a lower limit on vertex information ,Pick ; Create vertex cumulative information , the initial value is ; Create vertex information to be processed , the initial value is ;create , each vertex object There is only one corresponding ; Create calculation thread code , each computing thread has a unique corresponding computing thread code ; Create damping coefficient , according to the characteristics of the target graph data ; Create a lower limit on vertex information , according to the characteristics of the target graph data ; Create thread status identifier , initially in false state; create thread operation state identifier , initially in false state; create a list of vertex objects , where each element contains information encoded by the vertex and vertex objects Composition; create a list of vertex object memory addresses , where each element corresponds to a vertex object The corresponding memory address , element information is provided by the vertex object list Get; create a list of directed edge weights , where each element contains information about the target vertex object and directed edge weights , where the directed edge weights By vertex object The out-degree is determined by taking the vertex object One out-degree of Vertex Object Mapping Table , where it is assumed that the vertex object memory address list The number of elements of , list the vertex object memory addresses All elements are evenly distributed computing threads, each computing thread corresponds to Vertex objects.

[0093] S2: Thread state identifier of each computing thread Enable is true, All computing threads are in the runnable state.

[0094] S21: When Thread state identifier of each computing thread When all are in true state, the control thread marks the thread operation status of all computing threads Enable is true, at which point all computing threads begin to call the core computing function to calculate the vertex cumulative information of all vertex objects. , each computing thread traverses the mapping table corresponding to the computing thread , traverse each vertex object First determine the vertex object The amount of vertex information to be processed Is it less than or equal to the lower limit of vertex information? , if it is less than or equal to the lower limit of vertex information , then the vertex object No calculation is performed and the next vertex object is determined. ; If it is greater than the lower limit of vertex information , then calculate the vertex object The amount of vertex cumulative information , where the calculation process steps are as follows:

[0095] Ⅰ: Vertex Object As the current source vertex object : Get the current source vertex object of , the current source vertex object The amount of vertex information to be processed Increase the amount of accumulated information at its vertex And make temporary storage variables , clear the vertex pending information , release the current source vertex object of ;

[0096] Ⅱ: Traverse the current source vertex object Corresponding list , for the list Each target vertex object in Do the same process: Get the target vertex object of , the damping coefficient The result increases to the amount of information to be processed at the vertex , release the target vertex object of .

[0097] The entire traversal process is completed until all vertex objects are The amount of vertex cumulative information calculation till now.

[0098] S22: The control thread loops through the vertex object memory address list , check the amount of vertex information to be processed for all vertex objects , if both are less than the lower limit of vertex information , then the vertex cumulative information of all vertex objects is After the calculation is completed, the control thread ends the loop and marks the thread status of all calculation threads Set to false to shut down all computation threads.

[0099] S3: Calculate the PageRank values ​​of all vertex objects based on the vertex cumulative information of all vertex objects described in step S2 , output the PageRank vector containing the PageRank values ​​of all vertex objects.

[0100] The total number of non-sunken vertices of the target graph data .

[0101] The output PageRank vector containing the PageRank values ​​of all vertex objects is , Corresponding vertex object The PageRank value of the academic literature is higher. The higher the PageRank value, the more cited the academic literature is and the greater its importance is. The top ten academic literatures in terms of importance can be screened out, as shown in Table 6 below:

[0102] Table 6 Top ten academic papers ranked by importance

[0103]

[0104] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A distributed parallel PageRank algorithm for large-scale graphs, characterized by: The following steps are involved: S1: Determine the parallelism of computing threads based on the target graph data scale and distributed hardware environment conditions , and initialize the target graph data to obtain the initialized variables and initialized list; S2: The parallelism of the calculation threads obtained in step S1 , started The computing thread is then started, and the control thread is used to The calculation thread is controlled, and the vertex cumulative information of all vertex objects in the target graph data is calculated by combining the initialization variables and initialization list obtained in step S1; S3: Calculate the PageRank values ​​of all vertex objects based on the vertex cumulative information of all vertex objects obtained in step S2, and output a PageRank vector containing the PageRank values ​​of all vertex objects; In step S1, the initialization process includes the following steps: S11: Determine the required initialization variables and initialization list, where the initialization variables include the damping coefficient , the lower limit of vertex information , vertex object related variables, computing thread related variables and control object related variables, where vertex object related variables include vertex cumulative information , the amount of vertex information to be processed and ; Calculation thread related variables include calculation thread code ; The control object related variables include atomic Boolean thread state identifiers and atomic Boolean thread operation status flags ; S12: Create initialization variables and initialization lists; In step S12, the creation of initialization variables and initialization lists specifically includes the creation of damping coefficients , determined according to the specific application scenario; create a lower limit for vertex information , according to the required result accuracy; create vertex cumulative information , the initial value is ; Create vertex information to be processed , the initial value is ;create , each vertex object There is only one corresponding ; Create calculation thread code , each computing thread has a unique corresponding computing thread code ; Create thread status identifier , initially in false state; create thread operation state identifier , initially in false state; create a list of vertex objects , where each element contains information encoded by the vertex and vertex objects Composition; create a list of vertex object memory addresses , where each element corresponds to a vertex object The corresponding memory address , element information is provided by the vertex object list Get; create a list of directed edge weights , where each element contains information about the target vertex object and directed edge weights , where the directed edge weights By vertex object The out-degree is determined by taking the vertex object One out-degree of Vertex Object Mapping Table , where it is assumed that the vertex object memory address list The number of elements of , list the vertex object memory addresses All elements are evenly distributed computing threads, each computing thread corresponds to Vertex objects; The calculation of the vertex object The amount of vertex cumulative information The following steps are involved: Ⅰ: Vertex Object As the current source vertex object , get the current source vertex object of , the current source vertex object The amount of vertex information to be processed Increase the amount of accumulated information at its vertex And make temporary storage variables , clear the vertex pending information , release the current source vertex object of ; Ⅱ: Traverse the current source vertex object Corresponding list , for the list Each target vertex object in Do the same process: Get the target vertex object of , the damping coefficient The result increases to the amount of information to be processed at the vertex , release the target vertex object of ; In step S3, the formula for calculating the PageRank values ​​of all vertex objects is: Assuming that the vertex object The corresponding PageRank value is , is the total number of non-sunken vertices in the target graph data.

2. The distributed parallel PageRank algorithm for large-scale graphs according to claim 1, characterized in that In step S1, the vertex objects in the target graph data ,in Represents vertex object encoding, vertex object There are two states: one is the current source vertex object ,in , and another as the target vertex object ,in ; All vertex objects All through directed edges Connect, when a vertex object As the current source vertex object When it passes the directed edge All other vertex objects pointed to Both are called target vertex objects , and its corresponding relationship combined with the directed edge is recorded as , that is, the current source vertex object Points to the target vertex object ; When the vertex object As the current source vertex object When it points to the target vertex object The total number of vertex objects The out-degree of the vertex object As the target vertex object When it is pointed to, the current source vertex object The total number of vertex objects The in-degree.

3. The distributed parallel PageRank algorithm for large-scale graphs according to claim 1, characterized in that The initialization list in step S11 includes a vertex object list , Vertex object memory address list List of directed edge weights , and computation threads Vertex Object Mapping Table .

4. The distributed parallel PageRank algorithm for large-scale graphs according to claim 1, characterized in that In step S2, the startup The computation thread consists of Thread state identifier of each computing thread Enable is true, The computing threads are all in the runnable state. Controlling a computing thread includes the following steps: S21: When Thread state identifier of each computing thread When all are in true state, the control thread marks the thread operation status of all computing threads Enable is true, at which point all computing threads begin to call the core computing function to calculate the vertex cumulative information of all vertex objects. ; S22: The control thread loops through the vertex object memory address list , check the amount of vertex information to be processed for all vertex objects , if both are less than the lower limit of vertex information , then the vertex cumulative information of all vertex objects is After the calculation is completed, the control thread ends the loop and marks the thread status of all calculation threads Set to false to shut down all computation threads.

5. The distributed parallel PageRank algorithm for large-scale graphs according to claim 4, characterized in that In step S21, the core calculation function also includes temporary storage variables , the core calculation function needs to traverse the mapping table corresponding to the calculation thread , that is, calculate all vertex objects corresponding to the calculation thread The amount of vertex cumulative information ; Traverse each vertex object First determine the vertex object The amount of vertex information to be processed Is it less than or equal to the lower limit of vertex information? , if it is less than or equal to the lower limit of vertex information , then the vertex object No calculation is performed and the next vertex object is determined. ; If it is greater than the lower limit of vertex information , then calculate the vertex object The amount of vertex cumulative information ; The entire traversal process is completed until all vertex objects are The amount of vertex cumulative information calculation till now.

6. The distributed parallel PageRank algorithm for large-scale graphs according to claim 1, characterized in that In step S3, the output contains the PageRank vector of all vertex objects PageRank values: , Corresponding vertex object PageRank value.

Images