Software repository parallel construction method and system based on community classification

Abstract

The invention discloses a software repository parallel construction method and system based on community classification, and the method comprises the steps: extracting a software package dependency relationship from software packages, and building a software package dependency relationship graph between the software packages based on the software package dependency relationship; carrying out clustering division on the software package dependency relationship graph based on a relationship graph community classification algorithm, and carrying out hierarchy determination on a community according to a software package dependency relationship; and performing parallel construction on the software repository according to the community of which the hierarchy is determined. According to the method, the software package community division is performed based on the software package dependency relationship information, the parallel construction of the software warehouse is realized, the support is provided for the management of the software warehouse, and the method has the advantages of convenience in application and easiness in implementation.

Description

Technical Field

[0001] The present invention relates to the technical field of software warehouse construction for Linux-type operating systems, and in particular discloses a software warehouse parallel construction method and system based on community classification. Background Art

[0002] There are relatively mature package management systems and some build tools, but building a large number of software repositories is not an easy task, especially in the current community development, which requires building operating system versions from scratch and compiling software packages for new versions. There are mainly the following challenges: (1) Huge number of software packages There are tens of thousands of source packages for mainstream Linux distributions. Kylin V4sp4 has 53,228 source packages, and Kylin V10.1 has 58,121 source packages. And these are only some of the source packages provided in the official distribution, not including other informal source packages and source packages in extras. So many packages bring a lot of difficulties to the construction. Maintaining a mature distribution abroad often requires the support of the entire community, and commercial distributions require the cooperation of many professionals.

[0003] (2) The package dependencies between packages are complex The existing package management system makes it easier for developers to use existing libraries, and the rapidly developing information network also allows developers around the world to obtain more and better open source libraries or frameworks. However, this also makes it increasingly complicated to maintain and reduce software dependencies. Taking KylinV4sp4 as an example, there are 204,422 direct package dependencies between 53,228 packages, which means that the average number of direct package dependencies for each source package is about 3.8, and the indirect dependencies of source packages are more.

[0004] (3) Long package dependency chains The compilation of software A depends on software package B, and the compilation of software package B depends on software package C, thus forming a simple software package dependency chain. When building a large number of software packages, it is often necessary to deal with such software package dependency chains, which are sometimes as deep as more than ten layers. The higher the application software is, the longer its software package dependency chain is.

[0005] (4) Circular Dependency If the compilation of software package A depends on software package B, and the compilation of software package B depends on software package C, but the compilation of software package C depends on software package A, then this kind of software package dependency is a circular dependency. There are many such dependencies in Linux systems, because Linux software reuse is very high, and mutual package dependencies between software are very common. And due to the continuous iterative evolution of open source systems, this kind of circular compilation dependency will increase. This is the main reason that affects the speed of operating system construction.

[0006] The traditional way to build a software warehouse is to compile and build in alphabetical order of source code packages. If the build is successful, the generated software package is added to the software warehouse and then the build continues. When the number of source code packages is huge, the construction of the software warehouse takes a long time. Summary of the invention

[0007] The present invention provides a method and system for parallel construction of software repositories based on community classification, aiming to solve at least one of the above-mentioned defects of the prior art.

[0008] One aspect of the present invention relates to a method for parallel construction of software repositories based on community classification, comprising the following steps: Extracting software package dependencies from software packages, and establishing a software package dependency graph between software packages based on the software package dependencies; Cluster the software package dependency graph based on the relationship graph community classification algorithm, and determine the level of the community according to the software package dependency relationship; Based on the hierarchical communities, the software repositories are built in parallel.

[0009] Furthermore, the steps of extracting software package dependencies from the software packages and establishing a software package dependency graph between the software packages based on the software package dependencies include: Based on the software package list file, a software warehouse is built, and the software warehouse records the software package version information and software package dependencies; Extract the software package version information triple from the software package version information; extract the software package dependency triple from the software package dependency; The package version information triplet is expressed by the following formula: N i =(P i ,Version,V i ) Among them, N i Represents the software package version information triplet, P i Represents software package A, V i Represents the version of software package A; Version represents the version; The package dependency triple is expressed by the following formula: R ij =(P i ,Depends,P j ) Among them, R ij Represents the package dependency triple, P i Represents software package A, P j Represents the dependent software package B, Depends represents the dependency relationship, and the software package dependency triplet indicates that software package A depends on software package B.

[0010] Furthermore, in the step of extracting software package dependencies from the software packages and establishing a software package dependency graph between the software packages based on the software package dependencies, the software packages constitute a software package dependency graph through the software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by software package version information triples, and the dependency edges are established by dependency triples. If software package A and software package B are identified, a directed edge from software package A to software package B is established. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

[0011] Furthermore, the software package dependency graph is clustered based on the relationship graph community classification algorithm, and the step of determining the level of the community according to the software package dependency includes: Based on the software package dependency graph G, the subgraph is divided into subgraphs, and the subgraph g⊆ G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where g represents a subgraph, G represents a software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community of the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the package community information triple, P i Represents software packages A, C i Represents the community to which software package A belongs. Belong represents belonging; The community level is determined according to the software package dependency relationship. The initial level of all communities is set to 0. The community level information triple is L i =(C i, Level, 0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on. After traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k}, and the demand information triple set Q = {Q0, Q1, ..., Q k}, where L i represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated level information triplet; L represents the set of all updated level information triples, Level represents the level, and Require represents the requirement.

[0012] Furthermore, in the step of parallel construction of software warehouses according to the communities determined by the hierarchy, the software warehouse is represented as G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E allRepresents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is equal to 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are built in parallel from high to low according to hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

[0013] Another aspect of the present invention relates to a software repository parallel construction system based on community classification, comprising: Establishing a module for extracting software package dependencies from software packages, and establishing a software package dependency graph between software packages based on the software package dependencies; A partitioning module is used to cluster and partition the software package dependency graph based on the relationship graph community classification algorithm, and to determine the level of the community according to the software package dependency relationship; The parallel build module is used to build the software repository in parallel according to the hierarchical communities.

[0014] Further, the building module includes: The building unit is used to build a software warehouse based on the software package list file, and the software warehouse records the software package version information and software package dependencies; An extraction unit, used to extract a software package version information triple from the software package version information; and extract a software package dependency triple from the software package dependency; The package version information triplet is expressed by the following formula: N i =(P i ,Version,V i ) Among them, N i Represents the software package version information triplet, P i Represents software package A, V i Represents the version of software package A; Version represents the version; The package dependency triple is expressed by the following formula: R ij =(P i ,Depends,P j ) Among them, R ij Represents the package dependency triple, P i Represents software package A, P jRepresents the dependent software package B, Depends represents the dependency relationship, and the software package dependency triplet indicates that software package A depends on software package B.

[0015] Furthermore, in the establishment module, software packages form a software package dependency graph through software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by software package version information triples, and the dependency edges are established by dependency triples. If software package A and software package B are identified, a directed edge from software package A to software package B is established. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

[0016] Furthermore, the division modules include: Partitioning unit, used to partition the subgraph based on the software package dependency graph G, subgraph g⊆ G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where g represents a subgraph, G represents a software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community of the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the package community information triple, P i Represents software packages A, C i Represents the community to which software package A belongs. Belong represents belonging; The level determination unit is used to determine the level of the community according to the software package dependency relationship. The initial level of all communities is set to 0. The community level information triplet is L i =(C i , Level, 0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P jIf they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on. After traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k}, and the demand information triple set Q = {Q0, Q1, ..., Q k}, where L i represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated level information triplet; L represents the set of all updated level information triples, Level represents the level, and Require represents the requirement.

[0017] Furthermore, in the parallel construction module, the software warehouse is represented as G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E all Represents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is equal to 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are built in parallel from high to low according to hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

[0018] The beneficial effects achieved by the present invention are: The present invention provides a method and system for parallel construction of software warehouses based on community classification, by extracting software package dependencies from software packages, establishing a software package dependency graph between software packages based on the software package dependencies; clustering and dividing the software package dependency graph based on the relationship graph community classification algorithm, and determining the level of the community according to the software package dependencies; and constructing the software warehouse in parallel according to the community after the level is determined. The method and system for parallel construction of software warehouses based on community classification provided by the present invention propose a method for automatically extracting software package dependency information by parsing a software package list file, which can obtain the dependency of the software package without installing the software package, thereby improving the efficiency of data processing; storing the software package dependency through a knowledge graph can quickly process the huge amount of data and complex software package relationships in the software warehouse, thereby improving the ability of graph algorithm analysis; performing software division through a community classification algorithm can effectively guide the parallel construction of the software warehouse, thereby improving the efficiency of software package construction. In summary, the present invention performs software package community division based on software package dependency information, realizes the parallel construction of software warehouses, provides support for the management of software warehouses, and has the advantages of convenient application and easy implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of the method for parallel construction of software repositories based on community classification according to the present invention. DETAILED DESCRIPTION

[0020] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0021] like Figure 1 As shown, the first embodiment of the present invention proposes a method for parallel construction of software warehouses based on community classification, comprising the following steps: Step S100: extracting software package dependencies from the software packages, and establishing a software package dependency graph between the software packages based on the software package dependencies.

[0022] Extract the software package dependencies from the software packages, and build a software package dependency graph between the software packages based on the software package dependencies. The software package dependency graph is a complex directed graph consisting of software package nodes and dependency edges.

[0023] Step S200: clustering the software package dependency graph based on the relationship graph community classification algorithm, and determining the level of the community according to the software package dependency.

[0024] The software package dependency graph is clustered based on the relationship graph community classification algorithm, and the software packages in the software repository are divided into a hierarchical community structure according to the software package dependencies.

[0025] Step S300: Building software repositories in parallel based on the communities whose levels are determined.

[0026] According to the software package community and hierarchical relationship, the software warehouse is built in parallel. It can improve the construction efficiency of the software warehouse, reduce the maintenance cost of the software warehouse, realize the dependency analysis and community division of the software package, improve the analysis and processing efficiency of the operating system development and construction, and provide data support for the construction and tailoring of the version. It has the advantages of simple management, convenient application and easy implementation.

[0027] Further, see Figure 1 In the method for building a software repository in parallel based on community classification provided in this embodiment, step S100 includes: Step S110: Building a software warehouse based on the software package list file, in which the software package version information and software package dependency are recorded.

[0028] The software repository is built based on the package list file, which records the package version information and dependencies.

[0029] Step S120: extracting a software package version information triple from the software package version information; and extracting a software package dependency triple from the software package dependency.

[0030] Extract the package version information triplet N from the "Version" field i =(P i ,Version,V i ), extract the package dependency triples from the "Depends" field R ij =(P i ,Depends,P j ), the software package version information triple is expressed by the following formula: N i =(P i ,Version,V i )(1) In formula (1), N i Represents the software package version information triplet, P i Represents software package A, V i Represents the version of software package A; Version represents the version.

[0031] The package dependency triple is expressed by the following formula: R ij =(P i ,Depends,P j )(2) In formula (2), R ijRepresents the package dependency triple, P i Represents software package A, P j Represents the dependent software package B, Depends represents the dependency relationship, and the software package dependency triplet indicates that software package A depends on software package B.

[0032] Preferably, see Figure 1 In the method for parallel construction of a community-classified software repository provided in this embodiment, in step S100, software packages form a software package dependency graph through software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by software package version information triples, and the dependency edges are established by dependency triples. If software package A and software package B are identified, a directed edge from software package A to software package B is established. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

[0033] Preferably, see Figure 1 The method for building a software repository in parallel based on community classification provided in this embodiment includes step S200: Step S210: divide the software package dependency graph G into subgraphs, where subgraph g⊆ G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where g represents a subgraph, G represents a software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community of the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the package community information triple, P i Represents software packages A, C i Indicates the community to which software package A belongs. Belong means belongs to.

[0034] Step S220: Determine the level of the community according to the software package dependency relationship, set the initial level of all communities to 0, and the community level information triplet is L i =(C i , Level, 0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C iand the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on. After traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k}, and the demand information triple set Q = {Q0, Q1, ..., Q k}, where L i represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated level information triplet; L represents the set of all updated level information triples, Level represents the level, and Require represents the requirement.

[0035] Further, see Figure 1 In the method for building a software warehouse in parallel based on community classification provided in this embodiment, in step S300, the software warehouse is represented by G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E allRepresents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is equal to 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are built in parallel from high to low according to hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

[0036] The following is a specific example of a method for building a software repository in parallel based on community classification provided by this embodiment: Unzip the Packages.gz file of the target build software repository to obtain the required package information list file Packages.

[0037] Read the software package information list file Packages, parse the software package information list file Packages to obtain detailed information of all software packages, and extract target information according to the attribute field. The dependencies are recorded in the "Depends" field. Some dependencies limit the version number, so the "Version" field that records the software package version information also needs to be extracted.

[0038] In order to facilitate information conversion into graph storage, triples are used to record software package information and dependency information. The software package version information triple N i =(P i ,Version,V i ). The set of all package version information triplets is N={N1, N2, ..., N k}. Package dependency triple R ij =(P i ,Depends,P j ), the set of all package dependency triples is R={R12,...,R 23 , ..., R ij}.

[0039] After extracting the software package and dependency triples, you can perform graph conversion and use the neo4j graph database to store the software package dependency directed graph.

[0040] Based on the software package version information triple N i =(P i ,Version,V i )Create a package node: create (n:Package {name:'P i ',version:'V i '}) return n This statement indicates that a file named P is created. i , version number is V i The package node.

[0041] R based on package dependency triples ij =(P i ,Depends,P j ) Create a dependency edge: match (a:Package {name:'P i '}), (b:Package {name:'P j '}) MERGE (a)-[:Depends]->(b) This statement indicates that the software package P is created. i Point to P j The directed edge of .

[0042] After creating the Neo4j graph database, you can perform graph algorithm analysis based on the generated software package dependency directed graph and use the Louvain community discovery algorithm to classify software package communities. This algorithm can discover hierarchical community structures.

[0043] Design a cypher query statement to call the Louvain algorithm for software package community classification: CALL gds.louvain.stream('dependsGraph') YIELD nodeId, communityId WITH communityId, collect(gds.util.asNode(nodeId).name) AS packages WITH *,size(packages) as groupsize RETURN * ORDER BY groupsize DESC This statement indicates that the Louvain algorithm is used to calculate the software package dependency directed graph 'dependsGraph', and the community IDs and community members are returned in descending order.

[0044] Use C = {C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where C i is the ith community in the community division. After the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where P i Represents software packages A, C i Represents the community to which software package A belongs. All software package community information triplets form a set B = {B1, B2, ..., B k}.

[0045] For the community collection C in C i , set the community level information triplet to Li = (Ci, Level, 0) and the community demand information triplet to Q i =(C i , Require, 0). Traverse the package dependency triples set R and add R ij =(P i ,Depends,P j ) in the software package P i Community C i and its dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j Not belonging to the same community, the dependent package P j Community C j The level of community C increases by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i , Require, 1). Similarly, after traversing all the package dependencies, we get the set of all updated level information triples L = {L0, L1, ..., L k} and the demand information triple set Q = {Q0, Q1, ..., Q k}.

[0046] If the final demand attribute in the demand information triple set Q is still equal to 0, it means that the community does not need to rely on other communities, and these communities can be built in parallel first.

[0047] Then, communities are constructed in parallel from high to low according to their hierarchical attributes. Communities with higher hierarchical levels will affect more communities. Prioritizing the compilation of communities with higher hierarchical levels can ensure the smooth compilation of subsequent communities.

[0048] Please see Figure 1 The present invention relates to a software warehouse parallel construction system based on community classification, comprising a building module, a partitioning module and a parallel construction module, wherein the building module is used to extract software package dependencies from software packages and to build a software package dependency graph between software packages based on the software package dependencies. The partitioning module is used to cluster and partition the software package dependency graph based on a relationship graph community classification algorithm and to determine the level of the community according to the software package dependencies. The parallel construction module is used to build the software warehouse in parallel according to the community after the level is determined.

[0049] The building module extracts the software package dependencies from the software packages, and builds a software package dependency graph between the software packages based on the software package dependencies. The software package dependency graph is a complex directed graph consisting of software package nodes and dependency edges.

[0050] The partitioning module clusters and partitions the software package dependency graph based on the relationship graph community classification algorithm, and divides the software packages in the software repository into a hierarchical community structure according to the software package dependencies.

[0051] The parallel construction module builds the software warehouse in parallel according to the software package community and hierarchical relationship. It can improve the construction efficiency of the software warehouse, reduce the maintenance cost of the software warehouse, realize the dependency analysis and community division of the software package, improve the analysis and processing efficiency of the operating system development and construction, provide data support for the construction and tailoring of the version, and has the advantages of simple management, convenient application and easy implementation.

[0052] Further, see Figure 1 The software warehouse parallel construction system for community classification provided in this embodiment includes a construction module and an extraction unit, wherein the construction unit is used to construct a software warehouse based on a software package list file, and the software warehouse records the software package version information and the software package dependency relationship. The extraction unit is used to extract the software package version information triple from the software package version information; and extract the software package dependency triple from the software package dependency relationship.

[0053] In the build unit, the software repository is built based on the software package list file, which records the software package version information and dependencies.

[0054] In the extraction unit, extract the software package version information triplet N from the "Version" field i =(P i ,Version,V i ), extract the package dependency triples from the "Depends" field R ij =(P i ,Depends,P j ), the software package version information triple is expressed by the following formula: N i =(P i ,Version,V i )(3) In formula (3), N i Represents the software package version information triplet, P i Represents software package A, V i Represents the version of software package A; Version represents the version.

[0055] The package dependency triple is expressed by the following formula: R ij =(P i ,Depends,P j )(4) In formula (4), R ij Represents the package dependency triple, P i Represents software package A, P j Represents the dependent software package B, Depends represents the dependency relationship, and the software package dependency triplet indicates that software package A depends on software package B.

[0056] Preferably, see Figure 1 In the software repository parallel building system for community classification provided in this embodiment, in the building module, software packages form a software package dependency graph through software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by software package version information triples, and the dependency edges are established by dependency triples. If software package A and software package B are identified, a directed edge from software package A to software package B is established. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

[0057] Further, see Figure 1 In the software warehouse parallel construction system for community classification provided in this embodiment, the partitioning module includes a partitioning unit and a level determination unit, wherein the partitioning unit is used to partition the subgraph based on the software package dependency graph G, and the subgraph g⊆ G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where g represents a subgraph, G represents a software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community of the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the package community information triple, P i Represents software packages A, C i Represents the community to which software package A belongs. Belong represents belonging. The level determination unit is used to determine the level of the community according to the software package dependency relationship. The initial level of all communities is set to 0. The community level information triplet is L i =(C i , Level, 0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on. After traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k}, and the demand information triple set Q = {Q0, Q1, ..., Q k}, where L i represents the community-level information triple, Q iRepresents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated level information triplet; L represents the set of all updated level information triples, Level represents the level, and Require represents the requirement.

[0058] Preferably, see Figure 1 In the parallel construction module of the software warehouse classified by the community provided in this embodiment, the software warehouse is represented by G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E all Represents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is equal to 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are built in parallel from high to low according to hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

[0059] The software warehouse parallel construction method and system based on community classification provided in this embodiment, compared with the prior art, extracts software package dependencies from software packages, establishes a software package dependency graph between software packages based on software package dependencies; clusters and divides the software package dependency graph based on the relationship graph community classification algorithm, and determines the level of the community according to the software package dependencies; and builds the software warehouse in parallel according to the community after the level is determined. The software warehouse parallel construction method and system based on community classification provided in this embodiment proposes a method for automatically extracting software package dependency information by parsing the software package list file, which can obtain the software package dependency without installing the software package, thereby improving the efficiency of data processing; storing the software package dependency by the knowledge graph can quickly process the huge amount of data and complex software package relationships in the software warehouse, and improve the ability of graph algorithm analysis; performing software division by the community classification algorithm can effectively guide the parallel construction of the software warehouse, thereby improving the efficiency of software package construction. In summary, this embodiment divides the software package community based on the software package dependency information, realizes the parallel construction of the software warehouse, provides support for the management of the software warehouse, and has the advantages of convenient application and easy implementation.

[0060] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for parallel construction of software warehouses based on community classification, characterized in that: The following steps are involved: Extracting software package dependencies from the software packages, and establishing a software package dependency graph between the software packages based on the software package dependencies; Clustering the software package dependency graph based on a relationship graph community classification algorithm, and determining the level of the community according to the software package dependency; Based on the hierarchical communities, the software repositories are built in parallel.

2. The method for parallel construction of software warehouses based on community classification according to claim 1, characterized in that: The step of extracting software package dependencies from the software packages and establishing a software package dependency graph between the software packages based on the software package dependencies comprises: Building a software warehouse based on the software package list file, wherein the software warehouse records software package version information and software package dependencies; Extracting a software package version information triple from the software package version information; extracting a software package dependency triple from the software package dependency; The software package version information triplet is expressed by the following formula: N i =(P i ,Version,V i ) Among them, N i Represents the software package version information triplet, P i Represents software package A, V i Represents the version of software package A; Version represents the version; The software package dependency triplet is expressed by the following formula: R ij =(P i ,Depends,P j ) Among them, R ij Represents the package dependency triple, P i Represents software package A, P j represents the dependent software package B, Depends represents the dependency relationship, and the software package dependency triplet indicates that software package A depends on software package B.

3. The method for parallel construction of software warehouses based on community classification according to claim 2, characterized in that: In the step of extracting software package dependencies from the software packages and establishing a software package dependency graph between the software packages based on the software package dependencies, the software packages constitute a software package dependency graph through the software package dependencies, and the software package dependency graph includes software package nodes and dependency edges, the software package nodes are established by the software package version information triples, and the dependency edges are established by the dependency triples. If software package A and software package B are identified, a directed edge from software package A to software package B is established, and this operation is performed for all dependencies to form a software package dependency graph G=(V, E), wherein G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

4. The method for parallel construction of software warehouses based on community classification according to claim 3, characterized in that: The step of clustering the software package dependency graph based on the relationship graph community classification algorithm and determining the level of the community according to the software package dependency comprises: Based on the software package dependency graph G, subgraph division is performed, and subgraph g G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k } represents the division of the graph G into a set of k communities, where g represents a subgraph, G represents a software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community of the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the package community information triple, P i Represents software packages A, C i Represents the community to which software package A belongs. Belong represents belonging; The community level is determined according to the software package dependency relationship, and the initial level of all communities is set to 0. The community level information triplet is L i =(C i , Level, 0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on. After traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k }, and the demand information triple set Q = {Q0, Q1, ..., Q k }, where L i represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated level information triplet; L represents the set of all updated level information triples, Level represents the level, and Require represents the requirement.

5. The method for parallel construction of software warehouses based on community classification according to claim 4, characterized in that: In the step of constructing the software warehouse in parallel according to the community determined by the hierarchy, the software warehouse is represented by G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E all Represents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is equal to 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are built in parallel from high to low according to hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

6. A software warehouse parallel construction system based on community classification, characterized in that: include: Establishing a module, used for extracting software package dependencies from the software packages, and establishing a software package dependency graph between the software packages based on the software package dependencies; A partitioning module, used for clustering the software package dependency graph based on a relationship graph community classification algorithm, and determining the level of the community according to the software package dependency; The parallel build module is used to build the software repository in parallel according to the hierarchical communities.

7. The software warehouse parallel construction system based on community classification according to claim 6, characterized in that: The establishment module includes: A construction unit, used to construct a software warehouse based on the software package list file, wherein the software warehouse records software package version information and software package dependency; An extraction unit, configured to extract a software package version information triple from the software package version information; and extract a software package dependency triple from the software package dependency; The software package version information triplet is expressed by the following formula: N i =(P i ,Version,V i ) Among them, N i Represents the software package version information triplet, P i Represents software package A, V i Represents the version of software package A; Version represents the version; The software package dependency triplet is expressed by the following formula: R ij =(P i ,Depends,P j ) Among them, R ij Represents the package dependency triple, P i Represents software package A, P j represents the dependent software package B, Depends represents the dependency relationship, and the software package dependency triplet indicates that software package A depends on software package B.

8. The software warehouse parallel construction system based on community classification according to claim 7, characterized in that: In the establishment module, the software packages form a software package dependency graph through the software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by the software package version information triples, and the dependency edges are established by the dependency triples. If software package A and software package B are identified, a directed edge from software package A to software package B is established. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

9. The software warehouse parallel construction system based on community classification according to claim 8, characterized in that: The division module comprises: A partitioning unit is used to partition the software package dependency graph G into subgraphs, where the subgraph g⊆ G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k } represents the division of the graph G into a set of k communities, where g represents a subgraph, G represents a software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community of the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the package community information triple, P i Represents software packages A, C i Represents the community to which software package A belongs. Belong represents belonging; A level determination unit is used to determine the level of the community according to the software package dependency relationship, and the initial level of all communities is set to 0. The community level information triple is L i =(C i , Level, 0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on. After traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k }, and the demand information triple set Q = {Q0, Q1, ..., Q k }, where L i represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated level information triplet; L represents the set of all updated level information triples, Level represents the level, and Require represents the requirement.

10. The software warehouse parallel construction system based on community classification according to claim 9, characterized in that: In the parallel construction module, the software warehouse is represented by G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E all Represents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is equal to 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are built in parallel from high to low according to hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

Images