A software warehouse parallel construction method and system based on community classification

Abstract

The present invention discloses a method and system for parallel construction of software repositories based on community classification. The method extracts package dependencies from software packages and establishes a package dependency graph between them based on the package dependency relationships. The graph is then clustered and divided using a community classification algorithm, and communities are hierarchically determined based on the package dependency relationships. Software repositories are then constructed in parallel based on the hierarchically determined communities. The method divides software packages into communities based on package dependency information, enabling parallel construction of software repositories and supporting software repositories management. The method is easy to apply and implement.

Description

Technical Field

[0001] The present invention relates to the technical field of software warehouse construction for Linux-type operating systems, and in particular discloses a method and system for parallel construction of software warehouses based on community classification. Background Art

[0002] While there are relatively mature package management systems and some build tools, building a large number of software repositories is not a simple task, especially in the current era of community development, where operating system versions must be built from scratch and software packages compiled for the new versions. The main challenges are:

[0003] (1) Huge number of software packages

[0004] Mainstream Linux distributions have tens of thousands of source code packages. Kylin V4sp4 has 53,228, and Kylin V10.1 has 58,121. And these are just a few of the source packages included in the official distribution, not counting unofficial source packages and those in extras. Such a large number of packages creates significant challenges when building a distribution. Maintaining a mature distribution abroad often requires the support of an entire community, and commercial distributions require the collaborative efforts of many professionals.

[0005] (2) The package dependencies between packages are complex

[0006] Existing package management systems make it easier for developers to use existing libraries, and the rapidly developing information network has also enabled developers around the world to access more and better open source libraries and frameworks. However, this also makes maintaining and reducing software dependencies increasingly complex. Taking Kylin V4sp4 as an example, there are 204,422 direct dependencies between 53,228 packages. This means that on average, each source package has approximately 3.8 direct dependencies, while the indirect dependencies are even greater.

[0007] (3) Very long package dependency chains

[0008] The compilation of software A depends on software package B, which in turn depends on software package C. This forms a simple software package dependency chain. When building a large number of software packages, you often need to deal with this kind of dependency chain. Sometimes, the dependency chain can be as deep as a dozen layers. The higher the application layer, the longer the dependency chain.

[0009] (4) Circular dependencies

[0010] If the compilation of package A depends on package B, which in turn depends on package C, and the compilation of package C depends on package A, then this is a circular dependency. Such dependencies are common in Linux systems because Linux software reuse is very high, and inter-package dependencies are common. Furthermore, due to the continuous iterative evolution of open source systems, these circular compilation dependencies are increasing in number. This is a major factor affecting operating system build speeds.

[0011] The traditional software repository construction method is to compile and build the source code packages in alphabetical order. If the build is successful, the generated software package is added to the software repository and the construction continues. When the number of source code packages is large, the software repository construction takes a long time. Summary of the Invention

[0012] The present invention provides a method and system for parallel construction of software repositories based on community classification, aiming to solve at least one of the above-mentioned deficiencies in the prior art.

[0013] One aspect of the present invention relates to a method for parallel construction of software repositories based on community classification, comprising the following steps:

[0014] Extracting software package dependencies from software packages, and building a software package dependency graph between software packages based on the software package dependencies;

[0015] Cluster the software package dependency graph based on the relationship graph community classification algorithm, and determine the community level according to the software package dependency relationship;

[0016] Based on the hierarchical communities, the software repositories are built in parallel.

[0017] Furthermore, the steps of extracting software package dependencies from the software packages and establishing a software package dependency graph between the software packages based on the software package dependencies include:

[0018] Based on the software package list file, a software warehouse is built, which records the software package version information and software package dependencies;

[0019] Extract the software package version information triple from the software package version information; extract the software package dependency triple from the software package dependency;

[0020] The software package version information triplet is expressed by the following formula:

[0021] N i =(P i , Version, V i )

[0022] Among them, Ni Represents the software package version information triplet, P i Represents software packages A, V i Represents the version of software package A; Version represents the version;

[0023] The package dependency triple is expressed by the following formula:

[0024] R ij =(P i ,Depends,P j )

[0025] Among them, R ij Represents the package dependency triple, P i Represents software package A, P j Represents the dependent package B, Depends represents the dependency relationship, and the package dependency triplet indicates that package A depends on package B.

[0026] Furthermore, in the step of extracting software package dependencies from the software packages and establishing a software package dependency graph between the software packages based on the software package dependencies, the software packages form a software package dependency graph through the software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by software package version information triples, and the dependency edges are established by dependency triples. If software package A and software package B are identified, a directed edge is established from software package A to software package B. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

[0027] Furthermore, the software package dependency graph is clustered based on the relationship graph community classification algorithm, and the steps of determining the level of the community according to the software package dependency include:

[0028] Based on the software package dependency graph G, the subgraph is divided into subgraphs, and the subgraph g⊆ G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where g represents the subgraph, G represents the software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community in the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the software package community information triple, P iRepresents software packages A and C i Indicates the community to which software package A belongs. Belong means belongs to.

[0029] The community level is determined according to the software package dependency relationship. The initial level of all communities is set to 0. The community level information triplet is L i =(C i ,Level,0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software Package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on, after traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k}, and the demand information triple set Q={Q0, Q1, ..., Q k}, where L i Represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated hierarchical information triplet; L represents the set of all updated hierarchical information triples, Level represents the level, and Require represents the requirement.

[0030] Furthermore, in the step of parallel construction of the software warehouse based on the community after the level is determined, the software warehouse is represented as G repo =(V all , Eall ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E all Represents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; then sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are constructed in parallel from high to low hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

[0031] Another aspect of the present invention relates to a software repository parallel construction system based on community classification, comprising:

[0032] Establish a module for extracting software package dependencies from software packages, and establishing a software package dependency graph between software packages based on the software package dependencies;

[0033] A partitioning module is used to cluster the software package dependency graph based on the relationship graph community classification algorithm and determine the level of the community according to the software package dependency relationship;

[0034] The parallel build module is used to build the software repository in parallel based on the community after the hierarchy is determined.

[0035] Furthermore, the building blocks include:

[0036] The build unit is used to build a software repository based on the software package list file, which records the software package version information and package dependencies;

[0037] An extraction unit, configured to extract a software package version information triple from the software package version information; and extract a software package dependency triple from the software package dependency relationship;

[0038] The package version information triplet is expressed by the following formula:

[0039] N i =(P i , Version, V i )

[0040] Among them, N i Represents the software package version information triplet, P i Represents software packages A, V i Represents the version of software package A; Version represents the version;

[0041] The package dependency triple is expressed by the following formula:

[0042] R ij =(P i ,Depends,P j )

[0043] Among them, R ij Represents the package dependency triple, P i Represents software package A, P j Represents the dependent package B, Depends represents the dependency relationship, and the package dependency triplet indicates that package A depends on package B.

[0044] Furthermore, in the establishment module, software packages form a software package dependency graph through software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by software package version information triples, and the dependency edges are established by dependency triples. If software package A and software package B are identified, a directed edge from software package A to software package B is established. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is the set of software package nodes, and E is the set of dependency edges.

[0045] Furthermore, the division modules include:

[0046] Partitioning unit, used to partition the subgraph based on the software package dependency graph G, subgraph g⊆ G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where g represents the subgraph, G represents the software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community in the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the software package community information triple, P i Represents software packages A and C i Indicates the community to which software package A belongs. Belong means belongs to.

[0047] The level determination unit is used to determine the level of the community according to the software package dependency relationship. The initial level of all communities is set to 0. The community level information triplet is L i =(C i,Level,0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software Package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on, after traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k}, and the demand information triple set Q={Q0, Q1, ..., Q k}, where L i Represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated hierarchical information triplet; L represents the set of all updated hierarchical information triples, Level represents the level, and Require represents the requirement.

[0048] Furthermore, in the parallel construction module, the software warehouse is represented as G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E allRepresents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; then sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are constructed in parallel from high to low hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

[0049] The beneficial effects achieved by the present invention are:

[0050] The present invention provides a method and system for parallel construction of software warehouses based on community classification, which extracts software package dependencies from software packages and establishes a software package dependency graph between software packages based on the software package dependencies; clusters and divides the software package dependency graph based on the relationship graph community classification algorithm, and determines the level of the community according to the software package dependencies; and constructs the software warehouse in parallel based on the community after the level is determined. The method and system for parallel construction of software warehouses based on community classification provided by the present invention propose a method for automatically extracting software package dependency information by parsing the software package list file, which can obtain the dependency of the software package without installing the software package, thereby improving the efficiency of data processing; storing the software package dependency through the knowledge graph can quickly process the huge amount of data and complex software package relationships in the software warehouse, thereby improving the ability of graph algorithm analysis; performing software division through the community classification algorithm can effectively guide the parallel construction of the software warehouse, thereby improving the efficiency of software package construction. In summary, the present invention divides the software package community based on the software package dependency information, realizes the parallel construction of the software warehouse, provides support for the management of the software warehouse, and has the advantages of convenient application and easy implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Schematic diagram of the process of the parallel construction method of software warehouse based on community classification of the present invention. DETAILED DESCRIPTION

[0052] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] like Figure 1 As shown, the first embodiment of the present invention proposes a method for parallel construction of software warehouses based on community classification, comprising the following steps:

[0054] Step S100: extracting software package dependencies from the software packages, and establishing a software package dependency graph between the software packages based on the software package dependencies.

[0055] Extract package dependencies from software packages and build a package dependency graph between packages based on the package dependencies. The package dependency graph is a complex directed graph consisting of package nodes and dependency edges.

[0056] Step S200: clustering the software package dependency graph based on the relationship graph community classification algorithm, and determining the level of the community according to the software package dependency.

[0057] The software package dependency graph is clustered based on the relationship graph community classification algorithm, and the software packages in the software warehouse are divided into a hierarchical community structure according to the software package dependency.

[0058] Step S300: Building software repositories in parallel based on the communities whose levels are determined.

[0059] Based on the software package communities and hierarchical relationships, software repositories are built in parallel. This improves software repository build efficiency and reduces maintenance costs. It can analyze software package dependencies and partition communities, improve the analysis and processing efficiency of operating system development and builds, and provide data support for version building and tailoring. It offers the advantages of simple management, convenient application, and ease of implementation.

[0060] Further, see Figure 1 In the method for building a software repository in parallel based on community classification provided in this embodiment, step S100 includes:

[0061] Step S110: Building a software warehouse based on the software package list file, and recording software package version information and software package dependency relationships in the software warehouse.

[0062] The software repository is built based on the package list file, which records the package version information and dependencies.

[0063] Step S120: extracting a software package version information triple from the software package version information; and extracting a software package dependency triple from the software package dependency.

[0064] Extract the package version information triplet N from the "Version" field i =(P i , Version, V i ), extract the package dependency triples R from the "Depends" field ij =(P i ,Depends,P j ), the software package version information triple is expressed by the following formula:

[0065] N i=(P i , Version, V i )(1)

[0066] In formula (1), N i Represents the software package version information triplet, P i Represents software packages A, V i Represents the version of software package A; Version represents the version.

[0067] The package dependency triple is expressed by the following formula:

[0068] R ij =(P i ,Depends,P j )(2)

[0069] In formula (2), R ij Represents the package dependency triple, P i Represents software package A, P j Represents the dependent package B, Depends represents the dependency relationship, and the package dependency triplet indicates that package A depends on package B.

[0070] Preferably, see Figure 1 In the method for parallel construction of a community-classified software repository provided in this embodiment, in step S100, software packages are connected to form a software package dependency graph through software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by software package version information triples, and the dependency edges are established by dependency triples. If software package A and software package B are identified, a directed edge is established from software package A to software package B. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

[0071] Preferably, see Figure 1 In the method for building a software repository in parallel based on community classification provided in this embodiment, step S200 includes:

[0072] Step S210: Divide the software package dependency graph G into subgraphs, where subgraph g⊆ G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where g represents the subgraph, G represents the software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community in the community division; after the software package is divided into communities, the software package community information triple B is establishedi =(P i , Belong, C i ), where B i Represents the software package community information triple, P i Represents software packages A and C i Indicates the community to which software package A belongs. Belong indicates belonging.

[0073] Step S220: Determine the level of the community according to the software package dependency relationship, set the initial level of all communities to 0, and the community level information triplet is L i =(C i ,Level,0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software Package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on, after traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k}, and the demand information triple set Q={Q0, Q1, ..., Q k}, where L i Represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L jRepresents the updated hierarchical information triplet; L represents the set of all updated hierarchical information triples, Level represents the level, and Require represents the requirement.

[0074] Further, see Figure 1 In the method for building a software warehouse in parallel based on community classification provided in this embodiment, in step S300, the software warehouse is represented by G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E all Represents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; then sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are constructed in parallel from high to low hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

[0075] The following describes the method for parallel construction of software repositories based on community classification provided by this embodiment with a specific embodiment:

[0076] Unzip the target build software warehouse Packages.gz to obtain all the required software package information list file Packages.

[0077] Read the package information list file Packages, parse the package information list file Packages to obtain detailed information of all packages, and extract target information based on the attribute fields. Dependencies are recorded in the "Depends" field. Some dependencies limit the version number, so the "Version" field that records the package version information also needs to be extracted.

[0078] In order to facilitate information conversion into graph storage, triples are used to record software package information and dependency information. The software package version information triple N i =(P i , Version, V i ). The set of all package version information triples is N={N1, N2, ..., N k Package dependency triplet R ij =(P i ,Depends,P j), the set of all package dependency triples is R={R12,...,R 23 , ..., R ij}.

[0079] After extracting the software package and dependency triples, we can convert them into a graph and use the neo4j graph database to store the software package dependency directed graph.

[0080] Based on the software package version information triplet N i =(P i , Version, V i )Create a package node:

[0081] create (n:Package {name:'P i ', version:'V i '}) return n

[0082] This statement indicates that a file named P is created. i , version number is V i The package node.

[0083] R based on package dependency triples ij =(P i ,Depends,P j ) Create dependency edges:

[0084] match (a:Package {name:'P i '}), (b:Package {name:'P j '}) MERGE (a)-[:Depends]->(b)

[0085] This statement indicates that the software package P is created. i Point to P j The directed edge of .

[0086] After creating a Neo4j graph database, you can perform graph algorithm analysis based on the generated directed graph of package dependencies and use the Louvain community discovery algorithm to classify package communities. This algorithm can discover hierarchical community structures.

[0087] Design a cypher query statement to call the Louvain algorithm to classify software packages into communities:

[0088] CALL gds.louvain.stream('dependsGraph')

[0089] YIELD nodeId, communityId

[0090] WITH communityId, collect(gds.util.asNode(nodeId).name) AS packages

[0091] WITH *,size(packages) as groupsize

[0092] RETURN *

[0093] ORDER BY groupsize DESC

[0094] This statement calculates the package dependency graph 'dependsGraph' using the Louvain algorithm, sorts the communities by size, and returns the community IDs and community members in descending order.

[0095] Use C={C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where C i is the ith community in the community division. After the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where P i Represents software packages A and C i Represents the community to which software package A belongs. All software package community information triples form a set B = {B1, B2, ..., B k}.

[0096] For the community collection C in C i , set the community level information triplet to Li=(Ci, Level, 0) and the community demand information triplet to Q i =(C i , Require, 0). Traverse the software package dependency triple set R and add R ij =(P i ,Depends,P j ) in the software package P i Community C i and its dependent packages NodeP j Conduct community C j Compare, if P i and P j Belong to the same community, the community level information does not change; if P i and P j Not belonging to the same community, the dependent package P jCommunity C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software Package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i , Require, 1). Similarly, after traversing all software package dependencies, we get the set of all updated hierarchical information triples L={L0, L1, ..., L k} and the demand information triple set Q={Q0, Q1, ..., Q k}.

[0097] If the final demand attribute in the demand information triple set Q is still equal to 0, it means that the community does not need to rely on other communities, and these communities can be built in parallel first.

[0098] Then, the communities are constructed in parallel from high to low according to the hierarchical attributes. The higher the level of the community, the more communities it affects. Prioritizing the compilation of high-level communities can ensure the smooth compilation of subsequent communities.

[0099] Please see Figure 1 The present invention relates to a software repository parallel construction system based on community classification, comprising a building module, a partitioning module, and a parallel construction module. The building module is configured to extract package dependencies from software packages and to build a package dependency graph based on the package dependencies. The partitioning module is configured to cluster the package dependency graph based on a relationship graph community classification algorithm and to hierarchically determine communities based on the package dependencies. The parallel construction module is configured to parallelly construct software repositories based on the hierarchically determined communities.

[0100] The establishment module extracts package dependencies from the software packages and establishes a package dependency graph between the software packages based on the package dependencies. The package dependency graph is a complex directed graph consisting of package nodes and dependency edges.

[0101] The partitioning module clusters and partitions the software package dependency graph based on the relationship graph community classification algorithm, and divides the software packages in the software repository into a hierarchical community structure according to the software package dependency.

[0102] The parallel build module builds software repositories in parallel based on the software package communities and hierarchical relationships. This improves software repository build efficiency and reduces maintenance costs. It can analyze software package dependencies and partition communities, improving the analysis and processing efficiency of operating system development and builds, and providing data support for version building and tailoring. It offers the advantages of simple management, convenient application, and ease of implementation.

[0103] Further, see Figure 1 This embodiment provides a parallel building system for community-classified software repositories. The building module includes a building unit and an extraction unit. The building unit is used to build a software repository based on a software package list file, where the software repository records software package version information and software package dependencies. The extraction unit is used to extract a software package version information triple from the software package version information and a software package dependency triple from the software package dependency.

[0104] In the build unit, the software repository is built based on the software package list file, which records the software package version information and dependencies.

[0105] In the extraction unit, extract the software package version information triplet N from the "Version" field i =(P i , Version, V i ), extract the package dependency triples R from the "Depends" field ij =(P i ,Depends,P j ), the software package version information triple is expressed by the following formula:

[0106] N i =(P i , Version, V i )(3)

[0107] In formula (3), N i Represents the software package version information triplet, P i Represents software packages A, V i Represents the version of software package A; Version represents the version.

[0108] The package dependency triple is expressed by the following formula:

[0109] R ij =(P i ,Depends,P j )(4)

[0110] In formula (4), R ij Represents the package dependency triple, P iRepresents software package A, P j Represents the dependent package B, Depends represents the dependency relationship, and the package dependency triplet indicates that package A depends on package B.

[0111] Preferably, see Figure 1 In the parallel construction system for community-classified software repositories provided in this embodiment, in the construction module, software packages are connected through package dependencies to form a package dependency graph. The package dependency graph includes package nodes and dependency edges. The package nodes are established by package version information triples, and the dependency edges are established by dependency triples. If package A and package B are identified, a directed edge is established from package A to package B. This operation is performed for all dependencies to form a package dependency graph G=(V, E), where G represents the package dependency graph, V is a set of package nodes, and E is a set of dependency edges.

[0112] Further, see Figure 1 The embodiment provides a parallel construction system for community-classified software warehouses. The partitioning module includes a partitioning unit and a level determination unit. The partitioning unit is used to partition the subgraph based on the software package dependency graph G. The subgraph g⊆ G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k} represents the division of the graph G into a set of k communities, where g represents the subgraph, G represents the software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community in the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the software package community information triple, P i Represents software packages A and C i Represents the community to which software package A belongs. Belong represents belonging. The level determination unit is used to determine the level of the community according to the software package dependency relationship. The initial level of all communities is set to 0. The community level information triplet is L i =(C i ,Level,0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if Pi and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software Package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on, after traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k}, and the demand information triple set Q={Q0, Q1, ..., Q k}, where L i Represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated hierarchical information triplet; L represents the set of all updated hierarchical information triples, Level represents the level, and Require represents the requirement.

[0113] Preferably, see Figure 1 In the parallel construction module of the software warehouse classified by the community provided in this embodiment, the software warehouse is represented by G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E allRepresents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; then sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are constructed in parallel from high to low hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

[0114] The method and system for parallel construction of software warehouses based on community classification provided in this embodiment, compared with the existing technology, extracts software package dependencies from software packages and establishes a software package dependency graph between software packages based on the software package dependencies; clusters and divides the software package dependency graph based on the relationship graph community classification algorithm, and determines the hierarchy of communities according to the software package dependencies; and constructs software warehouses in parallel based on the communities after the hierarchy is determined. The method and system for parallel construction of software warehouses based on community classification provided in this embodiment proposes a method for automatically extracting software package dependency information by parsing software package list files, which can obtain software package dependencies without installing software packages, thereby improving data processing efficiency; storing software package dependencies through knowledge graphs can quickly process the huge amount of data and complex software package relationships in software warehouses, thereby improving the ability of graph algorithm analysis; performing software division through community classification algorithms can effectively guide the parallel construction of software warehouses, thereby improving the efficiency of software package construction. In summary, this embodiment divides software package communities based on software package dependency information, realizes the parallel construction of software warehouses, provides support for the management of software warehouses, and has the advantages of convenient application and easy implementation.

[0115] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the invention. Thus, the present invention is intended to include such changes and modifications as fall within the scope of the claims and their equivalents.

Claims

1. A method for parallel construction of software warehouses based on community classification, characterized in that: The following steps are involved: Extracting software package dependencies from the software packages, and establishing a software package dependency graph between the software packages based on the software package dependencies; Clustering the software package dependency graph based on a relationship graph community classification algorithm, and determining the level of the community according to the software package dependency; Based on the hierarchical communities, software repositories are built in parallel; The step of clustering the software package dependency graph based on a relationship graph community classification algorithm and determining the level of the community according to the software package dependency comprises: Based on the software package dependency graph G, subgraph is divided, and subgraph g G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k } represents the division of the graph G into a set of k communities, where g represents the subgraph, G represents the software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community in the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the software package community information triple, P i Represents software packages A and C i Indicates the community to which software package A belongs. Belong means belongs to. The community level is determined according to the software package dependency relationship, and the initial level of all communities is set to 0. The community level information triplet is L i =(C i ,Level,0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software Package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on, after traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k }, and the demand information triple set Q={Q0, Q1, ..., Q k }, where L i Represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated hierarchical information triplet; L represents the set of all updated hierarchical information triples, Level represents the level, and Require represents the requirement.

2. The method for parallel construction of software warehouses based on community classification according to claim 1, characterized in that: The steps of extracting software package dependencies from the software packages and establishing a software package dependency graph between the software packages based on the software package dependencies include: Building a software warehouse based on the software package list file, wherein the software warehouse records software package version information and software package dependencies; Extracting a software package version information triple from the software package version information; extracting a software package dependency triple from the software package dependency; The software package version information triplet is expressed by the following formula: N i =(P i ,Version,V i ) Among them, N i Represents the software package version information triplet, P i Represents software packages A, V i Represents the version of software package A; Version represents the version; The software package dependency triple is expressed by the following formula: R ij =(P i ,Depends,P j ) Among them, R ij Represents the package dependency triple, P i Represents software package A, P j represents the dependent software package B, Depends represents the dependency relationship, and the software package dependency triplet indicates that software package A depends on software package B.

3. The method for parallel construction of software warehouses based on community classification according to claim 2, characterized in that: In the steps of extracting software package dependencies from the software packages and establishing a software package dependency graph between the software packages based on the software package dependencies, the software packages form a software package dependency graph through the software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by the software package version information triples, and the dependency edges are established by the dependency triples. If software package A and software package B are identified, a directed edge is established from software package A to software package B. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

4. The method for parallel construction of software warehouses based on community classification according to claim 1, characterized in that: In the step of constructing the software warehouse in parallel based on the community determined by the hierarchy, the software warehouse is represented by G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E all Represents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; then sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are constructed in parallel from high to low hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

5. A software warehouse parallel construction system based on community classification, characterized by: include: Establishing a module for extracting software package dependencies from software packages and establishing a software package dependency graph between software packages based on the software package dependencies; A partitioning module, configured to cluster the software package dependency graph based on a relationship graph community classification algorithm, and determine the level of the community according to the software package dependency; A parallel build module is used to build software repositories in parallel based on the communities determined by the hierarchy; Wherein, the division module includes: A partitioning unit is used to partition the software package dependency graph G into subgraphs, wherein the subgraph g G, the division is based on the community classification algorithm, using C={C1, C2, ..., C k } represents the division of the graph G into a set of k communities, where g represents the subgraph, G represents the software package dependency graph, and C represents the division of the software package dependency graph G into a set of k communities, C k is the kth community in the community division; after the software package is divided into communities, the software package community information triple B is established i =(P i , Belong, C i ), where B i Represents the software package community information triple, P i Represents software packages A and C i Indicates the community to which software package A belongs. Belong means belongs to. The level determination unit is used to determine the level of the community according to the software package dependency relationship, and the initial level of all communities is set to 0. The community level information triplet is L i =(C i ,Level,0), and at the same time establish the community demand information triple as Q i =(C i ,Require,0); each software package P i Community C i and the dependent packages nodeP j Conduct community C j Compare, if P i and P j belong to the same community, the community level information does not change; if P i and P j If they do not belong to the same community, the dependent software package node P j Community C j The level of community C is increased by 1. j The level information triplet is updated to L j =(C j ,Level,1),Software Package P i Community C i The demand of community C increases by 1. i The demand information triplet is updated to Q i =(C i ,Require,1), and so on, after traversing all the software package dependencies, we get the set of all updated level information triples L={L0, L1, ..., L k }, and the demand information triple set Q={Q0, Q1, ..., Q k }, where L i Represents the community-level information triple, Q i Represents the community demand information triple; P j Represents the software package node, L k represents the kth community level of the updated community level; L j Represents the updated hierarchical information triplet; L represents the set of all updated hierarchical information triples, Level represents the level, and Require represents the requirement.

6. The software warehouse parallel construction system based on community classification according to claim 5, characterized in that: The establishment module includes: A construction unit, configured to construct a software warehouse based on the software package list file, wherein the software warehouse records software package version information and software package dependencies; An extraction unit, configured to extract a software package version information triple from the software package version information; and extract a software package dependency triple from the software package dependency relationship; The software package version information triplet is expressed by the following formula: N i =(P i ,Version,V i ) Among them, N i Represents the software package version information triplet, P i Represents software packages A, V i Represents the version of software package A; Version represents the version; The software package dependency triple is expressed by the following formula: R ij =(P i ,Depends,P j ) Among them, R ij Represents the package dependency triple, P i Represents software package A, P j represents the dependent software package B, Depends represents the dependency relationship, and the software package dependency triplet indicates that software package A depends on software package B.

7. The software warehouse parallel construction system based on community classification according to claim 6, characterized in that: In the establishment module, software packages form a software package dependency graph through the software package dependencies. The software package dependency graph includes software package nodes and dependency edges. The software package nodes are established by the software package version information triples, and the dependency edges are established by the dependency triples. If software package A and software package B are identified, a directed edge is established from software package A to software package B. This operation is performed for all dependencies to form a software package dependency graph G=(V, E), where G represents the software package dependency graph, V is a set of software package nodes, and E is a set of dependency edges.

8. The software warehouse parallel construction system based on community classification according to claim 5, characterized in that: In the parallel building module, the software warehouse is represented by G repo =(V all , E all ), where G repo Represents the software warehouse, V all Represents the set of all packages that need to be built, E all Represents the set of all dependencies in the construction process; sort communities in descending order according to hierarchical attributes. Communities with higher hierarchical attributes have greater dependency weights, while communities with lower hierarchical attributes have smaller dependency weights; then sort communities in descending order according to demand attributes. Communities with higher demand attributes require more dependencies, while communities with lower demand attributes require fewer dependencies; if the demand attribute is 0, it means that the community does not need to rely on other communities, and communities with demand attributes equal to 0 are prioritized for parallel construction; secondly, communities are constructed in parallel from high to low hierarchical attributes. Communities with higher hierarchical attributes affect more communities.

Images