A processing method and device of an open source component, an electronic device, and a storage medium

Abstract

The application relates to the computer technology field, in particular to a processing method and device of an open source component, electronic equipment and a storage medium. The method comprises the following steps: receiving an open source project to be processed; the open source project comprises a plurality of open source components; core developer determination processing is performed on the plurality of open source components, and a core developer corresponding to each open source component is determined; based on a preset webpage ranking Pagerank algorithm and the core developer corresponding to each open source component, analysis component centrality determination processing is performed on each open source component, and a target core open source component is obtained; the analysis component centrality determination processing is used for comprehensively scoring the core developer corresponding to the open source component and a dependency relationship, and the target core open source component meeting a condition in comprehensive scoring is screened out. In this way, through refined analysis processing on the core developer in each open source component and the dependency relationship, the target core open source component of the open source project can be accurately determined.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, apparatus, electronic device and storage medium for processing open source components. Background Technology

[0002] Components that are widely used in open source components, i.e., highly centralized open source components, can be called core open source components, such as JUnit, Requests, and Mocha.

[0003] Core open-source components are the most important part of the entire open-source project. If the core open-source components cannot be accurately and efficiently identified, it may lead to inadequate maintenance of the core open-source components, which will have an impact on the entire project.

[0004] Therefore, how to accurately and efficiently identify the core open-source components has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, electronic device, and storage medium for processing open-source components, which can be used to provide a scheme for determining core open-source components.

[0006] In a first aspect, embodiments of the present invention provide a method for processing open-source components, the method comprising:

[0007] Receive open-source projects to be processed; the open-source projects include multiple open-source components;

[0008] The core developer determination process is performed on the multiple open-source components to identify the core developer corresponding to each open-source component.

[0009] Based on the preset PageRank algorithm for webpage ranking and the core developers corresponding to each open source component, the component centrality determination process is performed on each open source component to obtain the target core open source component; the component centrality determination process is used to comprehensively score the core developers and dependencies corresponding to the open source component, and to filter out the target core open source components that meet the comprehensive score conditions.

[0010] In one possible implementation, a core developer determination process is performed on the plurality of open-source components to determine the core developer corresponding to each open-source component, including:

[0011] For each open-source component, perform the following operations:

[0012] Clean the open-source component to obtain at least one source file corresponding to the open-source component;

[0013] Analyze the historical commit records of the open-source component to obtain information on all developers of the open-source component;

[0014] Based on the preset developer contribution rules, the component update logs corresponding to the open source components, the at least one source file, and all developer information, determine the file-level contribution of each developer to the at least one source file;

[0015] The core developers corresponding to the open-source component are determined based on each developer's file-level contribution to at least one source file; the core developers corresponding to the open-source component have the highest contribution to the creation and updating of multiple source files in the open-source component.

[0016] In one possible implementation, the method further includes:

[0017] Identify the data pairs in the historical submission record; each data pair includes email information and user identification information;

[0018] If the email information in the first data pair and the second data pair is the same, it is determined that the first data pair and the second data pair point to the same developer. The first data pair and the second data pair are then merged into one data pair as a single developer information entry.

[0019] If the Lewinstein distance between the third data pair and the fourth data pair is less than the first threshold, it is determined that the third data pair and the fourth data pair point to the same developer, and the third data pair and the fourth data pair are merged into one data pair as a developer information entry.

[0020] In one possible implementation, the preset developer contribution rule is obtained based on the following formula:

[0021] DA(m d f p )=α+β×FA(m d f p )+γ×DL(m d ,f p )-δ×ln(1+AC(m d f p ))

[0022] Where, m d Used to characterize developers, f p Used to characterize the source file, α, β, γ, and δ are used to characterize the weights, and FA is used to characterize the developer m. d Create source file f p The contribution of DA is used to characterize the developer m d For source file f pThe contribution of developers is used by deep learning to characterize the contribution of developers m. d For source file f p The number of updates, AC is used to represent the number of updates except for developer m d Other developers besides f have access to the source file. p Total number of updates.

[0023] In one possible implementation, the core developers corresponding to the open-source component are determined based on each developer's file-level contribution to the at least one source file, including:

[0024] The developer with the highest file-level contribution among all developers corresponding to each source file will be designated as the sub-core developer for each source file.

[0025] The sub-core developer with the most corresponding source files will be designated as the core developer of the open-source component.

[0026] In one possible implementation, based on a preset PageRank algorithm and the core developers corresponding to each open-source component, a component centrality determination process is performed on each open-source component to obtain the target core open-source component, including:

[0027] Dependency data is obtained from the package manager, and a mesh dependency graph is constructed based on the dependency data between open-source components; the package manager stores the dependency data of each open-source component, and the dependency data is used to indicate the dependency relationship between open-source components;

[0028] Based on the network dependency graph, remove all open-source components in the open-source project that have never been called by other components to obtain candidate open-source components; and calculate the score of each candidate open-source component based on the PageRank algorithm.

[0029] The target core open source component is determined based on the candidate open source components whose score value is greater than the second threshold.

[0030] In one possible implementation, the target core open-source component is determined based on candidate open-source components whose score values ​​are greater than a second threshold, including:

[0031] The core factor and score value corresponding to each candidate open source component whose score value is greater than the second threshold are determined respectively; wherein, the core factor is determined based on the number of sub-core developers whose corresponding source file count is greater than the third threshold;

[0032] If it is determined that the core factor corresponding to the first candidate open source component is less than the fourth threshold, and it is determined that the score value corresponding to the first candidate open source component is greater than the fifth threshold, then the first candidate open source component is taken as the target core open source component.

[0033] Secondly, embodiments of the present invention provide a processing apparatus for open-source components, the apparatus comprising:

[0034] A receiving unit is used to receive open-source projects to be processed; the open-source projects include multiple open-source components.

[0035] The determining unit is used to perform core developer determination processing on the multiple open source components, and to determine the core developer corresponding to each open source component.

[0036] The obtaining unit is used to perform component centrality determination processing on each open source component based on the preset PageRank algorithm and the core developers corresponding to each open source component to obtain the target core open source component; the component centrality determination processing is used to comprehensively score the core developers and dependencies corresponding to the open source component, and filter out the target core open source components that meet the comprehensive score conditions.

[0037] In one possible implementation, the determining unit is specifically used for:

[0038] For each open-source component, perform the following operations:

[0039] Clean the open-source component to obtain at least one source file corresponding to the open-source component;

[0040] Analyze the historical commit records of the open-source component to obtain information on all developers of the open-source component;

[0041] Based on the preset developer contribution rules, the component update logs corresponding to the open source components, the at least one source file, and all developer information, determine the file-level contribution of each developer to the at least one source file;

[0042] The core developers corresponding to the open-source component are determined based on each developer's file-level contribution to at least one source file; the core developers corresponding to the open-source component have the highest contribution to the creation and updating of multiple source files in the open-source component.

[0043] In one possible implementation, the apparatus further includes a processing unit for:

[0044] Identify the data pairs in the historical submission record; each data pair includes email information and user identification information;

[0045] If the email information in the first data pair and the second data pair is the same, it is determined that the first data pair and the second data pair point to the same developer. The first data pair and the second data pair are then merged into one data pair as a single developer information entry.

[0046] If the Lewinstein distance between the third data pair and the fourth data pair is less than the first threshold, it is determined that the third data pair and the fourth data pair point to the same developer, and the third data pair and the fourth data pair are merged into one data pair as a developer information entry.

[0047] In one possible implementation, the preset developer contribution rule is obtained based on the following formula:

[0048] DA(m d f p )=α+β×FA(m d f p )+γ×DL(m d f p )-δ×ln(1+AC(m d f p ))

[0049] Where, m d Used to characterize developers, f p Used to characterize the source file, α, β, γ, and δ are used to characterize the weights, and FA is used to characterize the developer m. d Create source file f p The contribution of DA is used to characterize the developer m d For source file f p The contribution of developers is used by deep learning to characterize the contribution of developers m. d For source file f p The number of updates, AC is used to represent the number of updates except for developer m d Other developers besides f have access to the source file. p Total number of updates.

[0050] In one possible implementation, the determining unit is specifically used for:

[0051] The developer with the highest file-level contribution among all developers corresponding to each source file will be designated as the sub-core developer for each source file.

[0052] The sub-core developer with the most corresponding source files will be designated as the core developer of the open-source component.

[0053] In one possible implementation, the obtaining unit is specifically used for:

[0054] Dependency data is obtained from the package manager, and a mesh dependency graph is constructed based on the dependency data between open-source components; the package manager stores the dependency data of each open-source component, and the dependency data is used to indicate the dependency relationship between open-source components;

[0055] Based on the network dependency graph, delete all open-source components in the open-source project that have never been called by other components to obtain candidate open-source components; and calculate the score of each candidate open-source component based on the PageRank algorithm.

[0056] The target core open source component is determined based on the candidate open source components whose score value is greater than the second threshold.

[0057] In one possible implementation, the obtaining unit is further configured to:

[0058] The core factor and score value corresponding to each candidate open source component whose score value is greater than the second threshold are determined respectively; wherein, the core factor is determined based on the number of sub-core developers whose corresponding source file count is greater than the third threshold;

[0059] If it is determined that the core factor corresponding to the first candidate open source component is less than the fourth threshold, and it is determined that the score value corresponding to the first candidate open source component is greater than the fifth threshold, then the first candidate open source component is taken as the target core open source component.

[0060] Thirdly, embodiments of the present invention provide an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any method provided by the first aspect of the present invention.

[0061] Fourthly, embodiments of the present invention provide a computer storage medium, wherein the computer-readable storage medium stores a computer program that causes a computer to perform any of the methods provided in the first aspect of the present invention.

[0062] Fifthly, embodiments of the present invention provide a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform any of the methods provided in the first aspect of the embodiments.

[0063] The beneficial effects of this invention are as follows:

[0064] In this embodiment of the invention, an open-source project to be processed can be received; the open-source project includes multiple open-source components; core developer determination processing is performed on the multiple open-source components to determine the core developers corresponding to each open-source component; based on a preset PageRank algorithm and the core developers corresponding to each open-source component, component centrality determination processing is performed on each open-source component to obtain target core open-source components; the component centrality determination processing is used to comprehensively score the core developers and dependencies corresponding to the open-source components, and to filter out target core open-source components that meet the comprehensive score conditions. It can be seen that by comprehensively scoring the core developers and dependencies corresponding to the open-source components, target core open-source components with high centrality and a low number of core developers can be filtered out, improving the accuracy of determining core open-source components.

[0065] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0066] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a schematic diagram of an application scenario in an embodiment of the present invention;

[0068] Figure 2 This is a schematic diagram of a processing method for an open-source component according to an embodiment of the present invention;

[0069] Figure 3 This is a schematic diagram of the composition structure of a processing device for an open-source component according to an embodiment of the present invention;

[0070] Figure 4 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. Unless otherwise specified, the embodiments and features in the embodiments of this invention can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0072] The term "comprising" and any variations thereof in the specification and claims of this invention are intended to cover non-exclusive protection. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0073] In this embodiment of the invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " is generally used to indicate that the preceding and following related objects have an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be used to represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0074] Furthermore, unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of the present invention are used to distinguish multiple objects and are not used to limit the size, content, order, timing, priority, or importance of multiple objects. For example, "first source file" and "second source file" are only used to distinguish source files and are not used to characterize the differences in content, size, priority, or importance of the two source files.

[0075] In the technical solution of this invention, the collection, dissemination, and use of data all comply with the requirements of relevant national laws and regulations.

[0076] As mentioned earlier, core open-source components are the most crucial element of the entire open-source project. Failure to accurately and efficiently identify these components could lead to inadequate maintenance and negatively impact the entire project. Therefore, accurately and efficiently identifying core open-source components has become a pressing issue that needs to be addressed.

[0077] In view of this, the present invention provides a method for processing open-source components. This method receives an open-source project to be processed; the open-source project includes multiple open-source components; it performs core developer determination processing on the multiple open-source components to identify the core developers corresponding to each open-source component; based on a preset PageRank algorithm and the core developers corresponding to each open-source component, it performs component centrality determination processing on each open-source component to obtain target core open-source components; the component centrality determination processing is used to comprehensively score the core developers and dependencies corresponding to the open-source components, and to filter out target core open-source components that meet the comprehensive score criteria. Therefore, by comprehensively scoring the core developers and dependencies corresponding to the open-source components, target core open-source components with high centrality and a low number of core developers can be filtered out, improving the accuracy of identifying core open-source components.

[0078] After introducing the design concept of the embodiments of the present invention, the following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of the present invention are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of the present invention and are not intended to limit it. In specific implementation, the technical solutions provided by the embodiments of the present invention can be flexibly applied according to actual needs.

[0079] The solutions provided in this invention can be applied to various real-world business projects built on open-source components.

[0080] See Figure 1 The diagram shown is a scenario illustration provided by an embodiment of the present invention. The scenario may include multiple terminal devices 101 and a server 102. A processing platform may be deployed on terminal devices 101-1, 101-2, ..., 101-n, and each terminal device is equipped with its own processing platform.

[0081] In this embodiment of the invention, when business personnel using terminal device 101 wish to know the core developers of an open-source project in order to better maintain the project, they can send the open-source project to be processed to server 102 through the processing platform deployed on terminal device 101. The open-source project includes multiple open-source components, and each component includes multiple source files. Then, the processing platform of the open-source components deployed on server 102 processes the open-source project to be processed, identifies the corresponding core developers, and provides feedback to terminal device 101. This allows business personnel to better formulate maintenance arrangements and plans for the open-source project, avoiding losses to the open-source project and related business operations due to the core developers' inability to maintain it.

[0082] In this embodiment of the invention, the terminal device 101 may be, for example, a mobile phone, a tablet computer (PAD), a personal computer (PC), a smart TV, a smart vehicle device, or a wearable device, etc., and this embodiment of the invention does not limit the device.

[0083] In this embodiment of the invention, server 102 may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, or it may be a physical server, but it is not limited to these.

[0084] In this embodiment, terminal device 101 and server 102, as well as various terminal devices 101, can communicate directly or indirectly through one or more networks 103. The network 103 can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network, or other possible networks. This embodiment of the invention does not limit the types of networks that can be used.

[0085] Of course, the method provided in the embodiments of the present invention is not limited to... Figure 1 The application scenarios shown can also be used in other possible scenarios, such as applications involving interaction between multiple terminal devices and multiple servers. This embodiment of the invention does not impose any limitations. Figure 1 The functions that each device in the application scenario shown can achieve will be described in the subsequent method embodiments, and will not be elaborated on here.

[0086] To further illustrate the technical solutions provided by the embodiments of the present invention, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present invention provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided by the embodiments of the present invention. In actual processing or when the device executes the method, it may be executed sequentially or in parallel according to the method shown in the embodiments or drawings.

[0087] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating a processing method for an open-source component according to an embodiment of the present invention. The method can be executed by an electronic device, which can be... Figure 1 The specific implementation process of this method on the server is as follows:

[0088] Step 201: Receive the open source project to be processed; the open source project includes multiple open source components.

[0089] Step 202: Perform core developer determination processing on multiple open source components to identify the core developers corresponding to each open source component.

[0090] In this embodiment of the invention, after the electronic device receives the open source project to be processed, it can perform core developer determination processing on the multiple open source components included in the open source project, and determine the core developer corresponding to each open source component.

[0091] In this embodiment of the invention, the electronic device may perform the following operations for each open-source component:

[0092] Step a: Clean the open-source component to obtain at least one source file corresponding to the open-source component.

[0093] In this embodiment of the invention, the electronic device can use tools such as software component analysis and code source tracing analysis to clean up all third-party libraries, code, frameworks, etc. in the open source component, and retain only the source files completed by the developer of the open source component, thereby obtaining at least one source file corresponding to the open source component.

[0094] Step b: Analyze the historical commit records of the open-source component to obtain all developer information for the open-source component.

[0095] For example, an open-source project is a software project hosting platform (GitHub). By accessing the historical commit records of an open-source project, one can obtain the historical commit records of an open-source component, thereby gaining access to all developer information for that open-source component.

[0096] In one possible implementation, to filter out duplicate developers, duplicate developer information can be identified using, but is not limited to, the following methods, and the duplicate developer information can be merged into a single developer profile. This avoids situations where a developer is identified as multiple developers due to using different accounts to create open-source components, thereby reducing the accuracy of determining the developer's core role in the open-source project and improving the accuracy of identifying core developers.

[0097] Method 1:

[0098] In this embodiment of the invention, the electronic device can identify each data pair in the historical submission record; each data pair includes email information and user identification information; if the email information in the first data pair and the second data pair is the same, it is determined that the first data pair and the second data pair point to the same developer, and the first data pair and the second data pair are merged into one data pair as a developer information entry.

[0099] Method 2:

[0100] In this embodiment of the invention, the electronic device can identify each data pair in the historical submission record; each data pair includes email information and user identification information; if the Lewinstein distance between the third data pair and the fourth data pair is less than a first threshold, it is determined that the third data pair and the fourth data pair point to the same developer, and the third data pair and the fourth data pair are merged into one data pair as a developer information entry. The first threshold is, for example, 4, but it can also be other values, and this embodiment of the invention does not impose any restrictions on it.

[0101] The Levenshtein distance can be determined using the following formula:

[0102]

[0103] in, This is an indicator function; its value is 1 when the i-th character of the third data pair 'a' and the j-th character of the fourth data pair 'b' are different; otherwise, it is 0.

[0104] In other words, in this embodiment of the invention, the email information or user identification information in the data pair can be identified separately. If any one of them is repeated, the two data pairs can be considered to be repeated. Alternatively, if the Lewinstein distance is less than a first threshold, the two data pairs are considered to have a high similarity due to their small distance. Therefore, the two data pairs are considered to be from the same developer and are considered to be repeated data pairs. This allows the data pairs considered to be repeated to be merged into one data pair, thereby improving the accuracy of identifying the core developer.

[0105] Step c: Based on the preset developer contribution rules, the component update logs corresponding to the open source component, at least one source file, and all developer information, determine the file-level contribution of each developer to at least one source file.

[0106] In this embodiment of the invention, the preset developer contribution rule is obtained based on the following formula:

[0107] DA(m d f p )=α+β×FA(m d f p )+γ×DL(m d f p )-δ×ln(1+AC(m d f p ))

[0108] Where, m d Used to characterize developers, f p Used to characterize the source file, α, β, γ, and δ are used to characterize the weights, and FA is used to characterize the developer m. d Create source file f p The contribution of DA is used to characterize the developer m d For source file f p The contribution of developers is used by deep learning to characterize the contribution of developers m. d For source file f p The number of updates, AC is used to represent the number of updates except for developer m d Other developers besides f have access to the source file. p The total number of updates. Optionally, α = 3.3, β = 1.0, γ = 0.2, δ = 0.3 can be set.

[0109] Step d: Determine the core developers corresponding to the open source component based on each developer's file-level contribution to at least one source file; the core developers corresponding to the open source component have the highest contribution to the creation and updating of multiple source files in the open source component.

[0110] In this embodiment of the invention, the electronic device can select the developer with the highest file-level contribution among all developers corresponding to each source file as the sub-core developer corresponding to each source file; and select the sub-core developer with the largest number of corresponding source files as the core developer corresponding to the open source component.

[0111] Step 203: Based on the preset PageRank algorithm and the core developers corresponding to each open source component, perform component centrality determination processing on each open source component to obtain the target core open source component; the component centrality determination processing is used to comprehensively score the core developers and dependencies corresponding to the open source component, and filter out the target core open source components that meet the comprehensive score conditions.

[0112] In this embodiment of the invention, the electronic device can obtain dependency data from a package manager and construct a mesh dependency graph between open-source components based on the dependency data. The package manager stores dependency data for each open-source component, which indicates the dependency relationships between open-source components. The package manager can be, for example, Maven, Npm, or PyPI. Optionally, the electronic device can construct "component-dependency-component" triples between open-source components, as well as components and their related attributes, based on the dependency data. Components are interconnected through dependency relationships, forming a mesh dependency graph of the software supply chain for each language. Further, based on the mesh dependency graph, all open-source components that have never been called by other components are deleted to obtain candidate open-source components. Then, based on the PageRank algorithm, the score value of each candidate open-source component is calculated; based on the candidate open-source components whose score value is greater than a second threshold, the target core open-source component is determined.

[0113] Based on the pre-defined PageRank algorithm, the component centrality of open-source component A is mainly determined by the following formula:

[0114]

[0115] Here, PR() represents the centrality of a certain open-source component, T1...T n is a component that depends on component A; d is a damping coefficient, which is generally set between 0 and 1, and is usually set to 0.85; C() represents the total number of other components that a certain open source component depends on.

[0116] In practice, the Neo4j database can be used to store the network dependency graph of open-source components. The centrality score of each open-source component can be obtained by directly calling the implementation of the PageRank algorithm in the Neo4j Graph Data Science Library, which is the aforementioned score value.

[0117] In this embodiment of the invention, the electronic device can determine the core factor and score corresponding to each candidate open-source component whose score is greater than a second threshold. The core factor is determined based on the number of sub-core developers whose corresponding source files exceed a third threshold. If the core factor corresponding to the first candidate open-source component is determined to be less than a fourth threshold, and the score corresponding to the first candidate open-source component is determined to be greater than a fifth threshold, the first candidate open-source component is designated as the target core open-source component. The fourth and fifth thresholds are determined based on the distribution of the core factors and score values ​​of each open-source component.

[0118] For example, taking JUnit, the most commonly used unit testing framework component in Java, as an example, JUnit's score is 2974, which is much greater than the fifth threshold of 10. Furthermore, JUnit's core factor is 3, which is less than the fourth threshold of 4. Therefore, JUnit is determined to be a core open-source component.

[0119] Optionally, the electronic device can employ a preset greedy algorithm to evaluate the number of core developers of the open-source component, thereby determining a core factor, such as a Truck factor. The preset greedy algorithm can be understood as follows: after the electronic device obtains the core developers of each source file, it can count the number of times each core developer is identified as a core developer of the source file. Based on this count, the ranking order of the core developers is determined. Further, the electronic device can determine a set of source files, such as a coverage set, and can delete the source files corresponding to the developers with the most counts. Specifically, it can iterate through all source files in the current open-source component. If a source file no longer has core developers after deleting that developer, then that source file is removed from the coverage set. The above steps continue until the number of source files in the coverage set is less than a third threshold, at which point the above steps stop, and the number of currently deleted developers is used as the core factor. The third threshold can be, for example, 0.5, or other values; this is not limited in this embodiment of the invention.

[0120] For example, suppose an open-source component includes 6 source files: source file A, source file B, source file C, source file D, source file E, and source file F. The core developer of source files A, C, and D is developer 1, the core developer of source file B is developer 2, and the core developer of source files E and F is developer 3. Then we can determine that developer 1 is a core developer 3 times, developer 2 is a core developer 1 time, and developer 3 is a core developer 2 times. Therefore, we can determine that the sorting order of developer 1 is 1, the sorting order of developer 3 is 2, and the sorting order of developer 2 is 3.

[0121] Furthermore, it can be determined that the coverage set includes source file A, source file B, source file C, source file D, source file E, and source file F. Assuming the third threshold is 2, first delete the source file corresponding to developer 1. At this time, the coverage set only includes 3 source files. Then, the source file corresponding to developer 3 can be deleted. At this time, the coverage set only includes 1 source file, which is less than the second threshold. Therefore, it can be determined to stop the aforementioned step of deleting the source files corresponding to the developers. That is to say, developer 1 and developer 3 have been deleted, that is, the number of developers deleted is 2. Therefore, 2 can be used as the Truck factor.

[0122] As can be seen, the embodiments of the present invention provide a complete evaluation method for determining core open-source components, thereby enabling more accurate and efficient determination of the core open-source components corresponding to each open-source project, as well as the core developers corresponding to the core open-source components, providing a good implementation foundation for the subsequent maintenance of core open-source components.

[0123] Based on the same inventive concept, embodiments of the present invention also provide a processing device for open-source components. For example... Figure 3 As shown, this is a schematic diagram of the structure of the processing device 300 for open-source components, which may include:

[0124] Receiving unit 301 is used to receive an open source project to be processed; the open source project includes multiple open source components;

[0125] The determining unit 302 is used to perform core developer determination processing on the plurality of open source components, and to determine the core developer corresponding to each of the open source components respectively.

[0126] The obtaining unit 303 is used to perform component centrality determination processing on each open source component based on the preset PageRank algorithm and the core developers corresponding to each open source component to obtain the target core open source component; the component centrality determination processing is used to comprehensively score the core developers and dependencies corresponding to the open source component, and filter out the target core open source components that meet the comprehensive score conditions.

[0127] In one possible implementation, the determining unit 302 is specifically used for:

[0128] For each open-source component, perform the following operations:

[0129] Clean the open-source component to obtain at least one source file corresponding to the open-source component;

[0130] Analyze the historical commit records of the open-source component to obtain information on all developers of the open-source component;

[0131] Based on the preset developer contribution rules, the component update logs corresponding to the open source components, the at least one source file, and all developer information, determine the file-level contribution of each developer to the at least one source file;

[0132] The core developers corresponding to the open-source component are determined based on each developer's file-level contribution to at least one source file; the core developers corresponding to the open-source component have the highest contribution to the creation and updating of multiple source files in the open-source component.

[0133] In one possible implementation, the apparatus further includes a processing unit for:

[0134] Identify the data pairs in the historical submission record; each data pair includes email information and user identification information;

[0135] If the email information in the first data pair and the second data pair is the same, it is determined that the first data pair and the second data pair point to the same developer. The first data pair and the second data pair are then merged into one data pair as a single developer information entry.

[0136] If the Lewinstein distance between the third data pair and the fourth data pair is less than the first threshold, it is determined that the third data pair and the fourth data pair point to the same developer, and the third data pair and the fourth data pair are merged into one data pair as a developer information entry.

[0137] In one possible implementation, the preset developer contribution rule is obtained based on the following formula:

[0138] DA(m d f p )=α+β×FA(m d f p )+γ×DL(m d ,f p )-δ×ln(1+AC(m d f p ))

[0139] Where, m d Used to characterize developers, f p Used to characterize the source file, α, β, γ, and δ are used to characterize the weights, and FA is used to characterize the developer m. d Create source file f p The contribution of DA is used to characterize the developer m d For source file f p The contribution of developers is used by deep learning to characterize the contribution of developers m. d For source file f p The number of updates, AC is used to represent the number of updates except for developer md Other developers besides f have access to the source file. p Total number of updates.

[0140] In one possible implementation, the determining unit 302 is specifically used for:

[0141] The developer with the highest file-level contribution among all developers corresponding to each source file will be designated as the sub-core developer for each source file.

[0142] The sub-core developer with the most corresponding source files will be designated as the core developer of the open-source component.

[0143] In one possible implementation, the obtaining unit 303 is specifically used for:

[0144] Dependency data is obtained from the package manager, and a mesh dependency graph is constructed based on the dependency data between open-source components; the package manager stores the dependency data of each open-source component, and the dependency data is used to indicate the dependency relationship between open-source components;

[0145] Based on the network dependency graph, remove all open-source components in the open-source project that have never been called by other components to obtain candidate open-source components; and calculate the score of each candidate open-source component based on the PageRank algorithm.

[0146] The target core open source component is determined based on the candidate open source components whose score value is greater than the second threshold.

[0147] In one possible implementation, the obtaining unit 303 is further configured to:

[0148] The core factor and score value corresponding to each candidate open source component whose score value is greater than the second threshold are determined respectively; wherein, the core factor is determined based on the number of sub-core developers whose corresponding source file count is greater than the third threshold;

[0149] If it is determined that the core factor corresponding to the first candidate open source component is less than the fourth threshold, and it is determined that the score value corresponding to the first candidate open source component is greater than the fifth threshold, then the first candidate open source component is taken as the target core open source component.

[0150] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this invention, the functions of each module (or unit) can be implemented in one or more software or hardware components.

[0151] Having introduced the processing method and apparatus for open-source components according to exemplary embodiments of the present invention, the electronic device according to another exemplary embodiment of the present invention will now be described.

[0152] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0153] Based on the same inventive concept as the above method embodiments, this invention also provides an electronic device. In this embodiment, the structure of the electronic device can be as follows: Figure 4 As shown, the electronic device is, for example, the aforementioned Figure 1 Server 102 in the middle, such as Figure 4 As shown, the electronic device in this embodiment of the invention includes at least one processor 401, a memory 402 connected to the at least one processor 401, and a communication interface 403. This embodiment of the invention does not limit the specific connection medium between the processor 401 and the memory 402. Figure 4 Taking the connection between processor 401 and memory 402 via system bus 400 as an example, the system bus 400 is... Figure 4 The thick lines are used for characterization. The connection methods between other components are for illustrative purposes only and are not intended to be limiting. The system bus 400 can be divided into address bus, data bus, control bus, etc., for ease of characterization. Figure 4 A single thick line is used for characterization, but it is not used to characterize a single bus or a single type of bus.

[0154] In this embodiment of the invention, the memory 402 stores instructions that can be executed by at least one processor 401. By executing the instructions stored in the memory 402, at least one processor 401 can perform the steps included in the aforementioned processing method of the open-source component.

[0155] The processor 401 is the control center of the electronic device. It can connect to various parts of the fault detection device through various interfaces and lines. By running or executing instructions stored in the memory 402 and calling data stored in the memory 402, it can realize various functions of the electronic device. Optionally, the processor 401 may include one or more processing units. The processor 401 may integrate an application processor and a modem processor. The processor 401 mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401. In some embodiments, the processor 401 and the memory 402 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0156] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0157] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 402 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 402 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In embodiments of the present invention, memory 402 may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0158] Communication interface 403 is a transmission interface that can be used for communication. Data can be received or sent through communication interface 403.

[0159] In addition, the electronic device also includes a basic input / output system (I / O system) 404 that helps transmit information between various devices within the electronic device, and a large-capacity storage device 408 for storing the operating system 405, application programs 406 and other program modules 407.

[0160] The basic input / output system 404 includes a display 409 for displaying information and an input device 410 for user input, such as a mouse or keyboard. Both the display 409 and the input device 410 are connected to the processor 401 via the basic input / output system 404 connected to the system bus 400. The basic input / output system 404 may also include an input / output controller for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller also provides output to a display screen, printer, or other types of output devices.

[0161] Specifically, mass storage device 408 is connected to processor 401 via a mass storage controller (not shown) connected to system bus 400. Mass storage device 408 and its associated computer-readable media provide non-volatile storage for the server package. That is, mass storage device 408 may include computer-readable media (not shown) such as hard disks or CD-ROM drives.

[0162] According to various embodiments of the present invention, the electronic device can also be connected to a remote computer on a network such as the Internet. That is, the electronic device can be connected to the network 411 via the communication interface 403 connected to the system bus 400, or the communication interface 403 can be used to connect to other types of networks or remote computer systems (not shown).

[0163] This invention also provides a computer storage medium, wherein the computer-readable storage medium stores a computer program, the computer program being used to cause a computer to execute the technical solution of the processing method of the open-source component described above.

[0164] This invention also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it enables the computer to implement the processing method of the open-source component described in the above embodiments.

[0165] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program instructions are computer programs, which can be stored in a computer-readable storage medium. When executed, the computer program performs the steps of the above method embodiments. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0166] The program product of embodiments of the present invention may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a computing device. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0167] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0168] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0169] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0170] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0171] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for processing open-source components, characterized in that, The method includes: Receive open-source projects to be processed; the open-source projects include multiple open-source components; The core developer determination process is performed on the multiple open-source components to identify the core developer corresponding to each open-source component. Based on the preset PageRank algorithm and the core developers corresponding to each open source component, the component centrality determination process is performed on each open source component to obtain the target core open source component; the component centrality determination process is used to comprehensively score the core developers and dependencies corresponding to the open source component, and to filter out the target core open source components that meet the comprehensive score conditions. The step of analyzing the component centrality of each open-source component based on the preset PageRank algorithm and the core developers corresponding to each open-source component to obtain the target core open-source component includes: Dependency data is obtained from the package manager, and a mesh dependency graph is constructed based on the dependency data between open-source components; the package manager stores the dependency data of each open-source component, and the dependency data is used to indicate the dependency relationship between open-source components; Based on the network dependency graph, remove all open-source components in the open-source project that have never been called by other components to obtain candidate open-source components; and calculate the score of each candidate open-source component based on the PageRank algorithm. The target core open-source component is determined based on the core factors and scores of the candidate open-source components whose scores are greater than the second threshold; wherein the core factors are determined based on the corresponding core developers.

2. The method as described in claim 1, characterized in that, The core developer determination process is performed on the multiple open-source components to identify the core developers corresponding to each open-source component, including: For each open-source component, perform the following operations: Clean the open-source component to obtain at least one source file corresponding to the open-source component; Analyze the historical commit records of the open-source component to obtain information on all developers of the open-source component; Based on the preset developer contribution rules, the component update logs corresponding to the open source components, the at least one source file, and all developer information, determine the file-level contribution of each developer to the at least one source file; The core developers corresponding to the open-source component are determined based on each developer's file-level contribution to at least one source file; the core developers corresponding to the open-source component have the highest contribution to the creation and updating of multiple source files in the open-source component.

3. The method as described in claim 2, characterized in that, The method further includes: Identify the data pairs in the historical submission record; each data pair includes email information and user identification information; If the email information in the first data pair and the second data pair is the same, it is determined that the first data pair and the second data pair point to the same developer. The first data pair and the second data pair are then merged into one data pair as a single developer information entry. If the Lewinstein distance between the third data pair and the fourth data pair is less than the first threshold, it is determined that the third data pair and the fourth data pair point to the same developer, and the third data pair and the fourth data pair are merged into one data pair as a developer information entry.

4. The method as described in claim 2 or 3, characterized in that, The preset developer contribution rule is obtained based on the following formula: in, Used to characterize developers, Used to characterize the source file. , , , Used to characterize weights, FA is used to characterize developers. Create source file The contribution of DA is used to characterize the developer's contribution. For the source file DL is used to characterize the developer's contribution. For the source file The number of updates, AC is used to represent the number of updates except for developers. Other developers besides [the developer] have access to the source files. Total number of updates.

5. The method as described in claim 2 or 3, characterized in that, Based on each developer's file-level contribution to the at least one source file, the core developers corresponding to the open-source component are determined, including: The developer with the highest file-level contribution among all developers corresponding to each source file will be designated as the sub-core developer for each source file. The sub-core developer with the most corresponding source files will be designated as the core developer of the open-source component.

6. The method as described in claim 5, characterized in that, Based on the core factors and scores corresponding to the candidate open-source components whose scores are greater than the second threshold selected from the candidate open-source components, the target core open-source components are determined, including: The core factor and score value corresponding to each candidate open source component whose score value is greater than the second threshold are determined respectively; wherein, the core factor is determined based on the number of sub-core developers whose corresponding source file count is greater than the third threshold; If it is determined that the core factor corresponding to the first candidate open source component is less than the fourth threshold, and it is determined that the score value corresponding to the first candidate open source component is greater than the fifth threshold, then the first candidate open source component is taken as the target core open source component.

7. A processing device for an open-source component, characterized in that, The device includes: A receiving unit is used to receive open-source projects to be processed; the open-source projects include multiple open-source components. The determining unit is used to perform core developer determination processing on the multiple open source components, and to determine the core developer corresponding to each open source component. The obtaining unit is used to perform component centrality determination processing on each open source component based on the preset PageRank algorithm and the core developers corresponding to each open source component to obtain the target core open source component; the component centrality determination processing is used to comprehensively score the core developers and dependencies corresponding to the open source component, and filter out the target core open source components that meet the comprehensive score conditions. The obtaining unit is specifically used to: obtain dependency data from the package manager, and construct a network dependency graph between open source components based on the dependency data; the package manager stores the dependency data of each open source component, and the dependency data is used to indicate the dependency relationship between open source components; Based on the network dependency graph, remove all open-source components in the open-source project that have never been called by other components to obtain candidate open-source components; and calculate the score of each candidate open-source component based on the PageRank algorithm. The target core open-source component is determined based on the core factors and scores of the candidate open-source components whose scores are greater than the second threshold; wherein the core factors are determined based on the corresponding core developers.

8. An electronic device, characterized in that, The method includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to implement the method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The computer storage medium stores a computer program that enables the computer to perform the method as described in any one of claims 1-6.

Images