Method and system for parsing third-party dependencies using package manager based on Maven architecture

Abstract

The present invention discloses a method and system for parsing third-party dependencies in a package manager based on the Maven architecture. The method comprises the following steps: using a dependency parsing command to parse the dependency relationships of various third-party components in a project; when an abnormality occurs and the current parsing command cannot continue to be executed, if the abnormality is a direct dependency, the direct dependency list can be removed so that the parsing work can continue to be executed; if the abnormality is an indirect dependency, each dependency list is first separated and parsed separately, and then the ones that fail to parse are removed, and the ones that succeed are merged for comprehensive parsing, so that the comprehensive parsing result and the removed direct dependency information are the desired parsing result. Through the above method, not only the correctness of the parsing result can be effectively ensured, but also when an abnormality occurs in the parsing process, timely response measures can be taken to the abnormality so that the parsing process can continue to be executed, thereby effectively solving the shortcomings of the dependency parsing command in the package manager.

Description

Technical Field

[0001] The present invention relates to the technical field of software project third-party dependency component parsing, and in particular to a method and system for parsing third-party dependencies using a package manager based on the Maven architecture. Background Art

[0002] Software project development is a systematic undertaking. Every software project uses several source code components. To effectively improve software development efficiency, most software development today uses readily available open source components. However, while this helps improve software development efficiency, open source components can also introduce more risks and vulnerabilities to software projects. Therefore, it is necessary to test the open source components, or third-party dependencies, in software projects.

[0003] Currently, Software Composition Analysis (SCA) tools are commonly used in the market to analyze the composition of third-party components in a project and display information such as vulnerabilities and licensing. How can one identify the third-party dependencies in a project or application, especially for projects using package managers, such as Maven and Gradle for Java and NPM for JavaScript? Third-party dependency information is typically obtained by executing package manager commands. However, directly using the package manager's own commands for parsing has many limitations and drawbacks. For example, if an unknown error occurs in a dependency during the execution of existing package managers' parsing commands, the entire parsing process will cease, the corresponding parsing results will not be displayed, and it will be difficult to find the erroneous dependency component. Furthermore, accuracy cannot be guaranteed when using other parsing methods other than package manager commands. Therefore, it is crucial to ensure that third-party dependencies are correctly parsed while addressing the limitations and drawbacks of command lines. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for parsing third-party dependencies using a package manager based on the Maven architecture, which can effectively ensure the correct parsing of third-party dependencies and solve the abnormal problems that occur in the existing command parsing process.

[0005] To achieve the above object, the present invention discloses a method for parsing third-party dependencies using a package manager based on the Maven architecture, which includes:

[0006] Parse the dependency list in the total pom file of the current project through the dependency parsing command in the package manager based on the Maven architecture, and monitor the parsing process in real time;

[0007] When an error message appears during the parsing process, determine whether the error message comes from a direct dependency. If so, remove the dependency list corresponding to the error message from the pom file; if not, separate all dependency lists in the total pom file under the current project and store them separately to generate several temporary pom files with single dependency lists.

[0008] Each temporary pom file is parsed separately by the dependency parsing command. If the parsing is successful, the parsing success information is recorded. If the parsing fails, the dependency list is removed;

[0009] Merge several temporary pom files that have been successfully parsed individually to generate a new comprehensive pom file;

[0010] Use the above dependency parsing command to comprehensively parse the comprehensive pom file and record the parsing results.

[0011] Preferably, during the monitoring of the parsing process, if the same piece of download information stays for more than a preset value, the execution of the current dependency parsing command is interrupted, and all dependency lists in the total pom file under the current project are separated and stored separately to generate several temporary pom files with single dependency lists;

[0012] Each temporary pom file is parsed separately by the dependency parsing command. If the parsing is successful, the parsing success information is recorded. If the parsing fails, the dependency list is removed;

[0013] Merge several temporary pom files that have been successfully parsed individually to generate a new comprehensive pom file;

[0014] Use the above dependency parsing command to comprehensively parse the comprehensive pom file and record the parsing results.

[0015] Preferably, the number of removed dependency lists is accumulated and recorded. If the number of removed dependency lists exceeds a preset value, the execution of the current dependency parsing command is interrupted, and the temporary pom files that have been successfully parsed individually are merged to generate a new comprehensive pom file.

[0016] Use the above dependency parsing command to comprehensively parse the comprehensive pom file and record the parsing results.

[0017] Preferably, the execution process of the dependency parsing command is monitored through a log file to record parsing process information.

[0018] The present invention also discloses a system for parsing third-party dependencies using a package manager based on the Maven architecture, which includes a parsing module, a monitoring module, a first processing module, a second processing module, and a merging module;

[0019] The parsing module is used to parse the dependency list in the pom file under the current project through the dependency parsing command in the package manager based on the Maven architecture;

[0020] The monitoring module is used to monitor the parsing process of the parsing module in real time;

[0021] The first processing module is configured to remove the dependency list corresponding to the error message from the pom file when an error occurs during the execution of the parsing module and the error message comes from a direct dependency;

[0022] The second processing module is used to separate all dependency lists in the total pom file of the current project and store them separately to generate several temporary pom files with single dependency lists when an error occurs during the execution of the parsing module and the error information comes from an indirect dependency;

[0023] The merging module is used to merge several temporary pom files that have been successfully parsed individually to generate a new comprehensive pom file.

[0024] Preferably, a third processing module is also included, which is used to interrupt the execution of the current dependency parsing command if the same download information stays for more than a preset value during the process of monitoring the parsing process through the monitoring module, and call the second processing module and the merging module to process the current project.

[0025] Preferably, it also includes a counting module, which is used to accumulate and record the number of removed dependency lists; if the number recorded by the counting module exceeds a preset value, the execution of the current dependency parsing command is interrupted, and the second processing module and the merging module are called to process the current project.

[0026] Preferably, the monitoring module monitors the execution process of the dependency parsing command through a log file to record parsing process information.

[0027] The present invention also discloses a system for parsing third-party dependencies using a package manager based on the Maven architecture, which specifically includes:

[0028] one or more processors;

[0029] Memory;

[0030] and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including instructions for executing the method for resolving third-party dependencies by a package manager based on the Maven architecture as described above.

[0031] The present invention also discloses a computer-readable storage medium, which includes a computer program. The computer program can be executed by a processor to implement the method for resolving third-party dependencies by a package manager based on the Maven architecture as described above.

[0032] Compared with the prior art, the method for parsing third-party dependencies by the package manager of the present invention uses the dependency parsing command in the package manager to parse the dependency relationships of various third-party components in the project. When an exception occurs during the parsing process and the current parsing command cannot continue to execute, if it is a direct dependency exception, the direct dependency list can be removed so that the parsing work can continue to execute. If an indirect dependency exception occurs, each dependency list is first separated and parsed separately, and then the ones that fail to parse are removed, and the ones that succeed in parsing are merged for comprehensive parsing. Then, the comprehensive parsing result and the removed direct dependency information are the desired parsing result. It can be seen that through the above method, not only the correctness of the parsing result can be effectively ensured, but also when an exception occurs in the parsing process, timely response measures can be taken to the exception, and the dependency information lists with exceptions can be removed one by one, so that the parsing process can continue to execute, thereby effectively solving the shortcomings of the dependency parsing command in the package manager. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flow chart of a method for a package manager to resolve third-party dependencies in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to explain the technical content, structural features, achieved objectives and effects of the present invention in detail, the following is a detailed description in conjunction with the embodiments and the accompanying drawings.

[0035] This embodiment discloses a method for parsing third-party dependencies using a package manager based on the Maven architecture, so as to parse third-party dependencies (open source components) in a project for software development based on the Maven architecture.

[0036] In the prior art, in order to ensure the correctness of third-party dependency parsing, the pom file in the project is generally parsed through the dependency parsing command of the package manager (mvn dependency:tree). The pom file records the information of third-party dependencies used in the current project, generally in the dependencies tag. Each dependency (hereinafter referred to as dy) in it is a dependency list, but it is only a direct dependency. This dy may also depend on other dependency information, which is also configured through the pom file. That is, in the root directory of a project, there is a total pom file, which records all direct dependency information under the project. In each module of the project, there is also a pom file that records indirect dependency information.

[0037] The execution process of the dependency parsing command is:

[0038] First, parse the first dy, then download the pom file and jar package of the indirect dependencies required by the first dy, and then traverse them in sequence, and finally parse all the dependencies of the first dy, and then parse the second dy in order to get all the dependency information of the second dy. Similarly, get all the dy dependency information, and then use the Maven package manager mechanism to remove duplicate dependencies (for example, the indirect dependency of the first dependency and the indirect dependency of the second dependency are the same but the versions are different. At this time, they will be deduplicated according to the priority and dependency hierarchy. This is a rule of Maven dependency management).

[0039] It is worth noting that each time an indirect dependency is parsed, the pom file and jar package of the indirect dependency will be downloaded from the warehouse. If the pom file and jar package cannot be downloaded, the command will be stuck. If it still cannot be downloaded after waiting for a few minutes, the entire command execution will fail, and the user will not get any parsing information, making the command parsing efficiency relatively low. In order to solve this problem, Figure 1 The method for the package manager to resolve third-party dependencies in this embodiment includes the following steps:

[0040] S1: Use the dependency parsing command in the package manager based on the Maven architecture to parse the dependency list in the total pom file of the current project, and monitor the parsing process in real time;

[0041] S2: When an error message appears during the parsing process, determine whether the error message comes from a direct dependency. If so, proceed to S3; if not, proceed to S4.

[0042] S3: Remove the dependency list corresponding to the error message from the pom file;

[0043] S4: Separate all dependency lists from the total pom file under the current project and store them separately to generate several temporary pom files with a single dependency list;

[0044] S5: parse each temporary pom file separately through the dependency parsing command. If the parsing is successful, the parsing success information is recorded. If the parsing fails, the dependency list is removed.

[0045] S6: Merge several temporary pom files that have been successfully parsed individually to generate a new comprehensive pom file;

[0046] S7: Use the above dependency parsing command to perform comprehensive parsing on the comprehensive pom file and record the parsing results.

[0047] By using the above-mentioned method of parsing third-party dependencies through the package manager, the dependency parsing command in the package manager is used to parse the dependency relationships of various third-party components in the project, thereby effectively ensuring the accuracy of the parsing results. When an abnormality occurs during the parsing process and the current parsing command cannot continue to execute, if it is a direct dependency exception, the direct dependency list can be removed so that the parsing work can continue to execute. If it is an indirect dependency exception, each dependency list is first separated and parsed separately, and then the ones that failed to parse are removed, and the ones that successfully parse are merged for comprehensive parsing. The comprehensive parsing result and the removed direct dependency information are the desired parsing result. Therefore, when an abnormality occurs during the parsing process, timely response measures can be taken to the abnormality, and the dependency information lists with abnormalities can be removed one by one so that the parsing process can continue to execute, thereby effectively solving the shortcomings of the dependency parsing command in the package manager.

[0048] During the command line execution process, under the premise that no error message appears, if the same download information stays for more than the preset value, it means that the pom package and jar package of the indirect dependency on which the current direct dependency depends cannot be downloaded, then the same countermeasures as when the indirect dependency exception occurs are executed, that is, the execution of the current dependency parsing command is interrupted, and all dependency lists in the total pom file under the current project are separated and stored separately to generate temporary pom files with several single dependency lists, and each temporary pom file is parsed separately by the dependency parsing command. If the parsing is successful, the parsing success information is recorded. If the parsing fails, the dependency list is removed. Then, the several temporary pom files that have been successfully parsed individually are merged to generate a new comprehensive pom file. Finally, the comprehensive pom file is comprehensively parsed using the above-mentioned dependency parsing command, and the parsing results are recorded.

[0049] Under normal circumstances, if a project's command line parsing fails with five compilations, it is definitely related to the environment, such as being unable to connect to the external network or incorrect configuration information, etc., and there is no need to spend time processing subsequent dependencies. Therefore, in another preferred embodiment of the method for parsing third-party dependencies by a package manager of the present invention, the number of removed dependency lists is cumulatively recorded. If the number of removed dependency lists exceeds a preset value, such as five, the execution of the current dependency parsing command is interrupted, and the temporary pom files that have been successfully parsed individually are merged to generate a new comprehensive pom file. The comprehensive pom file is then comprehensively parsed using the above-mentioned dependency parsing command, and the parsing results are recorded.

[0050] Furthermore, the execution process of the dependency parsing command is monitored through a log file to record parsing process information.

[0051] The present invention also discloses a system for parsing third-party dependencies using a package manager based on the Maven architecture, which specifically includes a parsing module, a monitoring module, a first processing module, a second processing module and a merging module.

[0052] The parsing module is used to parse the dependency list in the pom file under the current project through the dependency parsing command in the package manager based on the Maven architecture.

[0053] The monitoring module is used to monitor the parsing process of the parsing module in real time.

[0054] The first processing module is used to remove the dependency list corresponding to the error message from the pom file when an error occurs during the execution of the parsing module and the error message comes from a direct dependency.

[0055] The second processing module is used to separate all dependency lists in the total pom file under the current project and store them separately to generate several temporary pom files with single dependency lists when an error occurs during the execution of the parsing module and the error information comes from indirect dependencies.

[0056] The merge module is used to merge several temporary pom files that have been successfully parsed individually to generate a new comprehensive pom file.

[0057] Furthermore, the above-mentioned system also includes a third processing module, which is used to interrupt the execution of the current dependency parsing command if the same download information stays for more than a preset value during the process of monitoring the parsing process through the monitoring module, and call the second processing module and the merging module to process the current project.

[0058] Furthermore, the above system also includes a counting module, which is used to accumulate and record the number of dependency lists that are eliminated; if the number recorded by the counting module exceeds a preset value, the execution of the current dependency parsing command is interrupted, and the second processing module and the merging module are called to process the current project.

[0059] Furthermore, the monitoring module monitors the execution process of the dependency parsing command through a log file to record parsing process information.

[0060] The present invention also discloses another system for a package manager to resolve third-party dependencies, which includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the program includes instructions for executing the method for the package manager to resolve third-party dependencies as described above. The processor can adopt a general central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs to implement the functions required to be executed by the module in the system for the package manager to resolve third-party dependencies of the embodiment of the present application, or to execute the method for the package manager to resolve third-party dependencies of the embodiment of the method of the present application.

[0061] The present invention also discloses a computer-readable storage medium, which includes a computer program, and the computer program can be executed by a processor to complete the method of the package manager resolving third-party dependencies as described above. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a read-only memory (ROM), or a random access memory (RAM), or a magnetic medium, such as a floppy disk, a hard disk, a tape, a magnetic disk, or an optical medium, such as a digital versatile disc (DVD), or a semiconductor medium, such as a solid state disk (SSD).

[0062] The present application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the above-described method for resolving third-party dependencies by a package manager.

[0063] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the scope of the patent application of the present invention are still within the scope of the present invention.

Claims

1. A method for parsing third-party dependencies using a package manager based on the Maven architecture, characterized in that: include: Use the dependency parsing command in the Maven-based package manager to parse the dependency list in the total pom file of the current project and monitor the parsing process in real time; When an error message appears during the parsing process, determine whether the error message comes from a direct dependency. If so, remove the dependency list corresponding to the error message from the pom file; if not, separate all dependency lists in the total pom file under the current project and store them separately to generate several temporary pom files with single dependency lists. Each temporary pom file is parsed separately by the dependency parsing command. If the parsing is successful, the parsing success information is recorded. If the parsing fails, the dependency list is removed; Merge several temporary pom files that have been successfully parsed individually to generate a new comprehensive pom file; Use the above dependency parsing command to comprehensively parse the comprehensive pom file and record the parsing results.

2. The method for parsing third-party dependencies using a package manager based on the Maven architecture according to claim 1, wherein: During the monitoring of the parsing process, if the same download information stays for more than the preset value, the execution of the current dependency parsing command will be interrupted, and all dependency lists in the total pom file under the current project will be separated and stored separately to generate several temporary pom files with single dependency lists; Each temporary pom file is parsed separately by the dependency parsing command. If the parsing is successful, the parsing success information is recorded. If the parsing fails, the dependency list is removed; Merge several temporary pom files that have been successfully parsed individually to generate a new comprehensive pom file; Use the above dependency parsing command to comprehensively parse the comprehensive pom file and record the parsing results.

3. The method for parsing third-party dependencies using a package manager based on the Maven architecture according to claim 1, wherein: The number of removed dependency lists is accumulated and recorded. If the number of removed dependency lists exceeds the preset value, the execution of the current dependency parsing command is interrupted, and the temporary pom files that have been successfully parsed are merged to generate a new comprehensive pom file. Use the above dependency parsing command to comprehensively parse the comprehensive pom file and record the parsing results.

4. The method for parsing third-party dependencies using a package manager based on the Maven architecture according to claim 1, wherein: The execution process of dependency parsing commands is monitored through log files to record parsing process information.

5. A system for parsing third-party dependencies using a package manager based on the Maven architecture, characterized in that: It includes a parsing module, a monitoring module, a first processing module, a second processing module and a merging module; The parsing module is used to parse the dependency list in the pom file under the current project through the dependency parsing command in the package manager based on the Maven architecture; The monitoring module is used to monitor the parsing process of the parsing module in real time; The first processing module is configured to remove the dependency list corresponding to the error message from the pom file when an error message occurs during the execution of the parsing module and the error message comes from a direct dependency; The second processing module is used to separate all dependency lists in the total pom file under the current project and store them separately to generate several temporary pom files with single dependency lists when an error message occurs during the execution of the parsing module and the error message comes from an indirect dependency; The merging module is used to merge several temporary pom files that have been successfully parsed individually to generate a new comprehensive pom file.

6. The system for parsing third-party dependencies using a package manager based on the Maven architecture according to claim 5, characterized in that: It also includes a third processing module, which is used to interrupt the execution of the current dependency parsing command if the same download information stays for more than a preset value during the process of monitoring the parsing process through the monitoring module, and call the second processing module and the merging module to process the current project.

7. The system for parsing third-party dependencies using a package manager based on the Maven architecture according to claim 5, characterized in that: It also includes a counting module, which is used to accumulate and record the number of removed dependency lists; if the number recorded by the counting module exceeds a preset value, the execution of the current dependency resolution command is interrupted, and the second processing module and the merging module are called to process the current project.

8. The system for parsing third-party dependencies using a package manager based on Maven architecture according to claim 5, characterized in that: The monitoring module monitors the execution process of the dependency parsing command through a log file to record parsing process information.

9. A system for parsing third-party dependencies using a package manager based on the Maven architecture, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including instructions for executing the method for resolving third-party dependencies by a package manager based on the Maven architecture as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that The method comprises a computer program, which can be executed by a processor to implement the method for resolving third-party dependencies by a package manager based on the Maven architecture as described in any one of claims 1 to 4.

Images