A high-precision npm software supply chain dependency analysis method

Abstract

The application discloses a high-precision npm software supply chain dependency analysis method, comprising the following steps: acquiring metadata of all npm open source software packages, and performing real-time tracking and incremental updating on the data; extracting version and dependency information of the software packages from the metadata; performing concurrent dependency analysis on the npm software packages by using an improved dependency analysis algorithm according to the version and dependency information of the software packages, so as to obtain dependency analysis results and installation results of all the software packages; and storing the dependency analysis results of all the npm software packages and setting corresponding query interfaces. The method can automatically acquire and synchronize metadata in an npm official warehouse, can complete dependency analysis of all versions of all open source software packages in a short time, and can ensure real-time updating of the analysis results.

Description

Technical Field

[0001] This invention belongs to the field of open-source software supply chain security, and in particular relates to a high-precision method for npm software supply chain dependency analysis. Background Technology

[0002] With the rapid development of the internet and software industries, the functional requirements and complexity of software are constantly increasing, making the introduction of open-source third-party components an indispensable and crucial part of the software development process. Under this trend, JavaScript, with its advantages of low development difficulty, a vast number of open-source components, and a well-developed ecosystem, has become one of the core technologies of the internet. In 2022, JavaScript became the front-end programming language for 98% of websites on the internet, and with the support of technologies such as Node.js and Electron, it has become one of the main programming languages ​​in both the back-end and client-side domains. npm is the primary package manager for JavaScript, and its official repository currently contains over three million open-source packages, and this number continues to grow explosively.

[0003] Numerous researchers and vendors have proposed relevant methods and tools for npm software supply chain analysis. However, these tools all suffer from significant accuracy issues in the dependency analysis phase, impacting subsequent analysis work. For example: 1. Many existing studies use outdated official or third-party data, which cannot guarantee the completeness, timeliness, and authority of the data, thus affecting the timeliness and accuracy of the analysis results.

[0004] 2. Because JavaScript environments like Node.js rely on dependency trees for dependency management and loading, npm dependency resolution is inseparable from the construction of these dependency trees, affecting whether dependencies are installed and their versions. Furthermore, npm dependencies come in various types, including by default dependencies, peerDependencies, and optionalDependencies, each requiring different resolution and installation rules, further complicating the selection of npm dependencies and versions. Existing research and tools either only consider direct dependencies, ignoring dependency tree construction, or only consider common dependency types, ignoring complex dependency types like peer dependencies. This leads to significant discrepancies between dependency resolution results and those from official npm management tools, resulting in inaccurate results that fail to reflect the true state of npm packages during use.

[0005] 3. Some security tools directly use official npm tools. Official npm tools are inefficient and lack availability for comprehensive ecosystem analysis. Therefore, these tools only analyze specific packages, resulting in slow speeds, incomplete results, and an inability to provide comprehensive analysis and statistical data, making them unsuitable for open-source software supply chain analysis. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a high-precision method for npm software supply chain dependency analysis. This method can automatically acquire and synchronize metadata from the official npm repository, extract version and dependency-related information from the metadata in real time, and use an improved dependency resolution algorithm to parse all types of dependencies for all versions of all software packages. The parsing results are stored and a query interface is provided to the user.

[0007] According to a first aspect of the embodiments of this application, a high-precision method for npm software supply chain dependency analysis is provided, comprising: (1) Obtain the metadata of all npm open source packages and perform real-time data tracking and incremental updates; (2) Extract the package version and dependency information from the metadata; (3) Based on the version and dependency information of the package, use the improved dependency resolution algorithm to perform concurrent dependency resolution on the npm package to obtain the dependency resolution results and the proposed installation results of all packages; (4) Store the dependency resolution results of all npm packages and set up the corresponding query interface.

[0008] Further, step (1) includes: (1.1) Obtain all metadata information from the npm open-source repository and store it in the local metadata database; (1.2) Periodically repeat data tracking and metadata download, obtain updated data from the npm open source repository, and update the local metadata database to keep the local data up-to-date.

[0009] Further, step (2) includes: (2.1) Extract the tag information, version information, and metadata for each version of all npm packages from the metadata; (2.2) Store the tag information, version information, and metadata of each version into three key-value in-memory databases respectively.

[0010] Furthermore, in step (3), the dependency resolution result includes the dependency item, dependency version, dependency level and dependency path, and the proposed installation result includes the relative paths of all packages that need to be installed according to the dependency relationship after the target npm package is installed in an empty directory.

[0011] Further, in step (3), the improved dependency resolution algorithm includes: Create a node representing the version of the npm package to be parsed and add it to the queue of nodes to be parsed. While the queue of nodes to be parsed is not empty, the following operation is performed repeatedly: (i) Dequeue the first node; (i) For each invalid dependency edge of a dequeued node, load the target node of this edge and recursively load the peer dependency group of the target node, and add the edge and the target node to the task queue. Here, the target node of the edge is the dependent software package, the invalid dependency edge means that it does not point to a node, or the name or version of the target node does not meet the name and version range of the edge, and the peer dependency group refers to all direct or indirect dependencies of the node. (iii) For each edge and corresponding node in the task queue, the position of the node and its peer dependency group in the node tree is determined by the node tree position determination algorithm. The node and the node that regenerates the invalid dependency edge are added to the node queue to be parsed. The node tree is composed of parent-child relationships between nodes, representing the file tree to be installed.

[0012] Furthermore, the node tree position determination algorithm involves performing the following steps on each node and corresponding edge in the peer dependency group of the algorithm input node: Determine the initial target node: For an input edge and its corresponding input node, if the edge represents a peer dependency, the initial target node is the parent node of the source node of the input edge; otherwise, it is the source node. The source node of an edge is the software package that has the corresponding dependency of the edge. Traverse the parent nodes of the initial target node and execute the placement type judgment algorithm to determine the placement type of the input node under the current traversed node. The placement type includes OK, KEEP, REPLACE and CONFLICT. Take the last traversed node whose placement type is not CONFLICT, and perform node placement operations under that node according to the placement type.

[0013] Furthermore, the placement type determination algorithm includes: (I) If the target node has child nodes with the same name as the node to be placed: (i) If a node with the same name has the same version as the node to be placed, then the placement type is determined to be KEEP; (ii) If the node to be placed can replace a node with the same name, then the placement type is determined to be REPLACE; (iii) Otherwise, determine that the placement type is CONFLICT; (II) If the target node has no child nodes with the same name as the node to be placed: (i) If the node to be placed is placed as a peer dependency and will cause a conflict by blocking the dependency on the node path, then the placement type is determined to be CONFLICT; (ii) If all the recursive peer dependency groups of the node to be placed can be placed, then the placement type is determined to be OK; (iii) Otherwise, determine that the placement type is CONFLICT.

[0014] According to a second aspect of the embodiments of this application, a high-precision npm software supply chain dependency analysis apparatus is provided, comprising: The metadata acquisition module is used to acquire the metadata of all npm open source packages and perform real-time data tracking and incremental updates. An information extraction module is used to extract the version and dependency information of the software package from the metadata; The dependency resolution module is used to perform concurrent dependency resolution on npm packages based on the version and dependency information of the packages, using an improved dependency resolution algorithm to obtain the dependency resolution results and the proposed installation results for all packages. The results storage module is used to store the dependency resolution results of all npm packages and set up corresponding query interfaces.

[0015] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0016] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0017] The technical solutions provided by the embodiments of this application may include the following beneficial effects: This invention can automatically acquire and synchronize npm data. Through techniques such as update flow progress recording and data entry verification designed specifically for the npm database, it ensures data accuracy and traceability, thereby guaranteeing the integrity and timeliness of the analyzed data.

[0018] The algorithm design and implementation of this invention prioritize time efficiency. Through parallel execution of the improved dependency resolution algorithm, targeted database design optimizations, and the use of an in-memory database, this invention can complete dependency resolution for all versions of all open-source software packages in the entire ecosystem within a short time.

[0019] This invention designs a feedback update mechanism for the parsing results after data synchronization. By triggering incremental re-parsing and result updates through npm data updates, the parsing results are updated in real time, further improving the timeliness and accuracy of the analysis.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] Figure 1 This is a flowchart of a high-precision npm software supply chain dependency analysis method.

[0023] Figure 2 This is a schematic diagram of step (1).

[0024] Figure 3 This is a schematic diagram of step (2).

[0025] Figure 4 This is a schematic diagram of step (3).

[0026] Figure 5 This is a pseudocode diagram of the improved dependency resolution algorithm.

[0027] Figure 6 This is a pseudocode diagram illustrating the algorithm for determining the position of a node tree.

[0028] Figure 7 This is a pseudocode diagram of the placement type determination algorithm.

[0029] Figure 8 This is a schematic diagram of step (4).

[0030] Figure 9 This is a block diagram of a high-precision npm software supply chain dependency analysis device.

[0031] Figure 10 This is a schematic diagram of an electronic device. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0033] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0034] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0035] Figure 1 This is a flowchart illustrating a high-precision npm software supply chain dependency analysis method according to an exemplary embodiment, such as... Figure 1 As shown, the method may include the following steps: (1) Obtain the metadata of all npm open source packages and perform real-time data tracking and incremental updates; Specifically, such as Figure 2 As shown, this step includes two processes: a data tracking process and a metadata download process. These two processes will be executed sequentially in the following sub-steps: (1.1) Obtain all metadata information from the npm open-source repository and store it in the local metadata database; Specifically, the process begins with historical data tracking. The npm official database query API is used to retrieve the update stream information. Based on this update stream information, all data indexes established by the npm official database are obtained, triggering a metadata download to acquire all metadata information. The metadata download process downloads data according to the index, and then saves the acquired metadata to a local metadata database. Additional information such as the original revision and checksum of data entries is also stored for data checking and validation. The local metadata database uses a document-oriented database to ensure compatibility and integrity with the npm database and data.

[0036] (1.2) Periodically repeat data tracking and metadata download, obtain updated data from the npm open source repository, update the local metadata database to keep the local data up-to-date; Specifically, the npm official database query API is used to track data, obtain the data update index of the npm official database, perform metadata download based on the data update index, obtain the metadata of the npm package that needs to be updated, store it in the database, and record the index number of the updated npm package metadata; more specifically, records are made after the download and update are completed to save information such as update progress and index number, which will be used as the query starting point when this process is executed again.

[0037] It should be noted that this application continuously repeats the operation of step (1.2) to achieve real-time tracking and incremental updates of the database. If the data is updated, subsequent update actions such as metadata extraction and dependency information parsing are triggered when the data is updated.

[0038] (2) Extract the package version and dependency information from the metadata; Specifically, all the data required for dependency resolution is extracted from the local metadata database and stored in a more efficient database, such as... Figure 3 As shown, this step may include the following steps: (2.1) Extract the tag information, version information, and metadata for each version of all npm packages from the metadata; Specifically, information such as the name, dist-tags, and versions of npm packages is extracted from the metadata. The name-version information is split and reassembled before being updated to the version information database. The name-tag information is split and reassembled before being updated to the tag information database. The version-metadata information is split and cleaned before being updated to the dependency information database.

[0039] (2.2) Store the tag information, version information, and metadata for each version into three key-value in-memory databases respectively; Specifically, the data extracted in this step is the data source directly used in subsequent dependency resolution. The above-mentioned division of metadata information can reduce the amount of text processing tasks in the dependency resolution process, storing it in a key-value database can reduce the computational complexity of queries, and storing it in an in-memory database can reduce the hardware execution time of queries. These multiple processes ensure the conciseness of the extracted data and the efficiency of subsequent reading.

[0040] (3) Based on the version and dependency information of the package, use the improved dependency resolution algorithm to perform concurrent dependency resolution on the npm package to obtain the dependency resolution results and the proposed installation results of all packages; like Figure 4 As shown, this step may include the following sub-steps: (3.1) The improved dependency resolution algorithm is executed concurrently on all npm packages to be resolved. The required information for resolution is obtained by querying the name-version information database, name-tag information database and version-metadata information database in step (2); Specifically, dependency resolution is achieved through an improved dependency resolution algorithm. This improved algorithm executes a resolution algorithm for each npm package version in the npm ecosystem, and constructs a tree (node ​​tree) of files to be installed and a dependency graph (node ​​graph) based on the metadata stored in the database.

[0041] The main basic concepts of the improved dependency resolution algorithm are defined as follows: 1. Node: Represents an npm package, containing its name and version information.

[0042] 2. Edge: Represents the dependency relationship between nodes, containing name and dependency version range information; 1) The source node of an edge represents the software package with that dependency. The target node of the edge represents the software package that is depended upon. Outgoing edges of a node represent the node's dependencies. Ingoing edges of a node represent the node's dependencies. 2) A valid edge indicates that the edge points to a target node, and the name and version of the target node satisfy the name and version range of the edge. An invalid edge indicates that the edge does not point to a node, or the name or version of the target node does not satisfy the name and version range of the edge.

[0043] 3. Node tree: A tree composed of nodes and parent-child relationships between nodes, representing the file tree to be installed, that is, the node_modules directory tree after the actual dependency installation is performed.

[0044] 4. Node graph: A graph consisting of nodes and edges that represents the dependencies between packages.

[0045] 5. Peer dependency group: All direct or indirect dependencies of a node are called the peer dependency group of that node.

[0046] This algorithm is mainly divided into three sub-algorithms, which are called sequentially: Core logic sub-algorithm: The main body of the parsing algorithm, which is an improvement on breadth-first search.

[0047] Node tree position determination sub-algorithm: During the execution of the parsing algorithm, an algorithm that places a node in a suitable position in the node tree based on an edge and its corresponding target node.

[0048] Placement type judgment sub-algorithm: During the execution of the node tree position determination algorithm, this algorithm determines the placement method of a node at a specific position based on an edge and its corresponding target node. The placement type and its corresponding meaning are as follows: 1) OK means there are no existing nodes and it can be placed directly; 2) KEEP indicates that the existing nodes satisfy the aforementioned edge conditions, and there is no need to place the aforementioned nodes; 3) REPLACE indicates that the existing node does not satisfy the aforementioned edge, but the aforementioned node can replace the existing node; 4) CONFLICT indicates that the existing node does not satisfy the aforementioned edge, and the aforementioned node cannot replace the existing node, so it cannot be placed.

[0049] Specifically, the core logic of the parsing algorithm is based on an improvement upon breadth-first search, closely aligning with the actual installation process of npm packages and ensuring consistency with the parsing order of official npm tools, thus improving the accuracy of npm ecosystem analysis tools. The core logic of the parsing algorithm also provides targeted processing for ordinary dependencies and peer dependencies, further enhancing the accuracy of npm ecosystem analysis tools. The main logic of this part is as follows: Figure 5 As shown, specifically: (I) Nodeize the npm package version to be parsed and add it to the queue of nodes to be parsed; (II) When the queue of nodes to be resolved is not empty, perform the following operations repeatedly: (i) Dequeue the first node; (ii) For each invalid dependency edge of a dequeued node, load the target node of this edge and recursively load the peer dependency group of the target node, and add the edge and the target node to the task queue. (iii) For each edge and corresponding node in the task queue, execute the node tree position determination algorithm to determine the position of the node and its peer dependency group in the node tree, and add the node and the node that regenerates invalid dependency edges to the node queue to be parsed. Here, "node that regenerates invalid dependency edges" means that the node tree position determination algorithm may cause the dependency edges of existing nodes in the tree to change from valid to invalid.

[0050] Specifically, the node tree position determination algorithm is used to place nodes in the node tree based on edges. By simulating the path search logic of the node dependency loading process, the algorithm ensures its correctness and accuracy. Furthermore, the algorithm performs targeted processing and recursive execution for ordinary dependencies and peer dependencies, improving the robustness of the resolution process and enhancing the correctness and accuracy of resolving different types of dependencies. The main logic of this part is as follows: Figure 6 As shown, its main logic is to perform the following steps on each node and corresponding edge in the peer dependency group of the algorithm input node: (I) Determine the initial target node: For an input edge and its corresponding input node, if the edge represents a peer dependency, the initial target node is the parent node of the source node of the input edge; otherwise, it is the source node. (II) Traverse the parent nodes of the initial target node and execute the placement type judgment algorithm to determine the placement type of the input node under the current traversed node; (III) Take the last traversed node whose placement type is not CONFLICT, and perform node placement operation under this node according to the placement type. That is, the node with placement type OK is placed directly, the node with placement type KEEP is not placed, and the node with placement type REPLACE is placed by replacing the existing node.

[0051] Specifically, the placement type determination algorithm is used to determine the placement type of the node to be placed under the target node. The algorithm checks all possible cases to ensure no duplication or omission of the actual node tree situation; it also performs targeted processing and recursive execution for ordinary dependencies and peer dependencies, improving the correctness and accuracy of determining different dependency types. The main logic of this part is as follows: Figure 7 As shown, its main logic is: (I) If the target node has child nodes with the same name as the node to be placed: (i) If a node with the same name has the same version as the node to be placed, then the placement type is determined to be KEEP; (ii) If the node to be placed can replace the node with the same name (i.e., the node to be placed can satisfy all the incoming edges of the node with the same name, and the peer dependency group of the node to be placed can be placed or replaced recursively), then the placement type is determined to be REPLACE. (iii) Otherwise, determine that the placement type is CONFLICT; (II) If the target node has no child nodes with the same name as the node to be placed: (i) If the node to be placed is placed as a peer dependency and will cause a conflict by blocking the dependency on the node path, then the placement type is determined to be CONFLICT; (ii) If all the recursive peer dependency groups of the node to be placed can be placed, then the placement type is determined to be OK; (iii) Otherwise, determine that the placement type is CONFLICT.

[0052] Furthermore, the improved dependency resolution algorithm adds information such as tags and aliases, as well as preset information such as os, cpu, and glibc, to specify and filter the version for dependency resolution version determination; it improves the loading logic of package nodes, enhances the robustness of the loading process, reduces redundant data loading, and avoids problems such as null pointers and infinite loops in existing tools; it reduces redundant judgments and invalid processes, optimizes the data reading and processing process, and significantly improves time efficiency.

[0053] Furthermore, the algorithm is implemented in C++ in the embodiments, and focuses on optimizing time efficiency to meet the time requirements of full ecosystem dependency analysis. It can complete high-precision dependency analysis of all npm open source software within 100 hours.

[0054] (3.2) The parallelized dependency resolution process returns the resolution result data after the resolution is completed; Specifically, each parsing result includes dependency resolution results and proposed installation results. The dependency resolution results include dependencies, dependency versions, dependency levels, and dependency paths, etc.; the proposed installation results include the relative paths of all packages that need to be installed according to dependencies after installing the target npm package in an empty directory.

[0055] (4) Store the dependency resolution results of all npm packages and set up the corresponding query interface; Specifically, all npm package dependencies, versions, relationships, and levels are stored in a relational database, while all npm package dependency paths are stored in a document-oriented database. A query interface encapsulated from these two databases is provided.

[0056] In one embodiment, such as Figure 8 As shown, the results are processed and stored in the dependency information database and the dependency path database, respectively. The processing includes calculating dependency paths and dependency levels based on dependency relationships, compressing the proposed installation file tree, and serializing the dependency graph. The storage process includes storing the dependencies, dependency versions, and dependency levels in a structured manner in the dependency information database, and storing the dependency paths and proposed installation results in a documented manner in the dependency path database.

[0057] Upon receiving a query request, the system executes a packaged database query based on the request and returns the query results. It obtains information such as dependency versions, dependency relationships, and dependency levels of dependencies and their dependents through structured queries on the dependency information database, and also provides statistical information such as the number of dependencies, the number of dependents, and dependency rankings. Dependency path information is obtained by querying documents in the dependency path database.

[0058] In summary, this application has excellent prospects for widespread application: This invention can automatically acquire and synchronize npm data. Through techniques such as update flow progress recording and data entry verification designed specifically for the npm database, it ensures data accuracy and traceability, thereby guaranteeing the integrity and timeliness of the analyzed data. This invention improves existing supply chain dependency parsing algorithms by specifically parsing npm's dependency types such as dependencies, peerDependencies, and optionalDependencies, as well as version types such as tags, aliases, and semver, and by specifically filtering information such as os and cpu, thereby improving the accuracy and customizability of npm software supply chain analysis tools. The algorithm design and implementation of this invention prioritize time efficiency. Through parallel execution of the improved dependency resolution algorithm, targeted database design optimization, and the use of an in-memory database, this invention can complete dependency resolution for all versions of all open-source software packages in the entire ecosystem within a short time. This invention designs a feedback update mechanism for the parsing results after data synchronization. By triggering incremental re-parsing and result updates through npm data updates, the parsing results are updated in real time, further improving the timeliness and accuracy of the analysis.

[0059] Corresponding to the aforementioned embodiments of the high-precision npm software supply chain dependency analysis method, this application also provides embodiments of a high-precision npm software supply chain dependency analysis apparatus.

[0060] Figure 9 This is a block diagram illustrating a high-precision npm software supply chain dependency analysis apparatus according to an exemplary embodiment. (Refer to...) Figure 9 The device may include: The metadata acquisition module 21 is used to acquire the metadata of all npm open source packages and perform real-time data tracking and incremental updates. Information extraction module 22 is used to extract the version and dependency information of the software package from the metadata; The dependency resolution module 23 is used to perform concurrent dependency resolution on the npm packages based on the version and dependency information of the packages, using an improved dependency resolution algorithm to obtain the dependency resolution results and the proposed installation results for all packages. The result storage module 24 is used to store the dependency resolution results of all npm packages and set up the corresponding query interface.

[0061] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0062] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0063] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the high-precision npm software supply chain dependency analysis method described above. Figure 10 The diagram shown illustrates a hardware structure of any device with data processing capabilities, used in an embodiment of the present invention to provide a high-precision npm software supply chain dependency analysis method. (Except for...) Figure 10 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0064] Accordingly, this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the high-precision npm software supply chain dependency analysis method described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0065] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0066] It should be understood that this application is not limited to the precise steps described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A high-precision method for npm software supply chain dependency analysis, characterized in that, include: (1) Obtain the metadata of all npm open source packages and perform real-time data tracking and incremental updates; (2) Extract the package version and dependency information from the metadata; (3) Based on the version and dependency information of the package, use the improved dependency resolution algorithm to perform concurrent dependency resolution on the npm package to obtain the dependency resolution results and the proposed installation results of all packages; (4) Store the dependency resolution results of all npm packages and set up the corresponding query interface; In step (3), the improved dependency resolution algorithm includes: Create a node representing the version of the npm package to be parsed and add it to the queue of nodes to be parsed. While the queue of nodes to be parsed is not empty, the following operation is performed repeatedly: (i) Dequeue the first node; (i) For each invalid dependency edge of a dequeued node, load the target node of this edge and recursively load the peer dependency group of the target node, and add the edge and the target node to the task queue. Here, the target node of the edge is the dependent software package, the invalid dependency edge means that it does not point to a node, or the name or version of the target node does not meet the name and version range of the edge, and the peer dependency group refers to all direct or indirect dependencies of the node. (iii) For each edge and corresponding node in the task queue, the position of the node and its peer dependency group in the node tree is determined by the node tree position determination algorithm. The node and the node that regenerates the invalid dependency edge are added to the node queue to be parsed. The node tree is composed of parent-child relationships between nodes, representing the file tree to be installed.

2. The method according to claim 1, characterized in that, Step (1) includes: (1.1) Obtain all metadata information from the npm open-source repository and store it in the local metadata database; (1.2) Periodically repeat data tracking and metadata download, obtain updated data from the npm open source repository, and update the local metadata database to keep the local data up-to-date.

3. The method according to claim 1, characterized in that, Step (2) includes: (2.1) Extract the tag information, version information, and metadata for each version of all npm packages from the metadata; (2.2) Store the tag information, version information, and metadata of each version into three key-value in-memory databases respectively.

4. The method according to claim 1, characterized in that, In step (3), the dependency resolution result includes the dependency item, dependency version, dependency level and dependency path, and the proposed installation result includes the relative paths of all packages that need to be installed according to the dependency relationship after the target npm package is installed in an empty directory.

5. The method according to claim 1, characterized in that, The node tree position determination algorithm performs the following steps on each node and corresponding edge in the peer dependency group of the algorithm input node: Determine the initial target node: For an input edge and its corresponding input node, if the edge represents a peer dependency, the initial target node is the parent node of the source node of the input edge; otherwise, it is the source node. The source node of an edge is the software package that has the corresponding dependency of the edge. Traverse the parent nodes of the initial target node and execute the placement type judgment algorithm to determine the placement type of the input node under the current traversed node. The placement type includes OK, KEEP, REPLACE and CONFLICT. Take the last traversed node whose placement type is not CONFLICT, and perform node placement operations under that node according to the placement type.

6. The method according to claim 5, characterized in that, The placement type determination algorithm includes: (I) If the target node has child nodes with the same name as the node to be placed: (i) If a node with the same name has the same version as the node to be placed, then the placement type is determined to be KEEP; (ii) If the node to be placed can replace a node with the same name, then the placement type is determined to be REPLACE; (iii) Otherwise, determine that the placement type is CONFLICT; (II) If the target node has no child nodes with the same name as the node to be placed: (i) If the node to be placed is placed as a peer dependency and will cause a conflict by blocking the dependency on the node path, then the placement type is determined to be CONFLICT; (ii) If all the recursive peer dependency groups of the node to be placed can be placed, then the placement type is determined to be OK; (iii) Otherwise, determine that the placement type is CONFLICT.

7. A high-precision npm software supply chain dependency analysis device, characterized in that, include: The metadata acquisition module is used to acquire the metadata of all npm open source packages and perform real-time data tracking and incremental updates. An information extraction module is used to extract the version and dependency information of the software package from the metadata; The dependency resolution module is used to perform concurrent dependency resolution on npm packages based on the version and dependency information of the packages, using an improved dependency resolution algorithm to obtain the dependency resolution results and the proposed installation results for all packages. The results storage module is used to store the dependency resolution results of all npm packages and set up corresponding query interfaces; The improved dependency resolution algorithm in the dependency resolution module includes: Create a node representing the version of the npm package to be parsed and add it to the queue of nodes to be parsed. While the queue of nodes to be parsed is not empty, the following operation is performed repeatedly: (i) Dequeue the first node; (i) For each invalid dependency edge of a dequeued node, load the target node of this edge and recursively load the peer dependency group of the target node, and add the edge and the target node to the task queue. Here, the target node of the edge is the dependent software package, the invalid dependency edge means that it does not point to a node, or the name or version of the target node does not meet the name and version range of the edge, and the peer dependency group refers to all direct or indirect dependencies of the node. (iii) For each edge and corresponding node in the task queue, the position of the node and its peer dependency group in the node tree is determined by the node tree position determination algorithm. The node and the node that regenerates the invalid dependency edge are added to the node queue to be parsed. The node tree is composed of parent-child relationships between nodes, representing the file tree to be installed.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-6.

Images