Dependency package synchronization method, device, computer equipment and storage medium

By analyzing and consistently detecting the dependency packages of client projects in a private NPM warehouse, the problems of low synchronization efficiency and large resource utilization in the existing technology are solved, and efficient, accurate synchronization and consistency of the dependency packages are achieved, improving the efficiency, accuracy, and consistency of the dependency packages. Reliability of private NPM warehouses.

CN117971297BActive Publication Date: 2024-11-05GUANGZHOU DINGJIA COMPUTER TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311822782.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-27
Publication Date
2024-11-05
Estimated Expiration
2043-12-27

AI Technical Summary

Technical Problem

When establishing a private NPM warehouse, the dependency package synchronization efficiency is low and the resource occupies too much, and may cause inconsistency due to network problems, affecting the reliability of the private NPM warehouse.

Method used

By parsing the dependency configuration information of the client project, determine the dependency package sorting collection, and perform consistency detection in the private NPM repository. If the dependency package does not exist or the version is inconsistent, download and update from the public NPM repository to ensure that in the private NPM repository The consistency and integrity of the dependency package.

Benefits of technology

It improves the efficiency and consistency of dependent package synchronization in private NPM warehouses, reduces the use of storage space and network bandwidth, and ensures real-time synchronization and accuracy of dependent packages.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117971297B_ABST
    Figure CN117971297B_ABST
Patent Text Reader

Abstract

The present application relates to a dependency package synchronization method, device, computer equipment and storage medium, and relates to the field of software development technology. The method includes: parsing the dependency configuration information of the client project, determining the dependency package sorting set corresponding to the client project; when it is determined that the cache document of the private NPM warehouse does not contain the set identifier of the dependency package sorting set, for each dependency package included in the dependency package sorting set, if the dependency package does not exist in the private NPM warehouse, the dependency package is obtained from the public NPM warehouse and stored in the private NPM warehouse; after each dependency package corresponding to the client project in the private NPM warehouse passes the consistency test, a cached message is sent to the client containing the client project; the cached message is used to enable the client to obtain the dependency package corresponding to the client project from the private NPM warehouse. The use of this method can improve the efficiency of the dependency packages required for synchronization of private NPM warehouses.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of software development, and in particular to a dependency package synchronization method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] When developing software, different environments are built for different projects, which requires calling different dependency packages. Since the public NPM (Node Package Manager) repository needs to be accessed through the Internet, if the network environment is unstable or slow, the download and update of the package will become slow and unreliable. Based on this, a private NPM repository can be established and used as a transit station to download the required dependency packages from the public NPM repository to the private NPM repository, avoiding users from repeatedly downloading and updating from the public NPM repository.

[0003] In the related art, the server can periodically synchronize the dependency package mirrors of the public NPM repository to the private NPM repository through a script tool. However, this method requires a large amount of storage space, network bandwidth, and hardware resources, and may also cause inconsistencies in the synchronized dependency packages due to network problems, ultimately resulting in insufficient efficiency in synchronizing the dependency packages of the private NPM repository. Summary of the invention

[0004] Based on this, it is necessary to provide a dependency package synchronization method, apparatus, computer device, computer readable storage medium and computer program product to address the above technical issues.

[0005] In a first aspect, the present application provides a dependency package synchronization method. The method comprises:

[0006] Parsing the dependency configuration information of the client project to determine a sorted set of dependency packages corresponding to the client project; the sorted set of dependency packages includes names and version numbers of the dependency packages arranged in order;

[0007] In the case where it is determined that the cache document of the private NPM repository does not contain the set identifier of the dependency package sorting set, for each dependency package contained in the dependency package sorting set, if the dependency package does not exist in the private NPM repository, the dependency package is obtained from the public NPM repository and stored in the private NPM repository; the cache document contains the set identifier of the dependency package sorting set corresponding to the entered project;

[0008] After all dependency packages corresponding to the client project in the private NPM repository pass the consistency check, a cached message is sent to the client containing the client project; the cached message is used to enable the client to obtain the dependency packages corresponding to the client project from the private NPM repository.

[0009] In one embodiment, after parsing the dependency configuration information of the client project and determining the sorted set of dependency packages corresponding to the client project, the method further includes:

[0010] When it is determined that the cache document of the private NPM repository contains the set identifier of the sorted set of dependency packages, a cached message is returned to the client containing the client project, so that the client installs the dependency package corresponding to the client project based on the dependency package installation command and the address of the private NPM repository.

[0011] In one embodiment, parsing the dependency configuration information of the client project to determine the sorted set of dependency packages corresponding to the client project includes:

[0012] Obtaining dependency configuration information of the client project through the data interface of the client;

[0013] Based on a depth-first search strategy, the dependency configuration information is traversed to obtain the names and version numbers of multiple dependency packages arranged in sequence, thereby obtaining a sorted set of dependency packages corresponding to the client project.

[0014] In one embodiment, for each dependent package included in the dependent package sorting set, if the dependent package does not exist in the private NPM repository, obtaining the dependent package from the public NPM repository and storing it in the private NPM repository includes:

[0015] Traverse each dependent package corresponding to the dependent package sorting set, and if the private NPM repository does not contain the currently traversed dependent package, download the dependent package from the public NPM repository to the private NPM repository according to the name and version number of the dependent package;

[0016] If the private NPM repository contains the dependent package, traverse the next dependent package until the traversal of the sorted set of dependent packages is completed.

[0017] In one embodiment, before sending the cached message to the client including the client project, the method further comprises:

[0018] Traversing each dependent package corresponding to the client project in the private NPM repository, determining the historical version information of the directory where the currently traversed dependent package is located, the historical version information including multiple historical version numbers corresponding to the dependent package;

[0019] If the version number of the currently traversed dependent package is greater than the latest version number in the historical version information, then download the dependent package corresponding to the latest version number from the public NPM repository, store it in the directory corresponding to the dependent package, and update the latest version number to the historical version information;

[0020] If the historical version information does not contain the version number of the currently traversed dependent package, the dependent package corresponding to the version number is re-downloaded from the public NPM repository, and the version number is updated to the historical version information.

[0021] In one embodiment, before sending the cached message to the client including the client project, the method further comprises:

[0022] Traverse each dependency package corresponding to the client project in the private NPM repository, and determine the checksum of the dependency package currently traversed;

[0023] If the checksum of the dependent package is inconsistent with the checksum of the corresponding dependent package in the public NPM warehouse, the dependent package is downloaded again from the public NPM warehouse.

[0024] In one of the embodiments, after determining the sorted set of dependent packages corresponding to the client project, the method further includes:

[0025] Detect whether the cache document of the private NPM repository contains the hash value of the sorted set of dependency packages;

[0026] If the cache document includes the hash value of the dependent package sorted set, determining that the cache document includes the set identifier of the dependent package sorted set;

[0027] If the cache document does not include the hash value of the dependent package sorted set, determining that the cache document does not include the set identifier of the dependent package sorted set;

[0028] After sending the cached message to the client including the client project, the method further comprises:

[0029] Calculate a hash value of the ordered set of dependent packages, and store the hash value in the cache document.

[0030] In a second aspect, the present application also provides a dependency package synchronization device. The device comprises:

[0031] A parsing module, used to parse the dependency configuration information of the client project and determine a sorted set of dependency packages corresponding to the client project; the sorted set of dependency packages includes names and version numbers of the dependency packages arranged in order;

[0032] An acquisition module, for obtaining, when it is determined that the cache document of the private NPM repository does not contain the set identifier of the dependency package sorting set, for each dependency package contained in the dependency package sorting set, if the dependency package does not exist in the private NPM repository, the dependency package is acquired from the public NPM repository and stored in the private NPM repository; the cache document contains the set identifier of the dependency package sorting set corresponding to the entered project;

[0033] A sending module is used to send a cached message to a client including the client project after all dependency packages corresponding to the client project in the private NPM repository pass consistency detection; the cached message is used to enable the client to obtain the dependency package corresponding to the client project from the private NPM repository.

[0034] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect are implemented.

[0035] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0036] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0037] The above-mentioned dependency package synchronization method, device, computer equipment, storage medium and computer program product can determine whether the cache document in the private NPM warehouse stores the set identifier by obtaining the sorted set of dependency packages of the client project and determining the set identifier of the sorted set of dependency packages. If the cache document contains the set identifier, it is determined that the private NPM warehouse stores all the required dependency packages of the client project and has passed the consistency test, and the required dependency packages can be directly installed. If the cache document does not contain the set identifier, it is necessary to traverse each dependency package corresponding to the sorted set of dependency packages so that the private NPM warehouse synchronizes the required dependency packages from the public NPM warehouse, and it is also necessary to perform consistency testing on each required dependency package, so as to ensure that the private NPM warehouse can provide support for the client project. After completing the consistency test on each dependency package, the server sends a cached message to the client. Based on this, the client can obtain the dependency package corresponding to the client project from the private NPM warehouse and install the dependency package on the client. Finally, through the dependency configuration information of each client project, the cached documents of the private NPM repository can be updated in real time, and the dependency packages required by the client projects in the private NPM repository can be synchronized in real time, so there is no need to update all dependency packages of the public NPM repository. Only the dependency packages required by the client projects need to be updated, thereby reducing the storage space and network bandwidth occupied. In addition, the consistency rate of the dependency packages after synchronization can be improved through consistency detection, ultimately improving the efficiency of synchronizing dependency packages in the private NPM repository. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] FIG1 is a diagram of an application environment of a dependency packet synchronization method in one embodiment;

[0039] FIG2 is a schematic diagram of a flow chart of a dependency packet synchronization method in one embodiment;

[0040] FIG3 is a schematic diagram of a flow chart of a step of downloading a dependency package in one embodiment;

[0041] FIG4 is a schematic diagram of a flow chart of a dependency packet synchronization method in another embodiment;

[0042] FIG5 is a structural block diagram of a dependent packet synchronization device in one embodiment;

[0043] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] The dependency package synchronization method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the client 101 communicates with the server 102 through the network, and the server 102 can obtain the dependency configuration information of the client project from the client 101, and parse the dependency configuration information to obtain the dependency package sorting set of the client project and the set identifier of the dependency package sorting set. The server 102 obtains the cache document in the private NPM warehouse 103 and determines whether the set identifier is in the cache document. If not, the server 102 obtains the dependency package required by the client project from the public NPM warehouse 104 and synchronizes it to the private NPM warehouse 103. If the private NPM warehouse 103 contains the dependency package required by the client project, there is no need to download the dependency package from the public NPM warehouse 104. After the synchronization is completed, the private NPM warehouse 103 contains all the dependency packages required by the client project, and the client 101 can download all the dependency packages required by the client project from the private NPM warehouse 103. The private NPM warehouse 103 and the public NPM warehouse 104 can be integrated on the server 104, or placed on the cloud or other network servers. The client 101 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, etc. The server 102 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0046] In one embodiment, as shown in FIG. 2 , a dependency package synchronization method is provided, and the method is applied to the server in FIG. 1 as an example for description, including the following steps:

[0047] Step S202: parsing the dependency configuration information of the client project to determine a sorted set of dependency packages corresponding to the client project.

[0048] Among them, the client project is each software project in the software development process, and each project has a corresponding development environment; the dependency configuration information includes the dependency packages and dependency relationships required by the development environment corresponding to the client project; the dependency package sorting set includes the names and version numbers of the dependency packages arranged in order.

[0049] Specifically, the server obtains the dependency configuration information corresponding to each client project from the client interface request. And through the text parsing strategy, the name and version number of the required dependency package are obtained from the dependency configuration information. And based on the parsing order of the parsing strategy in this article, the names and version numbers of each dependency package are arranged in order, and the names and version numbers of each dependency package are stored in a set to obtain the sorted set of dependency packages corresponding to each client project.

[0050] Step S204, when it is determined that the cache document of the private NPM repository does not contain the set identifier of the dependency package sorting set, for each dependency package included in the dependency package sorting set, if the dependency package does not exist in the private NPM repository, the dependency package is obtained from the public NPM repository and stored in the private NPM repository.

[0051] Among them, the private NPM repository is an NPM repository for private internal access, which stores dependency packages of various names and versions disclosed by the public NPM repository; the cache document is used to record the sorted sets of dependency packages corresponding to each client project. The sorted sets of dependency packages that pass the consistency test can store their set identifiers in the cache document. Therefore, the cache document contains the set identifiers of the sorted sets of dependency packages corresponding to the entered project; the public NPM repository is an officially maintained and publicly accessible NPM repository that contains a large number of open source JavaScript packages.

[0052] Specifically, the server determines whether the cached document of the private NPM repository contains the set identifier of the sorted set of dependency packages corresponding to the client project. If the cached document of the private NPM repository does not contain the set identifier of the sorted set of dependency packages, it is necessary to detect all the dependency packages contained in the sorted set of dependency packages. Based on this, the server traverses each dependency package contained in the sorted set of dependency packages, and determines whether the currently traversed dependency package is stored in the private NPM repository. If the currently traversed dependency package is retrieved in the private NPM repository, no processing is performed; if the currently traversed dependency package is not retrieved in the private NPM repository, the currently traversed dependency package is obtained from the public NPM repository and stored in the private NPM repository.

[0053] Step S206, after all dependent packages corresponding to the client project in the private NPM repository pass the consistency check, a cached message is sent to the client including the client project.

[0054] The consistency check is used to determine whether the name and version number of the dependency package corresponding to the client project in the private NPM repository meets the requirements of the client project, and whether the files of the dependency packages corresponding to the client project in the private NPM repository are complete or damaged. The cached message is used to enable the client to obtain the dependency package corresponding to the client project from the private NPM repository.

[0055] Specifically, the server traverses each dependency package corresponding to the client project in the private NPM repository, and performs consistency checks on each dependency package. If the currently traversed dependency package does not pass the consistency check, the dependency package is downloaded again from the public NPM repository, and the re-downloaded dependency package is tested for consistency again until all dependency packages pass the consistency check. After all dependency packages corresponding to the client project in the private NPM repository pass the consistency check, the server sends a cached message to the client where the client project is stored. After receiving the cached message, the client can obtain the dependency packages required for the client project from the private NPM repository.

[0056] In the above dependency package synchronization method, the server can determine whether the cache document in the private NPM repository stores the collection identifier by obtaining the sorted set of dependency packages of the client project and determining the collection identifier of the sorted set of dependency packages. If the cache document contains the collection identifier, it is determined that the private NPM repository stores all the required dependency packages of the client project and has passed the consistency test, and the required dependency packages can be directly installed. If the cache document does not contain the collection identifier, it is necessary to traverse each dependency package corresponding to the sorted set of dependency packages so that the private NPM repository synchronizes the required dependency packages from the public NPM repository, and it is also necessary to perform consistency tests on each required dependency package to ensure that the private NPM repository can provide support for the client project. After completing the consistency test on each dependency package, the server sends a cached message to the client. Based on this, the client can obtain the dependency package corresponding to the client project from the private NPM repository and install the dependency package on the client. Finally, through the dependency configuration information of each client project, the cached documents of the private NPM repository can be updated in real time, and the dependency packages required by the client projects in the private NPM repository can be synchronized in real time, so there is no need to update all dependency packages of the public NPM repository. Only the dependency packages required by the client projects need to be updated, thereby reducing the storage space and network bandwidth occupied. In addition, the consistency rate of the dependency packages after synchronization can be improved through consistency detection, ultimately improving the efficiency of synchronizing dependency packages in the private NPM repository.

[0057] In one embodiment, the dependency package synchronization method further includes:

[0058] When it is determined that the cache document of the private NPM repository contains the set identifier of the sorted set of dependency packages, the cached message is returned to the client containing the client project, so that the client installs the dependency packages corresponding to the client project based on the dependency package installation command and the address of the private NPM repository.

[0059] Specifically, the server determines whether the cached document of the private NPM warehouse contains the set identifier of the dependency package sorting set corresponding to the client project. If the cached document of the private NPM warehouse contains the set identifier of the dependency package sorting set, it means that the private NPM warehouse contains the dependency packages corresponding to the dependency package sorting set, and each dependency package has passed the consistency test. Based on this, the server returns a cached message to the client. After the client receives the cached message, it downloads the dependency packages corresponding to the client project from the private NPM warehouse by executing the received dependency package installation command and the address of the private NPM warehouse. The names and version numbers of the downloaded dependency packages correspond to the dependency package sorting set. And install each dependency package to the client project in the client.

[0060] In this embodiment, by determining whether the cached document of the private NPM repository contains the set identifier of the sorted set of dependency packages corresponding to the client project, and when the cached document of the private NPM repository contains the set identifier of the sorted set of dependency packages, a cached message is directly sent to the client to enable the client to immediately install the dependency package. The dependency packages that pass the consistency check can be installed quickly, reducing the steps of repeatedly verifying the consistency of each dependency package, thereby improving the efficiency of installing the dependency package and improving the accuracy of synchronizing the dependency packages of the private NPM repository.

[0061] In one embodiment, the specific implementation process of the step of "parsing the dependency configuration information of the client project to determine the sorted set of dependency packages corresponding to the client project" includes:

[0062] Obtain the dependency configuration information of the client project through the client's data interface. Based on the depth-first search strategy, traverse the dependency configuration information to obtain the names and version numbers of multiple dependency packages arranged in order, thereby obtaining the sorted set of dependency packages corresponding to the client project.

[0063] The data interface may be a network interface for data interaction and communication between the client and the server, for example, the data interface may be a RESTFul interface. The depth-first search strategy may be a depth-first search DFS (Depth-First Search). DFS is a graph traversal algorithm. DFS starts from a vertex of the graph and traverses to the end along a path until it can no longer go on. Then it goes back to the previous node and continues traversing along another path until the entire graph is traversed.

[0064] Specifically, the server obtains the package.json file content of the client project through the RESTFul interface, parses the package.json file content through DFS, obtains the names and version numbers of multiple dependent packages arranged in the traversal order through graph traversal, and generates a sorted set of dependent packages corresponding to the client project. Among them, package.json is a file used to describe project information and manage project dependencies. Package.json contains information such as the name, version, author, dependent packages, and dependency relationships of the client project.

[0065] In this embodiment, a json file is obtained through a data interface, and parsed based on DFS to obtain a sorted set of dependency packages corresponding to the client project. All required dependency packages contained in the json file can be accurately obtained, thereby improving the accuracy of obtaining the dependency packages required by the client project.

[0066] In one embodiment, the specific implementation process of the step of "for each dependent package included in the dependent package sorting set, if the dependent package does not exist in the private NPM repository, obtain the dependent package from the public NPM repository and store it in the private NPM repository" includes:

[0067] Traverse each dependent package corresponding to the sorted set of dependent packages. If the private NPM repository does not contain the currently traversed dependent package, download the dependent package from the public NPM repository to the private NPM repository based on the name and version number of the dependent package.

[0068] If the private NPM repository contains dependent packages, traverse the next dependent package until the traversal of the sorted set of dependent packages is completed.

[0069] Specifically, the server traverses each dependent package corresponding to the dependent package sorting set, and searches the private NPM repository for the currently traversed dependent package based on the name and version number of the currently traversed dependent package. If the private NPM repository does not contain the currently traversed dependent package, then based on the name and version number of the dependent package, the corresponding dependent package is downloaded from the public NPM repository and stored in the private NPM repository. If the private NPM repository contains the currently traversed dependent package, there is no need to download the corresponding dependent package from the public NPM repository, and traverse the next dependent package until the traversal of each dependent package in the dependent package sorting set is completed.

[0070] In this embodiment, each dependency package is retrieved in the private NPM repository, and for the dependency packages that are not stored in the private NPM repository, the server downloads them from the public NPM repository according to the name and version number of the required dependency package, so that only the required versions of the dependency packages are synchronized to the private NPM repository, without synchronizing the dependency packages that are not required by the client project, thereby improving the accuracy and efficiency of the synchronized dependency packages.

[0071] In one embodiment, as shown in FIG3 , the dependency package synchronization method further includes:

[0072] Step S302, traverse each dependent package corresponding to the client project in the private NPM repository, and determine the historical version information of the directory where the currently traversed dependent package is located.

[0073] Among them, the directory where the dependency package is located refers to the file directory that stores the various versions of the dependency packages corresponding to the dependency package, that is, the directory where the dependency package is located contains multiple dependency packages with the same name but different version numbers, and the historical version information contains multiple historical version numbers corresponding to the dependency package. There is a one-to-one correspondence between each historical version number in the historical version information and each version of the dependency package contained in the directory where the dependency package is located.

[0074] Specifically, after the server traverses the sorted set of dependency packages corresponding to the client project and downloads the required dependency packages from the public NPM repository, it also needs to check whether the downloaded dependency packages have been downloaded and whether they have been downloaded correctly. The server traverses the dependency packages corresponding to the client project stored in the private NPM repository, obtains the directory where the currently traversed dependency package is located, and obtains the historical version information of the directory, that is, obtains the tgz files of each version corresponding to the currently traversed dependency package.

[0075] Step S304, if the version number of the currently traversed dependency package is greater than the latest version number in the historical version information, the dependency package corresponding to the latest version number is downloaded from the public NPM repository, stored in the directory corresponding to the dependency package, and the latest version number is updated to the historical version information.

[0076] The latest version number refers to the version number of the latest update in the private NPM repository. Since the private NPM repository stores each version of a dependency package in the directory where the dependency package is located, updates the historical version information corresponding to the directory, and updates the name and version number of the latest downloaded dependency package in the historical version information.

[0077] Specifically, the server compares the version number of the currently traversed dependency package with the latest version number of the dependency package in the private NPM warehouse. If the version number of the currently traversed dependency package is greater than the latest version number in the historical version information, it means that the dependency package corresponding to the latest version number has not been downloaded to the private NPM warehouse, that is, the currently traversed dependency package has previously been checked for "whether the dependency package is stored in the private NPM warehouse", and the dependency package has been downloaded from the public NPM warehouse to the private NPM warehouse. However, in the process of checking the historical version information, the version number of the dependency package is not the latest version number, that is, the dependency package of this version number has not been downloaded to the private NPM warehouse. Based on this, it is determined that the currently traversed dependency package has failed to download in the previous step, so it is necessary to download the dependency package corresponding to the latest version number from the public NPM warehouse again, and store it in the directory corresponding to the dependency package, and update the latest version number to the historical version information, so as to ensure that the currently traversed dependency package is completely downloaded successfully.

[0078] Step S306: If the historical version information does not contain the version number of the currently traversed dependent package, the dependent package corresponding to the version number is re-downloaded from the public NPM repository, and the version number is updated to the historical version information.

[0079] Specifically, the server uses the version number of the currently traversed dependency package to determine whether the version number is included in the historical version information. If the historical version information does not contain the version number of the currently traversed dependency package, it means that the private NPM repository does not contain the currently traversed dependency package, and the directory where the currently traversed dependency package is located does not store the dependency package with the version number. Based on this, the server downloads the currently traversed dependency package from the public NPM repository again, stores it in the current directory, and updates the historical version information. This step is used to prevent the dependency package that has been downloaded to the private NPM repository from being deleted due to unpredictable reasons, thereby increasing protection measures.

[0080] In this embodiment, by traversing each dependency package corresponding to the client project in the private NPM repository, it is determined that the version number of each dependency package can be recorded in the historical version information, thereby ensuring that each dependency package is downloaded to the private NPM repository, thereby improving the stability of dependency package synchronization and reducing synchronization failures.

[0081] In one embodiment, the dependency package synchronization method further includes:

[0082] Traverse the dependency packages corresponding to the client project in the private NPM repository and determine the checksum of the currently traversed dependency package. If the checksum of the dependency package is inconsistent with the checksum of the corresponding dependency package in the public NPM repository, download the dependency package from the public NPM repository again.

[0083] Among them, checksum is a method used to check the integrity and consistency of files. Checksum is a set of values ​​obtained by performing mathematical operations on the file content, which is used to verify the integrity of the file and determine whether the file content is consistent. Common checksum algorithms include MD5, SHA1, SHA256, etc. The above algorithms all obtain a fixed-length checksum by performing hash calculations on the file content, and different file contents will obtain different checksums.

[0084] Specifically, the server traverses each dependency package corresponding to the client project in the private NPM repository, and determines the checksum of the currently traversed dependency package through a checksum algorithm. The server compares the checksum of the currently traversed dependency package with the checksum of the corresponding dependency package in the public NPM repository. If the checksum of the dependency package is inconsistent with the checksum of the corresponding dependency package in the public NPM repository, the dependency package is downloaded again from the public NPM repository, and the checksum of the re-downloaded dependency package is calculated, and the checksum of the re-downloaded dependency package is compared again with the checksum of the corresponding dependency package in the public NPM repository until the checksums of the two are consistent.

[0085] In this embodiment, the checksum is used to determine whether the dependent package in the private NPM repository is damaged, and the dependent package is re-downloaded until the checksum is consistent with the checksum of the corresponding dependent package in the public NPM repository, thereby ensuring the integrity and consistency of each downloaded dependent package and improving the stability of the synchronized dependent packages.

[0086] In one embodiment, the dependency package synchronization method further includes:

[0087] Check whether the cached document of the private NPM repository contains the hash value of the sorted set of dependent packages;

[0088] If the cache document contains the hash value of the dependent package sorted set, determining that the cache document contains the set identifier of the dependent package sorted set;

[0089] If the cache document does not contain the hash value of the dependent package sorted set, it is determined that the cache document does not contain the set identifier of the dependent package sorted set.

[0090] The hash value of the dependency package sorting set is a hash value calculated based on the name and version number of each dependency package in the dependency package sorting set, so the hash values ​​of different dependency package sorting sets are different. Based on this, the hash value can be used as a set identifier to identify a unique dependency package sorting set.

[0091] Specifically, the server calculates the hash value of the dependent package sorting set through a hash algorithm. The server detects whether the cache document of the private NPM repository contains the hash value of the dependent package sorting set. If the cache document contains the hash value of the dependent package sorting set, it is determined that the cache document contains the set identifier of the dependent package sorting set, and it also indicates that the server has checked whether each dependent package of the dependent package sorting set is stored in the private NPM repository, and has performed consistency checks on each dependent package. If the cache document does not contain the hash value of the dependent package sorting set, it is determined that the cache document does not contain the set identifier of the dependent package sorting set, and it also indicates that the server has not performed relevant checks on the dependent package sorting set.

[0092] In this embodiment, by checking whether the cached document contains the hash value of the sorted set of dependency packages, it is possible to quickly determine whether the dependency packages of the private NPM repository need to be updated, and it is also possible to quickly enable the client to download the required dependency packages from the private NPM repository, thereby improving the efficiency of the client installing dependency packages.

[0093] In one embodiment, the dependency package synchronization method further includes: calculating a hash value of the dependency package sorted set, and storing the hash value in a cache document.

[0094] Specifically, after the server performs a consistency check on each dependent package in the dependent package sorting set and sends a cached message to the client, the hash value of the dependent package sorting set is calculated through a hash algorithm, and the hash value of the dependent package sorting set is stored in the cache document of the private NPM repository, so that subsequent identical client projects can obtain the cached message through the cache document, thereby directly obtaining the required dependent packages from the private NPM repository.

[0095] In this embodiment, by storing the hash value of the dependency package sorting set in the cache document, the judgment process of other clients calling the dependency package corresponding to the dependency package sorting set can be reduced, thereby improving the efficiency of the client installing the dependency package.

[0096] As shown in FIG4 , the specific execution process of the above dependency package synchronization method is described in detail below in conjunction with a specific embodiment, including the following steps:

[0097] The specific implementation process in the client project is as follows:

[0098] Step 1: Send the content of package.json corresponding to the client project to the private NPM repository through the RESTFul interface.

[0099] Step 2: Use the npm install command to install the dependency package and set the registry parameter to the private npm repository address.

[0100] Step 3, installation completed.

[0101] The specific execution process in the private NPM repository is as follows:

[0102] Step 1: Receive the content of package.json in the RESTFul interface.

[0103] Step 2: Parse and generate a sorted set P of dependent packages through DFS calculation.

[0104] Step 3: Check whether the Hash of sorted set P has been cached. If it has been cached, it means that the consistency has been checked and the correct result is returned directly.

[0105] Step 4: Otherwise, it is necessary to traverse all dependent package sets P.

[0106] Step 5: Check whether the installation package (dependency package) corresponding to the sorted set P already exists. If it does, proceed directly to the consistency check.

[0107] Step 6: If it does not exist, synchronize the corresponding installation package (dependency package) from the public NPM repository to the private NPM repository.

[0108] Step 7: Check the version number of the corresponding installation package (dependency package), the tgz file of the corresponding version number, and repair it if they are inconsistent with the content.

[0109] Step 8: Check the consistency of the version number. The server reads the package.json file (a json file that only contains the historical version information of the dependent package) in the directory where the corresponding installation package (dependent package) is located, and checks whether its latest version number is consistent with the currently required version number. If it is inconsistent, it is necessary to download it from the public NPM repository through a program and overwrite the original package.json file.

[0110] Step 9: Check the directory where the corresponding installation package is located according to the currently required version number. <package> - <version>Check whether the .tgz file does not exist or is incomplete. If so, you need to download it from the public NPM repository and save it to the current directory through a program, and check the consistency of the dependent package content by calculating the checksum of the file. If it is inconsistent, retry until it is consistent.

[0111] Step 10, after checking the consistency problem, cache the Hash of the sorted set P into the cache file.

[0112] Step 11, return the correct result to the corresponding RESTFul interface.

[0113] Based on this, the embodiment of the present application synchronizes the dependency packages of the public NPM repository to the private NPM repository on demand. There is no need to synchronize the complete dependency packages of each version. It only needs to calculate and synchronize all the installation packages required by the client, thereby reducing the storage space, network bandwidth and hardware resources occupied.

[0114] Before installing the dependency package on the client, all dependency packages are automatically calculated and analyzed through the RESTFul interface, and the download is automatically synchronized and possible consistency issues are repaired. There is no need to manually synchronize the dependency packages or manually repair errors that may occur during the installation of the dependency packages, thereby automatically repairing the dependency packages in the private NPM repository and saving manual maintenance time.

[0115] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0116] Based on the same inventive concept, the embodiment of the present application also provides a dependent package synchronization device for implementing the dependent package synchronization method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more dependent package synchronization device embodiments provided below can refer to the limitations of the dependent package synchronization method above, and will not be repeated here.

[0117] In one embodiment, as shown in FIG. 5 , a dependency package synchronization device 500 is provided, including: a parsing module 501 , an acquisition module 502 and a sending module 503 , wherein:

[0118] The parsing module 501 is used to parse the dependency configuration information of the client project and determine the dependency package sorting set corresponding to the client project; the dependency package sorting set includes the names and version numbers of the dependency packages arranged in order;

[0119] The acquisition module 502 is used for acquiring the dependent packages from the public NPM repository and storing them in the private NPM repository for each dependent package included in the dependent package sorting set if the dependent package does not exist in the private NPM repository when it is determined that the cache document of the private NPM repository does not include the set identifier of the dependent package sorting set; the cache document includes the set identifier of the dependent package sorting set corresponding to the entered project;

[0120] The sending module 503 is used to send a cached message to the client including the client project after all dependency packages corresponding to the client project in the private NPM repository pass the consistency check; the cached message is used to enable the client to obtain the dependency package corresponding to the client project from the private NPM repository.

[0121] Furthermore, the device also includes a message return module, which is specifically used to return a cached message to a client including a client project when it is determined that the cache document of the private NPM repository contains a set identifier of a sorted set of dependency packages, so that the client can install the dependency package corresponding to the client project based on the dependency package installation command and the address of the private NPM repository.

[0122] Furthermore, the parsing module 501 is specifically used for:

[0123] Obtain the dependency configuration information of the client project through the client's data interface;

[0124] Based on the depth-first search strategy, the dependency configuration information is traversed to obtain the names and version numbers of multiple dependency packages arranged in sequence, thereby obtaining a sorted set of dependency packages corresponding to the client project.

[0125] Furthermore, the acquisition module 502 is specifically configured to:

[0126] Traverse each dependent package corresponding to the dependent package sorting set. If the private NPM repository does not contain the currently traversed dependent package, download the dependent package from the public NPM repository to the private NPM repository according to the name and version number of the dependent package;

[0127] If the private NPM repository contains dependent packages, traverse the next dependent package until the traversal of the sorted set of dependent packages is completed.

[0128] Furthermore, the device also includes a dependency package download module, which is specifically used to:

[0129] Traverse the dependent packages corresponding to the client project in the private NPM repository, and determine the historical version information of the directory where the currently traversed dependent package is located. The historical version information includes multiple historical version numbers corresponding to the dependent package;

[0130] If the version number of the currently traversed dependency package is greater than the latest version number in the historical version information, the dependency package corresponding to the latest version number is downloaded from the public NPM repository, stored in the directory corresponding to the dependency package, and the latest version number is updated to the historical version information;

[0131] If the historical version information does not contain the version number of the currently traversed dependent package, the dependent package corresponding to the version number is re-downloaded from the public NPM repository and the version number is updated to the historical version information.

[0132] Furthermore, the device also includes a consistency detection module, which is specifically used to:

[0133] Traverse the dependency packages corresponding to the client project in the private NPM repository and determine the checksum of the currently traversed dependency package;

[0134] If the checksum of the dependency package is inconsistent with the checksum of the corresponding dependency package in the public NPM repository, download the dependency package again from the public NPM repository.

[0135] Furthermore, the device also includes an identifier detection module, which is specifically used to:

[0136] Check whether the cached document of the private NPM repository contains the hash value of the sorted set of dependent packages;

[0137] If the cache document contains the hash value of the dependent package sorted set, determining that the cache document contains the set identifier of the dependent package sorted set;

[0138] If the cache document does not contain the hash value of the dependent package sorted set, it is determined that the cache document does not contain the set identifier of the dependent package sorted set.

[0139] Furthermore, the device also includes an identifier storage module, which is specifically used to:

[0140] Calculate the hash value of the sorted set of dependent packages and store the hash value in the cache document.

[0141] Each module in the above-mentioned dependent package synchronization device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0142] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the dependent packages required by each client project. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a dependent package synchronization method is implemented.

[0143] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0144] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0145] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0146] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0148] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.< / version> < / package>

Claims

1. A dependency package synchronization method, characterized in that: The method comprises: Parsing the dependency configuration information of the client project to determine a sorted set of dependency packages corresponding to the client project; the sorted set of dependency packages includes names and version numbers of multiple dependency packages arranged in order; Detect whether the cache document of the private NPM repository contains the hash value of the dependent package sorted set; if the cache document contains the hash value of the dependent package sorted set, determine that the cache document contains the set identifier of the dependent package sorted set; if the cache document does not contain the hash value of the dependent package sorted set, determine that the cache document does not contain the set identifier of the dependent package sorted set; In the case where it is determined that the cache document of the private NPM repository does not contain the set identifier of the dependency package sorting set, for each dependency package contained in the dependency package sorting set, if the dependency package does not exist in the private NPM repository, the dependency package is obtained from the public NPM repository and stored in the private NPM repository; the cache document contains the set identifier of the dependency package sorting set corresponding to the entered project; Performing version number consistency check on each of the dependent packages, and after passing the version number consistency check, performing file consistency check on each of the dependent packages; the consistency check includes the version number consistency check and the file consistency check; the consistency check is used to determine whether the name and version number of each dependent package corresponding to the client project in the private NPM warehouse meet the requirements of the client project, and determine whether each dependent package file corresponding to the client project in the private NPM warehouse is complete and whether there is any damage; After all dependency packages corresponding to the client project in the private NPM repository pass the consistency check, a cached message is sent to the client including the client project; a hash value of the sorted set of dependency packages is calculated, and the hash value is stored in the cache document; the cached message is used to enable the client to obtain the dependency package corresponding to the client project from the private NPM repository; In the event that it is determined that the cache document of the private NPM repository contains a collection identifier of the ordered collection of dependent packages, a cached message is returned to the client containing the client project.

2. The method according to claim 1, characterized in that After parsing the dependency configuration information of the client project and determining the sorted set of dependency packages corresponding to the client project, the method further includes: When it is determined that the cache document of the private NPM repository contains the set identifier of the sorted set of dependency packages, a cached message is returned to the client containing the client project, so that the client installs the dependency package corresponding to the client project based on the dependency package installation command and the address of the private NPM repository.

3. The method according to claim 1, characterized in that The step of parsing the dependency configuration information of the client project to determine a sorted set of dependency packages corresponding to the client project includes: Obtaining dependency configuration information of the client project through the data interface of the client; Based on a depth-first search strategy, the dependency configuration information is traversed to obtain the names and version numbers of multiple dependency packages arranged in sequence, thereby obtaining a sorted set of dependency packages corresponding to the client project.

4. The method according to claim 1, characterized in that For each dependent package included in the dependent package sorting set, if the dependent package does not exist in the private NPM repository, obtaining the dependent package from the public NPM repository and storing it in the private NPM repository includes: Traverse each dependent package corresponding to the dependent package sorting set, and if the private NPM repository does not contain the currently traversed dependent package, download the dependent package from the public NPM repository to the private NPM repository according to the name and version number of the dependent package; If the private NPM repository contains the dependent package, traverse the next dependent package until the traversal of the sorted set of dependent packages is completed.

5. The method according to claim 1, characterized in that Before sending the cached message to the client including the client project, the method further comprises: Traversing each dependent package corresponding to the client project in the private NPM repository, determining the historical version information of the directory where the currently traversed dependent package is located, the historical version information including multiple historical version numbers corresponding to the dependent package; If the version number of the currently traversed dependent package is greater than the latest version number in the historical version information, then download the dependent package corresponding to the latest version number from the public NPM repository, store it in the directory corresponding to the dependent package, and update the latest version number to the historical version information; If the historical version information does not contain the version number of the currently traversed dependent package, the dependent package corresponding to the version number is re-downloaded from the public NPM repository, and the version number is updated to the historical version information.

6. The method according to claim 1, characterized in that Before sending the cached message to the client including the client project, the method further comprises: Traverse each dependency package corresponding to the client project in the private NPM repository, and determine the checksum of the dependency package currently traversed; If the checksum of the dependent package is inconsistent with the checksum of the corresponding dependent package in the public NPM warehouse, the dependent package is downloaded again from the public NPM warehouse.

7. A dependent package synchronization device, characterized in that: The device comprises: A parsing module, used to parse the dependency configuration information of the client project and determine a sorted set of dependency packages corresponding to the client project; the sorted set of dependency packages includes names and version numbers of multiple dependency packages arranged in order; An identifier detection module, used to detect whether the cache document of the private NPM repository contains the hash value of the dependent package sorting set; if the cache document contains the hash value of the dependent package sorting set, it is determined that the cache document contains the set identifier of the dependent package sorting set; if the cache document does not contain the hash value of the dependent package sorting set, it is determined that the cache document does not contain the set identifier of the dependent package sorting set; An acquisition module, for obtaining, when it is determined that the cache document of the private NPM repository does not contain the set identifier of the dependency package sorting set, for each dependency package contained in the dependency package sorting set, if the dependency package does not exist in the private NPM repository, the dependency package is acquired from the public NPM repository and stored in the private NPM repository; the cache document contains the set identifier of the dependency package sorting set corresponding to the entered project; A consistency detection module is used to perform version number consistency detection on each of the dependent packages, and after passing the version number consistency detection, perform file consistency detection on each of the dependent packages; the consistency detection includes the version number consistency detection and the file consistency detection; the consistency detection is used to determine whether the name and version number of each dependent package corresponding to the client project in the private NPM warehouse meet the requirements of the client project, and determine whether each dependent package file corresponding to the client project in the private NPM warehouse is complete and whether there is any damage; A sending module is used to send a cached message to the client including the client project after all dependency packages corresponding to the client project in the private NPM repository pass the consistency check; calculate the hash value of the sorted set of dependency packages and store the hash value in the cache document; the cached message is used to enable the client to obtain the dependency package corresponding to the client project from the private NPM repository; when it is determined that the cache document of the private NPM repository contains the set identifier of the sorted set of dependency packages, return the cached message to the client including the client project.

8. The device according to claim 7, characterized in that The analysis module is specifically used for: Obtain the dependency configuration information of the client project through the client's data interface; Based on the depth-first search strategy, the dependency configuration information is traversed to obtain the names and version numbers of multiple dependency packages arranged in sequence, thereby obtaining a sorted set of dependency packages corresponding to the client project.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.