A method and device for compressing and operating an image file
By calculating the summary information of Jar packages in Java projects and removing redundant Jar packages, the problems of excessive image volume and high network bandwidth requirements in distributed systems are solved, and the compression operation of mirror files is achieved, reducing transmission time and network congestion.
Patent Information
- Application Number
- CN202110473227.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-29
- Publication Date
- 2025-02-28
- Estimated Expiration
- 2041-04-29
AI Technical Summary
In distributed system development, the Docker image of each development module is too large due to redundant third-party Jar packages, which increases network bandwidth requirements and affects application system upgrades.
By calculating the Jar package summary information in Java projects, removing redundant Jar packages, and copying the redundant Jar packages into the image file, the compressed operation of the image file is achieved.
It reduces the transfer time of mirror files, alleviates network congestion, and solves the problem of excessive image packaging volume.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology and can be used in the financial field, and specifically is a method and device for compressing and operating an image file. Background Art
[0002] Currently, in the distributed system development scenario, developers often split the business logic into multiple independent development modules, and then develop each development module in the mode of independent Java projects. When developing the system according to this method, all projects and their dependent third-party Jar packages will be combined and packaged into a unified Docker image, and then the unified Docker image will be used to select the corresponding development module according to different image nodes to start the container for deployment.
[0003] This method has the following disadvantages:
[0004] 1. Each project carries its own referenced third-party Jar packages. Since these Jar packages may use the same technical architecture, there may be redundant identical Jar packages between projects.
[0005] 2. When all projects are combined and packaged into a Docker image, due to the above redundancy, each project introduces the same Jar package, and the image size will increase to N times the size of the Jar package.
[0006] 3. When the image container is started, a large number of application server nodes will pull the image at the same time, which requires high network bandwidth and easily causes network slowness and congestion, seriously affecting the application system upgrade. Summary of the invention
[0007] In view of the problems in the prior art, the present application provides a method and device for compressing and running an image file, which can remove redundant Jar packages in a Java project according to the summary information of the Jar package and realize the compression and operation of the image file.
[0008] In order to solve the above technical problems, this application provides the following technical solutions:
[0009] In a first aspect, the present application provides a method for compressing and operating an image file, comprising:
[0010] Obtain the Jar package corresponding to the Java project and calculate the summary information of the Jar package, where the Jar package includes the third-party Jar package and the project Jar package;
[0011] Removing redundant Jar packages according to the summary information;
[0012] The Jar package after redundancy removal is copied into the image file and the image file is run.
[0013] Furthermore, the calculation of summary information of the Jar package includes:
[0014] Create a package name summary information set for each Java project;
[0015] Calculate the summary information of the Jar package for each Java project respectively, and store the Jar package name of the third-party Jar package, the Jar package name of the project Jar package and the summary information into a corresponding package name summary information set;
[0016] The package name summary information sets corresponding to each Java project are merged and stored in a pre-created total set.
[0017] Further, the removing redundant Jar packages according to the summary information includes:
[0018] Update the package counter corresponding to the Jar package according to the Jar package name and the corresponding summary information recorded in the package name summary information set, and put each package counter into the package counter set;
[0019] Screening the Jar packages with the same summary information in the package counter set to obtain redundant Jar packages that should be removed;
[0020] According to the redundant Jar package, the pre-initialized engineering Jar package set is updated to remove the redundant Jar package;
[0021] Use the updated project Jar package set to update the pre-created project deduplication Jar package set.
[0022] Furthermore, after the pre-created project deduplication Jar package set is updated with the updated project Jar package set, the method further includes:
[0023] Concatenate the Jar packages in the project Jar package collection of each project to generate classpath data;
[0024] Copy the generated classpath data to the directory corresponding to the Java project.
[0025] Furthermore, copying the Jar package after removing redundancy into the image file and running the image file includes:
[0026] Call the Docker command to copy the Jar package after removing redundancy and the directory of the Java project into the image file;
[0027] The image file stored in the local image warehouse server is pushed to the remote image warehouse server, and the image file is run.
[0028] In a second aspect, the present application provides an image file compression and operation device, comprising:
[0029] A summary information calculation unit, used to obtain a Jar package corresponding to a Java project and calculate summary information of the Jar package, wherein the Jar package includes a third-party Jar package and a project Jar package;
[0030] A redundancy removal unit, used to remove redundant Jar packages according to the summary information;
[0031] The image running unit is used to copy the Jar package after redundancy is removed into the image file and run the image file.
[0032] Furthermore, the summary information calculation unit includes:
[0033] The package name summary information set creation module is used to create a package name summary information set for each Java project;
[0034] A summary information calculation module, used to calculate the summary information of the Jar package for each Java project, and store the Jar package name of the third-party Jar package, the Jar package name of the project Jar package and the summary information into a corresponding package name summary information set;
[0035] The merging and storing module is used to merge and store the package name summary information sets corresponding to each Java project into a pre-created total set.
[0036] Furthermore, the redundancy removal unit includes:
[0037] A package counter set updating module, used for updating the package counters corresponding to the Jar package according to the Jar package name and the corresponding summary information recorded in the package name summary information set, and putting each package counter into the package counter set;
[0038] A redundant screening module, used for screening the Jar packages with the same summary information in the package counter set, and obtaining the redundant Jar packages to be removed;
[0039] A redundancy removal module, used to update a pre-initialized engineering Jar package set according to the redundant Jar package, and remove the redundant Jar package;
[0040] The total set update module is used to update the pre-created project deduplicated Jar package set with the updated project Jar package set.
[0041] Furthermore, the redundancy removal unit further includes:
[0042] The splicing module is used to splice the Jar packages in the project Jar package set of each project to generate classpath data;
[0043] The copy module is used to copy the generated classpath data to the directory of the corresponding Java project.
[0044] Furthermore, the mirroring operation unit includes:
[0045] A copy module is used to call the Docker command to copy the Jar package after redundancy removal and the directory of the Java project into the image file;
[0046] The push module is used to push the image file stored in the local image warehouse server to the remote image warehouse server and run the image file.
[0047] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image file compression and operation method when executing the program.
[0048] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image file compression and operation method.
[0049] In response to the problems in the prior art, the image file compression operation method and device provided in the present application can solve the problem of excessively large image packaging volume of distributed system engineering modules, remove redundant Jar packages in Java projects according to the summary information of Jar packages, realize image file compression operation, reduce the image file transmission time, and alleviate the network congestion problem caused by this. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0051] Figure 1 This is one of the flow charts of the image file compression operation method in the embodiment of the present application;
[0052] Figure 2 A flowchart of calculating summary information of the Jar package in an embodiment of the present application;
[0053] Figure 3 This is a flowchart of removing redundant Jar packages according to the summary information in an embodiment of the present application;
[0054] Figure 4 This is the second flowchart of the image file compression operation method in the embodiment of the present application;
[0055] Figure 5 This is a flowchart of copying and running an image file in an embodiment of the present application;
[0056] Figure 6 This is one of the structural diagrams of the image file compression and operation device in the embodiment of the present application;
[0057] Figure 7 This is a structural diagram of a summary information calculation unit in an embodiment of the present application;
[0058] Figure 8 This is a structural diagram of a redundancy removal unit in an embodiment of the present application;
[0059] Fig. 9 This is a structural diagram of a redundancy removal unit in an embodiment of the present application;
[0060] Fig.10 This is a structural diagram of the mirror operation unit in the embodiment of the present application;
[0061] Fig.11 A schematic diagram of the structure of an electronic device in an embodiment of the present application;
[0062] Fig.12 This is a structural block diagram of a Docker image volume compression system based on a Maven project in an embodiment of the present application;
[0063] Fig.13 It is a structural diagram of the image packaging device in an embodiment of the present application. DETAILED DESCRIPTION
[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0065] It should be noted that the image file compression operation method and device provided in the present application can be used in the financial field, and can also be used in any field other than the financial field. The application field of the image file compression operation method and device provided in the present application is not limited.
[0066] See also Figure 1 In order to remove redundant Jar packages in a Java project according to the summary information of the Jar package and realize image file compression and operation, the present application provides an image file compression and operation method, including:
[0067] S101: Obtain the Jar package corresponding to the Java project and calculate summary information of the Jar package, where the Jar package includes a third-party Jar package and a project Jar package;
[0068] S102: removing redundant Jar packages according to the summary information;
[0069] S103: copying the Jar package after redundancy removal into a mirror file and running the mirror file.
[0070] Understandably, see Fig.12 The image file compression operation method provided in this application is implemented based on the Docker image volume compression system of the Maven project. The system includes a Java engineering development tool 1, a Maven server 2, a Git server 3, a compilation server 4, an image warehouse 5 and an application server group 6.
[0071] Specifically, the Java engineering development tool 1 can be an integrated development environment IDE (Integrated Development Environment) such as Eclipse or IDEA with Maven plug-ins or functions. The Java engineering development tool 1 is connected to the Git server 3 to download the source code of the distributed system module from the Git server 3 to the local, and the downloaded content includes the source code projects javaprj1, javaprj2, javaprj3...javaprjn. At the same time, the Java engineering development tool 1 is also connected to the Maven server 2 to pull the third-party Jar packages that each project depends on to the local warehouse and compile the project. After completing the code writing and local debugging locally, the code can be pushed to the Git server 3, and the Jar packages of the public projects can be compiled and pushed to the Maven server 2.
[0072] Maven server 2 is responsible for third-party Jar package management and provides Jar package download services for Java project development tool 1 and compilation server 4. Java project development tool 1 needs to connect to Maven server 2 to pull relevant Jar packages to local during code development stage and compilation server 4 needs to connect to Maven server 2 during code compilation stage.
[0073] The Git server 3 is responsible for source code version management of each module project of the distributed system. The Java project development tool 1 in the code development stage and the compile server 4 in the code compilation stage both need to connect to the Git server 3 to pull the source code of each module project of the distributed system.
[0074] The compilation server 4 is responsible for compiling the Java source code of each module project of the distributed system, deduplication calculation of Jar packages, generation of Java class path files, making images and storing them in the local warehouse, and pushing the images to the remote image warehouse, etc. Before executing the compilation, the compilation server 4 connects to the Git server 3 to download the source code, and connects to the Maven server 2 to download the dependent Jar packages. After the image is made, it pushes the image to the image warehouse server 5. All project modules and their dependent Jars are combined and packaged into the image file. Each project module corresponds to the module in the image file, javaprj1 in the Java project development tool 1 corresponds to module1, javaprj2 corresponds to module2, and so on for other projects.
[0075] The image warehouse server 5 is responsible for image storage. The image warehouse server 5 can receive the image pushed by the compilation server 4 and provide image pulling services for the application server cluster 6. When the application server cluster 6 deploys the service, it pulls the image from the image warehouse server 5 and starts the container.
[0076] The application server cluster 6 is responsible for pulling the image from the image warehouse server 5 to the application server node when the system is upgraded, and then selecting the corresponding module in the image to start according to the business service functions undertaken by different nodes. There can be multiple application server nodes, among which application server 1 selects module 1 in the image to start the container, application server 2 selects module 2 in the image to start the container, and so on for other application server nodes. When the container starts, it reads the classpath file of the module and loads the dependent Jar package in the classpath file to start.
[0077] In this Docker image volume compression system based on the Maven project, in order to compress the image volume, the embodiment of the present application needs to check the Jar packages contained in the Maven project for duplication and delete the duplicate and redundant Jar packages. To this end, the embodiment of the present application can first obtain the Jar package corresponding to the Java project and calculate the summary information of the Jar package, which includes the third-party Jar package and the project Jar package; then, remove the redundant Jar package according to the summary information; finally, copy the Jar package after the redundancy is removed into the image file and run the image file. Among them, the summary information can refer to the parameters that can mark the Jar package. In the embodiment of the present application, the MD5 value of the Jar package is used as its summary information.
[0078] The process of copying the Jar package after removing redundancy into the image file is usually performed by the compiling server 4 , and the process of running the image file is usually performed by the application server cluster 6 .
[0079] From the above description, it can be seen that the image file compression and operation method provided by the present application can solve the problem of excessive size of the image packaging of the distributed system engineering module, remove the redundant Jar packages in the Java project according to the summary information of the Jar package, realize the image file compression and operation, reduce the image file transmission time, and alleviate the network congestion problem caused by it.
[0080] See also Figure 2 , calculate the summary information of the Jar package, including:
[0081] S201: Create a package name summary information set for each Java project;
[0082] S202: Calculate the summary information of the Jar package for each Java project respectively, and store the Jar package name of the third-party Jar package, the Jar package name of the project Jar package and the summary information into a corresponding package name summary information set;
[0083] S203: merging and storing the package name summary information sets corresponding to the Java projects into a pre-created total set.
[0084] It is understandable that the embodiment of the present application can download the source code of each project from the Git server 3 to the compile server 4. When the compile server 4 starts to build the image, the Git command can be used to download the source code of the project of each module of the distributed system from the Git server 3 to the local. The directory examples of each project are as follows:
[0085]
[0086] After the project source code is downloaded, you can download the Jar packages that each project depends on from Maven server 2.
[0087] According to the POM (project object model) files in each source code project, you can use the mvn command to process each project one by one, and download the jar packages that each project depends on from Maven server 2 to the following directories (example):
[0088]
[0089] After the jar package is downloaded, you can compile the project source code. Loop through each project to compile the source code and copy the compiled project jar package to a directory such as / somepath / javaprji / lib, where i = 1, 2, 3...n.
[0090] Finally, calculate the MD5 value corresponding to the Jar package name of each project, and each MD5 value can be recorded using the Jar package record table. The same MD5 (and Jar package name) value means the same Jar package. In the subsequent steps, the same Jar package can be screened out based on the MD5 value and removed.
[0091] One implementation of the above process is to pre-set a total set MN and several package name summary information sets MI. The package name summary information set MI is used to store the name of each Jar package and its corresponding MD5 value. All Jar packages in each project correspond to a package name summary information set MI; the total set MN is used to store each package name summary information set MI.
[0092] When screening for the same MD5, you can retrieve the package name summary information set MI of the Jar package corresponding to the project by the project name. By retrieving the package name summary information set MI, you can get the MD5 of the corresponding Jar package.
[0093] From the above description, it can be seen that the image file compression and operation method provided by the present application can calculate the summary information of the Jar package.
[0094] See also Figure 3 , removing redundant Jar packages according to the summary information, including:
[0095] S301: updating the package counter corresponding to the Jar package according to the Jar package name and the corresponding summary information in the package name summary information set MI recorded in the total set MN, and storing the package counters in a unified manner in the package counter set MC;
[0096] S302: Screening Jar packages with the same summary information in the package counter set to obtain redundant Jar packages that should be removed;
[0097] S303: updating the pre-initialized engineering Jar package set PS according to the redundant Jar package, and removing the redundant Jar package;
[0098] S304: Use the updated project Jar package set PS to update the pre-created project deduplicated Jar package set PSS.
[0099] It can be understood that this step is to deduplicate Jar packages with the same Jar package name and MD5 value, and rename Jar packages with the same Jar package name but different MD5 values. Here, the MD5 value is not calculated based on the name of the Jar package, but the MD5 value is calculated for the content of the Jar package file (binary file); Maven manages third-party Jar packages, and there are multiple SNAPSHOT versions. Different versions may be downloaded at different times. Considering that the compilation process does not occur at a certain moment, but will continue for a period of time, different projects may obtain different SNAPSHOT versions at different times. In this way, the same Jar package name may correspond to different file contents, so the Jar packages with the same Jar package name but different MD5 values need to be renamed during the processing.
[0100] (1) For all projects, create a temporary directory to store the deduplicated Jar packages.
[0101] (2) Initialize a deduplicated Jar package set PSS for a project. Initialize a package counter set MC to record the number of times the same Jar package appears. Each Jar package corresponds to a package counter, and all package counters are stored in the package counter set MC; then initialize a temporary package name summary information set MM, one MM for all projects, and multiple package name summary information for each Jar package, which are stored in the temporary package name summary information set MM; then initialize multiple project Jar package sets PS to record all Jar packages in the project, and each project corresponds to a project Jar package set PS. The deduplicated Jar package set PSS for a project includes the Jar package sets PS for each project.
[0102] (3) Process each Jar package in each project in turn, retrieve the total set MN by the project name, obtain the package name summary information set MI, and retrieve the package name summary information set MI by recording the Jar package name to obtain the summary information (MD5 value) of all Jar packages. (Corresponding to step S301)
[0103] (4) This step describes a process of putting Jar packages with different summary information into a temporary package name summary information set MM:
[0104] If the Jar package does not exist in the package counter set MC, initialize a package counter, put the package counter corresponding to the Jar package into the package counter set MC, copy the Jar package to a temporary directory, put the summary information (MD5 value) corresponding to the Jar package into the temporary package name summary information set MM, and put the Jar package corresponding to the project into the project Jar package set PS.
[0105] (5) This step describes another process of putting Jar packages with different summary information into the temporary package name summary information set MM:
[0106] If the Jar package exists in the package counter set MC and the MD5 value of the Jar package exists in the package name summary information set MM, then the Jar package is taken out and the Jar package corresponding to the project is put into the project Jar package set PS.
[0107] (6) This step describes another process of putting Jar packages with different summary information into the temporary package name summary information set MM:
[0108] If the package counter set MC contains the package counter corresponding to the Jar package, and the summary information (MD5 value) of the Jar package does not exist in the package name summary information set MM, then take out the Jar package, add 1 to the corresponding package counter value, then put the package counter back into the package counter set MC, rename the Jar package to the original name of the Jar package - the counter value, put the summary information (MD5 value) of the Jar package into the package name summary information set MM, copy the Jar package to a temporary directory, and put the Jar package corresponding to the project into the project Jar package set PS.
[0109] (7) Put the updated project Jar package set PS into the project deduplicated Jar package set PSS.
[0110] After the above seven steps, the duplicate Jar packages in the entire project can be screened and deduplicated, and finally the PSS of the Jar packages after deduplication in the project can be updated. In the subsequent steps, the PSS of the Jar packages after deduplication in the project can be used to generate classpath data (file) to obtain the Jar packages after removing redundancy.
[0111] It can be seen from the above description that the image file compression and operation method provided by the present application can remove redundant Jar packages according to the summary information.
[0112] See also Figure 4 After the pre-created project deduplication Jar package set PSS is updated with the updated project Jar package set, it also includes:
[0113] S401: Splicing the Jar packages in the project Jar package set PS of each project to generate classpath data;
[0114] S402: Copy the generated classpath data to the directory corresponding to the Java project.
[0115] It is understandable that the embodiment of the present application can create a Jar package merge directory / somepath / mergelib / (example), and copy the Jar package in the temporary directory to the / somepath / mergelib / directory (example).
[0116] Then, generate the classpath file for each project and copy it to each project directory.
[0117] Specifically, the project-Jar package set PS in S301 to S303 is processed in a loop, and each Jar package name is concatenated to generate a classpath file. Then, the generated classpath file of each project is copied to the corresponding project directory (example):
[0118]
[0119] From the above description, it can be seen that the image file compression and operation method provided by the present application can copy the generated classpath data (file) to the directory of the corresponding Java project.
[0120] See also Figure 5 , copy the Jar package after removing redundancy into the image file and run the image file, including:
[0121] S501: Calling a Docker command to copy the Jar package after redundancy removal and the directory of the Java project into an image file;
[0122] S502: Push the image file stored in the local image repository server to the remote image repository server, and run the image file.
[0123] It is understandable that the third-party Jar package, project Jar package and classpath file after deduplication are packaged into the image directory.
[0124] For example, copy the project directory / somepath / javaprji / to the directory / somepathinimage / javaprji / in the image, i=1,2,3...n, copy / somepath / mergelib / to / somepathinimage / mergelib / , and the image will be in the following directory:
[0125]
[0126] Then, build and push the image to the image repository server 5.
[0127] Specifically, you can use the Docker command to build an image to a local image repository, and then push the image in the local image repository to a remote image repository server.
[0128] By adopting the image active compression method provided in this application, the image size is greatly reduced after implementing Jar package deduplication processing.
[0129] It can be seen from the above description that the image file compression and operation method provided in the present application can copy the Jar package after removing redundancy into the image file and run the image file.
[0130] Fig.13 It is a structural diagram of an image packaging device, which includes a Jar package downloading unit 201, a compiling unit 202, a class path calculating unit 203 and an image making unit 204.
[0131] The Jar package downloading unit 201 is connected to the Maven server 2, can send commands to the Maven server 2, and download the third-party Jar packages that each project depends on to the local computer, and store them according to the following rules:
[0132]
[0133] The compilation unit 202 connects to the Git server 3, and pulls the source code of each project of the distributed system to the compilation server, compiles it into javaprji.jar, i=1,2,3...n, and installs it to the local Maven warehouse, and moves it to the / somepath / javaprji / lib / directory, i=1,2,3...n.
[0134] The class path calculation unit 203 processes the Jar downloaded by the Jar package download unit 201 one by one, calculates the MD5 value for the Jar package of each project, and uses the Jar package record table to record its name and MD5 value. If neither of them exists in the record table, the Jar package is copied to / somepath / mergelib / , and its name and MD5 value are recorded in the record table. If only the name exists in the record table, the package counter corresponding to the Jar package is increased by 1, and then the original Jar package name is spliced with the counter value as the new name of the Jar package to modify the Jar package name, and the original Jar package name and its corresponding MD5 value are recorded in the record table. The merged paths of the Jar packages of each project are spliced and written into the classpath file until the Jar package of the project is processed, and the classpath file is copied to / somepath / javaprjn. The Jar packages of each project are processed in a loop until all Jar packages are processed.
[0135] The image production unit 204 copies the deduplicated Jar package directory and classpath file of the classpath calculation unit 203 to the image production directory, places them according to the following structure, and then produces them into an image and stores them in the local warehouse, and pushes them to the image warehouse server 5.
[0136] / somepathinimage / javaprj1 /
[0137] / somepathinimage / javaprj1 / classpath
[0138] / somepathinimage / javaprj2 /
[0139] / somepathinimage / javaprj2 / classpath
[0140] / somepathinimage / javaprj3 /
[0141] / somepathinimage / javaprj3 / classpath ...
[0142] / somepathinimage / javaprjn /
[0143] / somepathinimage / javaprjn / classpath
[0144] / somepathinimage / mergelib /
[0145] Based on the same inventive concept, the embodiments of the present application also provide a mirror file compression operation device, which can be used to implement the method described in the above embodiments, as described in the following embodiments. Since the principle of solving the problem by the mirror file compression operation device is similar to that of the mirror file compression operation method, the implementation of the mirror file compression operation device can refer to the implementation of the method based on the software performance benchmark determination, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.
[0146] See also Figure 6 In order to remove redundant Jar packages in a Java project according to the summary information of the Jar package and realize image file compression and operation, the present application provides an image file compression and operation device, including:
[0147] The summary information calculation unit 601 is used to obtain the Jar package corresponding to the Java project and calculate the summary information of the Jar package, wherein the Jar package includes the third-party Jar package and the project Jar package;
[0148] A redundancy removal unit 602, configured to remove redundant Jar packages according to the summary information;
[0149] The image running unit 603 is used to copy the Jar package after redundancy is removed into the image file and run the image file.
[0150] See also Figure 7 , the summary information calculation unit 601 includes:
[0151] A package name summary information set creation module 701 is used to create a package name summary information set for each Java project;
[0152] The summary information calculation module 702 is used to calculate the summary information of the Jar package for each Java project, and store the Jar package name of the third-party Jar package, the Jar package name of the project Jar package and the summary information into a corresponding package name summary information set;
[0153] The merging and storing module 703 is used to merge and store the package name summary information sets corresponding to each Java project into a pre-created total set.
[0154] See also Figure 8 , the redundancy removal unit 602 comprises:
[0155] A package counter set updating module 801 is used to update a package counter set corresponding to the Jar package according to the Jar package name and the corresponding summary information recorded in the package name summary information set;
[0156] A redundant screening module 802 is used to screen the Jar packages with the same summary information in the package counter set to obtain redundant Jar packages that should be removed;
[0157] A redundancy removal module 803, used to update a pre-initialized engineering Jar package set according to the redundant Jar package, and remove the redundant Jar package;
[0158] The total set updating module 804 is used to update the total set using the updated engineering Jar package set.
[0159] See also Fig. 9 , the redundancy removal unit 602 further includes:
[0160] A splicing module 901 is used to splice the Jar packages in the project Jar package set of each project to generate classpath data;
[0161] The copy module 902 is used to copy the generated classpath data to the directory corresponding to the Java project.
[0162] See also Fig.10 The mirror operation unit 603 includes:
[0163] A copy module 1001 is used to call a Docker command to copy the Jar package after redundancy removal and the directory of the Java project into an image file;
[0164] The push module 1002 is used to push the image file stored in the local image repository server to the remote image repository server and run the image file.
[0165] From the hardware level, in order to remove redundant Jar packages in a Java project according to the summary information of the Jar package and realize the image file compression operation, the present application provides an embodiment of an electronic device for realizing all or part of the contents of the image file compression operation method, and the electronic device specifically includes the following contents:
[0166] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the image file compression operation device and related devices such as core business systems, user terminals and related databases; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., and the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the image file compression operation method and the embodiment of the image file compression operation device in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.
[0167] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.
[0168] In practical applications, part of the image file compression operation method can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.
[0169] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.
[0170] Fig.11 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig.11 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig.11 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.
[0171] In one embodiment, the image file compression and operation method function may be integrated into the central processing unit 9100. The central processing unit 9100 may be configured to perform the following control:
[0172] S101: Obtain the Jar package corresponding to the Java project and calculate summary information of the Jar package, where the Jar package includes a third-party Jar package and a project Jar package;
[0173] S102: removing redundant Jar packages according to the summary information;
[0174] S103: copying the Jar package after redundancy removal into a mirror file and running the mirror file.
[0175] From the above description, it can be seen that the image file compression and operation method provided by the present application can solve the problem of excessive size of the image packaging of the distributed system engineering module, remove the redundant Jar packages in the Java project according to the summary information of the Jar package, realize the image file compression and operation, reduce the image file transmission time, and alleviate the network congestion problem caused by it.
[0176] In another embodiment, the image file compression operation device can be configured separately from the central processing unit 9100. For example, the data composite transmission device image file compression operation device can be configured as a chip connected to the central processing unit 9100, and the function of the image file compression operation method is realized through the control of the central processing unit.
[0177] like Fig.11As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig.11 In addition, the electronic device 9600 may also include Fig.11 For components not shown, reference may be made to the prior art.
[0178] like Fig.11 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.
[0179] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.
[0180] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.
[0181] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.
[0182] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).
[0183] The communication module 9110 is a transmitter / receiver 9110 that sends and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.
[0184] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.
[0185] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the image file compression and operation method in the above-mentioned embodiments, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the image file compression and operation method in the above-mentioned embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:
[0186] S101: Obtain the Jar package corresponding to the Java project and calculate summary information of the Jar package, where the Jar package includes a third-party Jar package and a project Jar package;
[0187] S102: removing redundant Jar packages according to the summary information;
[0188] S103: copying the Jar package after redundancy removal into a mirror file and running the mirror file.
[0189] From the above description, it can be seen that the image file compression and operation method provided by the present application can solve the problem of excessive size of the image packaging of the distributed system engineering module, remove the redundant Jar packages in the Java project according to the summary information of the Jar package, realize the image file compression and operation, reduce the image file transmission time, and alleviate the network congestion problem caused by it.
[0190] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0191] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0192] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0193] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0194] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A method for compressing and running an image file, characterized in that: include: Obtain the Jar package corresponding to the Java project and calculate the summary information of the Jar package, where the Jar package includes the third-party Jar package and the project Jar package; Removing redundant Jar packages according to the summary information; Copy the Jar package after removing redundancy into the image file and run the image file; The Java project development tool is also connected to the Maven server to pull the third-party Jar packages that each project depends on to the local warehouse and compile the project; Before executing the compilation, the compilation server connects to the Git server to download the source code, and connects to the Maven server to download the dependent Jar packages. After the image is made, it pushes the image to the image warehouse server; The step of calculating the summary information of the Jar package includes: Create a package name summary information set for each Java project; Calculate the summary information of the Jar package for each Java project respectively, and store the Jar package name of the third-party Jar package, the Jar package name of the project Jar package and the summary information into a corresponding package name summary information set; Merge and store the package name summary information sets corresponding to each Java project into a pre-created total set; The removing of redundant Jar packages according to the summary information includes: Update the package counter corresponding to the Jar package according to the Jar package name and the corresponding summary information recorded in the package name summary information set, and put each package counter into the package counter set; Screening the Jar packages with the same summary information in the package counter set to obtain redundant Jar packages that should be removed; According to the redundant Jar package, the pre-initialized engineering Jar package set is updated to remove the redundant Jar package; Use the updated project Jar package set to update the pre-created project deduplication Jar package set.
2. The image file compression and operation method according to claim 1, characterized in that: After the pre-created project deduplication Jar package set is updated with the updated project Jar package set, it also includes: Concatenate the Jar packages in the project Jar package collection of each project to generate classpath data; Copy the generated classpath data to the directory corresponding to the Java project.
3. The image file compression and operation method according to claim 2, characterized in that: The step of copying the Jar package after removing redundancy into the image file and running the image file includes: Call the Docker command to copy the Jar package after removing redundancy and the directory of the Java project into the image file; The image file stored in the local image warehouse server is pushed to the remote image warehouse server, and the image file is run.
4. A device for compressing and operating an image file, characterized in that: include: A summary information calculation unit, used to obtain a Jar package corresponding to a Java project and calculate summary information of the Jar package, wherein the Jar package includes a third-party Jar package and a project Jar package; A redundancy removal unit, used to remove redundant Jar packages according to the summary information; The image running unit is used to copy the Jar package after removing redundancy into the image file and run the image file; wherein the Java project development tool is also connected to the Maven server to pull the third-party Jar package that each project depends on to the local warehouse and compile the project; Before executing the compilation, the compilation server connects to the Git server to download the source code, and connects to the Maven server to download the dependent Jar packages. After the image is made, it pushes the image to the image warehouse server; Wherein, the summary information calculation unit includes: The package name summary information set creation module is used to create a package name summary information set for each Java project; A summary information calculation module, used to calculate the summary information of the Jar package for each Java project, and store the Jar package name of the third-party Jar package, the Jar package name of the project Jar package and the summary information into a corresponding package name summary information set; A merging and storing module is used to merge and store the package name summary information sets corresponding to each Java project into a pre-created total set; Wherein, the redundancy removal unit includes: A package counter set updating module, used for updating the package counter corresponding to the Jar package according to the Jar package name and the corresponding summary information recorded in the package name summary information set, and putting each package counter into the package counter set; A redundant screening module, used for screening the Jar packages with the same summary information in the package counter set, and obtaining the redundant Jar packages to be removed; A redundancy removal module, used to update a pre-initialized engineering Jar package set according to the redundant Jar package, and remove the redundant Jar package; The total set update module is used to update the pre-created project deduplicated Jar package set with the updated project Jar package set.
5. The image file compression and operation device according to claim 4, characterized in that: The redundancy removal unit further includes: The splicing module is used to splice the Jar packages in the project Jar package set of each project to generate classpath data; The copy module is used to copy the generated classpath data to the directory of the corresponding Java project.
6. The image file compression and operation device according to claim 5, characterized in that: The mirroring operation unit includes: A copy module is used to call the Docker command to copy the Jar package after redundancy removal and the directory of the Java project into the image file; The push module is used to push the image file stored in the local image warehouse server to the remote image warehouse server and run the image file.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the image file compression and operation method described in any one of claims 1 to 3 are implemented.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image file compression operation method described in any one of claims 1 to 3 are implemented.
Citation Information
Patent Citations
Virtual machine mirror image file storage and distribution method and device
CN104239575A
Method for deleting container mirror image data in cloud environment
CN107809467A
Container mirror image deduplication method and system, computer equipment and storage medium
CN112104725A