External library centralized control method, device, equipment, medium and product
By using the intermediate base version and Kubernetes dependency library in Java compiled language to share the external mount deployment method, the external package version dependency is decoupled, and the version conflict problem caused by the update of external package version in Java applications is solved, and the problem of version conflicts caused by the update of external package version in Java applications is achieved. External package version management is performed when updating the upper-level application code.
Patent Information
- Application Number
- CN202411065292.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-12
AI Technical Summary
Java compiled languages have version compatibility issues when referring to dependency libraries. Frequent updates of external package versions cause upper-level applications to passively update secure versions, and cannot resolve version conflicts without updating upper-level application code versions.
By packaging the obtained source code, set the program reference package to only reference the intermediate base version, build the application package, and run the intermediate base file as a container for the application package, use the Kubernetes dependency library to share the external mount deployment method, and use the Kubernetes dependency library to divide the The layer reference decouples the version dependencies, reads the intermediate layer base file pre-uploaded to external memory, and loads the external package into memory to run.
Without updating the upper-level application code version, the version conflict problem during external package update is solved, and flexible management and update of external package versions is achieved, avoiding the need for repackaging and rebuilding images.
Smart Images

Figure CN118939307A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of IT application technology, and in particular to an external library centralized control method, device, equipment, medium and product. Background Art
[0002] In the prior art, when the Java compiled language references dependent libraries, there are complex version compatibility issues between the dependent libraries, and the dependent libraries themselves are in a state of continuous iteration to deal with vulnerability repairs and function optimization, so the external package version is frequently updated; and because the version number of the external package is specified in the application, it is strongly coupled with the external library version, causing the upper-level application to passively update the security version even if the function is not updated. Although the external package loading path can be specified when Java is started, using only the external path can only achieve a semi-decoupled state, which cannot solve the version coupling problem and still has the problem of version conflict. Summary of the invention
[0003] In order to solve the above problems, the present invention proposes an external library centralized control method, device, equipment, medium and product, which can solve the version conflict problem when the external package is updated without updating the upper application code version.
[0004] An embodiment of the present invention provides an external library centralized control method, including:
[0005] Package the acquired source code, set the program reference package to reference only the middle-layer base version, and build the application package;
[0006] Run the image unit as a container of the application package and read the intermediate base file uploaded to the external storage in advance;
[0007] The external package pre-uploaded to the external memory is loaded into the internal memory and executed by reading the intermediate layer base file.
[0008] Preferably, the acquired source code is packaged, and the program reference package is set to reference only the middle-layer base version, and the application package is constructed, including:
[0009] Use Maven to obtain source code packaging, and use Maven to set the program reference package to only reference the middle-layer base version;
[0010] Use the Maven plugin exclusion function to exclude the middle-layer base files and external packages, and generate an application package that only includes the actual project code.
[0011] Preferably, the image unit is run as a container of the application package, and the intermediate layer base file of the external storage is read, including:
[0012] Use the kubernetes command image unit to run the container;
[0013] Read the startup parameters pre-added in the image text file;
[0014] The intermediate layer base file is read from the external memory according to the startup parameter.
[0015] Further, reading the intermediate layer base file from the external memory according to the startup parameter includes:
[0016] According to the path directory from the external storage system to the container defined in the startup parameters, all related pods are mounted to the corresponding storage directory by mounting the external persistent storage device through volume, and the intermediate layer base file is read.
[0017] Preferably, the process of uploading the intermediate layer base file and the external package to the external storage specifically includes:
[0018] Configure a whitelist of the latest available software versions in the whitelist server;
[0019] Match the versions of the intermediate base files and external packages to be uploaded with the whitelist;
[0020] When there is an unmatched middle-layer base file or external package, the unmatched middle-layer base file or external package is verified; the version is updated according to the verification result, and the version is re-matched with the whitelist;
[0021] When there is no unmatched intermediate-layer base file or external package, all intermediate-layer base files and external packages are uploaded to the external storage device.
[0022] Preferably, performing a version update according to the verification result includes:
[0023] When the unmatched intermediate base file or external package does not involve modification of the calling API, and the version does not conflict with the corresponding application package, the version of the unmatched intermediate base file or external package is updated in the whitelist;
[0024] When the unmatched intermediate layer base file or external package involves modification of the calling API, the upper layer application code of the corresponding application package is modified and the version of the whitelist is updated.
[0025] The embodiment of the present invention further provides an external library centralized control device, the device comprising:
[0026] The packaging module is used to package the acquired source code, set the program reference package to reference only the middle-layer base version, and build the application package;
[0027] The layering module is used to run the image unit as a container of the application package and read the intermediate base file uploaded to the external storage in advance;
[0028] The reference module is used to load the external package pre-uploaded to the external memory into the internal memory for execution by reading the intermediate layer base file.
[0029] An embodiment of the present invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, an external library centralized control method as described in any of the above embodiments is implemented.
[0030] An embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute an external library centralized control method as described in any of the above embodiments.
[0031] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.
[0032] Compared with the prior art, the present invention provides a method, device, equipment, medium and product for centralized control of external libraries, which packages the acquired source code and sets the program reference package to only reference the intermediate base version to build an application package; runs the image unit as a container of the application package, reads the intermediate base file pre-uploaded to the external memory; and loads the external package pre-uploaded to the external memory into the memory for execution through the read intermediate base file. The present application can solve the version conflict problem when the external package is updated without updating the upper application code version. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 It is a flow chart of a centralized control method for an external library provided by an embodiment of the present invention;
[0034] Figure 2 is another flow chart of the external library centralized control method provided by an embodiment of the present invention;
[0035] Figure 3 It is another flow chart of the external library centralized control method provided by an embodiment of the present invention;
[0036] Figure 4 It is a structural schematic diagram of an external library centralized control device provided by an embodiment of the present invention;
[0037] Figure 5 It is a structural diagram of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0039] It can be understood that the various embodiments described below can be combined or combined in a logical manner, and the embodiments of the present invention will not describe various combinations one by one.
[0040] At present, the packaging and running scheme of commonly used programs: the codes of some commonly used interpreted languages (such as Python) are executed line by line by the interpreter at runtime. This code does not need to be compiled into machine code, but is directly executed by the interpreter. Modules and libraries are dynamically loaded at runtime, and additional modules can be loaded as needed during program execution. Such external libraries can often be placed directly in external directories. However, as a special compiled language, Java language, in addition to compiling the code and external dependent libraries into a bytecode package: jar package, also requires that it must be run on a server with a Java virtual machine installed. The jar package will be loaded into the virtual machine memory for operation (at this time, the jar package already contains the full amount of external libraries). It is usually impossible to directly call other directories dynamically at runtime like interpreted languages. When Java is built into a jar package, the versions of all referenced external packages must be specified to ensure successful compilation. Therefore, Java is coupled with the external library version during packaging. Once there is an external library security vulnerability update version, the entire application will be repackaged and all images will be rebuilt. Although Java can exclude external packages and directly place external libraries in other directory locations and load them into memory at runtime, version updates of external libraries will still cause the upper-level application to update the pom file, resulting in rebuilding and deployment, which cannot solve the version coupling problem.
[0041] At present, the package decoupling solutions commonly used in Java programs are: the commonly used framework springboot in the industry uses spring-boot-starter and other methods to uniformly package commonly used libraries to resolve version conflicts of lower-level dependent packages. Customized projects often use many libraries that are not in their dependent library collections, and need to resolve version conflicts by themselves. Many projects will not use the libraries in the starter.
[0042] At present, when Java compiled languages reference dependent libraries, there are complex version compatibility issues between dependent libraries, and the dependent libraries themselves are in a state of continuous iteration to deal with vulnerability repairs and function optimization. At this time, external packages are open source public packages, and security vulnerabilities are frequently repaired and versions are frequently updated. Since the version number of the external package is specified in the application, it is strongly coupled with the external library version, causing the upper-level application to passively update the security version even if the function is not updated. Although the external package loading path can be specified when Java is started, using only the external path cannot solve the version coupling problem.
[0043] When using an integration solution such as spring-boot-starter, customized projects often use many libraries that are not included in its dependency set, and need to resolve version conflicts by themselves. Springboot provides some commonly used package sets, but also references a large number of packages that are not used in actual projects, with a high degree of redundancy. If you want to streamline, you also need to handle it yourself. The final package it prints has the disadvantage that subsequent images are often large in size, which is not conducive to storage and transmission. This solution only achieves a semi-decoupled state, and there is still a problem of version conflicts.
[0044] Therefore, in view of the above-mentioned defects of the prior art, the present invention provides a centralized control method for an external library, see Figure 1 , is a flow chart of a centralized control method for an external library provided by an embodiment of the present invention, the method comprising the following steps:
[0045] Step S1, packaging the acquired source code, and setting the program reference package to reference only the middle-layer base version to build an application package;
[0046] Step S2, running the image unit as a container of the application package, and reading the intermediate layer base file uploaded to the external storage in advance;
[0047] Step S3, loading the external package pre-uploaded to the external memory into the internal memory for execution by reading the intermediate layer base file.
[0048] When this embodiment is implemented, see Figure 2, is another flow chart of the external library centralized control method provided by an embodiment of the present invention. In the existing scheme, the application package web1.Jar is stored in the image 1 of kubernetes, and the dependency packages lib1.Jar and lib2.Jar are stored. Similarly, the corresponding application packages and dependency packages are stored in the corresponding images 2 and 3. When the image is run as a container, the complete package in the image is directly read. In addition to compiling the code and compiling the external dependent library into a bytecode package: jar package, it is also required to run on a server with a java virtual machine installed. The jar package will be loaded into the virtual machine memory for operation (the jar package at this time already contains the full amount of external libraries). It is usually impossible to directly call other directories dynamically during operation like an interpreted language. Java is coupled with the external library version during packaging. Once there is a security vulnerability in the external library, the updated version will cause the entire application to be repackaged and all images to be rebuilt.
[0049] In response to existing solutions, the solution proposed in this application is based on the method and system of shared external mount deployment of Kubernetes's dependency library, and uses a hierarchical reference approach to decouple version dependencies.
[0050] The Kubernetes image unit in the solution is used to store upper-layer application package units and provide a running environment;
[0051] Upper layer application package unit: application startup entry, referencing the middle layer unit;
[0052] The unified storage unit of the file system serving as the external storage is provided with a unified storage unit: used for storing the intermediate layer base package and the external library dependency package.
[0053] The unified storage unit is also provided with an intermediate layer unit, which is used to reference external dependent version packages and be referenced by upper layer applications, thereby decoupling the version coupling between the upper layer and the lower layer.
[0054] The external library unit is used to be referenced by the middle layer and indirectly provide a function library to the upper layer application.
[0055] In this case, a mapping directory between the application package and the external package of the external storage is constructed in advance in the image unit, and the intermediate layer base file and the external package are uploaded to the external storage.
[0056] When starting the jar package, you can specify the location of the external dependency package to read. When starting, the external dependency package and the package itself are loaded into the Java virtual machine memory. Therefore, the jar in the image can exclude the middle-layer base package and other external packages when packaging, leaving only the application content. When starting, load the upper-layer package of the application, the upper-layer package references the middle-layer base package of the specified path, and the base package references the external package of the specified version, and finally loads all required packages into the memory.
[0057] The packaged jar package only packages the program content itself, and other dependent packages are not included in the image. Only the external package with a unified storage path is read and loaded into the Java virtual machine at startup. The above features can be used to solve the problem of version coupling between the upper-level package and the external package, and the external package version can be updated without updating the image container.
[0058] By introducing the method of introducing the middle layer base, the versions of each referenced package can be adapted to generate the base version. The upper-layer application web only references the base layer and is only associated with the base version (when the external dependency package does not involve updating the API used by the upper layer). When the external dependency package referenced by base has a version update, only the base version reference description file needs to be updated. The upper-layer application does not need to be updated, which will not affect the upper-layer application. The upper-layer web application can be completely decoupled from the lower-layer external package. Therefore, this case can solve the version conflict problem when the external package is updated without updating the upper-layer application code version.
[0059] In another embodiment provided by the present invention, see Figure 3 , is another flow chart of the external library centralized control method provided by an embodiment of the present invention. The step S1 specifically includes:
[0060] When using the Maven tool to package, the source code package is obtained. When Maven sets the program reference package, only the middle-layer base version is set to reference it. That is, when constructing the image, the Gitlab-source code Maven tool sets the program reference package through Maven build to specify the use of the middle-layer base-1.0.0. When packaging, the Maven plug-in exclusion function is used to exclude all the middle-layer base and external dependency packages, and exclude lib / *.jar, so that the generated package only contains the actual code content of the project.
[0061] In another embodiment provided by the present invention, the step S2 specifically includes:
[0062] Use the kubernetes command image unit to run the container. When building the image, include the package excluding the external library, and add -Dloader.path= / filesys / app1 / v1 to the dockerfile startup parameter to reference the external dependency package, where app1 is the application name and v1 is the base version. / filesys is a directory in the local file system of the container when the container in the pod is started, and the directory is mapped to the external storage device system.
[0063] Use the kubernetes command to run the image as a container. The startup parameters can be automatically run at runtime to read the external package and the middle layer base in the external mount address, load them into the Java virtual machine memory to run the program.
[0064] In another embodiment provided by the present invention, when the intermediate layer base file is read from the external memory according to the startup parameter, it is performed according to the association of the storage system created in advance.
[0065] Define the external storage system volume in the Kubernetes pod definition file in advance and mount it to / filesys inside the container. When the container is started, the application package reads the external dependency package file through this path.
[0066] In specific implementation, the Kubernetes framework can mount external persistent storage devices through volumes so that all related pods can mount and share the same storage directory and read the intermediate layer base file.
[0067] In another embodiment of the present invention, the process of uploading the intermediate layer base file and the external package to the external storage device specifically includes:
[0068] Build a unified repository: Before uploading external packages, configure the latest available software version whitelist on the whitelist server. When uploading the package, the server will verify the matching status.
[0069] That is, the intermediate layer base files and external packages to be uploaded are matched with the whitelist in version.
[0070] Upload dependency library package whitelist service, including effective package version configurator, dependency package version verifier and storage system manager, for file package verification.
[0071] When there are mismatched middle-layer base files or external packages, the mismatched middle-layer base files or external packages are verified; and the version is updated according to the verification result.
[0072] After the whitelist, the middle-layer base file or the external package is updated, the version is matched with the whitelist again;
[0073] When there are no unmatched intermediate base files or external packages, that is, all intermediate base files or external packages meet the version matching conditions of the whitelist to ensure that there is no conflict between versions, all intermediate base files and external packages are uploaded to the external storage in batches.
[0074] The middle-layer base can be customized for the packages used in specific projects, and the external packages that need to be used can be flexibly determined. Based on the shared external storage volume characteristics of k8s, the layered package introduction and external storage volume directory management are used to completely decouple the upper-layer application from the lower-layer application.
[0075] In another embodiment provided by the present invention, the process of performing version update according to the verification result specifically includes:
[0076] Identify mismatched middle-layer base files or external packages, and inspect the mismatched middle-layer base files or external packages;
[0077] When the mismatched middle-layer base file or external package does not involve modifications to the API call, such as vulnerability fixes, and the version does not conflict with the corresponding application package, update the middle-layer base and external package, and update the whitelist.
[0078] When the lower-layer application version is updated, only the middle-layer base file and external packages need to be updated. Without updating the image, the existing image container can be redeployed and loaded.
[0079] When the mismatched middle-layer base file or external package involves modification of the API call, it is necessary to modify the upper-layer application code, modify the upper-layer application code of the corresponding application package, and update the version of the whitelist.
[0080] When there is no API change involved, you can redeploy the service without recompiling and packaging the jar package to upload the image.
[0081] Based on the characteristics of shared storage of k8s external storage volumes, this solution adopts a hierarchical package reference mechanism and a unified external path. The startup package specifies the middle-layer package and the external dependency package path to centrally manage the external dependency packages of Java programs, a special compiled language. The external dependency version whitelist is then used to verify safe and legal external packages, thereby decoupling Java applications from external program libraries.
[0082] See also Figure 4 , is a schematic diagram of the structure of an external library centralized control device provided by an embodiment of the present invention, the device comprising:
[0083] The packaging module is used to package the acquired source code, set the program reference package to reference only the middle-layer base version, and build the application package;
[0084] The layering module is used to run the image unit as a container of the application package and read the intermediate base file uploaded to the external storage in advance;
[0085] The reference module is used to load the external package pre-uploaded to the external memory into the internal memory for execution by reading the intermediate layer base file.
[0086] The external library centralized control device provided in this embodiment can execute all the steps and functions of the external library centralized control method provided in any of the above embodiments, and the specific functions of the device will not be described in detail here.
[0087] See also Figure 5 , is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. The terminal device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an external library centralized control program. When the processor executes the computer program, the steps in each of the above-mentioned external library centralized control method embodiments are implemented, such as Figure 1 Alternatively, the processor implements the functions of each module in the above-mentioned device embodiments when executing the computer program.
[0088] Exemplarily, the computer program may be divided into one or more modules, and the one or more modules are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device. For example, the computer program may be divided into several modules, and the specific functions of each module have been described in detail in an external library centralized control method provided in any of the above embodiments, and the specific functions of the device will not be repeated here.
[0089] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art may understand that the schematic diagram is only an example of a terminal device and does not constitute a limitation on a terminal device. The terminal device may include more or fewer components than shown in the figure, or may combine certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.
[0090] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect the various parts of the entire terminal device.
[0091] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the external library centralized control device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0092] Wherein, if the module integrated in the external library centralized control device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.
[0093] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of any method described in the above embodiments when executed by a processor.
[0094] It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A centralized control method for an external library, characterized in that: The method comprises: Package the acquired source code, set the program reference package to reference only the middle-layer base version, and build the application package; Run the image unit as a container of the application package and read the intermediate base file uploaded to the external storage in advance; The external package pre-uploaded to the external memory is loaded into the internal memory and executed by reading the intermediate layer base file.
2. The external library centralized control method according to claim 1, characterized in that: Package the acquired source code, set the program reference package to reference only the middle-layer base version, and build the application package, including: Use Maven to obtain source code packaging, and use Maven to set the program reference package to only reference the middle-layer base version; Use the Maven plugin exclusion function to exclude the middle-layer base files and external packages, and generate an application package that only includes the actual project code.
3. The external library centralized control method according to claim 1, characterized in that: Run the image unit as a container of the application package and read the intermediate base file in the external storage, including: Use the kubernetes command image unit to run the container; Read the startup parameters pre-added in the image text file; The intermediate layer base file is read from the external memory according to the startup parameter.
4. The external library centralized control method according to claim 3, characterized in that: Reading the intermediate layer base file from the external memory according to the startup parameter includes: According to the path directory from the external storage system to the container defined in the startup parameters, all related pods are mounted to the corresponding storage directory by mounting the external persistent storage device through volume, and the intermediate layer base file is read.
5. The external library centralized control method according to claim 1, characterized in that: The process of uploading the intermediate layer base file and the external package to the external storage specifically includes: Configure a whitelist of the latest available software versions in the whitelist server; Match the versions of the intermediate base files and external packages to be uploaded with the whitelist; When there is an unmatched middle-layer base file or external package, the unmatched middle-layer base file or external package is verified; the version is updated according to the verification result, and the version is re-matched with the whitelist; When there is no unmatched intermediate-layer base file or external package, all intermediate-layer base files and external packages are uploaded to the external storage device.
6. The external library centralized control method according to claim 5, characterized in that: Update the version based on the verification results, including: When the unmatched intermediate base file or external package does not involve modification of the calling API, and the version does not conflict with the corresponding application package, the version of the unmatched intermediate base file or external package is updated in the whitelist; When the unmatched intermediate layer base file or external package involves modification of the calling API, the upper layer application code of the corresponding application package is modified and the version of the whitelist is updated.
7. An external library centralized control device, characterized in that: The device comprises: The packaging module is used to package the acquired source code, set the program reference package to reference only the middle-layer base version, and build the application package; The layering module is used to run the image unit as a container of the application package and read the intermediate base file uploaded to the external storage in advance; The reference module is used to load the external package pre-uploaded to the external memory into the internal memory for execution by reading the intermediate layer base file.
8. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the external library centralized control method according to any one of claims 1 to 6 is implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the external library centralized control method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of any method described in claims 1 to 6 are implemented.
