Deployment method, device, server and storage medium of third-party application

Abstract

The application relates to the technical field of big data, in particular to a third-party application deployment method and device, a server and a storage medium, wherein the method comprises the following steps: obtaining a third-party application deployment request; obtaining an edge node image from an image warehouse according to the deployment request, establishing a shared path of the image and a big data cluster, and obtaining configuration information and program files of the third-party application from the big data cluster by using the shared path; constructing an edge node container in the image based on the configuration information and the program files; and deploying the third-party application in the edge node container. Therefore, the problem that related technologies cannot dynamically obtain configuration information changes of the big data cluster is solved, when the big data cluster information is updated, the configuration of the local environment needs to be repeatedly manually copied and modified, and the problems of high manual maintenance cost and poor migration and expansion convenience exist.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a method, apparatus, server and storage medium for deploying third-party applications. Background Technology

[0002] In big data computing scenarios, there are a large number of third-party applications that need to connect and configure with the Hadoop ecosystem framework to become edge nodes of big data clusters in order to complete different distributed computing and storage tasks. Especially in the current environment where container cloud platforms are popular, a large number of third-party applications are deployed on container cloud platforms.

[0003] In related technologies, configuration information is usually obtained first, and then manually configured into the application environment. However, when the information of the big data cluster is updated, it is necessary to repeatedly copy and modify the configuration of the local environment, and there is a problem of poor convenience for migration and expansion. Summary of the Invention

[0004] This application provides a method, device, server, and storage medium for deploying third-party applications to solve the problems of related technologies being unable to dynamically obtain changes in the configuration information of big data clusters. When the big data cluster information is updated, it is necessary to repeatedly manually copy and modify the configuration of the local environment, which results in high manual maintenance costs and poor convenience for migration and expansion.

[0005] The first aspect of this application provides a method for deploying a third-party application. The method is applied to a server and includes the following steps: obtaining a deployment request for the third-party application; obtaining an edge node image from an image repository according to the deployment request; establishing a shared path between the image and a big data cluster; obtaining configuration information and program files of the third-party application from the big data cluster using the shared path; constructing an edge node container in the image based on the configuration information and the program files; and deploying the third-party application in the edge node container.

[0006] Optionally, in one embodiment of this application, generating an edge node container in the image based on the configuration information and the program file includes: generating a container environment in the image based on the configuration information and the program file; creating a shared persistent volume in the container environment; and mounting the shared path on the shared persistent volume.

[0007] Optionally, in one embodiment of this application, after deploying the third-party application using the edge node container, the method further includes: obtaining update information of the big data cluster using a shared path mounted on the shared persistent volume; updating the container environment of the edge node container based on the update information; and configuring the third-party application based on the updated container environment.

[0008] Optionally, in one embodiment of this application, before obtaining the edge node image from the image repository according to the deployment request, the method further includes: obtaining a base container image; installing the program files of the big data cluster based on the base container image; and configuring the environment variables of the big data cluster to obtain the edge node image.

[0009] Optionally, in one embodiment of this application, obtaining the image of the edge node from the image repository according to the deployment request includes: querying the image repository using the edge node of the big data cluster as an index to obtain the image of the edge node, wherein if the image of the edge node is not found, the image of the edge node is customized.

[0010] A second aspect of this application provides a deployment apparatus for a third-party application. The apparatus is applied to a server and includes: a first acquisition module for acquiring a deployment request for a third-party application; a second acquisition module for acquiring an image of an edge node from an image repository according to the deployment request, establishing a shared path between the image and a big data cluster, and acquiring configuration information and program files of the third-party application from the big data cluster using the shared path; and a deployment module for building an edge node container in the image based on the configuration information and the program files, and deploying the third-party application in the edge node container.

[0011] Optionally, in one embodiment of this application, the deployment module is further configured to generate a container environment in the image based on the configuration information and the program file; create a shared persistent volume in the container environment; and mount the shared path on the shared persistent volume.

[0012] Optionally, in one embodiment of this application, it further includes: an update module, configured to obtain update information of the big data cluster by using a shared path mounted on the shared persistent volume after the third-party application is deployed on the edge node container; and a first configuration module, configured to update the container environment of the edge node container based on the update information, and configure the third-party application based on the updated container environment.

[0013] Optionally, in one embodiment of this application, it further includes: a third acquisition module, configured to acquire a base container image before acquiring the image of the edge node from the image repository according to the deployment request; and a second configuration module, configured to install the program files of the big data cluster based on the base container image and configure the environment variables of the big data cluster to obtain the image of the edge node.

[0014] Optionally, in one embodiment of this application, the second acquisition module is further configured to query the image repository using the edge node of the big data cluster as an index to obtain the image of the edge node, wherein if the image of the edge node is not found, the image of the edge node is customized.

[0015] A third aspect of this application provides a server, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a method for deploying a third-party application as described in the above embodiments.

[0016] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement a method for deploying a third-party application as described in the above embodiments.

[0017] Therefore, this application has at least the following beneficial effects:

[0018] This application embodiment can obtain the image of the edge node of the big data cluster based on the deployment request of a third-party application. Through sharing, it dynamically loads program files and configuration information from the big data cluster and builds a containerized application containing all cluster configuration information and program files. This ensures that the configuration information changes in real time with the big data cluster, increasing the convenience of cluster integration. It also guarantees the accuracy of the configuration information and the integrity of the edge node program files. Furthermore, when the application is migrated or scaled up, there is no need to separately monitor the integration configuration with the big data cluster. Therefore, it solves the problems of related technologies that cannot dynamically obtain changes in big data cluster configuration information, requiring repeated manual copying and modification of local environment configurations when big data cluster information is updated, resulting in high manual maintenance costs and poor migration and scaling convenience.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0021] Figure 1 This is a flowchart illustrating a method for deploying a third-party application according to an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of an edge node container construction process according to an embodiment of this application;

[0023] Figure 3This is an example diagram illustrating the integration of a big data cluster with a containerized edge node application according to an embodiment of this application.

[0024] Figure 4 This is a block diagram of a third-party application deployment device according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram of the structure of a server provided according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0027] The following description, with reference to the accompanying drawings, illustrates a method, apparatus, server, and storage medium for deploying third-party applications according to embodiments of this application. Addressing the problems mentioned in the background section, this application provides a method for deploying third-party applications. In this method, an image of the edge node of a big data cluster is obtained based on the deployment request of the third-party application. Through sharing, program files and configuration information are dynamically loaded from the big data cluster, and a containerized application containing all cluster configuration information and program files is constructed. This ensures that the configuration information changes in real time with the big data cluster, increasing the convenience of cluster integration. It also ensures the accuracy of the configuration information and the integrity of the edge node program files. Furthermore, when the application is migrated or expanded, there is no need to separately monitor the integration configuration with the big data cluster. Therefore, this solves the problems of related technologies that cannot dynamically obtain changes in the configuration information of the big data cluster, requiring repeated manual copying and modification of the local environment configuration when the big data cluster information is updated, resulting in high manual maintenance costs and poor ease of migration and expansion.

[0028] Specifically, Figure 1 This is a flowchart illustrating a method for deploying a third-party application, as provided in an embodiment of this application.

[0029] like Figure 1 As shown, the deployment method of this third-party application is applied to the server and includes the following steps:

[0030] In step S101, the deployment request of the third-party application is obtained.

[0031] In big data computing scenarios, there are a large number of third-party applications that need to connect to the Hadoop ecosystem framework and configure their information to become edge nodes of the big data cluster. In this embodiment, the deployment request of the third-party application can be obtained first, so that the configuration information of the big data cluster can be configured into its own application environment according to the request.

[0032] In step S102, the image of the edge node is obtained from the image repository according to the deployment request, a shared path between the image and the big data cluster is established, and the configuration information and program files of the third-party application are obtained from the big data cluster using the shared path.

[0033] In this embodiment, the main functions of the image repository are image storage, image management, and image distribution. Each repository can contain multiple images. Specifically, this embodiment can use the edge nodes of the big data cluster as indexes to query the image repository to obtain the images of the edge nodes. Based on the shared path between the images and the big data cluster, a server node containing all configuration information and program files is selected within the big data cluster. Using file sharing, the program files and configuration information are dynamically loaded from the big data cluster. Figure 2 As shown, in actual execution, if the image of the edge node is not found in the image storage, a custom image of the edge node is created.

[0034] In step S103, an edge node container is built in the image based on the configuration information and program files, and the third-party application is deployed in the edge node container.

[0035] It is understood that third-party applications in this application embodiment can use an image containing program files and configuration information of a big data cluster to build an edge node container. By quickly simulating the same operating environment through the container to run the application, the configuration information can be guaranteed to change in real time with the big data cluster, increasing the convenience of connecting to the cluster.

[0036] In one embodiment of this application, generating an edge node container in an image based on configuration information and program files includes: generating a container environment in an image based on configuration information and program files; creating a shared persistent volume in the container environment; and mounting a shared path on the shared persistent volume.

[0037] In this embodiment, a persistent volume is an abstraction of storage resources. Storage is defined as a resource that container applications can use. Persistent volumes are created and configured by the administrator. This embodiment can configure NFS file sharing services, open sharing permissions for specified files and configuration information paths, and create local shared persistent volumes for the cluster. Thus, big data cluster information is dynamically obtained using shared storage, enabling containerized edge nodes to dynamically load configuration information and program files at any time.

[0038] In one embodiment of this application, after deploying a third-party application using an edge node container, the method further includes: obtaining update information of the big data cluster using a shared path mounted on a shared persistent volume; updating the container environment of the edge node container based on the update information; and configuring the third-party application based on the updated container environment.

[0039] It is understood that, in this embodiment of the application, after obtaining the edge node image of the big data cluster, a containerized application containing all cluster configuration information and dependency files can be built, quickly becoming a containerized edge node of the cluster. This containerized edge node can dynamically load configuration information and program files at any time. When the big data cluster information is updated, there is no need to manually copy and modify the configuration of the local environment. At the same time, when the application is migrated and expanded, there is no need to pay extra attention to the interface configuration with the big data cluster.

[0040] In one embodiment of this application, before obtaining the image of the edge node from the image repository according to the deployment request, the method further includes: obtaining a base container image; installing the program files of the big data cluster based on the base container image, and configuring the environment variables of the big data cluster to obtain the image of the edge node.

[0041] The base container image can be a CentOS image or an Ubuntu image, without any specific restrictions.

[0042] Specifically, such as Figure 2 As shown in this embodiment, a clean CentOS 7 container image can be obtained from the official website. Based on this image, the dependent programs of the big data cluster, such as JDK and Python, are installed, and the relevant environment variables of the big data cluster are configured. After all the configurations are completed, the image of the edge node is stored in the image repository.

[0043] The following is combined with Figure 3 The flowchart shown below illustrates the integration process of big data clusters and containerized edge node applications.

[0044] 1. First, select a server node within the big data cluster that contains all configuration information and environment dependency files, and install the NFS file sharing service program;

[0045] 2. Configure NFS file sharing service, granting sharing permissions to specified files and paths configured in the configuration information;

[0046] 3. Kubernetes uses the shared path from step 2 to create a local shared persistent volume and provides it to the cluster;

[0047] 4. Obtain a clean CentOS 7 container image from the official website, install the dependent programs for the big data cluster based on the image, such as JDK and Python, and configure the relevant environment variables for the big data cluster. After completing all the configurations, repackage it into a customized image and push it to the image repository.

[0048] 5. Within the Kubernetes cluster, the shared persistent volume mounted on this image is used to create a big data edge node container. This container environment can access the big data cluster without any configuration or debugging.

[0049] 6. Containerized applications on Kubernetes: When defining the Dockerfile for the service, pull this image as the working container environment for the application, build it into a working container for big data edge nodes, and complete the access to the cluster.

[0050] In summary, the embodiments of this application can standardize the management of applications accessing big data clusters, eliminating the need for manual intervention in configuration information and dependency files, and reducing human maintenance costs.

[0051] The third-party application deployment method proposed in this application obtains the image of the edge node of the big data cluster based on the deployment request of the third-party application. Through sharing, program files and configuration information are dynamically loaded from the big data cluster, and a containerized application containing all cluster configuration information and program files is constructed. This ensures that the configuration information changes in real time with the big data cluster, increasing the convenience of cluster integration. It also ensures the accuracy of the configuration information and the integrity of the edge node program files. Furthermore, when the application is migrated or expanded, there is no need to pay additional attention to the integration configuration with the big data cluster. Therefore, this solves the problems of related technologies that cannot dynamically obtain changes in the configuration information of the big data cluster, and that when the big data cluster information is updated, it is necessary to repeatedly manually copy and modify the configuration of the local environment, resulting in high manual maintenance costs and poor convenience for migration and expansion.

[0052] Next, the deployment apparatus for a third-party application according to an embodiment of this application is described with reference to the accompanying drawings.

[0053] Figure 4 This is a block diagram of a third-party application deployment device according to an embodiment of this application.

[0054] like Figure 4 As shown, the deployment device 10 for the third-party application is applied to the server and includes: a first acquisition module 100 and a deployment module 300.

[0055] The first acquisition module 100 is used to acquire the deployment request of the third-party application; the first acquisition module 100 is used to acquire the image of the edge node from the image repository according to the deployment request, establish a shared path between the image and the big data cluster, and acquire the configuration information and program files of the third-party application from the big data cluster using the shared path; the deployment module 300 is used to build the edge node container in the image based on the configuration information and program files, and deploy the third-party application in the edge node container.

[0056] Optionally, in one embodiment of this application, the deployment module 300 is further configured to generate a container environment in the image based on configuration information and program files; create a shared persistent volume in the container environment; and mount a shared path on the shared persistent volume.

[0057] Optionally, in one embodiment of this application, it further includes an update module and a first configuration module.

[0058] The update module is used to obtain the update information of the big data cluster by using the shared path mounted on the shared persistent volume after the third-party application is deployed on the edge node container; the first configuration module is used to update the container environment of the edge node container based on the update information, and configure the third-party application based on the updated container environment.

[0059] In one embodiment of this application, the apparatus 10 of this application embodiment further includes: a third acquisition module and a second configuration module.

[0060] The third acquisition module is used to acquire the base container image before acquiring the edge node image from the image repository according to the deployment request; the second configuration module is used to install the program files of the big data cluster based on the base container image and configure the environment variables of the big data cluster to obtain the edge node image.

[0061] In one embodiment of this application, the second acquisition module 200 is further configured to query the image repository using the edge node of the big data cluster as an index to obtain the image of the edge node, wherein if no image of the edge node is found, the image of the edge node is customized.

[0062] It should be noted that the foregoing explanation of the method for deploying third-party applications also applies to the deployment device for third-party applications in this embodiment, and will not be repeated here.

[0063] The third-party application deployment device proposed in this application obtains the image of the edge node of the big data cluster based on the deployment request of the third-party application. Through sharing, it dynamically loads program files and configuration information from the big data cluster and constructs a containerized application containing all cluster configuration information and program files. This ensures that the configuration information changes in real time with the big data cluster, increasing the convenience of cluster integration. It also ensures the accuracy of the configuration information and the integrity of the edge node program files. Furthermore, when the application is migrated or expanded, there is no need to pay additional attention to the integration configuration with the big data cluster. Therefore, it solves the problems of related technologies that cannot dynamically obtain changes in the configuration information of the big data cluster, and that when the big data cluster information is updated, it is necessary to repeatedly manually copy and modify the configuration of the local environment, resulting in high manual maintenance costs and poor convenience for migration and expansion.

[0064] Figure 5 A schematic diagram of the structure of a server provided in an embodiment of this application. The server may include:

[0065] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0066] When processor 502 executes the program, it implements the deployment method for third-party applications provided in the above embodiments.

[0067] Furthermore, the server also includes:

[0068] Communication interface 503 is used for communication between memory 501 and processor 502.

[0069] The memory 501 is used to store computer programs that can run on the processor 502.

[0070] The memory 501 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0071] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0072] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0073] Processor 502 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of this application.

[0074] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the deployment method of the third-party application described above.

[0075] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0076] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0077] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0078] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0079] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0080] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for deploying a third-party application, characterized in that, The method is applied to a server, and the method includes the following steps: Obtain deployment requests from third-party applications; According to the deployment request, the image of the edge node is obtained from the image repository, a shared path is established between the image and the big data cluster, and the configuration information and program files of the third-party application are obtained from the big data cluster using the shared path; Based on the configuration information and the program files, an edge node container is built in the image, and the third-party application is deployed in the edge node container. The step of building an edge node container in the image based on the configuration information and the program file includes: generating a container environment in the image based on the configuration information and the program file; creating a shared persistent volume in the container environment; and mounting the shared path on the shared persistent volume. After deploying the third-party application to the edge node container, the method further includes: obtaining update information of the big data cluster using a shared path mounted on the shared persistent volume; updating the container environment of the edge node container based on the update information; and configuring the third-party application based on the updated container environment.

2. The method for deploying a third-party application according to claim 1, characterized in that, Before retrieving the image of the edge node from the image repository according to the deployment request, the process also includes: Obtain the base container image; The program files for the big data cluster are installed based on the base container image, and the environment variables of the big data cluster are configured to obtain the image of the edge node.

3. The method for deploying a third-party application according to claim 1 or 2, characterized in that, The step of obtaining the edge node image from the image repository according to the deployment request includes: Using the edge nodes of the big data cluster as indexes, the image repository is queried to obtain the images of the edge nodes. If no images of the edge nodes are found, images of the edge nodes are customized.

4. A deployment device for a third-party application, characterized in that, The device is used in a server, wherein the device includes: The first acquisition module is used to acquire deployment requests from third-party applications; The second acquisition module is used to acquire the image of the edge node from the image repository according to the deployment request, establish a shared path between the image and the big data cluster, and use the shared path to acquire the configuration information and program files of the third-party application from the big data cluster. The deployment module is used to build an edge node container in the image based on the configuration information and the program file, and deploy the third-party application in the edge node container; The deployment module is further configured to: generate a container environment in the image based on the configuration information and the program file; create a shared persistent volume in the container environment; and mount the shared path on the shared persistent volume. The update module is used to obtain the update information of the big data cluster by using the shared path mounted on the shared persistent volume after the third-party application is deployed to the edge node container. The first configuration module is used to update the container environment of the edge node container based on the update information, and to configure the third-party application based on the updated container environment.

5. A server, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the deployment method of a third-party application as described in any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the deployment method of the third-party application as described in any one of claims 1-3.

Images