A development environment configuration method, device, electronic device and storage medium based on K8S

Through the service mesh, the interconnection relationship of interactive development environment containers is established in the K8S cluster, and the problem of single-machine restrictions in the existing technology is solved, multi-machine and multi-card interconnection and network isolation are realized, and the flexibility and capability of the development environment are improved. Maintainability, supports AI algorithm development.

CN118170494BActive Publication Date: 2024-11-08BEIJING LUCHEN TECH CO LTD
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
CN202410332974.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-22
Publication Date
2024-11-08
Estimated Expiration
2044-03-22

AI Technical Summary

Technical Problem

The existing K8S-based development environment configuration method only supports stand-alone users, cannot support multi-machine interconnection and network isolation, and it is difficult to meet the needs of large-scale model training tasks.

Method used

The service mesh obtains the configuration reference information of the interactive development environment container in the K8S cluster, establishes the interconnection relationship between the interactive development environment containers, realizes the configuration of the interactive development environment on the cloud with multiple machines and multiple cards, and supports network isolation.

Benefits of technology

It improves the flexibility, versatility and maintainability of the K8S-based development environment configuration, can better support AI algorithm development, realize multi-user joint development and network on-demand isolation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118170494B_ABST
    Figure CN118170494B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a development environment configuration method, device, electronic device and storage medium based on K8S, the method comprising: obtaining container configuration reference information of an associated interactive development environment container in a K8S cluster; configuring a target interactive development environment container for the current interactive development environment container according to the container associated configuration information of the associated interactive development environment container; establishing an interconnection relationship between the current interactive development environment container and the target interactive development environment container. The technical solution of the embodiment of the present invention can improve the flexibility, versatility and maintainability of the development environment configuration based on K8S.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the fields of computer application and cloud computing technology, and in particular to a development environment configuration method, device, electronic device and storage medium based on K8S. Background Art

[0002] As the demand for computing power in algorithm development grows, cloud computing power, with its on-demand use and flexible deployment features, has become increasingly popular among people in related fields. In this context, many cloud-based interactive development environments have emerged, such as JupyterHub and VSCode Server (Visual Studio Code, a cross-platform source code editor).

[0003] The current mainstream cloud development environment products mainly include JupyterHub, VSCode Server and KubeflowNotebook Operator. Among them, JupyterHub is a product based on Jupyter Notebook that supports multi-user development and can create, manage and proxy multiple Jupyter Notebook instances on the same cloud host. VSCodeServer is built by the popular text editor VSCode (Visual Studio Code) and can be installed as a service on a remote host to provide a similar experience to VSCode in the browser. Kubeflow in Kubeflow Notebook Operator is an open source cloud machine learning platform based on Google's internal machine learning workflow. NotebookOperator is a component that manages the interactive development environment. It can create and manage Jupyter Notebook containers on the container platform K8S (kubernetes, an open source, used to manage containerized applications on multiple hosts in the cloud platform).

[0004] In the process of implementing the present invention, the inventors found that the prior art has the following defects: as the model size grows, the training task needs to be expanded to multiple machines and multiple cards, while the current solutions are basically designed based on the premise of single-machine operation. For example, the design goal of JupyterHub is "a multi-user Notebook that is good enough to use", and the matching degree of multi-machine requirements is extremely low. It only has a single-machine mode and does not natively support the network interconnection of multiple machines, which is very inconvenient to use. Although JupyterHub can support multiple users, there is no isolation between users, and it is only suitable for computing environments that trust each other. VSCode Server is designed as a single-machine development environment on the cloud. It does not need to consider upper-level issues such as multi-machine / cloud scheduling, and it only supports single machines / single users. Kubeflow Notebook Operator is designed with K8S as the operating environment, and does not support multi-machine interconnection and network isolation, because it only requires a single-machine Notebook when it is designed, and it is only optimized for Jupyter Notebook, and cannot support VSCode Server and other third-party products. Summary of the invention

[0005] The embodiments of the present invention provide a K8S-based development environment configuration method, device, electronic device and storage medium, which can improve the flexibility, versatility and maintainability of the K8S-based development environment configuration.

[0006] According to one aspect of the present invention, a development environment configuration method based on K8S is provided, which is applied to a service grid, and the method includes:

[0007] Get the container configuration reference information of the interactive development environment container associated with the K8S cluster;

[0008] According to the container associated configuration information of the associated interactive development environment container, a target interactive development environment container is configured for the current interactive development environment container;

[0009] An interconnection relationship between the current interactive development environment container and the target interactive development environment container is established.

[0010] According to another aspect of the present invention, a development environment configuration device based on K8S is provided, which is configured in a service grid, and the device includes:

[0011] The container configuration reference information acquisition module is used to obtain the container configuration reference information of the interactive development environment container associated with the K8S cluster;

[0012] A development environment container configuration module, configured to configure a target interactive development environment container for the current interactive development environment container according to the container associated configuration information of the associated interactive development environment container;

[0013] The interconnection relationship establishing module is used to establish an interconnection relationship between the current interactive development environment container and the target interactive development environment container.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the K8S-based development environment configuration method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the K8S-based development environment configuration method described in any embodiment of the present invention when executed.

[0019] The embodiment of the present invention obtains the container configuration reference information of the associated interactive development environment container in the K8S cluster through the service grid, and configures the target interactive development environment container for the current interactive development environment container according to the obtained container associated configuration information of the associated interactive development environment container, thereby establishing an interconnection relationship between the current interactive development environment container and the target interactive development environment container, thereby solving the problem that the existing K8S-based development environment configuration method only supports single-machine users or cannot support network isolation, and the like, and realizes the configuration of a cloud-based interactive development environment with multiple machines and multiple cards interconnected, which can improve the flexibility, versatility and maintainability of the K8S-based development environment configuration, and can better support AI algorithm development.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1This is a flow chart of a development environment configuration method based on K8S provided in Example 1 of the present invention;

[0023] Figure 2 This is a flow chart of a development environment configuration method based on K8S provided in Embodiment 2 of the present invention;

[0024] Figure 3 This is a schematic diagram of the overall framework of a development environment based on K8S provided in the second embodiment of the present invention;

[0025] Figure 4 It is a schematic diagram of a development environment configuration device based on K8S provided in Embodiment 3 of the present invention;

[0026] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "comprises" and "has" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatus.

[0029] Embodiment 1

[0030] Figure 1 This is a flowchart of a development environment configuration method based on K8S provided in the first embodiment of the present invention. This embodiment is applicable to the configuration of an interactive development environment on the cloud that uses a service grid to interconnect multiple machines and multiple cards. The method can be executed by a development environment configuration device based on K8S. The device can be implemented by software and / or hardware, and can generally be integrated in an electronic device. The electronic device can be a terminal device or a server device. As long as the development environment based on K8S can be configured, the embodiment of the present invention does not limit the specific device type of the electronic device. Accordingly, Figure 1 As shown, the method includes the following operations:

[0031] S110. Obtain container configuration reference information of an interactive development environment container associated with the K8S cluster.

[0032] Among them, the interactive development environment built in the K8S cluster can be any type of development environment supported by the K8S cluster, for example, it can include but not limited to JupyterHub, VSCode Server and Kubeflow Notebook Operator. Correspondingly, the interactive development environment container can be a container for carrying the interactive development environment. It is understandable that an interactive development environment container can correspondingly carry one interactive development environment. The associated interactive development environment containers can be multiple interactive development environment containers that need to be interconnected and configured. The container configuration reference information can be the relevant configuration information of the interactive development environment container.

[0033] In an embodiment of the present invention, each interactive development environment built in the K8S cluster can be configured with a corresponding service grid. As an infrastructure layer built into the application, the service grid can record the interaction mode of the service, so as to optimize the communication and avoid downtime more easily. Optionally, the service grid can be Istio. Istio is an infrastructure based on K8S. By adding a network proxy container in front of the existing container, it can transparently provide protection, connection, routing and monitoring functions for distributed applications. Optionally, Istio can use Envoy (a high-performance non-blocking service proxy developed in C++) agent as a network proxy container. The Envoy agent is a high-performance, lightweight network proxy server that can provide access control and observability to facilitate the adjustment of overall performance.

[0034] Correspondingly, the service grids configured corresponding to each interactive development environment container can configure the interconnection and interaction information of the interactive development environment container. Optionally, the development environment can be configured in parallel between different service grids, that is, the development environment configuration processes between the service grids are not affected by each other. Correspondingly, when one or more service grids in the K8S cluster configure the associated interactive development environment built in the K8S cluster, the container configuration reference information of the associated interactive development environment container in the K8S cluster can be first obtained. Among them, the service grid currently performing the development environment configuration can first determine the associated interactive development environment container. For example, the current interactive development environment container and other interactive development environment containers that have interconnection requirements with the current interactive development environment container can be used as associated interactive development environment containers, and the container configuration information of the associated interactive development environment container can be obtained as the container configuration reference information.

[0035] S120: Configure a target interactive development environment container for the current interactive development environment container according to the container associated configuration information of the associated interactive development environment container.

[0036] The current interactive development environment container may be an interactive development environment container managed by a service grid that is currently configuring a development environment, and the target interactive development environment container may be an interactive development environment container that has interconnection requirements with the current interactive development environment container.

[0037] Correspondingly, after obtaining the container association configuration information of the associated interactive development environment container, the service grid can parse the container association configuration information of the associated interactive development environment container to respectively determine the container configuration information of the current interactive development environment container and the container configuration information of the target interactive development environment container, thereby configuring the target interactive development environment container for the current interactive development environment container according to the container configuration information of the current interactive development environment container and the container configuration information of the target interactive development environment container.

[0038] Exemplarily, the container configuration information may include, but is not limited to, a container identifier, a device where the container is located, and related interfaces used for connecting the container.

[0039] Exemplarily, the service grid can determine the current interactive development environment container and the target interactive development environment container based on the container identifier in the container configuration information, and establish a mapping relationship between the container identifier of the current interactive development environment container and the container identifier of the target interactive development environment container, thereby configuring the target interactive development environment container for the current interactive development environment container.

[0040] S130: Establish an interconnection relationship between the current interactive development environment container and the target interactive development environment container.

[0041] Correspondingly, after configuring the target interactive development environment container for the current interactive development environment container, the service grid can establish an interconnection relationship between the current interactive development environment container and the target interactive development environment container, completing the configuration process of the multi-machine and multi-card cloud development environment.

[0042] Exemplarily, the service grid can obtain the container identifier of the target interactive development environment container, the device where the container is located, and the relevant interfaces used for container connection based on the container configuration information of the target interactive development environment container, so as to establish a communication connection with the target interactive development environment container through the container identifier of the target interactive development environment container, the device where the container is located, and the relevant interfaces used for container connection, thereby realizing multi-machine and multi-card interconnection in the cloud development environment.

[0043] It is understandable that interactive development environment containers that are interconnected can communicate with each other to exchange data and jointly execute training tasks of related algorithm models, etc., so as to better support the algorithm development of AI (Artificial Intelligence).

[0044] It can be seen that the embodiment of the present invention can meet the configuration requirements of a multi-machine multi-card development environment by interconnecting and configuring each interactive development environment container using a service grid. At the same time, the service grid only interconnects and configures the interactive development environment containers with interconnection requirements, while the interactive development environment containers without interconnection requirements can be effectively isolated. That is, interactive development environment containers that have not established an interconnection relationship cannot communicate with each other. Therefore, the development environment configuration method based on K8S provided by the embodiment of the present invention can not only support the development environment of multiple machines and multiple cards, realize multi-user joint development, but also support network on-demand isolation, and can efficiently realize various requirements such as internal systems and online service systems according to application scenarios, with high flexibility. At the same time, it also has high versatility and high maintainability, and can support various types of development environments such as JupyterNotebook and VSCode Server, and the engineering complexity is relatively small. Therefore, the development environment configuration method based on K8S provided by the embodiment of the present invention improves the flexibility, versatility and maintainability of the development environment configuration based on K8S.

[0045] The embodiment of the present invention obtains the container configuration reference information of the associated interactive development environment container in the K8S cluster through the service grid, and configures the target interactive development environment container for the current interactive development environment container according to the obtained container associated configuration information of the associated interactive development environment container, thereby establishing an interconnection relationship between the current interactive development environment container and the target interactive development environment container, thereby solving the problem that the existing K8S-based development environment configuration method only supports single-machine users or cannot support network isolation, and the like, and realizes the configuration of a cloud-based interactive development environment with multiple machines and multiple cards interconnected, which can improve the flexibility, versatility and maintainability of the K8S-based development environment configuration, and can better support AI algorithm development.

[0046] Embodiment 2

[0047] Figure 2 This is a flow chart of a development environment configuration method based on K8S provided in the second embodiment of the present invention. This embodiment is specific based on the above embodiment. In this embodiment, multiple specific optional implementation methods for obtaining container configuration reference information of an interactive development environment container associated with a K8S cluster are provided. Figure 2 As shown, the method of this embodiment may include:

[0048] S210: Determine user configuration information of a current user that matches the current interactive development environment container.

[0049] The current user is a user who has development authority for the interactive development environment in the current interactive development environment container. The user configuration information may include, but is not limited to, user identification and user development authority and other related configuration information.

[0050] In the embodiment of the present invention, fine-grained network configuration of the user's development environment permissions can be performed through the service grid. Figure 3 This is a schematic diagram of the overall framework of a development environment based on K8S provided in the second embodiment of the present invention. In a specific example, Figure 3 As shown, a centralized management service can be provided to end users based on the K8S cluster. The management service can be used to create, manage, and delete multi-machine and multi-card development environments on the cloud. End users can manage all development environments and resource groups in the K8S cluster based on the management interface of the management service. Correspondingly, end users can also control the service grid based on the management interface provided by the management service, for example, sending instructions to the service grid to establish interconnections between containers of different interactive development environments.

[0051] It is understandable that a user can have development permissions for multiple interactive development environments. Figure 3 As shown, multiple interactive development environment containers (not limited to 3) in the same level can constitute an interactive development environment, and the interactive development environment opens development permissions to a user.

[0052] S220. Determine the associated interactive development environment container from each interactive development environment container in the K8S cluster according to the user configuration information of the current user.

[0053] In an embodiment of the present invention, the interactive development environment containers within a user's interactive development environment can be interconnected as needed, but the interactive development environments of different users cannot be connected to each other unless authorized. Therefore, when determining the associated interactive development environment container that can be interconnected with the current interactive development environment container, the user configuration information of the current user can be obtained first, and then based on the user configuration information of the current user, the interactive development environment container for which the user has development permissions is determined from the interactive development environment containers in the K8S cluster as the associated interactive development environment container. Optionally, the user having development permissions can be understood as the user having the permission to operate the interactive development environment container; it can also be possible for the user to have the permission to operate other users' interactive development environment containers after static authorization.

[0054] In an optional embodiment of the present invention, the user configuration information of the current user may include the user identification of the current user; determining the associated interactive development environment container from each interactive development environment container in the K8S cluster based on the user configuration information of the current user may include: obtaining user identification information of each interactive development environment container in the K8S cluster; matching the user identification of the current user with the user identification information of each interactive development environment container in the K8S cluster; and determining the interactive development environment container of the current user from each interactive development environment container in the K8S cluster as the associated interactive development environment container based on the user identification matching result.

[0055] Optionally, users can configure the interconnection relationship between the interactive development environment containers in their interactive development environment based on the service grid. That is, the connections between the interactive development environment containers in the same interactive multi-machine development environment can be configured according to user needs, so as to run multi-machine and multi-card tasks. When configuring the connection between the interactive development environment containers in the same interactive multi-machine development environment, the service grid can obtain the user identification information of each interactive development environment container in the K8S cluster, thereby matching the user identification of the current user with the user identification information of each interactive development environment container in the K8S cluster, and screening out the interactive development environment containers for which the current user can have development permissions without authorization from other users, as the associated interactive development environment containers. Exemplarily, the user identification of the current user may include, but is not limited to, unique identification information such as a user account or user ID.

[0056] That is, among the interactive development environment containers of the current user, all interactive development environment containers except the current interactive development environment container can be used as associated interactive development environment containers, so that the current user can further filter the target interactive development environment container corresponding to the current interactive development environment container from the associated interactive development environment containers.

[0057] In an optional embodiment of the present invention, the user configuration information of the current user may also include the cross-user development permissions of the current user; determining the associated interactive development environment container from the interactive development environment containers in the K8S cluster based on the user configuration information of the current user may include: determining the target user identifier based on the cross-user development permissions of the current user; matching the target user identifier with the user identifier information of each of the interactive development environment containers in the K8S cluster; and determining, based on the user identifier matching result and the configuration instructions of the current user, from the interactive development environment containers in the K8S cluster, the interactive development environment container of the target user authorized by the current user as the associated interactive development environment container.

[0058] The cross-user development permission can be understood as the current user having the development permission for the interactive development environment container of other users. The target user can be a user who authorizes the current user to use his interactive development environment container. The target user ID is also the ID of the target user.

[0059] Optionally, users can also configure the interconnection relationship between the interactive development environment container in their own interactive development environment and the interactive development environment containers authorized by other users based on the service grid. That is, after authorization, the interactive development environment containers in different interactive multi-machine development environments can also be configured according to user needs to run multi-machine multi-card tasks. When configuring connections between interactive development environment containers in different interactive multi-machine development environments, the service grid can obtain the user identification information of each interactive development environment container in the K8S cluster, thereby filtering out the target user identification from each user identification according to the cross-user development permissions of the current user, and further matching the target user identification with the user identification information of each interactive development environment container in the K8S cluster, thereby filtering out the interactive development environment container for which the current user has development permissions after being authorized by the target user, and using it as an associated interactive development environment container at the same time as one's own interactive development environment container.

[0060] As an optional embodiment, the target user can also open up development permissions to some interactive development environment containers in his own interactive development environment. That is, the current user determines the interactive development environment container of the target user authorized by the current user from the interactive development environment containers in the K8S cluster through the service grid according to the user identification matching result. It can be part or all of the interactive development environment containers in the interactive development environment of the target user, and the embodiment of the present invention does not limit this.

[0061] It can be seen that the above technical solution provides an optional network isolation function, that is, different interactive development environments cannot be connected to each other, or only different interactive development environment containers of the same user can be connected to each other. A user cannot connect to another user's interactive development environment unless authorized. In order to further improve the security of the network environment and network data, the interactive development environment and other types of resources in the K8S cluster (such as sensitive data storage areas, etc.) can also be isolated.

[0062] S230: Obtain container configuration reference information of the associated interactive development environment container.

[0063] Correspondingly, after determining each associated interactive development environment container, the service grid can obtain container configuration reference information of each associated interactive development environment container, so as to configure the multi-machine multi-card development environment using the obtained container configuration reference information.

[0064] S240: Configure a target interactive development environment container for the current interactive development environment container according to the container associated configuration information of the associated interactive development environment container.

[0065] S250: Establish an interconnection relationship between the current interactive development environment container and the target interactive development environment container.

[0066] In an optional embodiment of the present invention, the K8S cluster can be configured with a stateful load controller; the stateful load controller can be used to: obtain cluster device association attribute information of the K8S cluster; determine the K8S resource scheduling strategy based on the cluster device association attribute information of the K8S cluster.

[0067] The cluster device associated attribute information may be the related attribute information of the devices in the K8S set, for example, including but not limited to the device identification, the device geographic location information, and scheduling authority, etc. The K8S resource scheduling strategy may be the strategy for scheduling the cluster devices.

[0068] Currently, existing mainstream products have shortcomings for scenarios running on the container platform K8S. For example, JupyterHub has few optional parameters for creating a working container and does not fully utilize K8S's resource scheduling capabilities. As a product for the general public, it must ensure compatibility on bare metal, cloud hosts, and K8S, so it can only take the greatest common divisor. VSCodeServer also does not include any K8S scheduling capabilities.

[0069] To solve the scheduling problem of an interactive development environment, in a specific example, Figure 3 As shown in the figure, each interactive development environment in the K8S cluster can be configured with a stateful load controller Statefulset. As a deployment unit of workloads on the K8S platform, Statefulset is used to manage stateful Pods (the smallest schedulable and deployable computing unit on the K8S platform, consisting of a group of containers) and provide persistent storage and identifiers for these Pods.

[0070] That is, the interactive development environment can be composed of a K8S resource group, including a StatefulSet and several Istios, and each Istio can be configured with corresponding network rules. Each Pod of Statefulset can be composed of an interactive development environment container and an envoy proxy container. Accordingly, Statefulset can obtain the cluster device associated attribute information of the K8S cluster to determine the K8S resource scheduling strategy based on the obtained cluster device associated attribute information of the K8S cluster. For example, Statefulset can constrain devices beyond a certain distance range to not schedule each other.

[0071] In an optional embodiment of the present invention, the cluster device association attribute information of the K8S cluster may include cluster device geographic location information and device processor identification; the stateful load controller may also be used to: determine the geographical constraint range of device interconnection based on the cluster device geographic location information; determine the authority of processor interconnection based on the device processor identification.

[0072] The device processor identifier may be an identifier of a cluster device processor, and the geographic constraint range may be a constraint range determined according to geographic location information.

[0073] In an embodiment of the present invention, Statefulset can be used to determine the geographical constraint range of device interconnection using cluster device geographic location information, and cluster devices beyond the geographical constraint range cannot be interconnected. For example, if the distance between city A and city B exceeds 1,000 kilometers, which exceeds the geographical constraint range of 100 kilometers, interconnection scheduling is not allowed between cluster devices in city A and city B. At the same time, Statefulset can also determine the permissions for interconnection scheduling between different processors. Specifically, Statefulset can flexibly configure the permissions for interconnection of each processor based on the device processor identifier.

[0074] In an optional embodiment of the present invention, the K8S cluster may also be configured with a scheduler plug-in; the scheduler plug-in may be used to: determine the batch scheduling strategy of the K8S cluster; and perform batch scheduling of the resources of the K8S cluster in conjunction with the stateful load controller according to the batch scheduling strategy of the K8S cluster.

[0075] The batch scheduling strategy may be a strategy for batch scheduling. The so-called batch scheduling can be understood as: tasks are divided into batches for execution. Batch scheduling is usually used for tasks that need to be executed sequentially.

[0076] The embodiment of the present invention can also introduce a scheduler plug-in into the K8S cluster ( Figure 3The scheduler plug-in can be used in conjunction with Statefulset, specifically for batch scheduling of K8S cluster resources in conjunction with the stateful load controller Statefulset according to the batch scheduling strategy of the K8S cluster.

[0077] It can be seen that by introducing Statefulset and scheduler plug-ins, rich scheduling functions can be provided for the interactive development environment. Containers can be scheduled based on resources and affinity in a manner completely supported by K8S, and all-or-nothing batch scheduling can be performed through the installed scheduler plug-in.

[0078] In addition, Figure 3 As shown, users can access various interactive development environments through the istio Ingress gateway deployed in the cluster, or through the K8S NodePort. Among them, the istioIngress gateway can provide services using sub-URL (Uniform Resource Locator) or sub-domain forwarding, such as providing a Jupyter Notebook browser interface, or an API (Application Programming Interface) server. It can save IP / port resources and facilitate monitoring. K8S native NodePort can provide complete IP+port services, or SSH (Secure Shell Protocol) such as pure TCP (Transmission Control Protocol) connections for users to use, thereby delegating the greatest control to users and ensuring that there are no services that cannot be supported by the development environment.

[0079] Optionally, K8S can assign a virtual domain name to each container so that they can communicate with each other.

[0080] In summary, the interactive development environment in the embodiment of the present invention is composed of a K8S resource group, including a StatefulSet and several Istios. Each Pod of Statefulset can be composed of an interactive development environment container and an envoy proxy gateway container. Istio's network rules can act on each envoy proxy container, allowing containers of the same interactive development environment to communicate with each other, prohibiting cross-development environment / cross-user communication, or sniffing other services in the cluster. At the same time, Statefulset is used to release the native scheduling capabilities of K8S. By configuring K8S Statefulset, the scheduling of each interactive development environment container can be fine-grained, and scheduling plug-ins such as batch scheduling (Gang Scheduling) can be used to promote flexible allocation and effective use of computing power. Using Istio service grid and Statefulset, an interactive development environment on the cloud with multi-machine and multi-card interconnection is realized, which has high versatility and maintainability, can run all browser-based development environments, and has flexible network configuration. User authentication can be realized on demand, different user network isolation, and development environment and cluster The rest of the network isolation function. It also supports high-performance network stacks such as RDMA (Remote Direct Memory Access) to accelerate algorithm development. It can be seen that the above-mentioned K8S-based development environment configuration method solves the network isolation and resource scheduling problems, and can better support AI algorithm development. The method is simple to implement, and the interconnection and isolation are effective, meeting the application requirements.

[0081] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information (such as user identification information, etc.) are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0082] It should be noted that any arrangement and combination of the technical features in the above embodiments also falls within the protection scope of the present invention.

[0083] Embodiment 3

[0084] Figure 4 Schematic diagram of a K8S-based development environment configuration device provided in Embodiment 3 of the present invention, the device is configured in a service grid, such as Figure 4 As shown, the device includes: a container configuration reference information acquisition module 310, a development environment container configuration module 320 and an interconnection relationship establishment module 330, wherein:

[0085] The container configuration reference information acquisition module 310 is used to obtain the container configuration reference information of the associated interactive development environment container in the K8S cluster;

[0086] A development environment container configuration module 320, configured to configure a target interactive development environment container for the current interactive development environment container according to the container associated configuration information of the associated interactive development environment container;

[0087] The interconnection relationship establishing module 330 is used to establish an interconnection relationship between the current interactive development environment container and the target interactive development environment container.

[0088] The embodiment of the present invention obtains the container configuration reference information of the associated interactive development environment container in the K8S cluster through the service grid, and configures the target interactive development environment container for the current interactive development environment container according to the obtained container associated configuration information of the associated interactive development environment container, thereby establishing an interconnection relationship between the current interactive development environment container and the target interactive development environment container, thereby solving the problem that the existing K8S-based development environment configuration method only supports single-machine users or cannot support network isolation, and the like, and realizes the configuration of a cloud-based interactive development environment with multiple machines and multiple cards interconnected, which can improve the flexibility, versatility and maintainability of the K8S-based development environment configuration, and can better support AI algorithm development.

[0089] Optionally, the container configuration reference information acquisition module 310 is specifically used to: determine the user configuration information of the current user that matches the current interactive development environment container; determine the associated interactive development environment container from each interactive development environment container in the K8S cluster based on the user configuration information of the current user; and obtain the container configuration reference information of the associated interactive development environment container.

[0090] Optionally, the user configuration information of the current user includes the user identification of the current user; the container configuration reference information acquisition module 310 is specifically used to: obtain the user identification information of each of the interactive development environment containers in the K8S cluster; match the user identification of the current user with the user identification information of each of the interactive development environment containers in the K8S cluster; determine the interactive development environment container of the current user from the interactive development environment containers in the K8S cluster according to the user identification matching result as the associated interactive development environment container.

[0091] Optionally, the user configuration information of the current user also includes the cross-user development permissions of the current user; the container configuration reference information acquisition module 310 is specifically used to: determine the target user identifier based on the cross-user development permissions of the current user; match the target user identifier with the user identifier information of each interactive development environment container in the K8S cluster; determine the interactive development environment container of the target user authorized for the current user from each interactive development environment container in the K8S cluster based on the user identifier matching result, as the associated interactive development environment container.

[0092] Optionally, the K8S cluster is configured with a stateful load controller; the stateful load controller is used to: obtain cluster device association attribute information of the K8S cluster; and determine the K8S resource scheduling strategy based on the cluster device association attribute information of the K8S cluster.

[0093] Optionally, the cluster device association attribute information of the K8S cluster includes cluster device geographic location information and device processor identification; the stateful load controller is also used to: determine the geographical constraint range of device interconnection based on the cluster device geographic location information; determine the processor interconnection authority based on the device processor identification.

[0094] Optionally, the K8S cluster is also configured with a scheduler plug-in; the scheduler plug-in is used to: determine the batch scheduling strategy of the K8S cluster; and perform batch scheduling of the resources of the K8S cluster in conjunction with the stateful load controller according to the batch scheduling strategy of the K8S cluster.

[0095] The above-mentioned K8S-based development environment configuration device can execute the K8S-based development environment configuration method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, please refer to the K8S-based development environment configuration method provided by any embodiment of the present invention.

[0096] Since the K8S-based development environment configuration device introduced above is a device that can execute the K8S-based development environment configuration method in the embodiment of the present invention, based on the K8S-based development environment configuration method introduced in the embodiment of the present invention, the technical personnel in this field can understand the specific implementation method of the K8S-based development environment configuration device of this embodiment and its various variations, so how the K8S-based development environment configuration device implements the K8S-based development environment configuration method in the embodiment of the present invention will not be introduced in detail here. As long as the technical personnel in this field implement the device adopted by the K8S-based development environment configuration method in the embodiment of the present invention, it belongs to the scope of protection of this application.

[0097] Embodiment 4

[0098] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0099] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0100] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0101] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a development environment configuration method based on K8S.

[0102] Optionally, a K8S-based development environment configuration method is applied to a service grid, the method comprising: obtaining container configuration reference information of an associated interactive development environment container in a K8S cluster; configuring a target interactive development environment container for a current interactive development environment container according to the container associated configuration information of the associated interactive development environment container; and establishing an interconnection relationship between the current interactive development environment container and the target interactive development environment container.

[0103] In some embodiments, the K8S-based development environment configuration method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the K8S-based development environment configuration method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the K8S-based development environment configuration method by any other appropriate means (e.g., by means of firmware).

[0104] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0106] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0109] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

Claims

1. A development environment configuration method based on K8S, It is characterized in that Applied to a service grid, the method comprises: Get the container configuration reference information of the interactive development environment container associated with the K8S cluster; According to the container association configuration information of the associated interactive development environment container, a target interactive development environment container is configured for the current interactive development environment container; wherein the target interactive development environment container is an interactive development environment container that has an interconnection requirement with the current interactive development environment container; Establishing an interconnection relationship between the current interactive development environment container and the target interactive development environment container; The obtaining of the container configuration reference information of the interactive development environment container associated with the K8S cluster includes: Determine user configuration information of a current user that matches the current interactive development environment container; Determine the associated interactive development environment container from each interactive development environment container in the K8S cluster according to the user configuration information of the current user; Obtaining container configuration reference information of the associated interactive development environment container; The configuring the target interactive development environment container for the current interactive development environment container according to the container associated configuration information of the associated interactive development environment container includes: Determine the current interactive development environment container and the target interactive development environment container based on the container identifier in the container configuration information; A mapping relationship is established between the container identifier of the current interactive development environment container and the container identifier of the target interactive development environment container, so as to configure the target interactive development environment container for the current interactive development environment container.

2. The method according to claim 1, It is characterized in that The user configuration information of the current user includes the user identifier of the current user; The determining the associated interactive development environment container from each interactive development environment container in the K8S cluster according to the user configuration information of the current user includes: Obtain user identification information of each interactive development environment container in the K8S cluster; Match the user identification of the current user with the user identification information of each interactive development environment container in the K8S cluster; According to the user identification matching result, the interactive development environment container of the current user is determined from each interactive development environment container in the K8S cluster as the associated interactive development environment container.

3. The method according to claim 2, It is characterized in that The user configuration information of the current user also includes the cross-user development authority of the current user; The determining the associated interactive development environment container from each interactive development environment container in the K8S cluster according to the user configuration information of the current user includes: Determine the target user identifier according to the cross-user development authority of the current user; Match the target user identifier with the user identifier information of each interactive development environment container in the K8S cluster; According to the user identification matching result, an interactive development environment container of the target user authorized by the current user is determined from each interactive development environment container in the K8S cluster as the associated interactive development environment container.

4. The method according to claim 1, It is characterized in that The K8S cluster is configured with a stateful load controller; the stateful load controller is used to: Obtain cluster device associated attribute information of the K8S cluster; Determine the K8S resource scheduling strategy based on the cluster device associated attribute information of the K8S cluster.

5. The method according to claim 4, It is characterized in that The cluster device associated attribute information of the K8S cluster includes cluster device geographic location information and device processor identification; the stateful load controller is also used for: Determine the geographical constraint range of device interconnection according to the geographical location information of the cluster devices; The processor interconnection authority is determined according to the device processor identification.

6. The method according to claim 4, It is characterized in that The K8S cluster is also configured with a scheduler plug-in; the scheduler plug-in is used to: Determine the batch scheduling strategy of the K8S cluster; According to the batch scheduling strategy of the K8S cluster, the stateful load controller is combined to perform batch scheduling on the resources of the K8S cluster.

7. A development environment configuration device based on K8S, It is characterized in that Configured in a service grid, the device includes: The container configuration reference information acquisition module is used to obtain the container configuration reference information of the interactive development environment container associated with the K8S cluster; A development environment container configuration module, configured to configure a target interactive development environment container for the current interactive development environment container according to the container association configuration information of the associated interactive development environment container; wherein the target interactive development environment container is an interactive development environment container that has an interconnection requirement with the current interactive development environment container; An interconnection relationship establishing module, used to establish an interconnection relationship between the current interactive development environment container and the target interactive development environment container; The container configuration reference information acquisition module is also used to: determine the user configuration information of the current user that matches the current interactive development environment container; determine the associated interactive development environment container from each interactive development environment container in the K8S cluster according to the user configuration information of the current user; and obtain the container configuration reference information of the associated interactive development environment container; The development environment container configuration module is also used to: determine the current interactive development environment container and the target interactive development environment container based on the container identifier in the container configuration information; A mapping relationship is established between the container identifier of the current interactive development environment container and the container identifier of the target interactive development environment container, so as to configure the target interactive development environment container for the current interactive development environment container.

8. An electronic device, It is characterized in that The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the K8S-based development environment configuration method described in any one of claims 1-6.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the processor to implement the K8S-based development environment configuration method described in any one of claims 1-6 when executed.

Citation Information

Patent Citations

  • Development environment construction method, apparatus, platform system and storage medium

    CN109104467A

  • An interactive model development environment system and method based on a cloud environment

    CN109901823A

  • Development and test environment deployment method and system, electronic equipment and medium

    CN113448837A

  • Container management method and device of target application, electronic equipment and storage medium

    CN114489951A

  • Multi-cluster communication method and device, electronic equipment and storage medium

    CN115767786A