Big data platform tenant management system, methods, storage media and electronic devices

Abstract

This application relates to the field of big data platform technology, specifically to a big data platform tenant management system, method, storage medium, and electronic device. The system includes a comprehensive management subsystem, used to send alarm information to an alarm subsystem when the user's tenant resource usage on the big data platform meets the conditions for creating a new tenant; the alarm subsystem, used to obtain the first resources required by the user to create a new tenant after receiving the alarm information; and an automatic deployment tool, used to deploy service components for the new tenant created by the user on the obtained first resources. This application, by monitoring tenant resource usage, triggers the new tenant creation process when the conditions for creating a new tenant are met, automatically creating a new tenant and automatically deploying the required service components for the new tenant. This achieves automated management of tenant resource monitoring, early warning, and the new tenant creation process, improving management efficiency, saving labor costs, and providing strong early warning capabilities.

Description

Technical Field

[0001] This invention relates to the field of big data platform technology, specifically to a big data platform tenant management system, method, storage medium, and electronic device. Background Technology

[0002] With the development of big data platform technology, big data services are increasingly being applied to various scenarios such as data analysis and data pattern research across industries. For example, the MapReduce service (MRS) provided by a big data platform is an enterprise-grade Hadoop big data cluster cloud service that is compatible with open source and fully controllable by tenants. MRS enables massive data analysis scenarios. For instance, in studying the weather patterns of a city, the big data platform's MRS service can quickly analyze the weather trends of that city over the past year or several years. The basic data used by the big data platform to analyze weather trends may come from weather service applications (APPs) or browser pages. Similarly, in analyzing customer shopping preferences, the big data platform's MRS service can quickly analyze customer search preferences to determine their shopping habits, enabling more accurate product recommendations and higher sales conversion rates. The basic data used to analyze customer shopping preferences may also come from shopping apps or browser pages.

[0003] Illustrative scenarios of big data platforms providing data services, such as Figure 1 As shown, the big data platform 100 may include a big data platform tenant management system 110 and a big data platform data service system 120. The big data platform tenant management system 110 is used to manage tenants opened by users on the big data platform 100, including managing basic tenant information (including the tenant's system name, ID, etc.), configuring tenant resources, monitoring tenant resources, and reclaiming idle resources, etc. Figure 1As shown, the tenant management system 110 of the big data platform manages tenants 1-n, which can have a one-to-one or many-to-one correspondence with users. Each tenant n has a unique system name and ID. The big data platform data service system 120 provides various data services required by users to their respective users through tenants via data collection, data storage, data analysis, data modeling, and data display. Users of the big data platform can be individuals (e.g., researchers, software developers with specific needs), organizations (e.g., research institutions, government agencies), or developers in various internet industries (e.g., smart device developers, app developers). Different users have different usage needs and service resource requirements for the big data platform. Some users only need to open one tenant on the big data platform to meet their needs, while others have more usage needs and higher service resource requirements, requiring multiple tenants to meet their needs. Since the resource capacity of each tenant is limited, when the resource usage of an existing tenant reaches or exceeds the total resource capacity of that tenant, a new tenant needs to be created to respond to the user's service request. Summary of the Invention

[0004] This application provides a big data platform tenant management system and method. By automatically detecting and monitoring tenant resource usage, the system determines whether the conditions for creating a new tenant are met. When the tenant's resource usage meets the conditions, the system automatically sends an alarm notification to the user and automatically triggers the start of the new tenant creation process. It automatically creates a new tenant, automatically deploys the necessary service components for the new tenant, and automatically applies to the public cloud to create a cloud service instance. After the new tenant is ready, the system automatically enables the new tenant to respond to new APP access requests. This achieves automated management of tenant resource monitoring, early warning, and the new tenant creation process, greatly improving management efficiency, saving labor costs, and providing strong early warning capabilities. It effectively avoids many problems caused by tenant resource usage exceeding limits, improving the user experience of the big data platform.

[0005] In a first aspect, embodiments of this application provide a big data platform tenant management system, the system comprising: a comprehensive management subsystem, configured to send alarm information to an alarm subsystem when it is detected that the user's tenant resource usage on the big data platform meets the conditions for creating a new tenant; an alarm subsystem, configured to obtain a first resource required by the user to create a new tenant after receiving the alarm information; and an automatic deployment tool, configured to deploy service components for the new tenant created by the user on the obtained first resource.

[0006] For example, the tenant management module of the integrated management subsystem manages tenants and automatically detects their resource usage. The configuration management module configures capacity or utilization thresholds for tenant resources and monitors whether the tenant's resource usage meets the conditions for creating a new tenant. When the conditions for creating a new tenant are met, the configuration management module sends an alarm message to the alarm subsystem. The alarm access module of the alarm subsystem receives the alarm message and the resource request module within the alarm subsystem obtains the virtual machine resources required to create a new tenant (e.g., the first resource is virtual machine resources).

[0007] In one possible implementation of the first aspect above, if the tenant's resource usage is greater than or equal to a capacity threshold, the user's tenant resource usage meets the conditions for creating a new tenant; or if the tenant's resource utilization rate is greater than or equal to a utilization rate threshold, the user's tenant resource usage meets the conditions for creating a new tenant, wherein the tenant's resource utilization rate is the ratio between the tenant's resource usage and the tenant's total resource capacity.

[0008] For example, if the configuration management module of the integrated management subsystem is set to a capacity threshold, then an alarm message will be sent to the alarm subsystem when the tenant's resource usage is greater than or equal to the capacity threshold; if the configuration management module of the integrated management subsystem is set to a utilization rate threshold, then an alarm message will be sent to the alarm subsystem when the tenant's resource utilization rate is greater than or equal to the utilization rate threshold.

[0009] In one possible implementation of the first aspect above, the alarm subsystem acquires the first resource in the following manner: when the user's vacant resources on the big data platform are less than the first resource, the alarm subsystem sends a resource acquisition request to an external entity of the big data platform, wherein the resource acquisition request is used to request the acquisition of a second resource, the second resource being determined based on the difference between the first resource and the vacant resource; when the user's vacant resources on the big data platform are more than or equal to the first resource, the alarm subsystem acquires the vacant resource as the first resource.

[0010] For example, if the vacant resources are vacant virtual machine resources, the resource request module in the alarm subsystem can query the vacant virtual machine resource status in the resource pool of the big data platform. If the vacant virtual machine resources in the resource pool are insufficient for a new tenant, it will call the resource request API of the external virtual machine resource library to request new virtual machine resources (e.g., the second resource is a new virtual machine resource) as a supplement. If the vacant virtual machine resources in the resource pool are sufficient, the resource request module can send a resource sufficiency feedback message to the integrated management subsystem. This resource acquisition method helps improve the resource utilization efficiency of the big data platform.

[0011] In one possible implementation of the first aspect above, the automatic deployment tool deploys service components for the new tenant created by the user on the acquired first resource in the following manner: filtering third resources required by the service components from the first resource; creating a server cluster and server cluster topology for the service components on the acquired third resource; installing the software required by the service components on the created cluster; starting the software and completing the initial parameter configuration of the software.

[0012] For example, the automated deployment tool's automatic fetch script retrieves service packages for service components from the software repository on a big data platform. These service packages include filters for the corresponding service components, cluster topology requirement descriptions, software binaries, and automated scripts. The automated cluster building module of the automated deployment tool can parse these service packages and use the filters within them to select the necessary virtual machine resources (i.e., third resources) for the corresponding service components from a first resource (e.g., virtual machine resources prepared for new tenants). The automated cluster building module then creates server clusters and server cluster topologies for the service components on the selected third resources. The automated installation process control module of the automated deployment tool can control the execution of the automated scripts, installing the software from the service packages on the created clusters, and then starting the installed software after installation.

[0013] In one possible implementation of the first aspect above, the automatic deployment tool deploys service components for the new tenant created by the user on the acquired first resource in a manner that further includes: when there are dependencies between the service components, constructing a tree-like dependency relationship between the service components based on the dependencies; and, according to the tree-like dependency relationship, sequentially installing the software required by each service component from the root of the tree-like dependency relationship upwards; and after the software required by each service component is installed, sequentially starting the software of each service component from the root of the tree-like dependency relationship upwards; when there are no dependencies between the service components, the automatic deployment tool installs the software required by each service component in a parallel manner, and after the software required by each service component is installed, starts the software of each service component in a parallel manner.

[0014] For example, in a tree-like dependency relationship, the service component that is depended upon is closer to the root of the dependency tree. When deploying service components, the automatic deployment tool installs the software of the dependent service component first, and then installs the software of other service components sequentially upwards from the root of the dependency tree. After all service component software is installed, the automatic deployment tool first starts and runs the software of the dependent service component, and then starts and runs the software of other service components sequentially upwards from the root of the dependency tree.

[0015] In one possible implementation of the first aspect above, when the user's tenant resource usage on the big data platform is detected to meet the conditions for creating a new tenant, the integrated management subsystem is further configured to generate identification information for the new tenant, wherein the identification information includes at least the system name or identification number of the new tenant in the big data platform; and, when the service component is deployed, the integrated management subsystem sends the identification information of the new tenant to the service component to activate the service component; and, when the service component is activated, the integrated management subsystem sends an instance request to the public cloud, wherein the instance request is used to request the creation of a cloud service instance for the new tenant in the public cloud.

[0016] For example, the tenant management module in the integrated management subsystem automatically generates a new tenant's system name or identity document (ID) at the start of the new tenant creation process. It can also generate both simultaneously to distinguish the new tenant from existing tenants. After the automated deployment tool deploys the service component, it can send the service component's access address (URL) back to the integrated management subsystem. The tenant management module then sends the new tenant's identification information based on this address to activate the service component. Upon receiving the new tenant's identification information, the service component prepares to respond to service requests, such as app access requests, according to the tenant's identification information.

[0017] In one possible implementation of the first aspect above, the resources required by the tenant include at least one of virtual machine resources, physical machine resources, and container resources, and the resources required by the tenant include the first resource.

[0018] For example, the first resource, the vacant resource, the second resource, and the third resource mentioned above are all virtual machine resources.

[0019] Secondly, embodiments of this application provide a method for managing tenant resources on a big data platform. The method includes: sending an alarm message when it is detected that the usage of tenant resources by a user on the big data platform meets the conditions for creating a new tenant; wherein the alarm message can trigger the acquisition of a first resource required by the user to create a new tenant; and deploying service components for the new tenant created by the user on the acquired first resource.

[0020] In one possible implementation of the second aspect above, the method further includes: when the tenant's resource usage is greater than or equal to a preset capacity threshold, the user's tenant resource usage meets the conditions for creating a new tenant; or when the tenant's resource utilization rate is greater than or equal to a utilization rate threshold, the user's tenant resource usage meets the conditions for creating a new tenant, wherein the tenant's resource utilization rate is the ratio between the tenant's resource usage and the tenant's total resource capacity.

[0021] In one possible implementation of the second aspect above, the method further includes: obtaining the first resource required by the user to create a new tenant in the following manner: if the user's vacant resources on the big data platform are less than the first resource, sending a resource acquisition request to an external part of the big data platform, wherein the resource acquisition request is used to request the acquisition of a second resource, the second resource being determined based on the difference between the first resource and the vacant resources; if the user's vacant resources on the big data platform are more than or equal to the first resource, acquiring the vacant resources as the first resource.

[0022] In one possible implementation of the second aspect above, the step of deploying a service component for a new tenant created for the user on the acquired first resource includes: filtering a third resource required for the service component from the first resource; creating a server cluster and server cluster topology for the service component on the acquired third resource; installing the software required for the service component on the created cluster; starting the software and configuring the initial parameters of the software.

[0023] In one possible implementation of the second aspect above, the deployment of service components for the new tenant created for the user on the acquired first resource further includes: if there are dependencies between the service components, constructing a tree-like dependency relationship between the service components based on the dependencies, and sequentially installing the software required by each service component from the root of the tree-like dependency relationship upwards according to the tree-like dependency relationship; and after the software required by each service component is installed, sequentially starting the software of each service component from the root of the tree-like dependency relationship upwards according to the tree-like dependency relationship; if there are no dependencies between the service components, installing the software required by each service component in a parallel manner, and starting the software of each service component in a parallel manner after the software required by each service component is installed.

[0024] In one possible implementation of the second aspect above, the method further includes: the alarm information triggering the big data platform to generate identification information for the new tenant, the identification information including at least the system name of the new tenant in the big data platform or the identity number of the new tenant; and, upon completion of the deployment of the service component, sending the identification information of the new tenant to the service component to activate the service component; and, upon completion of the activation of the service component, the big data platform sending an instance request to the public cloud, the instance request being used to request the creation of a cloud service instance for the new tenant in the public cloud.

[0025] In one possible implementation of the second aspect above, the method further includes: the resources required by the tenant include at least one of virtual machine resources, physical machine resources, and container resources, and the resources required by the tenant include the first resource.

[0026] Thirdly, embodiments of this application provide a tenant management method for a big data platform, the method comprising: receiving alarm information when the user's tenant resource usage on the big data platform meets the conditions for creating a new tenant; obtaining a first resource required by the user to create a new tenant based on the obtained alarm information; wherein the service components of the tenant can be deployed on the first resource.

[0027] For example, after receiving alarm information from the integrated management subsystem, the alarm subsystem obtains the virtual machine resources required by the user to create a new tenant (for example, the first resource is virtual machine resources).

[0028] In one possible implementation of the third aspect above, the method further includes: when the tenant's resource usage is greater than or equal to a preset capacity threshold, the user's tenant resource usage meets the conditions for creating a new tenant; or when the tenant's resource utilization rate is greater than or equal to a utilization rate threshold, the user's tenant resource usage meets the conditions for creating a new tenant, wherein the resource utilization rate is the ratio between the tenant's resource usage and the tenant's total resource capacity.

[0029] In one possible implementation of the third aspect above, the method further includes obtaining the first resource required by the user to create a new tenant in the following manner: if the user's vacant resources on the big data platform are less than the first resource, sending a resource acquisition request to an external entity of the big data platform, wherein the resource acquisition request is used to request the acquisition of a second resource, the second resource being determined based on the difference between the first resource and the vacant resources; if the user's vacant resources on the big data platform are more than or equal to the first resource, acquiring the vacant resources as the first resource.

[0030] In one possible implementation of the third aspect above, the resources required by the tenant include at least one of virtual machine resources, physical machine resources, and container resources, and the resources required by the tenant include the first resource.

[0031] Fourthly, embodiments of this application provide a method for managing tenant resources of a big data platform, the method comprising: obtaining a call instruction; and, in response to the call instruction, deploying a service component for a new tenant created by a user of the big data platform on a first resource.

[0032] For example, after receiving a call instruction from the integrated management subsystem, the automated deployment tool responds to the call instruction by deploying service components for the new tenant on the prepared virtual machine resources.

[0033] In one possible implementation of the fourth aspect above, the step of deploying a service component for a new tenant created for the user on the acquired first resource includes: filtering a third resource required for the service component from the first resource; creating a server cluster and server cluster topology for the service component on the acquired third resource; installing the software required for the service component on the created cluster; starting the software and completing the initial parameter configuration of the software.

[0034] In one possible implementation of the fourth aspect above, the deployment of service components for the new tenant created for the user on the acquired first resource further includes: when there are dependencies between the service components, constructing a tree-like dependency relationship between the service components based on the dependencies, and sequentially installing the software required by each service component from the root of the tree-like dependency relationship upwards according to the tree-like dependency relationship; and after the software required by each service component is installed, sequentially starting the software of each service component from the root of the tree-like dependency relationship upwards according to the tree-like dependency relationship; when there are no dependencies between the service components, installing the software required by each service component in a parallel manner, and starting the software of each service component in a parallel manner after the software required by each service component is installed.

[0035] Fifthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the aforementioned big data platform tenant management method.

[0036] Sixthly, embodiments of this application provide an electronic device, including: one or more processors; one or more memories; the one or more memories storing one or more programs, which, when executed by the one or more processors, cause the electronic device to execute the above-described big data platform tenant management method.

[0037] Seventhly, embodiments of this application provide an apparatus included in an electronic device, which has the function of implementing the behaviors of the electronic device in the above aspects and possible implementations thereof. The functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or subsystems corresponding to the above functions. For example, a comprehensive management subsystem (such as a processor with the functions of such a subsystem), an automatic deployment tool (such as a controller with the functions of such a tool), etc. For example, the comprehensive management subsystem is used to manage tenants, configure capacity thresholds, automatically detect and monitor the resource usage of tenants, issue alarm information when the resource usage of a tenant exceeds the capacity threshold, and generate identification information of a new tenant at the start of the new tenant creation process; it is also used to apply for cloud service instances from the public cloud for new tenants, etc.; the automatic deployment tool is used to obtain service software packages, deploy service components for new tenants, etc. Attached Figure Description

[0038] Figure 1 The image shows a schematic diagram illustrating a scenario where a big data platform provides data services.

[0039] Figure 2 The diagram shown is an application scenario illustration of the big data platform tenant management system and method according to an embodiment of this application.

[0040] Figure 3 The diagram shown is a structural schematic of the big data platform tenant management system according to an embodiment of this application.

[0041] Figure 4 The diagram shown is a flowchart of a big data platform tenant management method according to an embodiment of this application.

[0042] Figure 5 The diagram shown is a schematic of the operation and maintenance interface of the big data platform tenant management system 110 according to an embodiment of this application.

[0043] Figure 6 A system block diagram of a computer according to an embodiment of this application is shown. Detailed Implementation

[0044] As described above, the big data platform 100 includes a big data platform tenant management system 110 and a big data platform data service system 120. When the remaining available resources of an existing tenant are insufficient, it is necessary to open a new tenant to expand resources to meet the user's needs. Embodiments of this application disclose a big data platform tenant management method. This method, based on the big data platform tenant management system 110, can automatically monitor the resource usage of existing tenants and automatically open a new tenant to expand resources to meet the user's needs when the remaining available resources of an existing tenant are insufficient.

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of the embodiments of this application will be further described in detail below with reference to the accompanying drawings and implementation schemes.

[0046] To avoid conceptual confusion, in this embodiment of the application, the user of the big data platform 100 is hereinafter referred to as "user," and the specific APP user is hereinafter referred to as "customer." This distinction is explained here. For example, for a shopping APP, the application developer who develops the shopping APP can be the user, while the user who downloads the shopping APP to make purchases is the customer.

[0047] exist Figure 1 In the illustrated scenario of a big data platform providing data services, the data collection methods in the big data platform data service system 120 can be varied, including but not limited to log collection and data synchronization from data sources, which are not limited here. Taking the collection of APP log data as an example, a dedicated statistical software development kit (SDK) can be developed for APP log data collection. That is to say, after the APP successfully connects to the big data platform 100, various log data generated during the APP's operation can be continuously pushed into the big data platform 100. Generally speaking, the relevant data during the APP's operation has strong business characteristics and high customization requirements. Therefore, in addition to some basic data such as the application environment, APP data is collected more from the perspective of "by event," such as login events and business operation events. Data storage in the big data platform data service system 120 stores the various data collected during the data collection process for further data analysis, data modeling, and data visualization. Users, based on their needs or the purpose of using the data, send data requests to the big data platform 100 through a tenant on the user interface. The big data platform's data service system 120 then performs data analysis, modeling, or visualization based on the massive amounts of stored data to obtain results that match the data request and feed them back to the user. For example, based on data related to a shopping app accessed by a tenant, the big data platform 100 can analyze a customer's shopping preferences based on log data generated from customer product searches, and can also analyze the relationship between the customer's shopping preferences and age / gender by combining information such as age and gender from the customer's registration information. It is understood that the format in which the big data platform's data service system 120 displays the data service results to the user can be set by the user themselves, and there are no restrictions here.

[0048] As described above, in this embodiment of the application, the tenants on the big data platform 100 are managed by the big data platform tenant management system 110.

[0049] Figure 2This illustration shows an application scenario diagram of a big data platform tenant management system 110 according to an embodiment of this application. For example... Figure 2 As shown, in the big data platform tenant management system 110, user 111 has tenant 1111, user 112 has tenants 1121-1122, and user 113 has tenants 1131-1133. It is understood that in some other embodiments, each user may also have other numbers of tenants. Each user can apply for one or more tenants on the big data platform 100 according to their own needs, and there is no restriction here.

[0050] It is understood that the resources of the aforementioned tenants are isolated from each other, and the tenants respond to various user requests through their service components and cloud service instances. The tenant's resources are mainly used to respond to various user requests. For ease of understanding, the technical solution of this application will be described in detail below using the tenant's resources responding to the APP's access request as an example. Here, the APP's resource usage can be quantified as the number of APPs accessed by the tenant.

[0051] Understandably, due to limited tenant resources, the number of app access requests each tenant can respond to is also limited. Once an app access request is responded to, the app successfully connects to the tenant. When a user's existing tenants are about to reach their limit for responding to app access requests, the user needs to create a new tenant to respond to new app access requests. It's important to note that when the new tenant's resources are ready, the user will enable the new tenant to respond to new app access requests and simultaneously stop using the existing tenants to respond to new app access requests. The number of apps connected to the existing tenants will be the same as the number of apps connected up to the point when the new tenant's resources are ready.

[0052] like Figure 2As shown, assume each tenant can respond to 500 app access requests. When a user has sufficient existing tenant resources, new app access requests will be responded to by existing tenants. For example, if user 111 has 300 apps that need to access the tenant, then all 300 app access requests will be responded to by tenant 1111, and tenant 1111 will still have enough resources to respond to 200 app access requests. When a user's existing tenant resources are insufficient, the user needs multiple tenants to respond to app access requests. For example, user 112 has two tenants, 1121 and 1122. When 800 apps need to access the tenant, user 112 needs to complete the new tenant creation process before its tenant 1121 accesses the 500th app, resulting in a ready new tenant 1122. Simultaneously, tenant 1122 is enabled to respond to new app access requests, while tenant 1121 stops responding to new app access requests. If tenant 1122 completes its creation and is ready when tenant 1121 has responded to the access request of the 499th APP, then from this point onwards, tenant 1121 will stop responding to new APP access requests, and tenant 1122 will continue to respond to the access requests of the 500th APP and subsequent new APPs.

[0053] Similarly, user 113 has three tenants 1131-1133. When 1110 apps need to connect to the tenant, user 113 must complete the new tenant creation process before tenant 1131 connects to the 500th app to obtain a ready new tenant 1132 to continue responding to the new app's access request. For example, if tenant 1131 has already responded to the access request of the 501st app, and tenant 1132 is created and ready, then from this point on, tenant 1132 continues to respond to the access request of the 502nd app, while tenant 1131 stops responding to the new app's access request. As the number of apps connected to tenant 1132 continues to increase, user 113 must complete the new tenant creation process before tenant 1132 connects to the 1002nd app to obtain a ready new tenant 1133 to continue responding to the new app's access request. For example, if tenant 1133 is newly created and ready when tenant 1132 has responded to the access request of the 1000th app, then from this point onwards, tenant 1133 will continue to respond to the access request of the 1001st app, while tenant 1132 will stop responding to new app access requests. However, if a user's app access demand exceeds the app access capacity of the user's existing tenants, a new tenant needs to be applied for for the user. For example, if user 113 has 2000 apps that need to access a tenant, then a new tenant capable of supporting 500 apps needs to be applied for for user 113.

[0054] It's understandable that the number of apps a tenant can connect to largely depends on the resource requirements of each app. For example, some apps have a very large user base, so the resource requirements after connecting to a tenant will also be very large (these apps can be called large apps, such as WeChat); while some apps have a small user base, so the resource requirements after connecting to a tenant will be small (these apps can be called general apps or small apps, for example). For instance, the resource requirements of the WeChat app in the social networking category may be about 10 times that of other social networking apps. If a tenant responds to the access request of a large app (such as WeChat), the number of app access requests that the tenant can respond to may decrease.

[0055] Currently, monitoring tenant resource usage and the process of creating new tenants primarily rely on manual intervention. For example, the resource usage of each tenant is manually monitored through the big data platform's operations and maintenance interface. When a tenant's resource usage approaches or exceeds the total resource capacity, the operations and maintenance interface sends an alarm notification to the user interface. Simultaneously, the system monitors the real-time response volume of that tenant's access requests and continuously urges the user to apply for a new tenant as soon as possible. When a user applies to activate a new tenant, the process involves the user submitting a new tenant activation application, the big data platform confirming the application, and then the user manually submitting various resource requests, filling in information, and confirming authorization, completing the new tenant activation process step by step. This results in a lengthy activation time. If, during this process, there is a sudden surge in the data volume of existing tenants (for example, a surge in data volume caused by a batch of app access requests requiring tenant responses), it may lead to resource overload of existing tenants before the new tenant is activated, resulting in reduced operating efficiency or even server crashes, leading to a poor user experience.

[0056] To address the aforementioned technical issues, this application provides a big data platform tenant management system and method capable of automatically detecting and monitoring tenant resource usage and automatically completing the new tenant creation process. By automatically detecting and monitoring tenant resource usage, the system determines whether the conditions for creating a new tenant are met. When the tenant's resource usage meets the conditions, the system automatically sends an alarm notification to the user and automatically triggers the start of the new tenant creation process. It automatically creates a new tenant, automatically deploys the necessary service components for the new tenant, and automatically applies to the public cloud to create a cloud service instance. Once the new tenant is ready, the system automatically enables the new tenant to respond to new APP access requests. This achieves automated management of tenant resource monitoring, early warning, and the new tenant creation process, significantly improving management efficiency, saving labor costs, and providing strong early warning capabilities, effectively avoiding many problems caused by tenant resource usage exceeding limits.

[0057] Specifically, the big data platform tenant management system and method of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] Figure 3 A schematic system structure block diagram of a big data platform tenant management system 110 according to an embodiment of this application is shown. Figure 3 As shown, the big data platform tenant management system 110 may include a comprehensive management subsystem 101, an alarm subsystem 102, a resource pool 103, an automatic deployment tool 104, and a software repository 105. The comprehensive management subsystem 101 may include a tenant management module 1011, a configuration management module 1012, and a resource metadata management module 1013. The alarm subsystem 102 may include an alarm access module 1021, an alarm sending module 1022, and a resource request module 1023. The resource pool 103 may include a resource import interface 1031, a virtual machine resource management module 1032, and a resource call interface 1033. The automatic deployment tool 104 may include an automatic script retrieval module 1041, an automatic cluster building module 1042, and an automatic installation process control module 1043. The functions of each module will be described in detail below.

[0059] It is understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the big data platform tenant management system 110. In other embodiments, the big data platform tenant management system 110 may include more or fewer modules or system structures than illustrated, or may split certain modules or system structures, or arrange different modules or system structures, etc. The illustrated structure may be implemented as a software module, or as hardware, or a combination of software and hardware.

[0060] The various interfaces on each system structure or module of the Big Data Platform Tenant Management System 110, as well as the interfaces called by the Big Data Platform Tenant Management System 110 when requesting cloud service instances from the public cloud 300, can be Application Programming Interfaces (APIs) with unique identification information. Similarly, APIs with unique identification information can also be used within the Big Data Platform Tenant Management System 110 to implement resource request, resource import, resource invocation, tool calls, and information transmission. These interfaces can also be other types of interfaces, software, or scripts with unique identification information; no restrictions are placed here.

[0061] In the aforementioned big data platform tenant management system 110, the functions of each system structure are described below. However, the following functional descriptions do not constitute a specific limitation on each system structure. Each system structure may have more functions and corresponding functional modules, which are not restricted here.

[0062] Integrated Management Subsystem 101

[0063] The integrated management subsystem 101 is used to manage tenants, configure capacity thresholds, automatically detect and monitor tenant resource usage, and manage resource metadata for each tenant on the big data platform 100. It is also used to request the creation of cloud service instances from the public cloud 300 for any tenant. Tenant resources include service component clusters (i.e., the aforementioned service components), cloud service instances on the public cloud, and hardware resources such as the central processing unit (CPU) and memory (mainly provided by virtual machines) upon which the service components depend. Only when all of these resources are ready (i.e., sufficient hardware resources, and the service components and cloud service instances are operational and available) can the tenant process various user requests. For example, a tenant can respond to user APP access requests. APP access requests are responded to by the tenant's service components or cloud service instances. After successful APP access, it continuously pushes various data generated during APP operation to the big data platform. This data can serve as the basic data source for data collection in the big data platform's data service system 120. The aforementioned service components and cloud service instances provide a virtual computing environment, including but not limited to basic computing components such as CPU, memory, operating system, bandwidth, and disk. The aforementioned resource metadata includes, but is not limited to, real-time usage status data of resources such as CPU and memory.

[0064] Specifically, in some embodiments, the integrated management subsystem 101 implements the above functions based on the following modules:

[0065] The tenant management module 1011 is used to manage tenants and automatically detect their resource usage. Specifically, the tenant management module 1011 manages tenants by registering or recording the identification information, resource configuration information, and resource usage of existing tenants, and automatically detecting their resource usage. The tenant management module 1011 also automatically triggers a new tenant creation process by executing a preset algorithm when the conditions for creating a new tenant are met. After the new tenant creation process begins, the new tenant's system name and ID are automatically generated and recorded in the tenant management module 1011 as part of the new tenant's unique identification information.

[0066] The configuration management module 1012 is used to configure capacity thresholds or utilization thresholds for tenant resources and monitor whether the tenant's resource usage exceeds the capacity threshold or utilization threshold. One or more capacity or utilization thresholds can be set; there is no restriction here. Specifically, taking setting a single capacity threshold as an example, the configuration management module 1012 can obtain the tenant's resource usage from the tenant management module 1011 and compare the obtained resource usage with the tenant's capacity threshold. When the tenant's resource usage exceeds the capacity threshold, the configuration management module 1012 sends an alarm message to the alarm subsystem 102 and the tenant management module 1011. After receiving the alarm message, the alarm subsystem 102 sends an alarm notification to the user and triggers an automatic resource application process. After receiving the alarm message, the tenant management module 1011 automatically triggers a new tenant creation process. The tenant management module 1011 receiving the alarm message indicates that the conditions for creating a new tenant have been met.

[0067] In addition, in some embodiments, the configuration management module 1012 can also set multiple capacity thresholds. For example, the configuration management module 1012 sets a first threshold, a second threshold, and a third threshold sequentially from low to high, wherein the difference between the first threshold and the second threshold is greater than the difference between the second threshold and the third threshold. When a tenant's resource usage exceeds the first threshold, the second threshold, or the third threshold, the configuration management module 1012 will send alarm information to the alarm subsystem 102 and the tenant management module 1011. Under this alarm mechanism, the alarm subsystem 102 can be configured to trigger the automatic resource request process only when the tenant's resource usage exceeds the third threshold, and to only send an alarm notification to the user when the tenant's resource usage exceeds the first or second threshold. This alarm mechanism can be understood as a phased alarm mechanism.

[0068] The resource metadata management module 1013 is used to record the usage status of resource metadata corresponding to each tenant on the big data platform 100 and to manage the resources of each tenant. The usage status of resource metadata includes, but is not limited to, parameters related to CPU running status and memory usage status. This usage status is used for querying by service components corresponding to each tenant on the big data platform 100, and also serves as the basis for processing requests for CPU, memory, and other resources, thereby enabling the management of tenant resources. For example, when a service component of a tenant needs to use a certain resource (such as using CPU for computation or using memory space to store data), but that resource is currently unavailable (e.g., no CPU is available for allocation) or insufficient (e.g., current memory usage is too high), the resource metadata management module 1013 will reject the request from that service component and send a message to the tenant management module 1011 indicating that the tenant's resources are insufficient. Upon receiving this message, the tenant management module 1011 can manage the tenant to suspend responding to service requests supported by the aforementioned service component, and will resume responding to the corresponding service request when the required resources become available. The resource metadata management module 1013 is equipped with an upper call interface for other systems (such as the alarm subsystem 102) to call.

[0069] Alarm Subsystem 102

[0070] The alarm subsystem 102 receives alarm information from the configuration management module 1012 of the integrated management subsystem 101. After receiving the alarm information, the alarm subsystem 102 sends an alarm notification to the user and triggers an automatic resource application process. This automatic resource application process is part of the new tenant creation process triggered by the tenant management module 1011. The automatic resource application process can prepare resources for the new tenant, including but not limited to virtual machine resources, physical machine resources, and container resources. In this embodiment, virtual machine resources are used as an example to illustrate the technical solution of this application. Specifically, in some embodiments, the alarm subsystem 102 implements the above functions based on the following modules:

[0071] The alarm access module 1021 is used to receive alarm information sent by the configuration management module 1012 and send the alarm information to other modules of the alarm subsystem.

[0072] The alarm sending module 1022 is used to receive alarm information sent by the alarm access module 1021 and generate an alarm notification to send to the user. In some other embodiments, the alarm sending module 1022 can also send the alarm notification to the operation and maintenance interface of the big data platform 100 to remind the operation and maintenance personnel of the big data platform 100 to pay attention to the resource usage of the tenants that are alerted.

[0073] The resource application module 1023 receives alarm information from the alarm access module 1021. Upon receiving the alarm information, the resource application module 1023 first queries the available virtual machine resources in the resource pool 103, such as the number of available virtual machines for the relevant user and their configuration parameters. When the available virtual machine resources in the resource pool 103 are insufficient to meet the resource needs of a new tenant (e.g., insufficient number of available virtual machines or configurations that do not meet the new tenant's requirements), the resource application module 1023 automatically calls the resource application API of the external virtual machine resource library to request new virtual machine resources as a supplement. If the available resources in the resource pool 103 are sufficient, the resource application module 1023 can send a resource sufficiency feedback message to the integrated management subsystem 101. By requesting new virtual machine resources in this manner, the resource application module 1023 effectively avoids the waste of available virtual machine resources within the big data platform 100 and improves the resource utilization rate within the big data platform 100.

[0074] The management system of the external virtual machine resource pool responds to the call to the resource request API by returning metadata of the new virtual machine resource. This metadata includes, but is not limited to, Internet Protocol Address (IP address), CPU, memory, and other information. When the resource request module 1023 receives the metadata of the new virtual machine resource returned by the resource request API, it indicates that the request for the new virtual machine resource has been successful. The resource request module 1023 then sends the received metadata of the new virtual machine resource to the resource pool 103.

[0075] Resource Pool 103

[0076] Resource pool 103 is used to manage the metadata of virtual machine resources within the big data platform 100, including receiving metadata of new virtual machine resources sent by resource request module 1023. Resource pool 103 registers the received new virtual machine resource metadata and manages it together with the metadata of the virtual machine resources in the big data platform 100. The metadata of the virtual machine resources includes, but is not limited to, the owner information of the virtual machine (i.e., which user owns the vacant virtual machine or the new virtual machine resource), its usage or vacancy status information, the virtual machine's IP address, CPU, and memory capacity, etc. Specifically, in some embodiments, resource pool 103 implements its functions based on the following modules:

[0077] The resource import interface 1031 accepts calls from the resource request module 1023 in the alarm subsystem 102, and connects internally with the virtual machine resource management module 1032 to transfer new virtual machine resource metadata sent by the resource request module 1023 to the virtual machine resource management module 1032.

[0078] The virtual machine resource management module 1032 is the core management module of the resource pool 103, used to effectively manage all available virtual machine resources within the big data platform 100. All available virtual machine resources within the big data platform 100 include existing virtual machine resources and newly acquired virtual machine resources. The virtual machine resource management module 1032 systematically registers the metadata of all available virtual machine resources within the big data platform 100, including but not limited to registering the owner information, usage or vacancy status information, IP address, CPU, and memory capacity of each virtual machine. After completing the registration of the metadata of a new virtual machine resource, the virtual machine resource management module 1032 sends a resource receipt confirmation message to the alarm subsystem 102, indicating that the new virtual machine resource has been successfully received.

[0079] Resource call interface 1033 is used for automatic deployment tool 104 to call. After the automatic deployment tool 104 calls resource call interface 1033, it can obtain virtual machine resource metadata required by the deployment service component from virtual machine resource management module 1032.

[0080] When the metadata of a virtual machine resource within the big data platform 100 is acquired and used, the virtual machine resource management module 1032 changes the vacancy status information in the virtual machine resource metadata to usage status information for identification.

[0081] Similarly, when a virtual machine resource within the big data platform 100 is no longer in use, the virtual machine resource management module 1032 changes the usage status information in the virtual machine resource metadata to vacancy status information as an identifier.

[0082] Automated Deployment Tool 104

[0083] The automated deployment tool 104 is used to obtain service packages from the software repository 105, and according to the resource filter descriptions in the service packages and automation scripts, obtain virtual machine resources, install the software corresponding to the service components, and start them to complete the deployment of the service components. Specifically, the automated deployment tool 104 implements its functions based on the following modules:

[0084] Automatic retrieval script 1041 is used to retrieve service packages from software repository 105. Each service package should correspond one-to-one with a service component. The service package includes the corresponding service component's filter, cluster topology requirement description text, software binary files, and automation scripts. These automation scripts include automated installation scripts, automated startup scripts, automated uninstallation scripts, automated upgrade scripts, etc. (and environment check scripts, etc.), used to complete the installation, startup, uninstallation, and upgrade operations involved in the process.

[0085] The automatic cluster building module 1042 parses the service software packages obtained by the automatic fetch script 1041, calls the resource call interface 1033 of the resource pool 103 to obtain virtual machine resources that meet the filter requirements from the resource pool 103, and automatically creates a cluster topology on the obtained virtual machines. Specifically, when obtaining virtual machine resources from the resource pool 103, the virtual machines are filtered using the filter in the aforementioned service software package to obtain those that meet the requirements. The automatic cluster building module 1042 installs an agent tool (an agent is a remote tool corresponding to the automatic deployment tool (server)) on each obtained virtual machine. Then, the automatic deployment tool 104 can use the agent to copy the software binaries, automation scripts, etc., from the service software package to the corresponding virtual machine resources for automated software installation and startup.

[0086] The automatic cluster building module 1042 can create a cluster topology that meets the requirements described in the service software package on the acquired virtual machine resources. The cluster topology represents the connection relationships between nodes (i.e., servers) in the cluster created on the virtual machine. Servers within this cluster topology can run automated scripts to automate software installation and startup.

[0087] The automatic component clustering module 1042 can also calculate and evaluate the relationship between the resources required by each service component. When the resources required by each service component can be provided by the same virtual machine resources (hereinafter referred to as resource co-location), the automatic component clustering module 1042 creates a cluster topology of multiple service components on the same virtual machine. Resource co-location can maximize the utilization of hardware resources and avoid resource waste.

[0088] The automatic installation process control module 1043 manages and controls the installation and execution process of the software binary files in the service package, as well as the startup process after software installation. The automatic installation process control module 1043 runs an automation script through the aforementioned agent, and performs automated software installation and startup according to the software installation and startup rules in the script. The software installation and startup rules set in the automation script include the following two cases:

[0089] (1) When there are dependencies between service components, a tree-like dependency relationship can be generated between the service components based on the dependencies. The installation rules in the automated installation script are set based on this tree-like dependency relationship. The automated installation process control module 1043 then controls the automated installation script to install the software of each service component sequentially upwards from the root of the tree-like dependency relationship according to the installation rules. Each software corresponds to one service component.

[0090] Accordingly, the startup rules in the automated startup script should also be set based on the above-mentioned tree-like dependency relationship. According to the startup rules, the automatic installation process control module 1043 controls the automated startup script to sequentially start the software of each installed service component from the root of the above-mentioned tree-like dependency relationship.

[0091] (2) When there are no dependencies between the service components, the installation rules in the automated installation script are set to install the software of each service component in parallel. The automated installation process control module 1043 controls the automated installation script to install the software of each service component in parallel according to the installation rules.

[0092] Accordingly, the startup rules in the automated startup script are also set to start the software of each service component in parallel. The automatic installation process control module 1043 controls the automated startup script to start the software of each service component in parallel according to these startup rules.

[0093] Software Repository 105

[0094] Software repository 105 stores the aforementioned service packages, each corresponding one-to-one with a service component that needs to be deployed. After the service packages of the big data platform 100 are published, they are automatically uploaded to software repository 105 so that the automatic deployment tool 104 can retrieve the corresponding service packages for the relevant service components. Service packages are generally compressed file packages; the files they contain and their uses have already been described in the relevant description of the automatic deployment tool 104 and will not be repeated here.

[0095] The big data platform tenant management system 110, through the aforementioned comprehensive management subsystem 101, alarm subsystem 102, resource pool 103, automatic deployment tool 104, and software repository 105, realizes the automated management of tenant resource monitoring, early warning, and new tenant creation processes. This enables the big data platform 100 to provide users with a package of services such as resource monitoring and new tenant creation, thereby improving user satisfaction with the services provided by the big data platform 100.

[0096] Below Figure 3 Based on the combination Figure 4 This application describes in detail the specific process of a big data platform tenant management method.

[0097] Figure 4 A flowchart illustrating a tenant management method for a big data platform according to this application is shown. Figure 4 As shown, the method includes the following steps:

[0098] S1: The big data platform tenant management system 110 (hereinafter referred to as system 110) automatically detects the resource usage of existing tenants. Specifically, as described above, the comprehensive management subsystem 101 in system 110 automatically detects the resource usage of existing tenants on the big data platform 100.

[0099] For example, Figure 5 A schematic diagram of the operation and maintenance interface of the big data platform tenant management system 110 according to an embodiment of this application is shown. Figure 5 As shown, the existing tenant's system name in the operation and maintenance interface of system 110 is test( Figure 5 The tenant name shown is the system name (automatically generated upon creation), the custom name is "test" (custom names can be set manually), the tenant ID is "p5," and the service component cluster is "cluster1." As an example, the total resource capacity of tenant "test" can be represented by the number of APP access requests that the tenant can respond to; for example, tenant "test" can respond to 500 APP access requests. System 110 automatically detects the number of APP access requests that tenant "test" has responded to. For example, if it detects that the tenant has responded to 350 APP access requests and completed the access of the 350th APP, then the tenant's resource utilization has reached 70% of the total resource capacity. The automatic detection interval can be reasonably set in System 110, for example, every 5 minutes. The interval for each automatic detection can also be set to longer or shorter; there is no restriction here.

[0100] S2: System 110 determines whether the resource usage of the existing tenant meets the conditions for creating a new tenant. If the resource usage of the existing tenant meets the conditions for creating a new tenant, proceed to step S3; if the resource usage of the existing tenant does not meet the conditions for creating a new tenant, proceed to step S1.

[0101] Specifically, whether the resource usage of existing tenants meets the conditions for new tenants can be determined in two ways:

[0102] (1) By setting a capacity threshold, the resource usage of existing tenants is compared with the capacity threshold. If the resource usage of existing tenants is greater than or equal to the capacity threshold, the resource usage of existing tenants meets the conditions for new tenants; otherwise, it does not meet the conditions.

[0103] (2) By setting a utilization rate threshold, the resource utilization rate of existing tenants is compared with the utilization rate threshold. When the resource utilization rate of existing tenants is greater than or equal to the utilization rate threshold, the resource utilization of existing tenants meets the conditions for new tenants; otherwise, it does not meet the conditions. The resource utilization rate of existing tenants is the ratio between the resource usage of tenants and the total resource capacity of tenants.

[0104] As described above, the integrated management subsystem 101 in system 110 can set a capacity threshold to monitor the resource usage of tenants, or it can set a utilization threshold to set a threshold for resource usage, without any restrictions.

[0105] For example, such as Figure 5 As shown, the maximum number of APP access requests that tenant test can respond to is 500. A capacity threshold can be set for the ratio between resource usage and total resource capacity, such as setting the capacity threshold to 90%; or a capacity threshold can be set for resource usage, such as setting the capacity threshold to 450 APP access requests.

[0106] It is understandable that there are multiple ways to set capacity thresholds. For example, users can set capacity thresholds for their tenants through the user interface on the Big Data Platform 100, and the operations and maintenance personnel of the Big Data Platform 100 can also set capacity thresholds for each tenant under each username through the operations and maintenance interface with user authorization. There are no restrictions on this. It is understood that the user interface and the operations and maintenance interface are the interfaces displayed by System 110 to users and operations and maintenance personnel, respectively. The capacity threshold for each tenant on System 110 can be set through the above methods.

[0107] S3: If system 110 determines that the resource usage of an existing tenant exceeds the capacity threshold, it issues an alarm notification and simultaneously triggers the creation of a new tenant process, preparing virtual machine resources for the new tenant. When the new tenant process is triggered, the system name of the new tenant is automatically generated. In some embodiments, the new tenant's ID and other identification information can also be automatically generated.

[0108] Specifically, when the integrated management subsystem 101 in system 110 determines that the resource usage of an existing tenant exceeds the capacity threshold during the monitoring of existing tenant resource usage, it sends an alarm message to the alarm subsystem 102 in system 110. Simultaneously, the integrated management subsystem 101 internally triggers the creation of a new tenant process. Upon receiving the alarm message, the alarm subsystem 102 sends alarm notifications externally, including but not limited to sending alarm notifications to users through the user interface of system 110 and to maintenance personnel through the maintenance interface of system 110. The integrated management subsystem 101 automatically triggers the creation of a new tenant process, and the system name and ID information of the new tenant are automatically generated.

[0109] During the operation of system 110, resource pool 103 can periodically send the quantity and configuration information of idle virtual machine resources of relevant users within the pool to alarm subsystem 102. Alarm subsystem 102 can also query the quantity and configuration information of idle virtual machine resources of relevant users within resource pool 103 by calling the interface of resource pool 103. Based on the quantity and configuration information of idle virtual machine resources of relevant users within resource pool 103, alarm subsystem 102 automatically triggers an algorithm to calculate the quantity of new virtual machine resources that relevant users need to replenish and the configuration requirements of the virtual machines while issuing alarm notifications, and then calls the resource request API of the external virtual machine resource library to request new virtual machine resources.

[0110] Understandably, after a user completes the initial creation of a tenant on system 110, when creating a new tenant, the system will configure virtual machine resources and deploy service components for the new tenant according to the resource configuration requirements of the tenant already created or the first one created by the user. Therefore, the alarm subsystem 102 can calculate the number of new virtual machine resources based on the configuration of the tenant already created or the first one created by the user. Users can also set the default resource configuration and service component configuration of the new tenant through the user interface, etc., without any restrictions.

[0111] After receiving the new virtual machine resource metadata from the external virtual machine resource library, the alarm subsystem 102 calls the resource import interface 1031 of the resource pool 103 to send the new virtual machine resource metadata to the resource pool 103. After successfully receiving the new virtual machine resource metadata, the resource pool 103 sends resource receipt confirmation information to the alarm subsystem 102.

[0112] The metadata of virtual machine resources or new virtual machine resources refers to the above. Figure 3 The relevant descriptions will not be repeated here. Resource pool 103 is available for other structures in system 110 to call through resource import interface 1031 and resource call interface, which can improve calling efficiency and save network resources.

[0113] For example, such as Figure 5As shown, under normal circumstances, tenant 'test' can respond to 500 app access requests. When tenant 'test' responds to the 450th app access request, its resource usage is judged to have reached 90% of the total resource capacity (i.e., the capacity threshold). At this time, the integrated management subsystem 101 sends an alarm message to the alarm subsystem 102. Simultaneously, the integrated management subsystem 101 automatically triggers the new tenant creation process to prepare for the creation of a new tenant. The integrated management subsystem 101 automatically generates the system name 'test-new' and the ID of the new tenant, for example, the ID of the new tenant is 'p6', and records the system name and ID of the new tenant in the tenant management module 1011 of the integrated management subsystem 101. Since the service component cluster of the new tenant 'test-new' has not been created at this time, the identification information of the service component cluster of the new tenant 'test-new' is displayed as blank, such as... Figure 5 As shown.

[0114] The virtual machine resource configuration required by the new tenant test-new is based on the virtual machine resource configuration of tenant test. Alarm subsystem 102 can calculate the number of virtual machines required by the new tenant test-new, as well as the CPU, memory, and other configuration requirements on the virtual machines, based on the virtual machine resources configured by tenant test. For example, if the new tenant test-new needs 4 virtual machines, with each virtual machine having a basic configuration of 16 CPUs, 32GB of memory, and 500GB of hard disk space, alarm subsystem 102 might need to call the resource request API to request 2 more virtual machines that meet the above basic configuration requirements. The external virtual machine resource library will respond to the call on the resource request API by returning the IP address, hostname, CPU, memory, tags, and other metadata of the 2 virtual machines that meet the above basic configuration requirements, and send them to alarm subsystem 102. After receiving the metadata of these 2 virtual machines, alarm subsystem 102 calls the resource import interface 1031 of resource pool 103 to send the metadata of these 2 virtual machines to resource pool 103 for registration and storage.

[0115] S4: The alarm subsystem 102 in system 110 sends the metadata of the new virtual machine resources to the comprehensive management subsystem 101. Specifically, after receiving the resource confirmation information from the resource pool 103, the alarm subsystem 102 sends the metadata of the new virtual machine resources to the resource metadata management module 1013 of the comprehensive management subsystem 101 for storage, so that it can be called and queried by various service components when performing tasks.

[0116] It is understandable that when the integrated management subsystem 101 receives the metadata of the new virtual machine resources, it indicates that the virtual machine resources for deploying service components for the new tenant are in place, and proceeds to step S5 below. After receiving the resource receipt information from the resource pool 103, the alarm subsystem 102 sends the metadata of the new virtual machine resources to the integrated management subsystem 101 to confirm that the virtual machine resources are in place. This process ensures the accuracy of information flow within system 110, thereby ensuring that each system or module structure within system 110 responds to accurate information and precisely triggers the next automated process, enabling system 110 to operate stably and orderly according to the established workflow.

[0117] It is understandable that if the number and configuration information of the available virtual machine resources of the relevant user in the resource pool 103 can meet the needs of the new tenant created by the user, the alarm subsystem 102 will directly send the feedback information that the resources are sufficient to the comprehensive management subsystem 101, without having to call the resource application API to apply for new virtual machine resources.

[0118] S5: When the integrated management subsystem 101 within system 110 receives metadata of new virtual machine resources, it automatically calls the automatic deployment tool 104 to deploy the required service components for the new tenant.

[0119] Specifically, the automated deployment tool 104 obtains the service package corresponding to the service component deployed by the new tenant through the automated fetch script 1041. The automated deployment tool 104 parses the obtained service package through the automated cluster building module 1042 and calls the resource call interface 1033 of the resource pool 103 to obtain virtual machine resources that meet the configuration requirements from the resource pool 103. The automated cluster building module 1042 further creates a cluster topology on the virtual machine according to the cluster topology requirements described in the service package. The automated deployment tool 104 controls the automated installation script in the service package to run and install the software through the automated installation process control module 1043. After successful software installation, the automated installation process control module 1043 then starts the software by controlling the automated startup script. Once the software starts running, the service component deployment is complete. The specific process of the automated deployment tool 104 deploying the service component is described above. Figure 3 The relevant descriptions will not be repeated here.

[0120] It is understandable that if the number and configuration information of the vacant virtual machine resources of the relevant user in the resource pool 103 can meet the needs of the new tenant created by the user, the integrated management subsystem 101 can also call the automatic deployment tool 104 to deploy the required service components for the new tenant after receiving the resource sufficiency feedback information sent by the alarm subsystem 102.

[0121] Understandably, new tenants need multiple service components to respond to different user service requests. For example, some service components provide task scheduling services to respond to user scheduling requests, such as scheduling a task to calculate weather data for a certain location over the past six months. Other service components provide monitoring process services to respond to user monitoring process requests, such as a user requesting to display customer activity data from a shopping app to observe the current activity level of customers on the app.

[0122] It is understandable that, since the resources required to deploy each service component are different, in order to maximize resource utilization, multiple service components can be deployed on the same virtual machine by resource co-location, provided there are no resource conflicts between the service components. For example, three service components can be deployed on two virtual machines, or three service components can be deployed on four virtual machines, or five service components can be deployed on three virtual machines. There are no restrictions here.

[0123] After the service components are successfully deployed, the automatic deployment tool 104 sends a notification message about the successful deployment of the service components and the access addresses (URLs) of these service components to the integrated management subsystem 101. In some other embodiments, the automatic deployment tool 104 may also send other identification and invocation information of the deployed service components to the integrated management subsystem 101, which is not limited here.

[0124] For example, if the new tenant test-new requires three service components A, B, and C, the automatic deployment tool 104 will obtain the service packages corresponding to these three service components. The automatic deployment tool 104 can obtain the service packages one by one or all three at once; there is no restriction on this. Each service package contains a filter for the corresponding service component, used to filter virtual machine resources to obtain virtual machine resources that meet the requirements of the corresponding service component. For example, service component A may specify the use of 5 CPUs, service component B may specify the tag service=B, and service component C may specify the use of 6 CPUs and 32GB of memory. Furthermore, there may be dependencies between service components A and C, such as service component C depending on service component B, and service component B depending on service component A. The automated deployment tool 104 will first select virtual machines with 5 CPUs through the filter for service component A, create a cluster and topology for service component A, and install the software for service component A. After the software installation of service component A is complete, it will then select virtual machines with service=B through the filter for service component B, create a cluster and topology for service component B, and install the software for service component B. After the software installation of service component B is complete, it will then select virtual machines with 6 CPUs and 32GB of memory through the filter for service component C, create a cluster and topology for service component C, and install the software for service component C. After all the software for service components AC is installed, it will first start the software for service component A, and then start the software for service components B and C in sequence to complete the deployment of service component AC.

[0125] After the service component AC is successfully deployed, the automatic deployment tool 104 sends the URL address of the service component AC to the integrated management subsystem 101.

[0126] S6: After receiving the URL address of the deployed service component from the automatic deployment tool 104, the integrated management subsystem 101 of system 110 sends the identification information of the new tenant to the deployed service component.

[0127] The integrated management subsystem 101 accesses the service component based on its URL address, sends the new tenant's identification information, and activates the service component. The new tenant's identification information includes the new tenant's system name, ID information, and may also include some additional information, which is not limited here. After receiving the new tenant's identification information, the service component completes activation and prepares to respond to user service requests. The new tenant then responds to user service requests through the service component.

[0128] For example, the integrated management subsystem 101 sends the system name test-new and ID information p6 of the new tenant test-new to the service component AC that has been deployed.

[0129] S7: The integrated management subsystem 101 of system 110 automatically sends task information that needs to be executed by each service component to each service component. Sending task information to service components includes the following two scenarios:

[0130] (1) When there are dependencies between service components, the tenant management module 1011 in the integrated management subsystem 101 will construct a tree-like dependency relationship describing the dependency relationship, and send the dependency task information to the service component located at the root according to the tree-like dependency relationship, so that the service component can execute the dependency task first; after the dependency task is successfully executed, the task information dependent on the above dependency task will be sent serially to the service components above the root and executed. In this case, the serial task sending and execution method can ensure the accuracy of the service components in executing tasks.

[0131] (2) When there are no dependencies between the service components, the integrated management subsystem 101 will send task information to each service component in parallel, and each service component will execute the task in parallel after receiving the task information. In this case, the parallel task issuance and execution method can improve the efficiency of the service components in executing tasks.

[0132] For example, if the dependency relationship between service components A and C is that service component C depends on service component B, and service component B depends on service component A, then service component A is located at the root in the tree-like dependency relationship constructed in the integrated management subsystem 101. Therefore, the integrated management subsystem 101 first sends the task information of service component A to service component A for execution; after service component A successfully executes the task, the integrated management subsystem 101 then sends the task information of service component B to service component B for execution; after service component B successfully executes the task, the integrated management subsystem 101 then sends the task information of service component C to service component C for execution.

[0133] S8: The integrated management subsystem 101 of system 110 calls the public cloud 300 service interface to request the creation of a cloud service instance for the new tenant. The public cloud 300 responds to the call to the above cloud service interface and provides the access address of the created cloud service instance to the new tenant in the integrated management subsystem 101.

[0134] The cloud service instance created by Public Cloud 300, together with the service components after executing the corresponding task information in step S7, serves as the response component for the user's service request.

[0135] It's understandable that once a service component is activated, it indicates that the new tenant's service component is ready. Applying for a cloud service instance at this point effectively avoids the situation where a cloud service instance is applied for while the new tenant creation process has failed. Since applying for a cloud service instance incurs a fee, this avoids the corresponding financial loss. Failure at any stage of the new tenant creation process can lead to the application's failure; for example, the service component failing to start after installation can cause the final deployment failure. The fees incurred during the cloud service instance application process after service component activation can be deducted from the user's account through a pre-signed pre-authorization agreement, or through other payment methods, which will not be elaborated upon here.

[0136] For example, if a new tenant, test-new, needs to use the MapReduce service on public cloud 300, then a corresponding cloud service instance for that service needs to be created on public cloud 300. After creation, public cloud 300 reports the cloud service instance's ID or access address to the integrated management subsystem 101 in system 110, so that the new tenant, test-new, can call it when submitting MapReduce tasks.

[0137] S9: All resources of the new tenant are ready, and system 110 automatically enables the new tenant to respond to new APP access requests.

[0138] Once the new tenant's resources are ready, including but not limited to the aforementioned service components and cloud service instances, the existing tenant from step S1 ceases responding to new app access requests, and system 110 automatically enables the new tenant to respond to new app access requests. At this point, the existing tenant may have already connected to 499 apps; in this case, the tenant continues to receive data pushed from the 499 connected apps and stores it in the user-specified location. If the existing tenant may have already connected to 501 apps, the tenant continues to receive data pushed from the 501 connected apps and stores it in the user-specified location. New app access requests are automatically responded to by the new tenant.

[0139] It is understood that steps S1-S9 above do not constitute a specific limitation on the tenant management method of the big data platform of this application. In other embodiments, steps S1-S9 above may be described as fewer or more steps, which is not limited here. Figure 3 As shown, the above steps S1-S9 can Figure 3 Based on the big data platform tenant management system shown, Figure 3 The diagram illustrates parts of steps S1-S9 above to aid in understanding the technical solution of this application. Figure 3 Steps S1-S9 shown above are referenced. Figure 4 The detailed descriptions of each step are omitted here.

[0140] It is understood that the aforementioned big data platform tenant management system 110 can run on a server cluster composed of multiple computers. The integrated management subsystem 101, alarm subsystem 102, resource pool 103, automatic deployment tool 104, and software repository 105 in the system 110 can run on different server clusters, or any two or three of them can run on the same server cluster. No restrictions are imposed here.

[0141] Figure 6 A system block diagram of a computer according to an embodiment of this application is shown. In one embodiment, system 600 may include one or more processors 604, system control logic 608 connected to at least one of the processors 604, system memory 612 connected to system control logic 608, non-volatile memory (NVM) 616 connected to system control logic 608, and network interface 620 connected to system control logic 608.

[0142] In some embodiments, processor 604 may include one or more single-core or multi-core processors. In some embodiments, processor 604 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where system 600 employs an evolved Node B (eNB) or RAN (Radio Access Network) controller, processor 604 may be configured to perform various conforming embodiments, such as... Figure 2-5 The example shown.

[0143] In some embodiments, system control logic 608 may include any suitable interface controller to provide any suitable interface to at least one suitable device or component in processor 604 that communicates with system control logic 608.

[0144] In some embodiments, system control logic 608 may include one or more memory controllers to provide an interface to system memory 612. System memory 612 may be used to load and store data and / or instructions. In some embodiments, memory 612 of system 600 may include any suitable volatile memory, such as suitable dynamic random access memory (DRAM).

[0145] NVM / Memory 616 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM / Memory 616 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of HDD (Hard Disk Drive), CD (Compact Disc) drive, and DVD (Digital Versatile Disc) drive.

[0146] The NVM616 may include a portion of the storage resources on the device on which system 600 is installed, or it may be accessible by the device, but is not necessarily part of the device. For example, the NVM616 may be accessed over a network via network interface 620.

[0147] Specifically, system memory 612 and NVM 616 may each include a temporary copy and a permanent copy of instruction 624. Instruction 624 may include, when executed by at least one of processors 604, causing system 600 to perform, as Figure 3-4 The instructions for the method shown. In some embodiments, instructions 624, hardware, firmware and / or their software components may additionally / alternatively be located in system control logic 608, network interface 620 and / or processor 604.

[0148] Network interface 620 may include a transceiver for providing a radio interface to system 600, thereby enabling communication with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 620 may be integrated into other components of system 600. For example, network interface 620 may be integrated into at least one of processor 604, system memory 612, NVM 616, and firmware device (not shown) with instructions that, when at least one of processor 604 executes the instructions, enable system 600 to implement... Figure 2-5 The method shown.

[0149] The network interface 620 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 620 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0150] In one embodiment, at least one of the processors 604 may be packaged together with the logic of one or more controllers for system control logic 608 to form a system in package (SiP). In another embodiment, at least one of the processors 604 may be integrated on the same die with the logic of one or more controllers for system control logic 608 to form a system-on-chip (SoC).

[0151] System 600 may further include an input / output (I / O) device 632. The I / O device 632 may include a user interface enabling a user to interact with system 600; the peripheral component interface is designed to allow peripheral components to also interact with system 600. In some embodiments, system 600 may also include sensors for determining at least one of environmental conditions and location information related to system 600.

[0152] In some embodiments, the user interface may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.) and a keyboard.

[0153] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0154] In some embodiments, the sensor may include, but is not limited to, a gyroscope sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit may also be part of or interact with the network interface 620 to communicate with components of the positioning network (e.g., Global Positioning System (GPS) satellites).

[0155] In this specification, the reference to "an embodiment" or "an embodiment" means that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technology disclosed in this application. The phrase "in an embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0156] The technical solutions disclosed in this application also relate to a means for performing operations in text. This means may be specifically constructed for the claimed purpose or may include a general-purpose computer selectively activated or reconfigured by a computer program stored in a computer. Such a computer program may be stored on a computer-readable medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, application-specific integrated circuits (ASICs), or any type of medium suitable for storing electronic instructions, and each may be coupled to a computer system bus.

[0157] The systems, process steps, and virtual machines presented herein do not relate to any specific computer or other device. Various general-purpose systems can also be used with programs based on the teachings herein, or constructing more specialized systems to perform one or more method steps may prove convenient. Structures for various such systems are discussed in the following description. Furthermore, any specific programming language sufficient to implement the techniques and embodiments disclosed herein can be used. Various programming languages ​​can be used to implement this disclosure, as discussed herein.

[0158] Furthermore, the language used in this specification has been primarily chosen for readability and instructional purposes and may not have been chosen to depict or limit the subject matter disclosed. Therefore, this application disclosure is intended to illustrate, rather than limit, the scope of the concepts discussed herein.

Claims

1. A big data platform tenant management system, characterized in that, include: The integrated management subsystem is used to send alarm information to the alarm subsystem when it is detected that the user's tenant resource usage on the big data platform meets the conditions for creating a new tenant; An alarm subsystem is configured to obtain the first resource required by the user to create a new tenant after receiving the alarm information; wherein the alarm subsystem obtains the first resource in the following manner: If the user's available resources on the big data platform are less than the first resource, the alarm subsystem sends a resource acquisition request to the outside of the big data platform. The resource acquisition request is used to request the acquisition of a second resource, which is determined based on the difference between the first resource and the available resources. When the user's vacant resources on the big data platform are greater than or equal to the first resource, the alarm subsystem acquires the vacant resources as the first resource. An automated deployment tool is used to deploy service components for the new tenant created by the user on the acquired first resource.

2. The system according to claim 1, characterized in that, If the tenant's resource usage is greater than or equal to the capacity threshold, the user's tenant resource usage meets the conditions for creating a new tenant; or If the resource utilization rate of a tenant is greater than or equal to the utilization rate threshold, the user's tenant resource utilization meets the conditions for creating a new tenant, wherein the resource utilization rate of the tenant is the ratio between the tenant's resource usage and the tenant's total resource capacity.

3. The system according to claim 1, characterized in that, The automated deployment tool deploys service components for the new tenant created by the user on the acquired first resource in the following manner: Filter the third resource required by the service component from the first resource; Create a server cluster and server cluster topology for the service component based on the acquired third resource; Install the software required for the service components on the created cluster; Start the software and complete the initial parameter configuration.

4. The system according to claim 3, characterized in that, The automatic deployment tool further includes the following method for deploying service components for the new tenant created by the user on the first resource obtained: When dependencies exist between the service components, a tree-like dependency relationship is constructed between the service components based on the dependencies, and... The automated deployment tool installs the software required for each service component sequentially, starting from the root of the tree dependency relationship and moving upwards. After the software required by each of the service components is installed, the software of each service component is started sequentially from the root of the tree dependency relationship upwards. In the absence of dependencies between the service components, the automatic deployment tool installs the software required by each service component in parallel, and after the software required by each service component is installed, starts the software of each service component in parallel.

5. The system according to claim 4, characterized in that, If the system detects that the user's tenant resource usage on the big data platform meets the conditions for creating a new tenant, the integrated management subsystem further generates identification information for the new tenant, wherein... The identification information includes at least the new tenant's system name or the new tenant's identification number in the big data platform; and... Once the service component is deployed, the integrated management subsystem sends the new tenant's identification information to the service component to activate it; and, Once the service component is activated, the integrated management subsystem sends an instance request to the public cloud. The instance request is used to request the creation of a cloud service instance for the new tenant in the public cloud.

6. The system according to any one of claims 1 to 5, characterized in that, The resources required by the tenant include at least one of virtual machine resources, physical machine resources, and container resources. The resources required by the tenant include the first resource.

7. A tenant management method for a big data platform, characterized in that, include: If the user's tenant resource usage on the big data platform meets the conditions for creating a new tenant, an alarm message is sent; wherein, The alarm information can trigger the acquisition of the first resource required by the user to create a new tenant; wherein, the first resource required by the user to create a new tenant is acquired in the following manner: If a user's available resources on the big data platform are less than the first resource, a resource acquisition request is sent to an external entity of the big data platform. The resource acquisition request is used to request a second resource, which is determined based on the difference between the first resource and the available resources. If the user's vacant resources on the big data platform are greater than or equal to the first resource, the vacant resources are acquired as the first resource. Deploy service components for the new tenant created for the user on the first resource obtained.

8. The method according to claim 7, characterized in that, If the tenant's resource usage is greater than or equal to a preset capacity threshold, the user's tenant resource usage meets the conditions for creating a new tenant; or If the resource utilization rate of a tenant is greater than or equal to the utilization rate threshold, the user's tenant resource utilization meets the conditions for creating a new tenant, wherein the resource utilization rate of the tenant is the ratio between the tenant's resource usage and the tenant's total resource capacity.

9. The method according to claim 7, characterized in that, The deployment service component for the new tenant created for the user on the acquired first resource includes: Filter the third resource required by the service component from the first resource; Create a server cluster and server cluster topology for the service component based on the acquired third resource; Install the software required for the service components on the created cluster; Start the software and complete the initial parameter configuration.

10. The method according to claim 9, characterized in that, The new tenant deployment service component created for the user on the acquired first resource also includes: When dependencies exist between the service components, a tree-like dependency relationship is constructed between the service components based on the dependencies, and... According to the tree-like dependency relationship, the software required for each of the service components is installed sequentially upwards from the root of the tree-like dependency relationship. After the software required by each of the service components is installed, the software of each service component is started sequentially from the root of the tree dependency relationship upwards. If there are no dependencies between the service components, the software required by each service component is installed in parallel, and after the software required by each service component is installed, the software of each service component is started in parallel.

11. The method according to claim 10, characterized in that, The method further includes: The alarm information can trigger the big data platform to generate identification information for the new tenant. This identification information includes at least the new tenant's system name or identification number within the big data platform. Once the service component is deployed, the identification information of the new tenant is sent to the service component to activate it; and, Once the service component is activated, the big data platform sends an instance request to the public cloud. The instance request is used to request the creation of a cloud service instance for the new tenant in the public cloud.

12. The method according to any one of claims 7 to 11, characterized in that, The resources required by the tenant include at least one of virtual machine resources, physical machine resources, and container resources. The resources required by the tenant include the first resource.

13. A method for managing tenants on a big data platform, characterized in that, include: An alarm message is received when the user's tenant resource usage on the big data platform meets the conditions for creating a new tenant; The first resource required by the user to create a new tenant is obtained based on the acquired alarm information; wherein, the first resource required by the user to create a new tenant is obtained in the following manner: If a user's available resources on the big data platform are less than the first resource, a resource acquisition request is sent to an external entity of the big data platform. The resource acquisition request is used to request a second resource, which is determined based on the difference between the first resource and the available resources. If the user's vacant resources on the big data platform are greater than or equal to the first resource, the vacant resources are acquired as the first resource. The tenant's service components can be deployed on the first resource.

14. The method according to claim 13, characterized in that, If the tenant's resource usage is greater than or equal to a preset capacity threshold, the user's tenant resource usage meets the conditions for creating a new tenant; or If the resource utilization rate of a tenant is greater than or equal to the utilization rate threshold, the user's tenant resource utilization meets the conditions for creating a new tenant, wherein the resource utilization rate of the tenant is the ratio between the tenant's resource usage and the tenant's total resource capacity.

15. The method according to claim 13, characterized in that, The resources required by the tenant include at least one of virtual machine resources, physical machine resources, and container resources. The resources required by the tenant include the first resource.

16. A method for managing tenants on a big data platform, characterized in that, include: Obtain the invocation instruction; In response to the invocation instruction, a service component is deployed on the first resource for the new tenant created by the user of the big data platform; wherein the first resource required by the user to create the new tenant is obtained in the following manner: If a user's available resources on the big data platform are less than the first resource, a resource acquisition request is sent to an external entity of the big data platform. The resource acquisition request is used to request a second resource, which is determined based on the difference between the first resource and the available resources. If a user's vacant resources on the big data platform are greater than or equal to the first resource, the vacant resources are acquired as the first resource.

17. The method according to claim 16, characterized in that, The new tenant deployment service component created for the users of the big data platform on the first resource includes: Filter the third resource required by the service component from the first resource; Create a server cluster and server cluster topology for the service component based on the acquired third resource; Install the software required for the service components on the created cluster; Start the software and complete the initial parameter configuration.

18. The method according to claim 17, characterized in that, The new tenant deployment service component created for the user on the acquired first resource also includes: When dependencies exist between the service components, a tree-like dependency relationship is constructed between the service components based on the dependencies, and... According to the tree-like dependency relationship, the software required for each of the service components is installed sequentially upwards from the root of the tree-like dependency relationship. After the software required by each of the service components is installed, the software of each service component is started sequentially from the root of the tree dependency relationship upwards. If there are no dependencies between the service components, the software required by each service component is installed in parallel, and after the software required by each service component is installed, the software of each service component is started in parallel.

19. A computer-readable storage medium, characterized in that, The storage medium stores instructions that, when executed on a computer, cause the computer to perform the big data platform tenant management method according to any one of claims 7 to 11, 13 to 15, and 16 to 18.

20. An electronic device, characterized in that, include: One or more processors; One or more memories; the one or more memories store one or more programs that, when executed by the one or more processors, cause the electronic device to perform the big data platform tenant management method according to any one of claims 13 to 15, 16 to 18.

Images