Big data component deployment method and device based on bare metal server cluster

Abstract

The invention provides a big data component deployment method and device based on a bare metal server cluster, and relates to the technical field of big data. The method comprises the steps of obtaining a big data component processing request, first component information of a first big data component installed by a first bare metal server and second component information of a target big data component; and according to the first component information and the second component information, performing target processing on the target big data component to obtain a deployment result. Wherein the big data component processing request is used for performing target processing on the first bare metal server, the target processing comprises newly adding a target big data component or unloading the target big data component, and the component information comprises an identifier of the component and hardware resources occupied by the component. According to the method, the resource utilization rate in the big data component deployment process is improved.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a method and apparatus for deploying big data components based on bare metal server clusters. Background Technology

[0002] Bare metal servers refer to physical servers without pre-installed hypervisors, allowing users to directly deploy operating systems and applications. With the continuous development of big data technology, the deployment of big data components based on bare metal servers is receiving increasing attention.

[0003] Currently, the common method for deploying big data is based on virtual machines. This involves selecting a suitable virtualization platform, creating multiple virtual machines on that platform, and then deploying big data components within those virtual machines.

[0004] However, existing technologies suffer from low resource utilization during the deployment of big data components. Summary of the Invention

[0005] This application provides a method and apparatus for deploying big data components based on bare metal server clusters, in order to solve the technical problem of low resource utilization in the deployment of big data components in the prior art.

[0006] Firstly, this application provides a method for deploying big data components based on a bare metal server cluster, including:

[0007] A big data component processing request is obtained. The big data component processing request is used to perform target processing on the first bare metal server. The target processing includes adding a target big data component or uninstalling the target big data component.

[0008] Obtain first component information of the first big data component installed on the first bare metal server, and second component information of the target big data component; wherein, the component information includes the component's identifier and the hardware resources occupied by the component;

[0009] Based on the first component information and the second component information, the target big data component is subjected to the target processing to obtain the deployment result.

[0010] Optionally, the step of performing the target processing on the target big data component based on the first component information and the second component information to obtain the deployment result includes:

[0011] Based on the second component information, the target big data component is subjected to the target processing to obtain the third component information of the second big data component currently installed on the first bare metal server;

[0012] The deployment result is determined based on the first component information, the second component information, and the third component information.

[0013] Optionally, determining the deployment result based on the first component information, the second component information, and the third component information includes:

[0014] If the target processing involves adding the target big data component, the deployment result is determined by judging whether the third component information includes the first component information and the second component information.

[0015] If the target processing is to uninstall the target big data component, the deployment result is determined by judging whether the first component information includes the second component information and the third component information.

[0016] Optionally, the method further includes:

[0017] After the deployment result indicates successful deployment, a service function check is performed on the target big data component to determine the check result;

[0018] Specifically, when the target processing involves adding the target big data component, the service function check is to check whether the service function of the target big data component is in a runnable state.

[0019] When the target processing involves uninstalling the target big data component, the service function check is to check whether the service function of the target big data component is in an inoperable state.

[0020] Optionally, the method further includes:

[0021] Obtain a bare metal server deletion request, which is used to delete the second bare metal server from the bare metal server cluster;

[0022] Obtain server information for all bare metal servers in the bare metal server cluster, including the identifier of the bare metal server and the hardware resources occupied by the bare metal server;

[0023] If the second bare metal server is the master node, then based on the server information of all bare metal servers, the big data component service of the bare metal server cluster is stopped.

[0024] If the second bare metal server is a worker node, then based on the server information of the second bare metal server, stop the big data component service of the second bare metal server.

[0025] Optionally, the method further includes:

[0026] Obtain a bare metal server add request, which is used to add a third bare metal server to the bare metal server cluster;

[0027] Obtain the hardware resource information of the third bare metal server;

[0028] Based on the hardware resource information of the third bare metal server, configure the corresponding hardware resources for the third bare metal server and add it to the bare metal server cluster.

[0029] Optionally, after the step of stopping the big data component service of the second bare metal server based on the server information of the second bare metal server if the second bare metal server is a worker node, the method further includes:

[0030] Restart the remaining bare metal servers in the bare metal server cluster.

[0031] Secondly, this application provides a big data component deployment device based on a bare metal server cluster, comprising:

[0032] The acquisition module is used to acquire big data component processing requests, which are used to perform target processing on the first bare metal server. The target processing includes adding a target big data component or uninstalling the target big data component.

[0033] The acquisition module is further configured to acquire first component information of the first big data component installed on the first bare metal server, and second component information of the target big data component; wherein, the component information includes the identifier of the component and the hardware resources occupied by the component;

[0034] The processing module is used to perform the target processing on the target big data component based on the first component information and the second component information, and obtain the deployment result.

[0035] Optionally, the device further includes: a determining module;

[0036] The processing module is further configured to perform the target processing on the target big data component based on the second component information;

[0037] The acquisition module is used to acquire the third component information of the second big data component currently installed on the first bare metal server;

[0038] The determining module is used to determine the deployment result based on the first component information, the second component information, and the third component information.

[0039] Optionally, the apparatus further includes: a determination module, configured to determine the deployment result by determining whether the third component information includes the first component information and the second component information if the target processing is to add the target big data component; and to determine the deployment result by determining whether the first component information includes the second component information and the third component information if the target processing is to uninstall the target big data component.

[0040] Optionally, the apparatus further includes: a checking module, configured to perform a service function check on the target big data component after the deployment result indicates successful deployment, and determine the check result; wherein, when the target processing is adding the target big data component, the service function check is to check whether the service function of the target big data component is in an operable state; when the target processing is uninstalling the target big data component, the service function check is to check whether the service function of the target big data component is in an inoperable state.

[0041] Optionally, the acquisition module is further configured to acquire a bare metal server deletion request, the bare metal server deletion request being used to delete the second bare metal server from the bare metal server cluster; acquire server information of all bare metal servers in the bare metal server cluster, the server information including the identifier of the bare metal server and the hardware resources occupied by the bare metal server;

[0042] The processing module is further configured to, if the second bare metal server is a master node, stop the big data component service of the bare metal server cluster based on the server information of all bare metal servers; and if the second bare metal server is a worker node, stop the big data component service of the second bare metal server based on the server information of the second bare metal server.

[0043] Optionally, the acquisition module is further configured to acquire a bare metal server addition request, the bare metal server addition request being used to add a third bare metal server to the bare metal server cluster; and acquire the hardware resource information of the third bare metal server;

[0044] The processing module is further configured to configure corresponding hardware resources for the third bare metal server based on the hardware resource information of the third bare metal server, and add it to the bare metal server cluster.

[0045] Optionally, the device further includes a restart module for restarting the remaining bare metal servers in the bare metal server cluster.

[0046] Thirdly, this application provides a big data component deployment device based on a bare metal server cluster, comprising:

[0047] Processor and memory;

[0048] The memory stores computer-executed instructions;

[0049] The processor executes computer execution instructions stored in the memory to implement the big data component deployment method based on bare metal server clusters as described in the first aspect and various possible implementations of the first aspect above.

[0050] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the big data component deployment method based on a bare-metal server cluster as described in the first aspect and various possible implementations of the first aspect.

[0051] Fifthly, this application provides a computer program product comprising a computer program that, when executed by a processor, is used to implement the big data component deployment method based on a bare metal server cluster as described in the first aspect and various possible implementations thereof.

[0052] This application provides a method and apparatus for deploying big data components based on a bare metal server cluster. It obtains a big data component processing request for target processing on a first bare metal server, and acquires first component information of a first big data component installed on the first bare metal server, as well as second component information of the target big data component. Based on the first and second component information, it performs target processing on the target big data, either by installing the target big data component onto the first bare metal server or by uninstalling the target big data component from the first bare metal server. Users can customize and select the required target big data component and dynamically configure big data components according to their actual needs, thereby adding or uninstalling the installed big data component on the first bare metal server, achieving customized deployment. Furthermore, by automatically deploying the target big data component on the first bare metal server, it not only reduces the deployment difficulty of the target big data component but also fully utilizes the hardware resources of the first bare metal server, improving resource utilization. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0054] Figure 1 This is an application scenario diagram illustrating the deployment method of big data components based on bare metal server clusters applicable to this application;

[0055] Figure 2The deployment method and process of big data components based on bare metal server clusters provided in this application Figure 1 ;

[0056] Figure 3 The deployment method and process of big data components based on bare metal server clusters provided in this application Figure 2 ;

[0057] Figure 4 The deployment method and process of big data components based on bare metal server clusters provided in this application Figure 3 ;

[0058] Figure 5 A schematic diagram of the structure of the big data component deployment device based on bare metal server cluster provided in this application;

[0059] Figure 6 The hardware structure diagram of the big data component deployment device based on bare metal server cluster provided in this application.

[0060] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0061] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0062] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0063] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0064] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0065] Modern enterprises and organizations generate massive amounts of data every day, which needs to be stored, processed, and analyzed efficiently. Big data deployment can help enterprises and organizations efficiently process and analyze massive amounts of data, extract valuable information, support business decisions and optimize operations, and improve user experience and competitiveness.

[0066] Currently, a common approach to big data deployment is virtual machine-based deployment. This involves creating multiple virtual machines on a virtualization platform and then deploying big data components within them. The virtualization platform is the software layer that manages and runs these virtual machines, allowing the creation and management of multiple virtual machines on a single physical server. A virtual machine is a software-simulated computer system that runs on physical hardware but is isolated from it; each virtual machine has its own operating system and applications, functioning much like an independent physical computer.

[0067] Because virtualization platforms require computing resources to manage virtual machines, this resource overhead affects overall resource utilization. Furthermore, virtual machines running on virtualization platforms experience performance losses when accessing hardware resources, making it difficult to meet the demands of high-performance computing applications like big data. In practice, some virtual machines may have high resource utilization while others have limited resources, resulting in the physical server's performance not being fully utilized and overall resource utilization being low.

[0068] To address the aforementioned technical issues, the inventors, while researching the deployment process of big data components, discovered that virtual machine-based deployment introduces resource overhead, fails to fully utilize the performance of physical servers, and is complex and time-consuming. Therefore, considering that bare metal servers directly provide hardware resources to users, big data components can be deployed directly on bare metal servers. This eliminates the resource overhead of virtual machines while allowing direct access to hardware resources, improving resource utilization and performance.

[0069] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0070] Figure 1 This diagram illustrates an application scenario for the big data component deployment method based on bare metal server clusters applicable to this application. (Example:) Figure 1 As shown, this method is implemented based on a bare metal server cluster, a service control terminal, and a software repository.

[0071] The service control terminal, which is the control terminal for deploying services, is used to control the configuration file generation and deployment process of the entire bare metal server cluster, including host files, installation files and uninstallation files. Operation logs are also generated and processed by the service control terminal.

[0072] The software repository is mainly used to provide storage for all software packages and component installation packages required when deploying bare metal servers. These packages are placed in the software repository directory. The service control terminal will instruct the bare metal server to obtain the required installation packages from the software repository according to the actual installation needs and download them to the server for subsequent operations.

[0073] A bare metal server cluster consists of bare metal servers with actual big data components installed. Each bare metal server's node role and the required big data components are planned by the service control terminal according to deployment needs, and resources are allocated based on the actual situation of the big data components on that server. Depending on actual needs, bare metal servers can be planned as different nodes in a big data platform, such as master nodes and worker nodes; depending on the installed components, they may also become Hadoop nodes, ZooKeeper nodes, HBase nodes, etc.

[0074] Specifically, a Hadoop node refers to a bare-metal server with the Hadoop component installed. Hadoop is a distributed computing framework. A ZooKeeper node refers to a bare-metal server with the ZooKeeper component installed. ZooKeeper is a distributed coordination service typically used to manage the configuration and synchronization of distributed applications. An HBase node refers to a bare-metal server with the HBase component installed. HBase is a distributed NoSQL database that runs on top of Hadoop.

[0075] Figure 2 The deployment method and process of big data components based on bare metal server clusters provided in this application Figure 1 .like Figure 2As shown in the embodiment of this application, a method for deploying big data components based on a bare metal server cluster includes:

[0076] S201. Obtain a big data component processing request for target processing of the first bare metal server;

[0077] The execution subject of this application embodiment is a big data component deployment device based on a bare metal server cluster. This big data component deployment device based on a bare metal server cluster can be a terminal device or a server, such as... Figure 1 The service control terminal in the system. In practical applications, whether the deployment device for big data components based on bare metal server clusters is a terminal device or a server can be determined according to the actual situation, and there are no specific restrictions on this.

[0078] The target processing includes adding and uninstalling target big data components. The first bare metal server represents the bare metal server that needs to be added or uninstalled. Specifically, based on actual needs, the user selects the target big data component to be added or uninstalled and initiates the corresponding big data component processing request through a terminal device. Correspondingly, the big data component deployment device based on the bare metal server cluster receives the big data component processing request.

[0079] For example, if you need to process large-scale data, you can add a Hadoop component; if you need a distributed coordination service, you can add a ZooKeeper component; if you need a NoSQL database, you can add an HBase component.

[0080] Understandably, once a user identifies the target big data components to be processed, they need to specify and allocate resources for these components, specifically including hardware resource allocation and software configuration. Hardware resource allocation may include central processing unit (CPU) resources, memory resources, storage resources, network resources, etc., while software configuration may include configuring the big data components and optimizing their configuration parameters.

[0081] It's worth noting that each big data component typically has a set of default configuration parameters. These parameters define how system resources are allocated and used. If a user doesn't perform any custom configuration when deploying a big data component, it will use its default resource allocation and configuration parameters. These default configurations are preset by the big data component developers to provide a general setup so users can quickly start and run the component. While the default configurations offer a quick start-up method, in actual production environments, it's usually necessary to customize the configuration based on specific business needs and system resources to optimize performance and resource utilization.

[0082] S202. Obtain the first component information of the first big data component installed on the first bare metal server, and the second component information of the target big data component;

[0083] The component information includes the component's identifier and the hardware resources it occupies. Specifically, the first component information includes the identifier of the first component and the hardware resources it occupies, and the second component information of the target big data component includes the identifier of the second component and the hardware resources it occupies.

[0084] It should be understood that the second component information of the target big data component is the component information generated after the target big data component is specified and resources are allocated in S201, or it can be the component information determined according to the default configuration parameters of the target big data component.

[0085] Optionally, if the big data component processing request carries second component information of the target big data component, then the second component information of the target big data component carried in the big data component processing request is obtained; if the big data component processing request does not carry second component information of the target big data component, then the second component information of the target big data component is determined according to the default configuration parameters of the target big data component.

[0086] Optionally, the component information of the big data component may also include any of the following information to ensure the efficient operation of the system and meet business needs:

[0087] (1) Basic information, including component version number and supplier;

[0088] (2) Function and sub-component information, including main functions and included sub-components;

[0089] (3) System requirements, including operating system, dependent software, and hardware requirements;

[0090] (4) Deployment information, including deployment topology and installation path;

[0091] (5) Performance and optimization, including performance metrics and optimization suggestions.

[0092] S203. Based on the information of the first component and the information of the second component, perform target processing on the target big data component and obtain the deployment result.

[0093] In one possible implementation, S203 can be achieved through the following steps:

[0094] S2031. Perform target processing on the target big data component based on the second component information to obtain the third component information of the second big data component currently installed on the first bare metal server;

[0095] When the target processing involves adding a new target big data component, the target big data component is installed on the first bare metal server, and hardware resources are allocated to the target big data component according to the second component information; when the target processing involves unloading the target big data component, the target big data component is unloaded from the first bare metal server, and the hardware resources occupied by the target big data component are released according to the second component information.

[0096] After completing the target processing on the target big data component, obtain the third component information of the second big data component currently installed on the first bare metal server.

[0097] Specifically, when the target processing involves adding a new target big data component, if the target processing is successful, the second big data component should be the first big data component and the target big data component; when the target processing involves uninstalling a target big data component, the second big data component should be the remaining big data component in the first big data component excluding the target big data component.

[0098] S2032. Determine the deployment result based on the information of the first component, the second component, and the third component.

[0099] If the target processing is to add a new target big data component, the deployment result is determined by judging whether the information of the third component includes the information of the first and second components; if the target processing is to uninstall a target big data component, the deployment result is determined by judging whether the information of the first component includes the information of the second and third components.

[0100] In other words, when the target processing is to add a new target big data component, it is determined whether the second big data component is the first big data component and the target big data component; when the target processing is to uninstall the target big data component, it is determined whether the second big data component is the remaining big data component in the first big data component excluding the target big data component.

[0101] The deployment result refers to the processing result of performing the target operation on the target big data component, i.e., the deployment result, including: successful uninstallation, failed uninstallation, successful addition, or failed addition.

[0102] Optionally, when the deployment result is uninstallation failure or addition failure, the deployment result may also include the reason for the failure.

[0103] Optionally, in some embodiments, after the deployment result indicates successful deployment, a service function check is performed on the target big data component to determine the check result.

[0104] Specifically, when the target processing involves adding a new target big data component, the service function check checks whether the service functions of the target big data component are in a runnable state; when the target processing involves uninstalling a target big data component, the service function check checks whether the service functions of the target big data component are in a non-runnable state.

[0105] It should be understood that a deployment result indicating successful deployment refers to either successful uninstallation or successful addition. Successful deployment means that the relevant processing and analysis software and tools for the target big data component have been installed and configured on the first bare metal server to efficiently process, store, and analyze large amounts of data. Successful uninstallation means that the relevant processing and analysis software and tools for the target big data component have been removed from the first bare metal server, typically including deleting software packages, cleaning configuration files, and releasing occupied hardware resources.

[0106] This application provides a method for deploying big data components based on a bare metal server cluster. The method involves obtaining a big data component processing request and acquiring first component information of a first big data component installed on a first bare metal server, as well as second component information of a target big data component. Based on the first and second component information, the method performs target processing on the first bare metal server. Target processing includes adding or uninstalling a target big data component; component information includes the component's identifier and the hardware resources it occupies. Based on the second component information, the method performs target processing on the target big data component, obtaining third component information of the second big data component currently installed on the first bare metal server. The deployment result is determined based on the first, second, and third component information. Users can customize and select the required target big data component and dynamically configure big data components according to their actual needs. This allows for adding or uninstalling the target big data component on the first bare metal server, enabling customized deployment and dynamic allocation and adjustment of the hardware resources required by the big data component. This avoids resource overload or waste, ensuring that the target big data component fully utilizes the hardware resources of the first bare metal server during deployment, thus improving resource utilization.

[0107] Next, we will use a specific example to illustrate... Figure 2 The deployment method for big data components based on bare metal server clusters will be further explained.

[0108] Figure 3 The deployment method and process of big data components based on bare metal server clusters provided in this application Figure 2 ,like Figure 3 As shown, the deployment process of the target big data component specifically includes the following steps:

[0109] Step a1: The user initiates a big data component processing request.

[0110] Step a2: Determine whether the big data component is processing a request to add or uninstall the big data component.

[0111] If the big data component processes a request to add a new big data component, then

[0112] Step a201: The service control terminal automatically generates a host file based on the server information of the first bare metal server to be deployed.

[0113] In addition, the host file also includes the node roles of each bare metal server in the bare metal server cluster, as well as information on the hardware resources allocated to the existing big data components on each bare metal server.

[0114] Step a202: The service control terminal generates the installation files corresponding to the target big data components that need to be added according to user requirements.

[0115] The installation file includes component information of existing big data components on each bare metal server in the bare metal server cluster, as well as second component information of the target big data component.

[0116] Step a203: The service control terminal passes the host file and installation file to the pre-written installation script, and executes the installation script to deploy the new target big data on the first bare metal server.

[0117] Specifically, ensure that the host file, installation file, and installation script are in the same directory. The installation script will read the server information from the host file and the installation requirements from the installation file, and then execute the installation command on the first bare metal server.

[0118] Step a204: After the installation script is executed on the service control terminal, the first bare metal server will perform a service function check on the target big data component to ensure that the service function of the target big data component is in a runnable state.

[0119] Understandably, service functionality checks can also be performed on the big data components across the entire bare metal server cluster simultaneously to ensure the cluster functions correctly after the target big data component is installed on the first bare metal server. After completing the functionality checks on all bare metal servers and big data components, the results are fed back to the service control panel.

[0120] Step a205: Feedback the inspection results to the service control terminal. After receiving the feedback, the service control terminal generates the corresponding new log.

[0121] The newly added logs include information such as the execution node, the executed task, and the task result.

[0122] If you need to uninstall the target big data component, perform the following steps:

[0123] Step a211: Obtain the host file generated by the service control terminal.

[0124] Step a212: The service control terminal generates the uninstallation file corresponding to the target big data component that needs to be uninstalled according to the user's requirements.

[0125] The uninstallation file includes the second component information of the target big data component, as well as the remaining component information of the big data components that already exist on each bare metal server in the bare metal server cluster, excluding the second component information.

[0126] Step a213: The service control terminal passes the host file and uninstallation file to the pre-written uninstallation script, and executes the uninstallation script to uninstall the deployment of the target big data component on the first bare metal server.

[0127] Step a214: After the uninstallation script is executed on the service control terminal, the first bare metal server will perform a service function check on the target big data component to ensure that the service function of the target big data component is in an inoperable state.

[0128] Step a215: Feedback the inspection results to the service control terminal. After receiving the feedback, the service control terminal generates the corresponding uninstallation log.

[0129] Furthermore, while deploying big data components, in order to effectively manage and optimize bare metal server cluster resources, it is also necessary to consider whether to add or remove bare metal servers in the cluster.

[0130] For the bare metal server deletion operation, the first step is to obtain the bare metal server deletion request and the server information of all bare metal servers in the bare metal server cluster. The bare metal server deletion request is used to remove the second bare metal server from the cluster, and the server information includes the bare metal server's identifier and the hardware resources it occupies. Further, if the second bare metal server is a master node, the big data component service of the bare metal server cluster is stopped based on the server information of all bare metal servers; if the second bare metal server is a worker node, the big data component service of the second bare metal server is stopped based on its own server information.

[0131] Understandably, the master node is typically responsible for the management and coordination of the bare metal server cluster, while worker nodes are responsible for the actual data processing and storage. When the master node needs to be deleted, the coordination and management functions of the entire bare metal server cluster will be affected. Therefore, deleting the master node usually means stopping the big data component service of the entire bare metal server cluster to ensure data consistency and cluster stability. When a worker node stops service, only the data processing and storage tasks on that worker node are affected; other worker nodes and the master node can continue to operate normally. Therefore, deleting a worker node only requires stopping the big data component service on that worker node.

[0132] Optionally, after stopping the big data component service of the second bare metal server based on its server information, the remaining bare metal servers in the bare metal server cluster can be restarted. It's worth noting that bare metal server clusters typically have load balancing mechanisms to ensure efficient utilization of hardware resources and cluster stability. When a bare metal server is deleted, the load balancer needs to reallocate workloads and hardware resources, which may require restarting the remaining bare metal servers for the new configuration to take effect.

[0133] For adding a new cluster, the process first involves obtaining the bare metal server addition request and the hardware resource information of the third bare metal server. Then, based on the third bare metal server's hardware resource information, the appropriate hardware resources are configured for the third bare metal server, and it is added to the bare metal server cluster. This ensures that the newly added third bare metal server can seamlessly integrate with the existing bare metal server cluster and effectively handle the assigned tasks. The bare metal server addition request is used to add the third bare metal server to the bare metal server cluster.

[0134] Next, a specific example will be used to explain the process of adding or deleting bare metal servers in a cluster.

[0135] Figure 4The deployment method and process of big data components based on bare metal server clusters provided in this application Figure 3 ,like Figure 4 As shown, the management process of a bare metal server specifically includes the following steps:

[0136] Step b1: The user initiates a bare metal server processing request.

[0137] Step b2: Determine whether the bare metal server is processing a request to add or remove a bare metal server.

[0138] If you need to add a new bare metal server, then you need to determine which third bare metal server needs to be added to the bare metal server cluster.

[0139] Step b201: The service control terminal automatically generates a host file based on the server information of the newly added third bare metal server and the information of the big data components to be deployed on the third bare metal server.

[0140] In addition, the host file also includes the node roles of each bare metal server in the bare metal server cluster, as well as information on the hardware resources allocated to the existing big data components on each bare metal server.

[0141] Step b202: Determine whether the user has allocated hardware resources to the third bare metal server.

[0142] Specifically, if the user specifies the hardware resource configuration for the third bare metal server, a corresponding resource file will be generated based on the user's needs. This resource file includes the resource allocation quotas for the big data components required by the third bare metal server. If the user does not specify the hardware resource configuration for the third bare metal server, the default resource allocation rules will be used.

[0143] Step b203: The service control terminal passes the host file to the pre-written cluster initialization script.

[0144] Specifically, the cluster initialization script will initialize the environment configuration for the newly added third bare metal server, including system environment variables, downloading and decompressing the installation package, and other operations.

[0145] Step b204: After completing the initialization configuration, the service control terminal passes the host file to the pre-written installation script, and executes the installation script to add a third bare metal server in the bare metal server cluster.

[0146] Specifically, the installation script installs relevant components based on the node type of the third bare metal server and the planned big data components included in the host file.

[0147] Step b205: After the installation script is executed, the service control terminal performs a component function check on the newly added third bare metal server to ensure that the service functions of the big data components deployed on the third bare metal server can start normally.

[0148] Step b206: After obtaining the inspection results, the service control terminal generates the corresponding new log.

[0149] If you need to remove a bare metal server, then identify the second bare metal server that needs to be removed from the bare metal server cluster.

[0150] Step b211: Obtain the host file generated by the service control terminal.

[0151] Step b212: Determine the node type of the second bare metal server.

[0152] Step b213: If it is the master node, execute the pre-written uninstallation script through the service control terminal to stop the big data component service on all bare metal servers in the bare metal server.

[0153] This can be understood as unloading the big data component services from the entire bare metal server cluster.

[0154] Step b214: After stopping the big data component services on all bare metal servers, the service control terminal continues to clear the configuration files and stored data of all big data components in the bare metal server cluster.

[0155] Step b215: After the service control terminal completes the task of stopping the service and clearing the components, it generates the corresponding deletion log.

[0156] Step b216: If the second bare metal server is a worker node, execute the uninstallation script through the service control terminal to stop all big data component services on the second bare metal server.

[0157] Step b217: After stopping all big data component services on the second bare metal server, the service control terminal continues to clear the configuration files and stored data of all big data components on the second bare metal server.

[0158] Step b218: After the service control terminal completes the tasks of stopping services and clearing components, it restarts the services on the remaining bare metal servers in the bare metal server cluster and generates the corresponding deletion logs.

[0159] This can be understood as restarting the services on the master node and other worker nodes to ensure information synchronization between these nodes.

[0160] This application provides a method for deploying big data components based on a bare metal server cluster. It obtains a big data component processing request and acquires first component information of a first big data component installed on a first bare metal server, as well as second component information of a target big data component. Based on the first and second component information, the method performs target processing on the first bare metal server. Target processing includes adding or uninstalling a target big data component. Component information includes the component's identifier and the hardware resources it occupies. Based on the first and second component information, the method executes corresponding installation or uninstallation scripts to install or uninstall the target big data component from the first bare metal server. Users can customize and select the required target big data component and dynamically configure big data components according to their actual needs. This allows for adding or uninstalling the target big data component on the first bare metal server, achieving customized deployment. Furthermore, deploying big data components by executing corresponding scripts reduces the deployment difficulty and improves the deployment efficiency of big data components. In addition, in order to make full use of the hardware resources in the bare metal server cluster, bare metal servers can be added or deleted as needed, thereby achieving dynamic management of the bare metal server cluster, which can improve resource utilization, simplify deployment and management, and improve cluster performance.

[0161] Figure 5 This is a schematic diagram of the structure of the big data component deployment device based on a bare metal server cluster provided in this application. Figure 5 As shown, the big data component deployment device 500 based on bare metal server cluster includes: an acquisition module 501 and a processing module 502.

[0162] Among them, the acquisition module 501 is used to acquire big data component processing requests. The big data component processing requests are used to perform target processing on the first bare metal server. The target processing includes adding a target big data component or uninstalling a target big data component.

[0163] The acquisition module 501 is also used to acquire the first component information of the first big data component installed on the first bare metal server, and the second component information of the target big data component; wherein, the component information includes the component's identifier and the hardware resources occupied by the component;

[0164] The processing module 502 is used to perform target processing on the target big data component based on the first component information and the second component information, and obtain the deployment result.

[0165] Optionally, the big data component deployment device 500 based on bare metal server clusters also includes: a determination module 503;

[0166] The processing module 502 is also used to perform target processing on the target big data component based on the second component information;

[0167] The acquisition module 501 is used to acquire the third component information of the second big data component currently installed on the first bare metal server;

[0168] The determination module 503 is used to determine the deployment result based on the information of the first component, the information of the second component, and the information of the third component.

[0169] Optionally, the big data component deployment device 500 based on bare metal server cluster further includes: a judgment module 504, used to determine the deployment result by judging whether the third component information includes the first component information and the second component information if the target processing is to add a new target big data component; and to determine the deployment result by judging whether the first component information includes the second component information and the third component information if the target processing is to uninstall a target big data component.

[0170] Optionally, the big data component deployment device 500 based on bare metal server cluster further includes: a checking module 505, used to perform a service function check on the target big data component after the deployment result indicates successful deployment, and determine the check result; wherein, when the target processing is to add a target big data component, the service function check is to check whether the service function of the target big data component is in an operable state; when the target processing is to uninstall the target big data component, the service function check is to check whether the service function of the target big data component is in an inoperable state.

[0171] Optionally, the acquisition module 501 is also used to acquire a bare metal server deletion request, which is used to delete the second bare metal server from the bare metal server cluster; and to acquire server information of all bare metal servers in the bare metal server cluster, including the identifier of the bare metal server and the hardware resources occupied by the bare metal server.

[0172] The processing module 502 is also used to stop the big data component service of the bare metal server cluster based on the server information of all bare metal servers if the second bare metal server is the master node; and to stop the big data component service of the second bare metal server based on the server information of the second bare metal server if the second bare metal server is the worker node.

[0173] Optionally, the acquisition module 501 is also used to acquire a bare metal server addition request, which is used to add a third bare metal server to the bare metal server cluster; and to acquire the hardware resource information of the third bare metal server.

[0174] The processing module 502 is also used to configure the corresponding hardware resources for the third bare metal server according to the hardware resource information of the third bare metal server, and add it to the bare metal server cluster.

[0175] Optionally, the device further includes a restart module 506 for restarting the remaining bare metal servers in the bare metal server cluster.

[0176] The big data component deployment apparatus based on bare metal server clusters provided in this application embodiment can be used to execute the big data component deployment method based on bare metal server clusters in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0177] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. Additionally, these modules can be fully or partially integrated together, or implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions.

[0178] Figure 6 This is a hardware structure diagram of the big data component deployment device based on a bare-metal server cluster provided in this application. Figure 6 As shown, the big data component deployment device 600 based on bare metal server cluster includes:

[0179] Processor 601 and memory 602;

[0180] The memory stores the instructions that the computer executes;

[0181] The processor executes the computer execution instructions stored in memory 602, causing the big data component deployment device based on bare metal server cluster to perform the big data component deployment method based on bare metal server cluster described above.

[0182] It should be understood that the processor 601 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The memory 602 may include high-speed random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disc, etc.

[0183] Optionally, the big data component deployment device 600 based on bare metal server clusters can be implemented as... Figure 1 The service control terminal shown.

[0184] The big data component deployment device based on bare metal server cluster provided in this application embodiment can be used to execute the big data component deployment method based on bare metal server cluster provided in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0185] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the above-described method for deploying big data components based on a bare-metal server cluster.

[0186] This application also provides a computer program product, which includes a computer program that, when executed by a processor, is used to implement the above-described method for deploying big data components based on a bare metal server cluster.

[0187] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0188] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0189] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0190] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0191] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0192] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0193] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0194] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0195] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for deploying big data components based on a bare metal server cluster, characterized in that, include: A big data component processing request is obtained. The big data component processing request is used to perform target processing on the first bare metal server. The target processing includes adding a target big data component or uninstalling the target big data component. Obtain first component information of the first big data component installed on the first bare metal server, and second component information of the target big data component; wherein, the component information includes the component's identifier and the hardware resources occupied by the component; Based on the first component information and the second component information, the target big data component is subjected to the target processing to obtain the deployment result.

2. The method according to claim 1, characterized in that, The step of performing the target processing on the target big data component based on the first component information and the second component information to obtain the deployment result includes: Based on the second component information, the target big data component is subjected to the target processing to obtain the third component information of the second big data component currently installed on the first bare metal server; The deployment result is determined based on the first component information, the second component information, and the third component information.

3. The method according to claim 2, characterized in that, Determining the deployment result based on the first component information, the second component information, and the third component information includes: If the target processing involves adding the target big data component, the deployment result is determined by judging whether the third component information includes the first component information and the second component information. If the target processing is to uninstall the target big data component, the deployment result is determined by judging whether the first component information includes the second component information and the third component information.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: After the deployment result indicates successful deployment, a service function check is performed on the target big data component to determine the check result; Specifically, when the target processing involves adding the target big data component, the service function check is to check whether the service function of the target big data component is in a runnable state. When the target processing involves uninstalling the target big data component, the service function check is to check whether the service function of the target big data component is in an inoperable state.

5. The method according to any one of claims 1-3, characterized in that, The method further includes: Obtain a bare metal server deletion request, which is used to delete the second bare metal server from the bare metal server cluster; Obtain server information for all bare metal servers in the bare metal server cluster, including the identifier of the bare metal server and the hardware resources occupied by the bare metal server; If the second bare metal server is the master node, then based on the server information of all bare metal servers, the big data component service of the bare metal server cluster is stopped. If the second bare metal server is a worker node, then based on the server information of the second bare metal server, stop the big data component service of the second bare metal server.

6. The method according to any one of claims 1-3, characterized in that, The method further includes: Obtain a bare metal server add request, which is used to add a third bare metal server to the bare metal server cluster; Obtain the hardware resource information of the third bare metal server; Based on the hardware resource information of the third bare metal server, configure the corresponding hardware resources for the third bare metal server and add it to the bare metal server cluster.

7. The method according to claim 5, characterized in that, After the statement "If the second bare metal server is a worker node, then based on the server information of the second bare metal server, stop the big data component service of the second bare metal server" is made, the method further includes: Restart the remaining bare metal servers in the bare metal server cluster.

8. A big data component deployment device based on a bare metal server cluster, characterized in that, include: The acquisition module is used to acquire big data component processing requests, which are used to perform target processing on the first bare metal server. The target processing includes adding a target big data component or uninstalling the target big data component. The acquisition module is further configured to acquire first component information of the first big data component installed on the first bare metal server, and second component information of the target big data component; wherein, the component information includes the identifier of the component and the hardware resources occupied by the component; The processing module is used to perform the target processing on the target big data component based on the first component information and the second component information, and obtain the deployment result.

9. A big data component deployment device based on a bare metal server cluster, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the big data component deployment method based on bare metal server clusters as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the big data component deployment method based on a bare metal server cluster as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, is used to implement the big data component deployment method based on a bare metal server cluster as described in any one of claims 1 to 7.

Images