Distributed database deployment method and device, electronic equipment and storage medium

Abstract

The invention relates to a distributed database deployment method and device, electronic equipment and a storage medium. The method comprises the steps that a deployment request sent by a cloud platform is received; configuring a distributed controller cluster according to the data information; issuing the deployment request to a distributed controller cluster, so that the distributed controller cluster constructs a corresponding distributed database in response to the deployment request; and obtaining a distributed database constructed by a distributed controller cluster, and storing the data of the cloud platform into distributed nodes of the distributed database to complete deployment of the distributed database, so that a client does not need to pay attention to complicated request processing in the distributed database, and only needs to send a deployment or operation and maintenance instruction, and the deployment of the distributed database is completed. Deployment and application of the distributed database in the public cloud environment can be easily realized; in addition, the distributed database constructed through the distributed controller cluster can automatically carry out operation and maintenance management under the condition that a client does not need to intervene, and stable operation of the distributed database is ensured.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a distributed database deployment method, device, electronic device and storage medium. Background Art

[0002] Cloud native is based on the cloud environment and focuses on optimizing application development and deployment in the cloud environment. It uses the elasticity, flexibility, and automation of cloud computing to package applications into lightweight, portable images, and uses container orchestration tools (such as Kubernetes) to automatically deploy, scale, and manage applications to improve efficiency and reliability. In addition, cloud native decomposes applications into independent microservices for flexible development and maintenance, and uses CI / CD processes to automate build, testing, and deployment to speed up delivery and reduce errors.

[0003] However, public clouds face some challenges in practical applications, especially when the amount of user data is very large and highly concurrent. It becomes particularly difficult to deploy distributed databases on public cloud platforms and to implement the deployment and maintenance of different types of distributed databases.

[0004] Therefore, there is an urgent need to develop a distributed database deployment method, device, electronic device and storage medium to solve one or more of the above-mentioned problems. Summary of the invention

[0005] In view of this, in order to solve the above technical problems or part of the technical problems, the embodiments of the present invention provide a distributed database deployment method, device, electronic device and storage medium.

[0006] In a first aspect, the present application provides a distributed database deployment method, the method comprising:

[0007] Receiving a deployment request sent by a cloud platform, the deployment request is used to request deployment of a distributed database on the cloud platform, and the deployment request carries data information of the cloud platform;

[0008] According to the data information, a distributed controller cluster is configured, wherein the distributed controller cluster includes at least one type of distributed controller;

[0009] Sending the deployment request to a distributed controller cluster so that the distributed controller cluster builds a corresponding distributed database in response to the deployment request;

[0010] A distributed database constructed by a distributed controller cluster is obtained, and the data of the cloud platform is stored in the distributed nodes of the distributed database to complete the deployment of the distributed database.

[0011] In a possible implementation, configuring a distributed controller cluster according to the data information includes:

[0012] Parsing the data information to determine the data type of the cloud platform and the data content of each type of data;

[0013] According to the data type of the cloud platform, controller type requirements are obtained;

[0014] According to the data content of each type of data, the quantity requirement of each type of controller is obtained;

[0015] Based on the controller type requirements and the quantity requirements of each type of controller, a matching distributed controller cluster is configured.

[0016] In a possible implementation, the method further includes:

[0017] Receiving monitoring information of the distributed database fed back by the distributed controller cluster;

[0018] The distributed controller cluster is adjusted according to the monitoring information, and the deployment request is sent to the adjusted distributed controller cluster, so that the adjusted distributed controller cluster builds a new distributed database in response to the deployment request.

[0019] In a possible implementation, adjusting the distributed controller cluster according to the monitoring information includes:

[0020] According to the monitoring information, determining whether there is data update in the distributed database;

[0021] When there is data update in the distributed database, the distributed controller cluster is adjusted according to the data update situation.

[0022] In a possible implementation, when there is data update in the distributed database, adjusting the distributed controller cluster according to the data update situation includes:

[0023] When there is new data in the distributed database, obtaining data information of the new data;

[0024] According to the data information of the newly added data, configure the corresponding newly added distributed controller;

[0025] The newly added distributed database is added to the distributed controller cluster, and the adjustment of the distributed controller cluster is completed.

[0026] In a possible implementation, when there is data update in the distributed database, adjusting the distributed controller cluster according to the data update situation includes:

[0027] In the case where data is deleted in the distributed database, obtaining data information of the deleted data;

[0028] Determining, according to the data information of the deleted data, a distributed controller in the distributed controller cluster corresponding to the deleted data;

[0029] Obtaining the working status of the distributed controller;

[0030] When the working state of the distributed controller is idle, the distributed controller is deleted from the distributed controller cluster, and the adjustment of the distributed controller cluster is completed.

[0031] In a possible implementation manner, before sending the deployment request to the distributed controller cluster, the method further includes:

[0032] Obtaining a preset distributed database template, wherein the distributed database template includes key parameters required for constructing a distributed database;

[0033] The distributed database template is sent to a distributed controller cluster, so that the distributed controller cluster constructs a distributed database matching the distributed database template according to key parameters.

[0034] In a second aspect, the present application provides a distributed database deployment device, the device comprising:

[0035] A receiving module, used to receive a deployment request sent by a cloud platform, wherein the deployment request is used to request deployment of a distributed database on the cloud platform, and the deployment request carries data information of the cloud platform;

[0036] A configuration module, configured to configure a distributed controller cluster according to the data information, wherein the distributed controller cluster includes at least one type of distributed controller;

[0037] A sending module, used for sending the deployment request to a distributed controller cluster, so that the distributed controller cluster builds a corresponding distributed database in response to the deployment request;

[0038] The deployment module is used to obtain the distributed database constructed by the distributed controller cluster and store the data of the cloud platform in the distributed nodes of the distributed database to complete the deployment of the distributed database.

[0039] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the distributed database deployment method described in any one of the embodiments of the first aspect are implemented.

[0040] In a fourth aspect, the present application further provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the distributed database deployment method described in any embodiment of the first aspect are implemented.

[0041] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the distributed database deployment method provided by the embodiment of the present application parses the deployment request sent by the cloud platform and sends the deployment request to the corresponding distributed controller cluster, and then the data of the cloud platform is stored in the distributed database built by the distributed controller cluster, so that the customer does not need to pay attention to the complex request processing inside the distributed database, and only needs to issue deployment or operation and maintenance instructions to easily realize the deployment and application of the distributed database in the public cloud environment; in addition, the distributed database built by the distributed controller cluster can automatically perform operation and maintenance management without customer intervention to ensure its stable operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0045] Figure 1 A schematic diagram of a distributed database deployment method provided in an embodiment of the present application;

[0046] Figure 2 Call relationship diagram of kubernetes components provided for the embodiment of this application

[0047] Figure 3A schematic diagram of the steps of a distributed database deployment method provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of the structure of a distributed database deployment device provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0051] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0052] In order to solve the problem that the amount of data of public cloud users in the prior art is very large and highly concurrent, the distributed database is deployed on the public cloud platform, and the deployment and maintenance of different types of distributed databases become particularly difficult. The present application provides a distributed database deployment method, device, electronic device and storage medium, which can enable users to only focus on simple deployment requests and processing results without having to care about complex k8s requests of different distributed database systems, thereby simplifying user operations and improving ease of use; and managing in namespaces, when the number of requests decreases within a certain period of time, the system will automatically delete idle k8s controllers; conversely, when the number of requests suddenly increases, the webservice will calculate and create additional k8s controllers that meet the needs based on the current request volume, and distribute the requests to these controllers. Through this mechanism, the system realizes dynamic concurrent processing of distributed database requests on the public cloud platform, thereby ensuring the efficiency and stability of the overall system.

[0053] Figure 1 A schematic diagram of a distributed database deployment method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method specifically includes:

[0054] S101, receiving a deployment request sent by a cloud platform, wherein the deployment request is used to request deployment of a distributed database on the cloud platform, and the deployment request carries data information of the cloud platform;

[0055] In a cloud platform, a deployment request is an instruction sent by a user or system to a cloud platform, requesting the cloud platform to allocate resources and configure the environment to support the deployment of a specific application or service.

[0056] In the deployment request, in addition to clearly indicating that the user requests to deploy a distributed database on the cloud platform, detailed data information of the cloud platform will also be carried. This data information includes but is not limited to the type of cloud platform (such as public cloud, private cloud or hybrid cloud), geographical location (such as the country or city where the data center is located), available resources (such as CPU, memory, storage space, etc.) and security policies (such as data encryption, access control, etc.). The cloud platform deployment plan is formulated and the cloud platform resource configuration is optimized based on this data information.

[0057] In this embodiment, webservice is the carrier of the present invention. Webservice interacts with the public cloud platform and receives call requests from the public cloud platform. Different module interfaces are called according to the interface purpose. According to the execution status of the request, the request result is fed back to the public cloud platform.

[0058] In addition, when the cloud platform receives a deployment request, it will perform a series of pre-checks to ensure the validity and legality of the request. Pre-checks generally include verifying the identity of the user, checking whether the format of the request complies with the specifications, and evaluating whether the current resource status of the cloud platform can meet the deployment requirements. Once confirmed to be correct, the cloud platform will start planning a specific deployment plan based on the data information in the request, and retrieve the type and quantity of resources that need to be allocated, as well as the deployment schedule, etc.

[0059] S102. According to the data information, configure a distributed controller cluster, wherein the distributed controller cluster includes at least one type of distributed controller;

[0060] In this embodiment, each user has different requirements for distributed databases. The user can deploy the required distributed database type, corresponding distributed controller version, persistent volume directory, computing resources and other configuration parameters in the webservice's ConfigMap (an API object used to store non-encrypted configuration data in k8s), and submit a database deployment request to the public cloud platform.

[0061] Specifically, after receiving a request from the public cloud platform, the webservice will configure a suitable distributed database and its corresponding controller based on the parameters configured by the user in ConfigMap. The distributed control cluster includes various types of distributed controllers corresponding to different distributed database requirements.

[0062] S103, sending the deployment request to the distributed controller cluster, so that the distributed controller cluster builds a corresponding distributed database in response to the deployment request;

[0063] In this embodiment, the Kubernetes controller listens to relevant requests in Kubernetes, puts the requests into its own processing queue, processes the distributed database requests in the queue in turn, and after sending the deployment request to the distributed controller cluster, starts the complex and orderly process of building a distributed database.

[0064] In addition, the Kubernetes controller listens to the Kubernetes request, processes the request according to its own internal logic, implements the deployment and maintenance of the distributed database, and updates the current distributed database task processing status to the Kubernetes resources; the webservice will subsequently analyze the resource and determine the request result.

[0065] Specifically, the distributed controller cluster will first parse the received deployment request; in this process, the controller identifies key information in the request, such as the type of database (such as key-value storage, column storage, document database, etc.), expected storage capacity, performance requirements (such as read and write speed, latency requirements), data consistency and availability levels, as well as specific security policies and backup requirements; then based on the parsed information, the controller cluster will make a preliminary plan to determine the amount of resources required to build a distributed database, including computing resources (CPU, memory), storage resources (hard disk space, I / O performance) and network resources (bandwidth, latency). At the same time, according to the current state and load of the cluster, the most suitable distributed node is selected to carry different components of the database. After the planning is completed, the distributed controller cluster starts to allocate resources, reserve sufficient storage space for the distributed nodes, allocate necessary computing resources, and configure the network to ensure effective communication between nodes; finally, after the resource allocation and preparation are completed, the distributed controller cluster will start to deploy the various components of the database so that the components can cooperate with each other correctly, ensuring that the distributed database system can run stably and meet business needs.

[0066] S104: Acquire a distributed database constructed by a distributed controller cluster, and store the data of the cloud platform in the distributed nodes of the distributed database to complete the deployment of the distributed database.

[0067] In this embodiment, a distributed database built by a distributed control cluster is obtained, and then the huge and complex data sets in the cloud platform are migrated to the distributed nodes of the distributed database. Each distributed node has data processing capabilities and storage capacity, which ensures the consistency and security of the data, realizes the comprehensive deployment of the distributed database, and lays a solid foundation for subsequent data management and analysis.

[0068] The storage process of cloud platform data includes the following steps:

[0069] Data extraction: Extract the data to be stored from various data sources on the cloud platform, where the data sources on the cloud platform may include relational databases, non-relational databases, file systems, or other forms of data storage systems.

[0070] Data conversion: The extracted data is converted and standardized to ensure that it meets the storage requirements of the distributed database. For example, JSON format data is converted into table format data supported by the distributed database.

[0071] Data loading: The converted data will be loaded into each node of the distributed database; this loading also needs to take into account factors such as data partitioning strategy, index optimization, and load balancing between nodes to ensure that the data can be stored efficiently and accurately in the specified location.

[0072] Data synchronization and backup: To ensure data consistency and reliability, distributed database systems usually implement data synchronization and backup mechanisms so that when data on a node changes, the data changes will be synchronized to other nodes in a timely manner and backed up to a secure storage medium at the same time.

[0073] In the market data processing method provided in this embodiment, webservice receives the deployment request sent by the cloud platform, selects the appropriate distributed database and its corresponding controller according to the parameters configured by the user in ConfigMap, configures the distributed controller cluster, and then sends the deployment request to the distributed controller cluster so that the distributed control cluster builds the corresponding distributed database. Finally, the data of the cloud platform is stored in the distributed nodes of the distributed database to complete the deployment of the system to the public cloud platform. This method allows users to only focus on simple deployment requests and processing results without having to care about complex k8s requests of different distributed database systems, thereby simplifying user operations and improving ease of use; webservice adopts a dynamic load management mechanism, which can monitor the load status of the k8s controller (distributed controller) in real time, and dynamically integrate the controller accordingly, manage it in namespaces, and automatically remove idle k8s controllers during periods when the number of requests decreases; on the contrary, when the number of requests surges, webservice calculates and generates the required additional k8s controllers according to the current number of requests, and distributes the requests to these controllers, thereby realizing dynamic concurrent processing of distributed database requests on the public cloud platform and ensuring the efficiency and stability of the entire system.

[0074] In an optional solution of the embodiment of the present invention, configuring a distributed controller cluster according to the data information includes:

[0075] Parsing the data information to determine the data type of the cloud platform and the data content of each type of data;

[0076] In this embodiment, the data information of the cloud platform is parsed, and different types of data are identified and distinguished by adopting file extension analysis, content parsing (such as regular expression matching), and metadata inspection, such as distinguishing structured data (such as tables in a database), semi-structured data (such as documents in JSON, XML, etc.), and unstructured data (such as text, pictures, videos, etc.);

[0077] For example, by checking the file extension (.txt, .jpg, .csv, etc.), you can preliminarily determine whether it is text, image, or table data; further, by parsing the file content, confirm its specific format and structure, and thus more accurately determine the data type.

[0078] For different types of data, the calculation method of byte length is also different. For example, for text data, the byte length is usually equal to the number of characters multiplied by the encoding length of the character (for example, under UTF-8 encoding, an English character occupies 1 byte, while a Chinese character may occupy 3 bytes). For image and video data, the byte length depends on factors such as the resolution and color depth of the image, as well as the frame rate and bit rate of the video; after determining the data type, determine the data content of each type of data, thereby determining the number of bytes of each type of data;

[0079] According to the data type of the cloud platform, controller type requirements are obtained;

[0080] In this embodiment, the controller type requirements corresponding to different data types are analyzed to further obtain the configuration requirements of the controller cluster. For example, for structured data, a SQL controller with efficient data processing capabilities is used; and for semi-structured data, a NoSQL controller with strong text parsing capabilities can be selected.

[0081] In addition, for unstructured data, the system will use an object storage controller with efficient storage and retrieval capabilities. Such object storage controllers can intelligently divide and index data according to the characteristics of the data to improve access efficiency. The system will also comprehensively consider the security and privacy requirements of the data and configure corresponding security policies for different types of data, by serving as the cloud platform controller type requirements.

[0082] According to the data content of each type of data, the quantity requirement of each type of controller is obtained;

[0083] In this embodiment, the required number of controllers is obtained according to the size of the data content and the processing complexity, and the stability and efficiency of the cloud platform service are guaranteed based on the data processing requirements and the expected load balancing requirements.

[0084] For example, for large-scale database operations, you may need to deploy multiple SQL controllers to achieve load balancing and failover.

[0085] Based on the controller type requirements and the quantity requirements of each type of controller, a matching distributed controller cluster is configured.

[0086] In this embodiment, after obtaining the controller type requirements and the quantity requirements of each type of controller, the distributed controller can be called according to the requirements and a matching distributed controller cluster can be configured to achieve efficient processing and storage of cloud platform data.

[0087] On this basis, the cluster configuration will fully consider the data growth trend of the cloud platform and the peak business demand, ensuring that the controller resources can be dynamically expanded to meet the changing business load of the cloud platform.

[0088] The configuration method of the distributed control cluster provided in this embodiment can effectively and automatically adjust controller resources based on the data information of the cloud platform, so that the deployment of the distributed database matches the dynamic growth of data in the cloud platform, greatly improving the flexibility and efficiency of data processing.

[0089] It should be noted that, in this embodiment, a matching distributed controller cluster may also be configured according to a pre-selected distributed database template, and the process specifically includes:

[0090] After the webservice is deployed, the operation and maintenance personnel select the type of distributed database used this time, select the corresponding webservice configuration template, and modify the configuration template content according to customer needs, such as the corresponding version of the distributed database and its controller. Then, according to the parameter requirements in the webservice configuration template, the corresponding controller cluster is automatically deployed and configured on the cloud platform.

[0091] Specifically, the webservice listens to the distributed database request. After receiving the request, it first reads the configuration file and selects the corresponding distributed database mode. According to different namespaces, a Kubernetes controller (distributed controller) and its required resources are created for each namespace.

[0092] In an optional solution of the embodiment of the present invention, the method further includes:

[0093] Receiving monitoring information of the distributed database fed back by the distributed controller cluster;

[0094] The distributed controller cluster is adjusted according to the monitoring information, and the deployment request is sent to the adjusted distributed controller cluster, so that the adjusted distributed controller cluster builds a new distributed database in response to the deployment request.

[0095] like Figure 2-3 As shown, in this embodiment, the Kubernetes controller interacts with Kubernetes; the Kubernetes controller listens to relevant requests in Kubernetes, the webservice will continuously monitor the status of these controllers, analyze the feedback of the distributed database, and report the task processing status of the distributed database to the public cloud platform; the webservice adjusts the Kubernetes controller cluster according to the monitoring information, and then re-sends the deployment information to the adjusted distributed controller cluster, so that the adjusted distributed controller cluster responds to the deployment request again, builds a new distributed database in real time, optimizes resource allocation, and improves system performance.

[0096] In an optional solution of the embodiment of the present invention, adjusting the distributed controller cluster according to the monitoring information includes:

[0097] According to the monitoring information, determining whether there is data update in the distributed database;

[0098] When there is data update in the distributed database, the distributed controller cluster is adjusted according to the data update situation.

[0099] In this embodiment, the monitoring information of the distributed controller is used to determine whether there is any data update in the distributed database deployed on the cloud platform. If there is a data update, the corresponding strategy adjustment of the distributed controller cluster is made in time, such as increasing or decreasing the number of controllers, optimizing resource allocation, and improving the efficiency and stability of the overall system.

[0100] Specifically, in order to cope with the large amount of data and high concurrency scenarios in the public cloud platform, webservice also adopts a dynamic load management mechanism, which monitors the load of the k8s controller in real time and dynamically integrates the controller accordingly;

[0101] Furthermore, management is performed in namespaces. When the number of requests decreases within a certain period of time, the system will automatically delete idle k8s controllers. Conversely, when the number of requests suddenly increases, the webservice will calculate and create additional k8s controllers that meet the demand based on the current request volume, and distribute the requests to these controllers. Through dynamic adjustment, the system realizes dynamic concurrent processing of distributed database requests on the public cloud platform, improving response speed and resource utilization.

[0102] In addition, if the subsequent distributed database system is in an unstable state, the Kubernetes controller will perform self-operation and maintenance according to the internal logic, and try to pull up the distributed database components that are currently in an unstable state, thereby restoring the distributed database system to a stable state.

[0103] In an alternative solution of the embodiment of the present invention, when there is data update in the distributed database, adjusting the distributed controller cluster according to the data update situation includes:

[0104] When there is new data in the distributed database, obtaining data information of the new data;

[0105] According to the data information of the newly added data, configure the corresponding newly added distributed controller;

[0106] The newly added distributed database is added to the distributed controller cluster, and the adjustment of the distributed controller cluster is completed.

[0107] In this embodiment, when it is monitored that there is new data in the distributed database, data information of the new data is obtained, and then a new distributed controller is configured according to the data information, and the new distributed controller is added to the distributed controller cluster, and the distributed control cluster is readjusted. Then, the adjusted distributed controller cluster is used to build and subsequently monitor the distributed database to achieve rapid response and efficient processing of the new data.

[0108] The specific steps include:

[0109] First, in the daily operation and maintenance of distributed databases by distributed controllers, real-time monitoring of data sources, capture and analysis of data change logs, and verification of data quality can be achieved. For example, advanced data stream processing technologies such as Apache Kafka or Flink can be integrated to achieve real-time capture and processing of data streams, ensuring that every detail of the new data is not missed.

[0110] After acquiring the new data, the data information of the new data can be mechanically parsed. Extract valuable information from the new data, such as data type, structure, source, timestamp, etc., to provide data support for subsequent controller processing and configuration. For example, for structured data, use SQL parser to extract table structure, field type and other information; for unstructured or semi-structured data, such as JSON or XML format, you may need to use regular expressions or special parsing libraries for parsing.

[0111] Finally, the corresponding new distributed controller is configured based on the data information of the new data. In the process of controller configuration, it is necessary to intelligently configure various parameters of the controller according to the characteristics and requirements of the data, such as load balancing strategy, data sharding rules, fault tolerance mechanism, etc.

[0112] The market data processing method provided in this embodiment monitors the new data in the distributed database. When new data appears, the system can respond quickly and automatically complete the adjustment and optimization of the controller.

[0113] In an alternative solution of the embodiment of the present invention, when there is data update in the distributed database, adjusting the distributed controller cluster according to the data update situation includes:

[0114] In the case where data is deleted in the distributed database, obtaining data information of the deleted data;

[0115] Determining, according to the data information of the deleted data, a distributed controller in the distributed controller cluster corresponding to the deleted data;

[0116] Obtaining the working status of the distributed controller;

[0117] When the working state of the distributed controller is idle, the distributed controller is deleted from the distributed controller cluster, and the adjustment of the distributed controller cluster is completed.

[0118] In this embodiment, when data deletion is detected in the distributed database, the corresponding distributed controller in the distributed controller cluster is locked by obtaining data information of the deleted data, and the operation of the distributed controller is stopped. When there are no other monitoring nodes in the distributed controller, the distributed controller is deleted from the distributed control cluster to complete the adjustment of the distributed controller cluster.

[0119] The specific adjustment process includes the following steps:

[0120] First, the data type of the deleted data is obtained, including key attributes such as the unique identifier of the data (such as the primary key), the data type, and the deletion timestamp. Then, the distributed controller in the distributed controller cluster responsible for processing the data is determined based on the data information.

[0121] When data deletion occurs in a distributed database, the first task is to accurately obtain the data information of the deleted data. This includes but is not limited to key attributes such as the unique identifier of the data (such as the primary key), data type, and deletion timestamp. The accurate acquisition of this information is the basis for subsequent steps. They are like key pieces in a jigsaw puzzle, guiding us to gradually restore the outline of the entire data deletion event.

[0122] Based on the deleted data information obtained, the next task is to determine the distributed controller in the distributed controller cluster that is responsible for processing this data. In a distributed system, data is usually stored in multiple nodes in pieces and managed and scheduled by the corresponding distributed controllers. This process is similar to quickly locating the bookshelf and location of a book in a large library based on its number. Through precise matching algorithms, we can quickly lock the distributed controller related to the deleted data.

[0123] In a distributed system, the node's load condition, fault status, and the length of the current task queue are key factors that determine whether it can execute new tasks. After obtaining the distributed controller, check the working status of the distributed controller to ensure that the selected distributed controller is idle, that is, it is not currently executing any critical tasks and has sufficient resources to cope with the upcoming deletion operation, avoiding service interruption or data loss due to improper operation.

[0124] After confirming that the distributed controller is in an idle state, the distributed controller is deleted to ensure that other running services or data are not affected when the distributed controller is deleted.

[0125] In an optional solution of the embodiment of the present invention, before sending the deployment request to the distributed controller cluster, the method further includes:

[0126] Obtaining a preset distributed database template, wherein the distributed database template includes key parameters required for constructing a distributed database;

[0127] The distributed database template is sent to a distributed controller cluster, so that the distributed controller cluster constructs a distributed database matching the distributed database template according to key parameters.

[0128] In this embodiment, a pre-designed template containing key parameters required for building a distributed database is obtained, wherein the key parameters include but are not limited to database architecture, node configuration, data distribution strategy, fault tolerance mechanism, and performance optimization options.

[0129] The template is then sent to the distributed controller cluster, which automatically builds a distributed database that matches the template based on the key parameters in the template. This process includes but is not limited to node initialization, network configuration, data partitioning and replication, index creation, and security policy implementation. For example, after the webservice is deployed, the operation and maintenance personnel select the type of distributed database used this time, select the corresponding webservice configuration template, and modify the configuration template content according to customer needs, such as the corresponding version of the distributed database and its controller, the persistent volume directory, the size of computing resources, etc., and deploy it on Kubernetes in the form of configmap.

[0130] Figure 4 A schematic diagram of a distributed database deployment system provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the system specifically includes:

[0131] The receiving module 401 is used to receive a deployment request sent by a cloud platform, wherein the deployment request is used to request the deployment of a distributed database on the cloud platform, and the deployment request carries data information of the cloud platform;

[0132] A configuration module 402, configured to configure a distributed controller cluster according to the data information, wherein the distributed controller cluster includes at least one type of distributed controller;

[0133] The sending module 403 is used to send the deployment request to the distributed controller cluster, so that the distributed controller cluster builds a corresponding distributed database in response to the deployment request;

[0134] The deployment module 404 is used to obtain the distributed database constructed by the distributed controller cluster, and store the data of the cloud platform in the distributed nodes of the distributed database to complete the deployment of the distributed database.

[0135] In one possible implementation, the configuration module 402 is also used to parse the data information, determine the data type of the cloud platform and the data content of each type of data; obtain the controller type requirement based on the data type of the cloud platform; obtain the quantity requirement of each type of controller based on the data content of each type of data; and configure a matching distributed controller cluster based on the controller type requirement and the quantity requirement of each type of controller.

[0136] In one possible implementation, the device also includes a monitoring module 405, which is used to receive monitoring information of the distributed database fed back by the distributed controller cluster; adjust the distributed controller cluster according to the monitoring information, and send the deployment request to the adjusted distributed controller cluster, so that the adjusted distributed controller cluster builds a new distributed database in response to the deployment request.

[0137] In a possible implementation, the monitoring module 405 is further configured to determine whether there is any data update in the distributed database according to the monitoring information; and if there is any data update in the distributed database, adjust the distributed controller cluster according to the data update situation.

[0138] In one possible implementation, the monitoring module 405 is also used to obtain data information of new data when there is new data in the distributed database; configure the corresponding new distributed controller according to the data information of the new data; add the new distributed database to the distributed controller cluster, and complete the adjustment of the distributed controller cluster.

[0139] In one possible implementation, the monitoring module 405 is also used to obtain data information of deleted data when data deletion exists in the distributed database; determine the distributed controller in the distributed controller cluster corresponding to the deleted data based on the data information of the deleted data; obtain the working status of the distributed controller; and when the working status of the distributed controller is idle, delete the distributed controller from the distributed controller cluster, thereby completing the adjustment of the distributed controller cluster.

[0140] In one possible implementation, the sending module 403 is also used to obtain a preset distributed database template, which includes key parameters required for building a distributed database; and send the distributed database template to a distributed controller cluster so that the distributed controller cluster builds a distributed database that matches the distributed database template based on the key parameters.

[0141] The distributed database deployment device provided in this embodiment can be as follows Figure 4 The distributed database deployment device shown in , can be executed as follows Figure 1-3 All steps of distributed database deployment in Figure 1-3 The technical effects of the distributed database deployment shown in the figure are as follows. For details, please refer to Figure 1-3 For the sake of brevity, the relevant description is not repeated here.

[0142] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, an embodiment of the present application provides an electronic device, including a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504; the memory 503 is used to store computer programs; the processor 501 is used to execute the program stored in the memory 503, and implement the steps of the distributed database deployment method provided by any of the above method embodiments:

[0144] Receive a deployment request sent by a cloud platform, the deployment request is used to request the deployment of a distributed database on the cloud platform, and the deployment request carries data information of the cloud platform; configure a distributed controller cluster according to the data information, the distributed controller cluster includes at least one type of distributed controller; send the deployment request to the distributed controller cluster, so that the distributed controller cluster builds a corresponding distributed database in response to the deployment request; obtain the distributed database built by the distributed controller cluster, and store the data of the cloud platform in the distributed nodes of the distributed database, so as to complete the deployment of the distributed database.

[0145] In one possible implementation, the data information is parsed to determine the data type of the cloud platform and the data content of each type of data; based on the data type of the cloud platform, the controller type requirement is obtained; based on the data content of each type of data, the quantity requirement of each type of controller is obtained; based on the controller type requirement and the quantity requirement of each type of controller, a matching distributed controller cluster is configured.

[0146] In one possible implementation, monitoring information of the distributed database fed back by the distributed controller cluster is received; based on the monitoring information, the distributed controller cluster is adjusted, and the deployment request is sent to the adjusted distributed controller cluster, so that the adjusted distributed controller cluster builds a new distributed database in response to the deployment request.

[0147] In a possible implementation, it is determined whether there is any data update in the distributed database according to the monitoring information; if there is any data update in the distributed database, the distributed controller cluster is adjusted according to the data update situation.

[0148] In one possible implementation, when there is new data in the distributed database, data information of the new data is obtained; according to the data information of the new data, a corresponding new distributed controller is configured; the new distributed database is added to the distributed controller cluster, and the adjustment of the distributed controller cluster is completed.

[0149] In one possible implementation, when data deletion exists in the distributed database, data information of the deleted data is obtained; based on the data information of the deleted data, the distributed controller corresponding to the deleted data in the distributed controller cluster is determined; the working status of the distributed controller is obtained; and when the working status of the distributed controller is idle, the distributed controller is deleted from the distributed controller cluster, thereby completing the adjustment of the distributed controller cluster.

[0150] In one possible implementation, a preset distributed database template is obtained, wherein the distributed database template includes key parameters required for constructing a distributed database; the distributed database template is sent to a distributed controller cluster, so that the distributed controller cluster constructs a distributed database that matches the distributed database template according to the key parameters. Through the description of the above implementation, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0151] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0152] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A distributed database deployment method, characterized in that: include: Receiving a deployment request sent by a cloud platform, the deployment request is used to request deployment of a distributed database on the cloud platform, and the deployment request carries data information of the cloud platform; According to the data information, a distributed controller cluster is configured, wherein the distributed controller cluster includes at least one type of distributed controller; Sending the deployment request to a distributed controller cluster so that the distributed controller cluster builds a corresponding distributed database in response to the deployment request; A distributed database constructed by a distributed controller cluster is obtained, and the data of the cloud platform is stored in the distributed nodes of the distributed database to complete the deployment of the distributed database.

2. The method according to claim 1, characterized in that The configuring a distributed controller cluster according to the data information includes: Parsing the data information to determine the data type of the cloud platform and the data content of each type of data; According to the data type of the cloud platform, controller type requirements are obtained; According to the data content of each type of data, the quantity requirement of each type of controller is obtained; Based on the controller type requirements and the quantity requirements of each type of controller, a matching distributed controller cluster is configured.

3. The method according to claim 1, characterized in that The method further comprises: Receiving monitoring information of the distributed database fed back by the distributed controller cluster; The distributed controller cluster is adjusted according to the monitoring information, and the deployment request is sent to the adjusted distributed controller cluster, so that the adjusted distributed controller cluster builds a new distributed database in response to the deployment request.

4. The method according to claim 3, characterized in that The step of adjusting the distributed controller cluster according to the monitoring information includes: According to the monitoring information, determining whether there is data update in the distributed database; When there is data update in the distributed database, the distributed controller cluster is adjusted according to the data update situation.

5. The method according to claim 4, characterized in that When there is data update in the distributed database, adjusting the distributed controller cluster according to the data update situation includes: When there is new data in the distributed database, obtaining data information of the new data; According to the data information of the newly added data, configure the corresponding newly added distributed controller; The newly added distributed database is added to the distributed controller cluster, and the adjustment of the distributed controller cluster is completed.

6. The method according to claim 4, characterized in that When there is data update in the distributed database, adjusting the distributed controller cluster according to the data update situation includes: In the case where data is deleted in the distributed database, obtaining data information of the deleted data; Determining, according to the data information of the deleted data, a distributed controller in the distributed controller cluster corresponding to the deleted data; Obtaining the working status of the distributed controller; When the working state of the distributed controller is idle, the distributed controller is deleted from the distributed controller cluster, and the adjustment of the distributed controller cluster is completed.

7. The method according to claim 1, characterized in that Before sending the deployment request to the distributed controller cluster, the method further includes: Obtaining a preset distributed database template, wherein the distributed database template includes key parameters required for constructing a distributed database; The distributed database template is sent to a distributed controller cluster, so that the distributed controller cluster constructs a distributed database matching the distributed database template according to key parameters.

8. A distributed database deployment device, characterized in that: include: A receiving module, used to receive a deployment request sent by a cloud platform, wherein the deployment request is used to request deployment of a distributed database on the cloud platform, and the deployment request carries data information of the cloud platform; A configuration module, configured to configure a distributed controller cluster according to the data information, wherein the distributed controller cluster includes at least one type of distributed controller; A sending module, used for sending the deployment request to a distributed controller cluster, so that the distributed controller cluster builds a corresponding distributed database in response to the deployment request; The deployment module is used to obtain the distributed database constructed by the distributed controller cluster and store the data of the cloud platform in the distributed nodes of the distributed database to complete the deployment of the distributed database.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the distributed database deployment method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the distributed database deployment method according to any one of claims 1 to 7 are implemented.

Images