Method and device for building ambari-based Redis database architecture, and electronic device

Abstract

The application discloses a building method and device of a Redis database architecture based on ambari and electronic equipment, and the building method comprises the following steps: determining the attribute name of the current service and the attribute value associated with the attribute name, rendering the preset configuration template of each preset component based on the attribute name and the attribute value, starting the preset component in the case of rendering completion, determining the database architecture type adapted to the current service based on the component quantity of each preset component, and building the preset database based on the database architecture type. The application solves the technical problem that the efficiency of manually modifying the database architecture configuration is low and the database required by the service cannot be built in time in the related art.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a method, apparatus, and electronic device for building a Redis database architecture based on Ambari. Background Technology

[0002] Redis is a high-performance, open-source key-value in-memory database. Due to its speed and excellent performance, it is widely used in the internet industry. Redis offers three different cluster architectures: single-machine, Sentinel, and cluster. Each architecture has its advantages and uses, and the choice of architecture is generally determined by the business requirements. Furthermore, changes in business structure may necessitate adjustments to the Redis architecture.

[0003] In related technologies, Redis starts as a single-machine architecture after downloading and starting, listening on port 6379 by default. Redis Sentinel is a solution to the single-point-of-failure problem inherent in Redis's single-machine architecture. It uses the Redis-sentinel component to check the health of Redis. If a problem occurs, it switches the primary Redis instance to the backup, and the backup switches back to the primary to continue providing service. Compared to traditional Redis master-slave architectures, it offers automatic failover, read load balancing, and other features. Redis Cluster addresses the limitation of Redis Sentinel in horizontal scaling. In the case of Redis Sentinel, the primary and backup data are completely consistent, and data capacity is limited by the hardware capacity of a single machine. Redis Cluster uses a hash consistency algorithm to distribute data evenly across multiple machines, thereby reducing the write pressure on a single machine and preventing it from exceeding its capacity.

[0004] However, configuring the aforementioned Redis architectures is quite complex, with each architecture requiring its own specific configuration. Furthermore, as business scales and needs change, Redis often needs to be transitioned from a single machine to a sentinel and then to a cluster. This requires manual modification of the Redis configuration on each machine to complete the changes and deployment.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This invention provides a method, apparatus, and electronic device for building a Redis database architecture based on Ambari, to at least solve the technical problem in related technologies where manual modification of database architecture configuration is inefficient and cannot build the database required for business in a timely manner.

[0007] According to one aspect of the present invention, a method for building a Redis database architecture based on Ambari is provided, comprising: determining the attribute name of the current business and the attribute value associated with the attribute name; rendering a preset configuration template for each preset component based on the attribute name and the attribute value, and starting the preset component upon completion of rendering, wherein the preset component is a node in a preset database, and the preset configuration template is a configuration template for pre-configured node attributes; determining a database architecture type adapted to the current business based on the number of components in each preset component; and building the preset database based on the database architecture type, wherein the preset database is used to store the business data of the current business in the database architecture type.

[0008] Optionally, before determining the attribute name of the current service and the attribute value associated with the attribute name, the method further includes: constructing a metadata file, wherein the metadata file includes at least: a preset service and the preset components carried by the preset service, the component types of the preset components include: a primary node component, a backup node component, and a sentinel component, the backup node component is a backup component of the primary node component, and the sentinel component is used to monitor the component status of the primary node component and the backup node component; and registering the preset service and the preset components carried by the preset service with a preset server based on the metadata file.

[0009] Optionally, before determining the attribute name of the current service and the attribute value associated with the attribute name, the method further includes: determining the attributes managed by the preset server; using the attribute name of each attribute as a template symbol, wherein the template symbol is used to place the attribute; configuring a default value for each template symbol; and constructing the preset configuration template based on the template symbol and the default value associated with the template symbol.

[0010] Optionally, before determining the attribute name of the current business and the attribute value associated with the attribute name, the method further includes: constructing an attribute acquisition function, wherein the attribute acquisition function is used to acquire the attribute name declared by the preset configuration template and the default value associated with the attribute name; constructing a component acquisition function, wherein the component acquisition function is used to acquire the number of components for each preset component; and constructing a preset attribute configuration file based on the attribute acquisition function and the component acquisition function.

[0011] Optionally, the step of rendering a preset configuration template for each preset component based on the attribute name and the attribute value includes: querying the template symbol corresponding to the attribute name from the preset configuration template based on the preset attribute configuration file; replacing the default value associated with the template symbol with the attribute value associated with the attribute name, thereby completing the rendering of the preset configuration template.

[0012] Optionally, the step of determining the database architecture type adapted to the current business based on the number of components in each preset component includes: obtaining the number of components in each preset component based on the preset attribute configuration file; determining the database architecture type as a cluster structure if the number of components in the master node component is greater than a first preset threshold; and determining the database architecture type as a sentinel structure if the number of components in the sentinel component is greater than a second preset threshold.

[0013] Optionally, the step of building a preset database based on the database architecture type includes: when the current node is the master node and the first node, controlling the current node to create a database cluster of the preset database, wherein the first node is the first server in the preset database; controlling other nodes to join the database cluster, wherein when the other nodes are backup nodes, controlling the other nodes to join the database cluster as backup nodes; allocating preset hash slots to all nodes in the database cluster to complete the construction of the preset database, wherein the preset hash slots are used to locate the node to which the business data belongs.

[0014] According to another aspect of the present invention, an apparatus for building a Redis database architecture based on Ambari is also provided, comprising: a first determining unit, configured to determine the attribute name of the current business and the attribute value associated with the attribute name; a rendering unit, configured to render a preset configuration template of each preset component based on the attribute name and the attribute value, and, upon completion of rendering, start the preset component, wherein the preset component is a node in a preset database, and the preset configuration template is a configuration template of pre-configured node attributes; a second determining unit, configured to determine a database architecture type adapted to the current business based on the number of components in each preset component; and a building unit, configured to build the preset database based on the database architecture type, wherein the preset database is used to store the business data of the current business in accordance with the database architecture type.

[0015] Optionally, the setup device further includes: a first construction module, configured to construct a metadata file before determining the attribute name of the current service and the attribute value associated with the attribute name, wherein the metadata file includes at least: a preset service and the preset components carried by the preset service, the component types of the preset components include: a primary node component, a backup node component, and a sentinel component, the backup node component is a backup component of the primary node component, and the sentinel component is used to monitor the component status of the primary node component and the backup node component; and a first registration module, configured to register the preset service and the preset components carried by the preset service with a preset server based on the metadata file.

[0016] Optionally, the setup device further includes: a first determining module, configured to determine the attributes managed by the preset server before determining the attribute name of the current service and the attribute value associated with the attribute name; a first serving module, configured to use the attribute name of each attribute as a template symbol, wherein the template symbol is used to place the attribute; a first configuration module, configured to configure a default value for each template symbol; and a second construction module, configured to construct the preset configuration template based on the template symbol and the default value associated with the template symbol.

[0017] Optionally, the building device further includes: a third building module, configured to build an attribute acquisition function before determining the attribute name of the current business and the attribute value associated with the attribute name, wherein the attribute acquisition function is used to acquire the attribute name declared by the preset configuration template and the default value associated with the attribute name; a fourth building module, configured to build a component acquisition function, wherein the component acquisition function is used to acquire the number of components for each preset component; and a fifth building module, configured to build a preset attribute configuration file based on the attribute acquisition function and the component acquisition function.

[0018] Optionally, the rendering unit includes: a first query module, used to query the template symbol corresponding to the attribute name from the preset configuration template based on the preset attribute configuration file; and a first replacement module, used to replace the default value associated with the template symbol with the attribute value associated with the attribute name, thereby completing the rendering of the preset configuration template.

[0019] Optionally, the second determining unit includes: a first obtaining module, configured to obtain the number of components of each preset component based on the preset attribute configuration file; a second determining module, configured to determine that the database architecture type is a cluster structure when the number of components of the master node component is greater than a first preset threshold; and a third determining module, configured to determine that the database architecture type is a sentinel structure when the number of components of the sentinel component is greater than a second preset threshold.

[0020] Optionally, the setup unit includes: a first creation module, used to control the current node to create a database cluster of the preset database when the current node is the master node and the first node, wherein the first node is the first server in the preset database; a first joining module, used to control other nodes to join the database cluster, wherein when the other nodes are backup nodes, the other nodes are controlled to join the database cluster as backup nodes; and a first allocation module, used to allocate preset hash slots to all nodes in the database cluster to complete the setup of the preset database, wherein the preset hash slots are used to locate the node to which the business data belongs.

[0021] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the above-described method for building a Redis database architecture based on Ambari.

[0022] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method for building a Redis database architecture based on Ambari.

[0023] In this disclosure, the attribute names and associated attribute values ​​of the current business are determined. A preset configuration template for each preset component is rendered based on the attribute names and values. Upon completion of rendering, the preset component is started. Based on the number of components in each preset component, the database architecture type suitable for the current business is determined. Based on the database architecture type, a preset database is built. This disclosure simplifies the configuration process of preset components by using preset configuration templates and automatically builds the preset database based on the determined database architecture type. This improves the efficiency of modifying database architecture configurations and solves the technical problem in related technologies where manual modification of database architecture configurations is inefficient and cannot promptly build the database required for the business. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 This is a flowchart of an optional method for building a Redis database architecture based on Ambari according to an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of an optional Redis control script logic according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of an optional Redis_slave control script logic according to an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of an optional apparatus for building a Redis database architecture based on Ambari according to an embodiment of the present invention;

[0029] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for building a Redis database architecture based on Ambari, according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0033] It's important to note that Ambari, as a supplement to Hadoop (a distributed system infrastructure) clusters, allows for the rapid setup of a Hadoop big data technology stack through its user interface and wizard-driven interface. Starting Ambari and following the wizard's steps to select hosts and the services and components to be installed on them automatically builds the cluster. However, Redis is not included in the Hadoop big data technology stack and is therefore not supported by default. Code and configuration files need to be written to extend Ambari to support Redis.

[0034] This invention combines Ambari and Redis, leveraging Ambari's centralized configuration management features to quickly and efficiently build a Redis cluster. It can automatically transition from a single machine to a sentinel or cluster when the number of components reaches a certain level. This invention simplifies configuration operations and cluster setup by using Ambari's API (Application Program Interface) and corresponding script templates to customize Redis configuration files.

[0035] The present invention will now be described in detail with reference to various embodiments.

[0036] Example 1

[0037] According to an embodiment of the present invention, an embodiment of a method for building a Redis database architecture based on Ambari is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0038] Figure 1 This is a flowchart illustrating an optional method for building a Redis database architecture based on Ambari, according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0039] Step S101: Determine the attribute name of the current business and the attribute value associated with the attribute name.

[0040] Step S102: Render the preset configuration template of each preset component based on the attribute name and attribute value, and start the preset component after rendering is complete. The preset component is a node in the preset database, and the preset configuration template is a configuration template of the pre-configured node attributes.

[0041] Step S103: Determine the database architecture type that is compatible with the current business based on the number of components in each preset component.

[0042] Step S104: Based on the database architecture type, build a preset database, wherein the preset database is used to store the business data of the current business according to the database architecture type.

[0043] Through the above steps, the attribute names and associated attribute values ​​of the current business can be determined. Based on the attribute names and values, a preset configuration template for each preset component is rendered. Upon completion of rendering, the preset component is started. Based on the number of components in each preset component, the database architecture type suitable for the current business is determined. Based on the database architecture type, a preset database is built. In this embodiment of the invention, the preset configuration template for each preset component is first rendered according to the attribute names and associated attribute values ​​of the current business. Then, the preset component is started. Based on the number of components in each preset component, the corresponding database architecture type is determined. Then, based on the database architecture type, a preset database is built to store the business data of the current business according to the current database architecture type. The preset configuration template simplifies the configuration process of the preset components and automatically builds the preset database based on the determined database architecture type, improving the efficiency of modifying the database architecture configuration. This solves the technical problem in related technologies where manual modification of the database architecture configuration is inefficient and cannot promptly build the database required for the business.

[0044] The embodiments of the present invention will now be described in detail with reference to the steps described above.

[0045] In an embodiment of the present invention, optionally, before determining the attribute name of the current service and the attribute value associated with the attribute name, the method further includes: constructing a metadata file, wherein the metadata file includes at least: a preset service and preset components carried by the preset service, the component types of the preset components include: a master node component, a backup node component, and a sentinel component, the backup node component is a backup component of the master node component, and the sentinel component is used to monitor the component status of the master node component and the backup node component; based on the metadata file, registering the preset service and the preset components carried by the preset service with a preset server.

[0046] In this embodiment of the invention, a metadata file (such as metainfo.xml, which contains the metadata information of the components) can be constructed first. This metadata file includes at least: a preset service (i.e., Redis) and preset components carried by the preset service. The component types of the preset components include: master node components, slave node components, and sentinel components. The slave node components are backup components of the master node components, and the sentinel components are used to monitor the component status (i.e., the health status of the components) of the master node components and slave node components. The metadata file is used to register the declared service and its components with Ambari (i.e., the preset server) (i.e., registering the preset service and its preset components with the preset server based on the metadata file). The declared service is Redis, and the components are Redis (master node in single-machine and cluster) (i.e., master node component), Redis_slave (sentinel, slave node in cluster) (i.e., slave node component), and Redis_sentinel (sentinel node within a sentinel) (i.e., sentinel component).

[0047] Optionally, before determining the attribute name of the current business and the attribute value associated with the attribute name, the method further includes: determining the attributes managed by the preset server; using the attribute name of each attribute as a template symbol, wherein the template symbol is used as a placeholder attribute; configuring a default value for each template symbol; and constructing a preset configuration template based on the template symbol and the default value associated with the template symbol.

[0048] In this embodiment of the invention, a template (i.e., a preset configuration template, such as Redis.xml and Redis-sentinel.xml, which are configuration file templates for the Redis component and the Redis Sentinel component, respectively) can be configured first. Specifically, the attributes managed by the preset server are determined first, and then the attribute name of each attribute is used as a template symbol (this template symbol is used as a placeholder attribute). Then, a default value is configured for each template symbol. After that, the preset configuration template is constructed based on the template symbol and the default value associated with the template symbol. This preset configuration template is used to declare to Ambari the configuration file templates and configuration attributes that need its help to manage. In the template file, the attributes that need to be managed by Ambari can be declared using template symbols (such as {{attribute name}}), and in the XML file, the property attribute (used to use a method as an attribute) is used to declare the attribute name and the default value of the attribute.

[0049] Optionally, before determining the attribute name and the attribute value associated with the current business, the method further includes: constructing an attribute retrieval function, wherein the attribute retrieval function is used to retrieve the attribute name declared in the preset configuration template and the default value associated with the attribute name; constructing a component retrieval function, wherein the component retrieval function is used to retrieve the number of components for each preset component; and constructing a preset attribute configuration file based on the attribute retrieval function and the component retrieval function.

[0050] In this embodiment of the invention, the file name and script control type (such as Python control type) can be declared in the metadata file. A preset attribute configuration file can be constructed using Python control. Specifically, an attribute retrieval function can be constructed first to obtain the attribute names declared in the preset configuration template and the default values ​​associated with those attribute names. A component retrieval function can then be constructed to obtain the number of components for each preset component. Finally, based on the attribute retrieval function and the component retrieval function, the preset attribute configuration file is constructed.

[0051] In this embodiment, the logic in the preset attribute configuration file is as follows: The built-in function `script.get_config` (i.e., the attribute retrieval function) provided by Ambari is used to obtain the attribute names and their values ​​declared in the preset configuration template. Then, a built-in Ambari function (i.e., the component retrieval function) is used to obtain the number of each component in the Ambari cluster. Based on the number, it is determined whether to expand to a cluster or a sentinel model. This number is declared in `Redis.xml` and managed by Ambari, and can be configured through the Ambari web interface. When the number of `Redis_sentinel` and `Redis` exceeds this number, a Redis cluster will be automatically built.

[0052] Step S101: Determine the attribute name of the current business and the attribute value associated with the attribute name.

[0053] In this embodiment of the invention, the attribute name of the current business and the attribute value associated with the attribute name can be determined first, so as to build a database with the corresponding architecture according to the current business requirements.

[0054] Step S102: Render the preset configuration template of each preset component based on the attribute name and attribute value, and start the preset component after rendering is complete. The preset component is a node in the preset database, and the preset configuration template is a configuration template of the pre-configured node attributes.

[0055] Optionally, the step of rendering the preset configuration template of each preset component based on the attribute name and attribute value includes: querying the template symbol corresponding to the attribute name from the preset configuration template based on the preset attribute configuration file; replacing the default value associated with the template symbol with the attribute value associated with the attribute name, and completing the rendering of the preset configuration template.

[0056] In this embodiment of the invention, a preset configuration template for each preset component (a preset component is a node in a preset database used to operate business data) can be rendered according to the attribute name and attribute value. (The preset configuration template is a configuration template of pre-configured node attributes; updating the attribute value of the template can generate the current configuration file of the component.) After rendering is completed, the preset component is started. Specifically, the attribute name and its attribute value can be obtained from the preset configuration template through the attribute retrieval function in the preset attribute configuration file. Then, the template symbol corresponding to the attribute name of the current business is queried. The default value associated with the template symbol is replaced with the attribute value associated with the attribute name to complete the rendering of the preset configuration template, generate the configuration file of the preset component, and thus start the preset component.

[0057] Step S103: Determine the database architecture type that is compatible with the current business based on the number of components in each preset component.

[0058] Optionally, the step of determining the database architecture type suitable for the current business based on the number of components of each preset component includes: obtaining the number of components of each preset component based on a preset attribute configuration file; determining the database architecture type as a cluster structure when the number of components of the master node component is greater than a first preset threshold; and determining the database architecture type as a sentinel structure when the number of components of the sentinel component is greater than a second preset threshold.

[0059] In this embodiment of the invention, the database architecture type adapted to the current business can be determined based on the number of components in each preset component. Specifically, the number of components in each preset component can be obtained through the component acquisition function in the preset attribute configuration file. If the number of components in the master node component is greater than a first preset threshold (e.g., 3, which can be set according to actual conditions), the database architecture type can be determined to be a cluster structure; if the number of components in the sentinel component is greater than a second preset threshold (e.g., 3, which can be set according to actual conditions), the database architecture type can be determined to be a sentinel structure.

[0060] Step S104: Based on the database architecture type, build a preset database, wherein the preset database is used to store the business data of the current business according to the database architecture type.

[0061] Optionally, based on the database architecture type, the steps for building a preset database include: when the current node is the master node and the first node, controlling the current node to create a database cluster for the preset database, wherein the first node is the first server in the preset database; controlling other nodes to join the database cluster, wherein when other nodes are backup nodes, controlling other nodes to join the database cluster as backup nodes; and allocating preset hash slots to all nodes in the database cluster to complete the construction of the preset database, wherein the preset hash slots are used to locate the node to which the business data belongs.

[0062] In this embodiment of the invention, a preset database can be built according to the database architecture type to store the business data of the current business according to the current database architecture type. Specifically, it can be determined whether the node role is master (i.e., primary node) and whether it is the first node. If so, a cluster is created as the master through the command line (such as the Redis-cli command line). (i.e., when the current node is the primary node and the first node, the current node is controlled to create a database cluster of the preset database, and the first node is the first server in the preset database.) Other nodes in the configuration file will automatically join the cluster. If the role of other nodes is slave (i.e., backup node), they will join the cluster as slaves. (i.e., control other nodes to join the database cluster, and when other nodes are backup nodes, control other nodes to join the database cluster as backup nodes.) Redis will automatically allocate hash slots (i.e., preset hash slots) to the nodes in the cluster. (i.e., allocate preset hash slots to all nodes in the database cluster. These preset hash slots are used to locate the node to which the business data belongs) to complete the construction of the preset database.

[0063] The following describes in detail another optional implementation method.

[0064] This embodiment proposes a simple method for building a single-machine Redis Sentinel cluster based on Ambari, which includes the following three structural parts:

[0065] The first part of the structure is metainfo.xml, which is the component's metadata file. It is used to register the declared service and the components that come with the service with Ambari. The declared service is Redis, and the components are Redis (single machine, master node of cluster), Redis_slave (sentinel, backup node of cluster), and Redis_sentinel (sentinel node in the sentinel).

[0066] The second part of the structure consists of configuration templates, namely Redis.xml and Redis-sentinel.xml, which are the configuration file templates for the Redis component and the Redis Sentinel component, respectively. They are used to declare to Ambari the configuration file templates and configuration properties that need its help to manage. In the template file, the properties that need to be managed by Ambari need to be declared using template symbols, such as {{property name}}, and the property name and default value of the property are declared using the property attribute in the XML file.

[0067] The third part consists of the component's control script files. The file names and script control types used can be declared in the metadata file. There are three control files and one attribute configuration file, which can be controlled by Python.

[0068] The logic in the property configuration file is as follows: It retrieves the property names and values ​​declared in the configuration template using the built-in function `script.get_config` provided by Ambari. It then uses Ambari's built-in functions to obtain the number of each component in the cluster. Based on this number, it determines whether to expand to a cluster or a sentinel model. This number is declared in `Redis.xml` and managed by Ambari, and can be configured through Ambari's web interface. When the number of `Redis_sentinel` and `Redis` objects exceeds this number, a Redis cluster will be automatically built.

[0069] The logic of the three control file scripts is as follows: Each script has a specific function to control the lifecycle of the component, namely the configura function for template rendering, the install script for installing components, the start function for starting the service, the stop function for stopping the service, and the status function for the service. Ambari controls the lifecycle of the component by calling each method in the script.

[0070] Figure 2 This is a schematic diagram of an optional Redis control script logic according to an embodiment of the present invention, such as... Figure 2 As shown, the control script logic in Redis is as follows: first, render the configuration template, then start the component, determine whether a cluster needs to be built, if so, call the function in the Redis cluster control script and indicate the role as the master node in the parameters, otherwise end directly.

[0071] Figure 3 This is a schematic diagram of an optional Redis_slave control script logic according to an embodiment of the present invention, such as... Figure 3As shown, the control script logic for Redis_slave is as follows: first, render the configuration template, then start the component, determine whether a cluster needs to be built, if so, call the function in the Redis cluster control script and indicate the role as a slave node in the parameters, otherwise end directly.

[0072] The logic of the Redis cluster control script is as follows: First, it determines whether the node role is master and whether it is the first node. If it is, it creates the cluster as the master via the command line. Other nodes in the configuration file will automatically join the cluster. If the role of other nodes is slave, they will join the cluster as slaves. Redis will automatically allocate hash slots to the nodes in the cluster.

[0073] The application scenario of this embodiment can be: when using Redis, you want to quickly build Redis clusters with different architectures according to the business scale, and you want Redis configuration to be simpler and more convenient.

[0074] In this embodiment of the invention, by combining the component control features and configuration template features brought by Ambari, the process of Redis configuration for operation and maintenance personnel is simplified, and the cluster building process is automated, making Redis use more convenient and faster.

[0075] The following is a detailed description with reference to another embodiment.

[0076] Example 2

[0077] The apparatus for building a Redis database architecture based on Ambari provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in Embodiment 1 above.

[0078] Figure 4 This is a schematic diagram of an optional Redis database architecture based on Ambari, according to an embodiment of the present invention. Figure 4 As shown, the assembly device may include: a first determining unit 40, a rendering unit 41, a second determining unit 42, and an assembly unit 43, wherein...

[0079] The first determining unit 40 is used to determine the attribute name of the current business and the attribute value associated with the attribute name;

[0080] Rendering unit 41 is used to render the preset configuration template of each preset component based on the attribute name and attribute value, and to start the preset component when the rendering is completed. The preset component is a node in the preset database, and the preset configuration template is a configuration template of the node attributes that is pre-configured.

[0081] The second determining unit 42 is used to determine the database architecture type that is compatible with the current business based on the number of components in each preset component.

[0082] Unit 43 is used to build a preset database based on the database architecture type. The preset database is used to store the business data of the current business according to the database architecture type.

[0083] The aforementioned setup device can determine the attribute name and associated attribute value of the current business through the first determining unit 40, render the preset configuration template of each preset component based on the attribute name and attribute value through the rendering unit 41, and start the preset component after rendering is complete. The second determining unit 42 determines the database architecture type suitable for the current business based on the number of components in each preset component, and the setup unit 43 builds the preset database based on the database architecture type. In this embodiment of the invention, the preset configuration template of each preset component can be rendered first according to the attribute name and associated attribute value of the current business, then the preset component can be started, the corresponding database architecture type can be determined according to the number of components in each preset component, and then the preset database can be built according to the database architecture type to store the business data of the current business according to the current database architecture type. The preset configuration template simplifies the configuration process of the preset components and can automatically build the preset database according to the determined database architecture type, improving the efficiency of modifying the database architecture configuration. This solves the technical problem in related technologies where manual modification of database architecture configuration is inefficient and cannot build the database required for the business in a timely manner.

[0084] Optionally, the setup device further includes: a first construction module, used to construct a metadata file before determining the attribute name of the current service and the attribute value associated with the attribute name, wherein the metadata file includes at least: a preset service and preset components carried by the preset service, the component types of the preset components include: a master node component, a backup node component, and a sentinel component, the backup node component is a backup component of the master node component, and the sentinel component is used to monitor the component status of the master node component and the backup node component; and a first registration module, used to register the preset service and the preset components carried by the preset service with the preset server based on the metadata file.

[0085] In this embodiment of the invention, a metadata file (such as metainfo.xml, which contains the metadata information of the components) can be constructed first. This metadata file includes at least: a preset service (i.e., Redis) and preset components carried by the preset service. The component types of the preset components include: master node components, slave node components, and sentinel components. The slave node components are backup components of the master node components, and the sentinel components are used to monitor the component status (i.e., the health status of the components) of the master node components and slave node components. The metadata file is used to register the declared service and its components with Ambari (i.e., the preset server) (i.e., registering the preset service and its preset components with the preset server based on the metadata file). The declared service is Redis, and the components are Redis (master node in single-machine and cluster) (i.e., master node component), Redis_slave (sentinel, slave node in cluster) (i.e., slave node component), and Redis_sentinel (sentinel node within a sentinel) (i.e., sentinel component).

[0086] Optionally, the setup device further includes: a first determining module, used to determine the attributes managed by the preset server before determining the attribute name of the current business and the attribute value associated with the attribute name; a first serving module, used to use the attribute name of each attribute as a template symbol, wherein the template symbol is used as a placeholder attribute; a first configuration module, used to configure a default value for each template symbol; and a second building module, used to build a preset configuration template based on the template symbol and the default value associated with the template symbol.

[0087] In this embodiment of the invention, a template (i.e., a preset configuration template, such as Redis.xml and Redis-sentinel.xml, which are configuration file templates for the Redis component and the Redis Sentinel component, respectively) can be configured first. Specifically, the attributes managed by the preset server are determined first, and then the attribute name of each attribute is used as a template symbol (this template symbol is used as a placeholder attribute). Then, a default value is configured for each template symbol. After that, the preset configuration template is constructed based on the template symbol and the default value associated with the template symbol. This preset configuration template is used to declare to Ambari the configuration file templates and configuration attributes that need its help to manage. In the template file, the attributes that need to be managed by Ambari can be declared using template symbols (such as {{attribute name}}), and in the XML file, the property attribute (used to use a method as an attribute) is used to declare the attribute name and the default value of the attribute.

[0088] Optionally, the setup device further includes: a third construction module, used to construct an attribute retrieval function before determining the attribute name of the current business and the attribute value associated with the attribute name, wherein the attribute retrieval function is used to retrieve the attribute name declared in the preset configuration template and the default value associated with the attribute name; a fourth construction module, used to construct a component retrieval function, wherein the component retrieval function is used to retrieve the number of components for each preset component; and a fifth construction module, used to construct a preset attribute configuration file based on the attribute retrieval function and the component retrieval function.

[0089] In this embodiment of the invention, the file name and script control type (such as Python control type) can be declared in the metadata file. A preset attribute configuration file can be constructed using Python control. Specifically, an attribute retrieval function can be constructed first to obtain the attribute names declared in the preset configuration template and the default values ​​associated with those attribute names. A component retrieval function can then be constructed to obtain the number of components for each preset component. Finally, based on the attribute retrieval function and the component retrieval function, the preset attribute configuration file is constructed.

[0090] In this embodiment, the logic in the preset attribute configuration file is as follows: The built-in function `script.get_config` (i.e., the attribute retrieval function) provided by Ambari is used to obtain the attribute names and their values ​​declared in the preset configuration template. Then, a built-in Ambari function (i.e., the component retrieval function) is used to obtain the number of each component in the Ambari cluster. Based on the number, it is determined whether to expand to a cluster or a sentinel model. This number is declared in `Redis.xml` and managed by Ambari, and can be configured through the Ambari web interface. When the number of `Redis_sentinel` and `Redis` exceeds this number, a Redis cluster will be automatically built.

[0091] Optionally, the rendering unit includes: a first query module, used to query template symbols corresponding to attribute names from a preset configuration template based on a preset attribute configuration file; and a first replacement module, used to replace the default values ​​associated with the template symbols with the attribute values ​​associated with the attribute names, thereby completing the rendering of the preset configuration template.

[0092] In this embodiment of the invention, a preset configuration template for each preset component (a preset component is a node in a preset database used to operate business data) can be rendered according to the attribute name and attribute value. (The preset configuration template is a configuration template of pre-configured node attributes; updating the attribute value of the template can generate the current configuration file of the component.) After rendering is completed, the preset component is started. Specifically, the attribute name and its attribute value can be obtained from the preset configuration template through the attribute retrieval function in the preset attribute configuration file. Then, the template symbol corresponding to the attribute name of the current business is queried. The default value associated with the template symbol is replaced with the attribute value associated with the attribute name to complete the rendering of the preset configuration template, generate the configuration file of the preset component, and thus start the preset component.

[0093] Optionally, the second determining unit includes: a first obtaining module, used to obtain the number of components for each preset component based on a preset attribute configuration file; a second determining module, used to determine the database architecture type as a cluster structure when the number of components in the master node component is greater than a first preset threshold; and a third determining module, used to determine the database architecture type as a sentinel structure when the number of components in the sentinel component is greater than a second preset threshold.

[0094] In this embodiment of the invention, the database architecture type adapted to the current business can be determined based on the number of components in each preset component. Specifically, the number of components in each preset component can be obtained through the component acquisition function in the preset attribute configuration file. If the number of components in the master node component is greater than a first preset threshold (e.g., 3, which can be set according to actual conditions), the database architecture type can be determined to be a cluster structure; if the number of components in the sentinel component is greater than a second preset threshold (e.g., 3, which can be set according to actual conditions), the database architecture type can be determined to be a sentinel structure.

[0095] Optionally, the setup unit includes: a first creation module, used to control the current node to create a database cluster of a preset database when the current node is the master node and the first node, wherein the first node is the first server in the preset database; a first joining module, used to control other nodes to join the database cluster, wherein when other nodes are backup nodes, it controls other nodes to join the database cluster as backup nodes; and a first allocation module, used to allocate preset hash slots to all nodes in the database cluster to complete the setup of the preset database, wherein the preset hash slots are used to locate the node to which the business data belongs.

[0096] In this embodiment of the invention, a preset database can be built according to the database architecture type to store the business data of the current business according to the current database architecture type. Specifically, it can be determined whether the node role is master (i.e., primary node) and whether it is the first node. If so, a cluster is created as the master through the command line (such as the Redis-cli command line). (i.e., when the current node is the primary node and the first node, the current node is controlled to create a database cluster of the preset database, and the first node is the first server in the preset database.) Other nodes in the configuration file will automatically join the cluster. If the role of other nodes is slave (i.e., backup node), they will join the cluster as slaves. (i.e., control other nodes to join the database cluster, and when other nodes are backup nodes, control other nodes to join the database cluster as backup nodes.) Redis will automatically allocate hash slots (i.e., preset hash slots) to the nodes in the cluster. (i.e., allocate preset hash slots to all nodes in the database cluster. These preset hash slots are used to locate the node to which the business data belongs) to complete the construction of the preset database.

[0097] The aforementioned construction device may also include a processor and a memory. The first determining unit 40, rendering unit 41, second determining unit 42, construction unit 43, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0098] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and by adjusting kernel parameters, a preset database can be built based on the database architecture type.

[0099] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0100] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: determining the attribute name of the current business and the attribute value associated with the attribute name; rendering a preset configuration template for each preset component based on the attribute name and attribute value; and, upon completion of rendering, starting the preset component; determining the database architecture type adapted to the current business based on the number of components in each preset component; and building a preset database based on the database architecture type.

[0101] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the above-described method for building a Redis database architecture based on Ambari.

[0102] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the above-described method for building a Redis database architecture based on Ambari.

[0103] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) according to an embodiment of the present invention for building a Redis database architecture based on Ambari. Figure 5 As shown, an electronic device may include one or more ( Figure 5 The processor 502 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 504 for storing data may also be included. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0104] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0105] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0107] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0109] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0110] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for building a Redis database architecture based on Ambari, characterized in that, include: Determine the attribute name of the current business and the attribute value associated with the attribute name; Render a preset configuration template for each preset component based on the attribute name and the attribute value, and start the preset component when rendering is complete. The preset component is a node in the preset database, and the preset configuration template is a configuration template for pre-configured node attributes. Based on the number of components in each preset component, determine the database architecture type that is compatible with the current business. Based on the database architecture type, the preset database is built, wherein the preset database is used to store the business data of the current business in accordance with the database architecture type; Before determining the attribute name of the current business and the attribute value associated with the attribute name, the process also includes: Construct a metadata file, wherein the metadata file includes at least: a preset service and a preset component carried by the preset service, wherein the component types of the preset component include: a primary node component, a backup node component, and a sentinel component, wherein the backup node component is a backup component of the primary node component, and the sentinel component is used to monitor the component status of the primary node component and the backup node component; Based on the metadata file, the preset service and the preset components carried by the preset service are registered with the preset server.

2. The construction method according to claim 1, characterized in that, Before determining the attribute name of the current business and the attribute value associated with the attribute name, the process also includes: Determine the attributes managed by the preset server; The attribute name of each attribute is used as a template symbol, wherein the template symbol is used as a placeholder for the attribute; Configure default values ​​for each template symbol; The preset configuration template is constructed based on the template symbol and the default value associated with the template symbol.

3. The construction method according to claim 2, characterized in that, Before determining the attribute name of the current business and the attribute value associated with the attribute name, the process also includes: Construct an attribute retrieval function, wherein the attribute retrieval function is used to retrieve the attribute name declared in the preset configuration template and the default value associated with the attribute name; Construct a component acquisition function, wherein the component acquisition function is used to acquire the number of components for each preset component; Based on the attribute retrieval function and the component retrieval function, a preset attribute configuration file is constructed.

4. The construction method according to claim 3, characterized in that, The steps of rendering the preset configuration template for each preset component based on the attribute name and the attribute value include: Based on the preset attribute configuration file, query the template symbol corresponding to the attribute name from the preset configuration template; The default value associated with the template symbol is replaced with the attribute value associated with the attribute name, thus completing the rendering of the preset configuration template.

5. The construction method according to claim 3, characterized in that, The step of determining the database architecture type suitable for the current business based on the number of components in each of the preset components includes: Based on the preset attribute configuration file, obtain the number of components for each preset component; If the number of components in the master node component is greater than a first preset threshold, the database architecture type is determined to be a cluster structure. If the number of components in the sentinel component is greater than a second preset threshold, the database architecture type is determined to be a sentinel structure.

6. The construction method according to claim 1, characterized in that, Based on the aforementioned database architecture type, the steps for building a preset database include: When the current node is the master node and the first node, control the current node to create the database cluster of the preset database, wherein the first node is the first server in the preset database; Control other nodes to join the database cluster, wherein, when the other nodes are standby nodes, control the other nodes to join the database cluster as standby nodes; The preset hash slots are assigned to all nodes in the database cluster to complete the construction of the preset database, wherein the preset hash slots are used to locate the node to which the business data belongs.

7. A device for building a Redis database architecture based on Ambari, characterized in that, include: The first determining unit is used to determine the attribute name of the current service and the attribute value associated with the attribute name; A rendering unit is used to render a preset configuration template for each preset component based on the attribute name and the attribute value, and to start the preset component when the rendering is complete. The preset component is a node in a preset database, and the preset configuration template is a configuration template for pre-configured node attributes. The second determining unit is used to determine the database architecture type that is compatible with the current business based on the number of components in each preset component; A setup unit is used to build the preset database based on the database architecture type, wherein the preset database is used to store the business data of the current business in accordance with the database architecture type; The device further includes: a first construction module, configured to construct a metadata file before determining the attribute name of the current service and the attribute value associated with the attribute name, wherein the metadata file includes at least: a preset service and preset components carried by the preset service, the component types of the preset components include: a master node component, a backup node component, and a sentinel component, the backup node component is a backup component of the master node component, and the sentinel component is used to monitor the component status of the master node component and the backup node component; and a first registration module, configured to register the preset service and the preset components carried by the preset service with the preset server based on the metadata file.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the method for building a Redis database architecture based on Ambari as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method for building a Redis database architecture based on Ambari as described in any one of claims 1 to 6.

Images