Method, device, equipment and product for automatically binding hardware resources to database

Abstract

The invention discloses a method, a device, equipment and a product for automatically binding a database with hardware resources, and the method comprises the following steps: analyzing a first installation configuration file by using a detection tool, and generating recommended binding configuration of a database instance and numa hardware resources; when each database instance is started, a configuration file including numa hardware resource binding parameters is read, and the corresponding numa hardware resources are automatically bound. According to the method for automatically binding the database with the hardware resources, a plurality of database instances can be deployed on a high-configuration server of an NUMA architecture; in addition, each database instance is automatically bound with numa hardware resources during installation, the numa hardware resources on one server can be fully used, each database instance corresponds to the bound numa hardware resources, the problem that the database instances frequently access memory resources across numa nodes is solved, and the operation performance of the database is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of databases, and in particular relates to a method, device, equipment and product for automatically binding hardware resources to a database. Background Art

[0002] When the database is deployed on a high-configuration server with NUMA architecture, only one database instance is often deployed on a server, which makes it impossible to effectively utilize the server hardware resources. For example, when the number of logical cores of the server CPU is greater than 40, the performance of a single task on a single database instance cannot be linearly improved as the number of logical cores of the CPU increases. At the same time, when the server adopts the NUMA architecture and has multiple NUMA nodes, a single database instance often frequently accesses memory resources across NUMA nodes, which also affects the performance of the database. Summary of the invention

[0003] In view of this, the present invention aims to overcome the defects in the prior art and proposes a method, device, equipment and product for automatically binding hardware resources to a database.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows: In a first aspect, the present invention discloses a method for automatically binding a database to hardware resources, comprising: After writing the IP addresses of all database instances into the first installation configuration file of the database, use the detection tool to parse the first installation configuration file to generate the recommended binding configuration of the database instance and the numa hardware resources; Writing the recommended binding configuration or the recommended binding configuration modified as needed into the first installation configuration file to obtain a second installation configuration file; The database is installed based on the second installation configuration file, and a configuration file including numa hardware resource binding parameters is provided in the configuration file directory of each database instance; When each database instance is started, it reads the configuration file including the NUMA hardware resource binding parameters and automatically binds the corresponding NUMA hardware resources.

[0005] In one embodiment of the present invention, the recommended binding configuration or the recommended binding configuration modified as needed is written into the first installation configuration file to obtain the second installation configuration file, including: using a detection tool to detect the second installation configuration file to determine whether the written recommended binding configuration is binding compliant.

[0006] In one embodiment of the present invention, the method also includes: when installing a new data server with expanded capacity, obtaining a second installation configuration file, writing the NUMA hardware resource binding parameters corresponding to the database instance in the new data server into the second installation configuration file, obtaining a third installation configuration file, and installing and expanding the new data server based on the third installation configuration file.

[0007] In one embodiment of the present invention, before installing and expanding the new data server based on the third installation configuration file, the method further includes: using a detection tool to detect the third installation configuration file to determine whether the written numa hardware resource binding parameters are binding compliant.

[0008] In one embodiment of the present invention, when each database instance is started, a configuration file including NUMA hardware resource binding parameters is read, and after automatically binding the corresponding NUMA hardware resources, it also includes: obtaining the configuration file including NUMA hardware resource binding parameters of each database instance, and determining that the NUMA hardware resource binding parameters are the same as the binding configuration in the second installation configuration file.

[0009] In one embodiment of the present invention, numa hardware resources include: CPU resources and memory resources.

[0010] In one embodiment of the present invention, the recommended binding configuration is that the numa hardware resources on a data server are evenly distributed and bound to several database instances on the data server.

[0011] In a second aspect, the present invention discloses a device for automatically binding a database to hardware resources, the device comprising: A recommended binding configuration module is used to write the IP addresses of all database instances into a first installation configuration file of the database, and then parse the first installation configuration file using a detection tool to generate a recommended binding configuration between the database instance and the numa hardware resource; A configuration file module, used for writing the recommended binding configuration or the recommended binding configuration modified as needed into the first installation configuration file to obtain a second installation configuration file; An installation module is used to install the database based on the second installation configuration file, and a configuration file including numa hardware resource binding parameters is provided in the configuration file directory of each database instance; The binding module is used to read the configuration file including the numa hardware resource binding parameters when each database instance is started, and automatically bind the corresponding numa hardware resources.

[0012] In a third aspect, the present invention discloses an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0013] In a fourth aspect, a computer program product is disclosed, comprising a computer program, which implements the above method when executed by a processor.

[0014] Compared with the prior art, the present invention has the following advantages: The present invention discloses a method, device, equipment and product for automatically binding hardware resources of a database, comprising: using a detection tool to parse a first installation configuration file, generating a recommended binding configuration of a database instance and a NUMA hardware resource; writing the recommended binding configuration or the recommended binding configuration modified as needed into the first installation configuration file to obtain a second installation configuration file; installing the database based on the second installation configuration file, wherein a configuration file directory of each database instance is provided with a configuration file including NUMA hardware resource binding parameters; when each database instance is started, reading the configuration file including NUMA hardware resource binding parameters, and automatically binding the corresponding NUMA hardware resource. The present invention discloses a method, device, equipment and product for automatically binding hardware resources of a database, wherein multiple database instances can be deployed on a high-configuration server of a NUMA architecture, and each database instance can be automatically bound to a NUMA hardware resource during installation, so that the NUMA hardware resources on a server can be fully used, the problem that the database instance frequently accesses memory resources across NUMA nodes can be improved, and the operation performance of the database can be greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0016] In the attached picture: Figure 1 A schematic diagram of an application scenario of a method for automatically binding hardware resources to a database according to an embodiment of the present invention; Figure 2 A schematic diagram of a method for automatically binding a database to hardware resources according to an embodiment of the present invention; Figure 3 A schematic diagram of a device for automatically binding a database to hardware resources according to an embodiment of the present invention; Figure 4 A schematic diagram of an electronic device for automatically binding hardware resources to a database according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0018] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0019] In the description of the present invention, it should be further explained that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0020] When the database is deployed on a high-configuration server with NUMA architecture, a single task on a single database instance cannot achieve linear performance improvement as the number of CPU logical cores increases. At the same time, when the server adopts NUMA architecture and has multiple NUMA nodes, a single database instance often frequently accesses memory resources across NUMA nodes, which also affects the performance of the database.

[0021] The present invention discloses a method, device, equipment and product for automatically binding hardware resources to a database, and the application scenarios are as follows: Figure 1 As shown, by using a detection tool to parse the first installation configuration file, a recommended binding configuration between the database instance and the numa hardware resource is generated; the recommended binding configuration or the recommended binding configuration modified as needed is written into the first installation configuration file; the database is installed based on the second installation configuration file; when each database instance is started, the configuration file including the numa hardware resource binding parameters is read to automatically bind the corresponding numa hardware resource. The method, device, equipment and product for automatically binding hardware resources to a database disclosed in the present invention can realize automatic binding of numa hardware resources to each database instance during installation, can make full use of the numa hardware resources on a server, improve the problem that the database instance frequently accesses memory resources across numa nodes, and greatly improve the operation performance of the database.

[0022] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] In one embodiment disclosed in the present invention, Figure 2 As shown, a method for automatically binding a database to hardware resources includes: Step S201, after writing the IP addresses of all database instances into the first installation configuration file of the database, the first installation configuration file is parsed using a detection tool to generate a recommended binding configuration of the database instance and the numa hardware resource; In this embodiment, numa hardware resources include: CPU resources and memory resources.

[0024] Exemplarily, the CPU resource may be a number of numbered CPU logic cores.

[0025] In this embodiment, the detection tool is the getNumaInfo.py script file, and the first installation configuration file is the demo.options file, both of which are provided by the GBase 8a MPP Cluster database software. The function of the getNumaInfo.py script file is to detect the demo.options file, obtain the numa hardware resource status, and generate a recommended binding configuration between the database instance and the numa hardware resources; Exemplarily, the command for parsing the first installation configuration file using the detection tool is as follows: python getNumaInfo.py --silent=demo.options --showNumaSuggest; The recommended binding configuration is to evenly distribute the NUMA hardware resources on a data server and bind them to several database instances on the data server.

[0026] Step S202, writing the recommended binding configuration or the recommended binding configuration modified as needed into the first installation configuration file to obtain a second installation configuration file; In this embodiment, the need to modify the recommended binding configuration means that the recommended binding configuration generated by the detection tool does not satisfy the user. The user can modify the binding configuration between the database instance and the numa hardware resource according to actual needs.

[0027] In this embodiment, writing into the first installation configuration file specifically means writing the recommended binding configuration or the recommended binding configuration modified as needed into the newly added parameters numaCpuBind and numaMemBind of the demo.options file; Step S203: installing the database based on the second installation configuration file, wherein a configuration file including numa hardware resource binding parameters is provided in the configuration file directory of each database instance; In this embodiment, the binding setting of the database instance and the NUMA hardware resources is completed during the installation phase. There is no need to manually modify the script after the database is running to adjust the binding relationship between the database instance and the NUMA hardware resources, which reduces the workload and improves the stability of the entire system.

[0028] In this embodiment, the configuration file directory is the config directory; the configuration file including the numa hardware resource binding parameters is the gbase_8a_numa.cnf file.

[0029] Step S204: When each database instance is started, a configuration file including numa hardware resource binding parameters is read to automatically bind the corresponding numa hardware resources.

[0030] In this embodiment, when starting each database instance, the gcluster_services script first reads the numa hardware resource binding parameters in gbase_8a_numa.cnf when starting the service, and uses the numa hardware resource binding parameters to start the database.

[0031] The present embodiment discloses a method for automatically binding hardware resources to a database, which can realize automatic binding of NUMA hardware resources to each database instance during installation, and can fully utilize the NUMA hardware resources on a server. Each database instance has corresponding bound NUMA hardware resources, which improves the problem of database instances frequently accessing memory resources across NUMA nodes, and greatly improves the operating performance of the database.

[0032] In another embodiment disclosed in the present invention, a recommended binding configuration or a recommended binding configuration modified as needed is written into a first installation configuration file to obtain a second installation configuration file, including: using a detection tool to detect the second installation configuration file to determine whether the written recommended binding configuration is binding compliant.

[0033] Exemplarily, the command for using the detection tool to detect the second installation configuration file is as follows: python getNumaInfo.py --silent=demo.options --checkNumaConfig; In this example, determining whether the written recommended binding configuration is binding compliant includes the following situations: ⑴ The CPU and memory configurations are different. For example, if a database instance is bound to a CPU resource of one numa and a memory resource of another numa at the same time, it is not compliant. (2) Binding a non-existent NUMA node. For example, if a database instance is bound to a non-existent NUMA node, it is not compliant. ⑶The database instance does not correspond to the NUMA node. For example, a database instance is not bound to a NUMA node, which is not compliant; In this embodiment, determining whether the written recommended binding configuration is binding compliant can timely monitor whether the second installation configuration file is correct and valid, discover possible errors in advance, improve the accuracy of automatic database binding of hardware resources, and improve data operation efficiency.

[0034] In one embodiment disclosed in the present invention, the method also includes: when installing a new data server with expanded capacity, obtaining a second installation configuration file, writing the NUMA hardware resource binding parameters corresponding to the database instance in the new data server into the second installation configuration file, obtaining a third installation configuration file, and installing and expanding the new data server based on the third installation configuration file.

[0035] In this embodiment, the scheduling node gcluster and the management node gcware are not bound to hardware resources.

[0036] In one embodiment disclosed in the present invention, before installing and expanding the new data server based on the third installation configuration file, the method further includes: using a detection tool to detect the third installation configuration file to determine whether the written numa hardware resource binding parameters are binding compliant; Whether binding compliance includes: ⑴ The CPU and memory configurations are different. For example, if a database instance is bound to a CPU resource of one numa and a memory resource of another numa at the same time, it is not compliant. (2) Binding a non-existent NUMA node. For example, if a database instance is bound to a non-existent NUMA node, it is not compliant. ⑶The database instance does not correspond to the NUMA node. For example, a database instance is not bound to a NUMA node, which is not compliant; If it is not in compliance, a prompt message will be printed and the system will exit with an error.

[0037] In this embodiment, the installation of the new data server for expansion is based on the second installation configuration file, and the third installation configuration file is obtained for installation, which ensures the consistency of the settings of the new data server and the original data server, and facilitates unified uninstallation and other operations later.

[0038] In one embodiment disclosed in the present invention, when each database instance is started, a configuration file including NUMA hardware resource binding parameters is read, and after automatically binding the corresponding NUMA hardware resources, it also includes: obtaining the configuration file including NUMA hardware resource binding parameters of each database instance, and determining that the NUMA hardware resource binding parameters are the same as the binding configuration in the second installation configuration file.

[0039] In this embodiment, it is determined that the NUMA hardware resource binding parameters are the same as the binding configuration in the second installation configuration file. After the database is running, it can be monitored whether the actual binding status of the database instance and the NUMA hardware resources is consistent with the settings, thereby ensuring the actual execution effect of the method and improving the performance of the database operation.

[0040] In this embodiment, the configuration file including the numa hardware resource binding parameters records the numa hardware resource binding parameters as follows: numactl --cpunodebind=+0,+1 --membind=+0,+1; like Figure 3 As shown, the present invention also discloses a device for automatically binding a database to hardware resources, comprising: The recommended binding configuration module 301 is used to write the IP addresses of all database instances into the first installation configuration file of the database, and then parse the first installation configuration file using a detection tool to generate a recommended binding configuration of the database instance and the numa hardware resource; Configuration file module 302, used for writing the recommended binding configuration or the recommended binding configuration modified as needed into the first installation configuration file to obtain a second installation configuration file; An installation module 303 is used to install the database based on the second installation configuration file, and a configuration file including numa hardware resource binding parameters is provided in the configuration file directory of each database instance; The binding module 304 is used to read the configuration file including the numa hardware resource binding parameters and automatically bind the corresponding numa hardware resources when each database instance is started.

[0041] The present invention also discloses an electronic device, such as Figure 4 As shown, an embodiment is disclosed, which is a block diagram of an electronic device suitable for the above-mentioned method of automatically binding hardware resources to a database.

[0042] The electronic device 40 of this embodiment includes a processor 401, which can perform various appropriate actions and processes according to the program stored in the ROM 402 or the program loaded from the storage part 408 into the RAM 403. The processor 401 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor, etc. The processor 401 may also include an onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0043] In RAM403, various programs and data required for the operation of electronic device 40 are stored. Processor 401, ROM402 and RAM403 are connected to each other through bus 404, and processor 401 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM402 and / or RAM403. It should be noted that the program can also be stored in one or more memories other than ROM402 and RAM403, and processor 401 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in one or more memories.

[0044] According to an embodiment of the present invention, the electronic device 40 may further include an I / O interface 405, which is also connected to the bus 404. The electronic device 40 may further include one or more of the following components connected to the I / O interface 405: an input portion 406 including a keyboard, a mouse, etc.; an output portion 407 including a cathode ray tube, a liquid crystal display, and a speaker; a storage portion 408 including a hard disk, etc.; and a communication portion 409 including a network interface card such as a LAN card, a modem, etc. The communication portion 409 performs communication processing via a network such as the Internet. A drive 4010 is also connected to the I / O interface 405 as needed. A removable medium 4011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 4010 as needed, so that a computer program read therefrom is installed into the storage portion 408 as needed.

[0045] Embodiments of the present invention also include a computer program product.

[0046] The computer program product includes a computer program, which contains program codes for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program codes are used to enable the electronic device to implement the method provided by the embodiment of the present invention.

[0047] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium. The program code included in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0048] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written by any combination of one or more programming languages, and specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages. Programming languages ​​include but are not limited to programming languages ​​such as Java, C++, python, C language or similar. The program code can be executed completely on the user computing device, partially on the user device, partially on the remote computing device, or completely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network or a wide area network, or can be connected to an external computing device.

[0049] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teaching of the present invention, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0050] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination. The scope of the present invention is limited by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for automatically binding a database to hardware resources, characterized in that: include: After writing the IP addresses of all database instances into the first installation configuration file of the database, the first installation configuration file is parsed using a detection tool to generate a recommended binding configuration between the database instance and the numa hardware resource; Writing the recommended binding configuration or the recommended binding configuration modified as needed into the first installation configuration file to obtain a second installation configuration file; The database is installed based on the second installation configuration file, and a configuration file including numa hardware resource binding parameters is provided in the configuration file directory of each database instance; When each database instance is started, the configuration file including the numa hardware resource binding parameters is read to automatically bind the corresponding numa hardware resources.

2. A method for automatically binding hardware resources to a database according to claim 1, characterized in that: Writing the recommended binding configuration or the recommended binding configuration modified as needed into the first installation configuration file to obtain the second installation configuration file includes: using the detection tool to detect the second installation configuration file to determine whether the written recommended binding configuration is binding compliant.

3. A method for automatically binding a database to hardware resources according to claim 1, characterized in that: The method also includes: when installing a new data server with expanded capacity, obtaining the second installation configuration file, writing the NUMA hardware resource binding parameters corresponding to the database instance in the new data server into the second installation configuration file, obtaining a third installation configuration file, and installing and expanding the new data server based on the third installation configuration file.

4. A method for automatically binding a database to hardware resources according to claim 3, characterized in that: Before installing and expanding the new data server based on the third installation configuration file, the method also includes: using the detection tool to detect the third installation configuration file to determine whether the written numa hardware resource binding parameters are bound in compliance.

5. The method for automatically binding a database to hardware resources according to claim 1, characterized in that: When each of the database instances is started, the configuration file including the NUMA hardware resource binding parameters is read, and after automatically binding the corresponding NUMA hardware resources, it also includes: obtaining the configuration file including the NUMA hardware resource binding parameters of each of the database instances, and determining that the NUMA hardware resource binding parameters are the same as the binding configuration in the second installation configuration file.

6. A method for automatically binding a database to hardware resources according to claim 1, characterized in that: The numa hardware resources include: CPU resources and memory resources.

7. The method for automatically binding a database to hardware resources according to claim 1, characterized in that: The recommended binding configuration is that the numa hardware resources on a data server are evenly distributed and bound to a plurality of the database instances on the data server.

8. A device for automatically binding a database to hardware resources, characterized in that: The device comprises: A recommended binding configuration module is used to write the IP addresses of all database instances into a first installation configuration file of the database, and then parse the first installation configuration file using a detection tool to generate a recommended binding configuration between the database instance and the numa hardware resource; A configuration file module, used for writing the recommended binding configuration or the recommended binding configuration modified as needed into the first installation configuration file to obtain a second installation configuration file; An installation module is used to install the database based on the second installation configuration file, and a configuration file directory of each database instance is provided with a configuration file including numa hardware resource binding parameters; The binding module is used to read the configuration file including the numa hardware resource binding parameters when each database instance is started, and automatically bind the corresponding numa hardware resources.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images