HBase table configuration method, electronic device, and computer-readable storage medium

Abstract

The present application discloses an HBase table configuration method, an electronic device, and a computer-readable storage medium, the method comprising: obtaining an association relationship between a pre-partition value and a variable value of an HBase table, and obtaining the type of a target object to be stored in the HBase table; wherein the variable value is at least related to a database storing the HBase table; generating a script corresponding to the HBase table based on the association relationship and the type of the target object; using the script to obtain the type of the target object to be matched, and the variable value to be matched of the HBase table, obtaining the target pre-partition value configured by the script for the HBase table, and obtaining the configured HBase table. The above scheme can improve the configuration efficiency of the HBase table.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an HBase table configuration method, an electronic device, and a computer-readable storage medium. Background Art

[0002] The pre-partition setting of HBase table can evenly distribute data to different partitions in the table, which can avoid the problem of imbalance caused by excessive data in some partitions. In addition, the pre-partition setting of HBase table can also enable data to be processed in parallel between multiple partitions, improving the throughput and concurrent processing capabilities of the processing terminal. However, HBase tables are often deployed in dozens or even hundreds of different scenarios. If the pre-partition value is set manually, the configuration efficiency of the HBase table will be very low. Therefore, how to improve the configuration efficiency of HBase table has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide an HBase table configuration method, an electronic device and a computer-readable storage medium, which can improve the configuration efficiency of the HBase table.

[0004] To solve the above technical problems, the first aspect of the present application provides an HBase table configuration method, comprising: obtaining an association relationship between a pre-partition value and a variable value of an HBase table, and obtaining the type of a target object to be stored in the HBase table; wherein the variable value is at least related to a database storing the HBase table; based on the association relationship and the type of the target object, generating a script corresponding to the HBase table; using the script to obtain the type of the target object to be matched, and the variable value to be matched of the HBase table, obtaining the target pre-partition value configured by the script for the HBase table, and obtaining the configured HBase table.

[0005] To solve the above technical problem, the second aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the method described in the first aspect above.

[0007] The above scheme obtains the association relationship between the pre-partition value and the variable value of the HBase table, and obtains the type of the corresponding target object that needs to be stored in the HBase table. Based on the association relationship between the pre-partition value and the variable value of the HBase table and the type of the target object, a script corresponding to the HBase table is generated, and the script is used to obtain the type of the target object to be matched, and the variable value to be matched of the HBase table to obtain the target pre-partition value configured for HBase by the script, thereby obtaining the configured HBase table. The pre-partition value of the HBase table can be quickly generated through the script and the replaceable variable value, thereby improving the configuration efficiency of the HBase table. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 This is a flowchart of an implementation method of the HBase table configuration method of the present application;

[0010] Figure 2 It is a flowchart of another implementation method of the HBase table configuration method of the present application;

[0011] Figure 3 It is a structural diagram of an implementation method of a pre-partition value script of an HBase table of the present application;

[0012] Figure 4 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0013] Figure 5 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0015] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0016] The HBase table configuration method provided in the present application is used to configure the pre-partition value of the HBase table. The corresponding execution subject is a processing terminal that can call the HBase table. The processing terminal includes but is not limited to a data middle-end system. The configured HBase table can be deployed to the corresponding end to be configured.

[0017] See also Figure 1 , Figure 1 This is a flow chart of an implementation method of the HBase table configuration method of the present application. The method includes:

[0018] S101: Acquire the association between the pre-partition value and the variable value of the HBase table, and acquire the type of the target object to be stored in the HBase table; wherein the variable value is at least related to the database storing the HBase table.

[0019] Specifically, the association relationship between the pre-partition value and the variable value of the HBase table is obtained, and the type of the corresponding target object that needs to be stored in the HBase table is obtained, wherein the variable value is at least related to the database storing the HBase table.

[0020] It should be noted that the target object type includes at least one, and each target object type is set with at least one association relationship.

[0021] In one application mode, the association relationship between the pre-partition value and the variable value of the HBase table is set by the usage object, the association relationship between the pre-partition value and the variable value of the HBase table set by the usage object is obtained, and the type of the target object to be stored in the HBase table is obtained.

[0022] In another application method, the pre-partition value corresponds to a preset number, and the association between the pre-partition value and the variable value of the HBase table is automatically set based on the preset number. The preset number of pre-partition values ​​and the variable value are obtained, the association between the pre-partition value and the variable value is generated, and the type of the target object to be stored in the HBase table is obtained.

[0023] It should be noted that the type of the target object is associated with the scene of the end to be configured. For example, in a road scene, the type of the target object includes at least motor vehicles and non-motor vehicles.

[0024] In some application scenarios, the variable value includes the number of servers corresponding to the database storing the HBase table.

[0025] In some application scenarios, the variable value also includes the number of corresponding target objects stored in the database when processing actual business.

[0026] In other application scenarios, the variable value is also related to other actual businesses corresponding to the HBase table, and this application does not impose any specific restrictions on this.

[0027] S102: Generate a script corresponding to the HBase table based on the association relationship and the type of the target object.

[0028] Specifically, based on the association between the pre-partition value of the HBase table and the variable value and the type of the target object, a script corresponding to the HBase table is generated.

[0029] In one application mode, the type of the association relationship and the target object is input into a script template pre-defined by the object, and a script corresponding to the HBase table is generated.

[0030] In another application mode, the type of the association relationship and the target object is input into an automatic generation tool, and a script corresponding to the HBase table is automatically generated using a specific script template.

[0031] It should be noted that the type of target object can be independently selected in the generated script according to the actual business needs of the end to be configured.

[0032] Optionally, the association relationship corresponds to an association source code, the association source code and the type of the target object are input into a script template, and the script template is used to generate a script corresponding to the HBase table.

[0033] S103: Using the script to obtain the to-be-matched type of the target object and the to-be-matched variable value of the HBase table, obtain the target pre-partition value configured by the script for the HBase table, and obtain the configured HBase table.

[0034] Specifically, the script is used to obtain the to-be-matched type of the target object and the to-be-matched variable value of the HBase table, so as to obtain the target pre-partition value configured for HBase by the script, thereby obtaining the configured HBase table.

[0035] In one application mode, a script is used to obtain the type of target object to be matched, and to obtain the variable value directly input by the object as the variable value to be matched of the HBase table, so as to obtain the target pre-partition value configured by the script for HBase, thereby obtaining the configured HBase table.

[0036] In another application method, a script is used to obtain the type of target object to be matched, and a parameter is pre-set on the script. When the object is used to select the parameter on the script, the script automatically generates a data acquisition instruction, and obtains the variable value to be matched of the HBase table according to the data acquisition instruction to obtain the target pre-partition value configured by the script for HBase, thereby obtaining the configured HBase table.

[0037] Optionally, the configured HBase table can be deployed to multiple different terminals to be configured.

[0038] The above scheme obtains the association relationship between the pre-partition value and the variable value of the HBase table, and obtains the type of the corresponding target object that needs to be stored in the HBase table. Based on the association relationship between the pre-partition value and the variable value of the HBase table and the type of the target object, a script corresponding to the HBase table is generated, and the script is used to obtain the type of the target object to be matched, and the variable value to be matched of the HBase table to obtain the target pre-partition value configured for HBase by the script, thereby obtaining the configured HBase table. The pre-partition value of the HBase table can be quickly generated through the script and the replaceable variable value, thereby improving the configuration efficiency of the HBase table.

[0039] See also Figure 2 , Figure 2 It is a flowchart of another implementation method of the HBase table configuration method of the present application. The HBase table configuration method includes:

[0040] S201: Acquire the association between the pre-partition value and the variable value of the HBase table, and acquire the type of the target object to be stored in the HBase table; wherein the variable value is at least related to the database storing the HBase table.

[0041] Specifically, the association relationship between the pre-partition value and the variable value of the HBase table is obtained, and the type of the corresponding target object that needs to be stored in the HBase table is obtained.

[0042] In one application, the step of obtaining the association relationship between the pre-partition value of the HBase table and the variable value in step S201 includes: determining the association relationship between the pre-partition value of the HBase table preset by the object and the variable value; and obtaining the relationship source code matching the association relationship.

[0043] Specifically, the object can predefine the association relationship between the pre-partition value of the HBase table and the variable value, and compile the association relationship into a matching relational source code to obtain the compiled relational source code.

[0044] In an application scenario, the pre-partition value of the HBase table is related to the number of servers corresponding to the database storing the HBase table. The user object can pre-define the association between the pre-partition value of the HBase table and the number of servers corresponding to the database storing the HBase table, and write it as a matching relationship source code.

[0045] In a specific application scenario, the data middle-office system supports the running of Java scripts, and the association between the pre-partition value of the HBase table pre-defined by the object and the number of servers corresponding to the database storing the HBase table can be: when the number of servers is less than 5, the pre-partition value of the HBase table is [10, 20, 30], when the number of servers is between 5-10, the pre-partition value of the HBase table is [100, 200, 300], when the number of servers is greater than 10, the pre-partition value of the HBase table is [a, b, c], where the pre-partition values ​​are all strings. At this time, the above association relationship is written as matching Java code. In other implementation scenarios, the data middle-office system can support the running of other programming languages, and the above association relationship can also be written as code in other programming languages. This application does not make specific restrictions on this.

[0046] In another application method, the pre-partition value of the HBase table corresponds to a preset number, and the step of obtaining the association relationship between the pre-partition value of the HBase table and the variable value includes: based on the preset number, obtaining the association relationship between the pre-partition value of the HBase table and the variable value; generating a relational source code matching the association relationship.

[0047] Specifically, the association relationship between the pre-partition value and the variable value can be automatically set, the association relationship between the pre-partition value and the variable value of the HBase table can be obtained based on a preset number, and a relational source code matching the association relationship can be automatically generated.

[0048] Optionally, you can use algorithms such as HexStringSplit or DecimalStringSplit and enter a preset number of pre-partitions to automatically obtain the association between the pre-partition values ​​of the HBase table and the variable values, and automatically generate a relationship source code that matches the association.

[0049] S202: Generate a script corresponding to the HBase table based on the association relationship and the type of the target object.

[0050] Specifically, based on the association between the pre-partition value of the HBase table and the variable value and the type of the target object, a script corresponding to the HBase table is generated.

[0051] In one application, step S202 specifically includes: obtaining data to be compiled based on the relationship source code corresponding to the association relationship and the type of the target object; compiling the data to be compiled to generate a script corresponding to the HBase table; wherein the script includes all association relationships corresponding to the type of the target object.

[0052] Specifically, based on the relationship source code corresponding to the association relationship and the type of the target object, the corresponding data to be compiled is obtained, and the data to be compiled is compiled to generate a script corresponding to the HBase table, wherein the script includes all association relationships corresponding to the type of the target object, thereby being able to meet various businesses in different scenarios and improving ease of use.

[0053] S203: Using the script to obtain the to-be-matched type of the target object and the to-be-matched variable value of the HBase table, obtain the target pre-partition value configured by the script for the HBase table, and obtain the configured HBase table.

[0054] Specifically, the script is used to obtain the to-be-matched type of the target object and the to-be-matched variable value of the HBase table, so as to obtain the target pre-partition value configured for HBase by the script, thereby obtaining the configured HBase table.

[0055] In one application mode, step S203 specifically includes: using a script to obtain a to-be-matched type of a target object, taking all association relationships corresponding to the to-be-matched type as candidate relationships, and obtaining a target relationship obtained from the candidate relationships; using a script to obtain a to-be-matched variable value of an HBase table; wherein the to-be-matched type, the target relationship, and the to-be-matched variable value constitute configuration information of the HBase table; obtaining a target pre-partition value configured for the HBase table by the script based on the configuration information, and obtaining a configured HBase table; wherein the configured HBase table is used to be configured to the end to be configured.

[0056] Specifically, the data middle-office system can automatically parse out the type of target object contained in the script, and each target object type corresponds to multiple association relationships. After using the script to obtain the type of target object to be matched, it continues to parse out all the corresponding association relationships in the type to be matched as candidate relationships, and obtains the target relationship confirmed from the candidate relationships. The data middle-office system has various built-in variables, which can be freely selected according to actual business. After using the script to obtain the value of the variable to be matched in the HBase table, the configuration information of the HBase table is composed of the type to be matched, the target relationship and the value of the variable to be matched.

[0057] In one application scenario, see Figure 3 , Figure 3It is a structural diagram of an implementation method of the pre-partition value script of the HBase table of the present application. The script includes the type to be matched, candidate relationships and parameter content. The data middle-end system can use the script to obtain the type to be matched of the target object, and take all corresponding association relationships as candidate relationships. After determining the target relationship from the candidate relationships, the variable value to be matched is entered in the parameter content, thereby forming the configuration information of the HBase table.

[0058] Furthermore, the script can generate different pre-partition values ​​based on different configuration information, so that the target pre-partition value to be configured can be selected according to different terminals to be configured that need to be deployed, which has great use value in the actual deployment process.

[0059] In one implementation scenario, the step of using a script to obtain the variable value to be matched of the HBase table specifically includes: in response to obtaining the number of servers corresponding to the database input from the script, using the number of servers as the variable value to be matched of the HBase table; in response to obtaining a preset configuration value, using a script to send a data acquisition instruction to the end to be configured, and obtaining the number of servers sent by the end to be configured in response to the data acquisition instruction.

[0060] Specifically, when the number of servers on the end to be configured is known, the number of servers on the end to be configured is obtained from the script and used as the variable value to be matched in the HBase table; when the number of servers on the end to be configured is unknown, after obtaining the preset configuration value input from the script, the script is used to send the corresponding data acquisition instruction to the end to be configured, and the end to be configured responds to the data acquisition instruction and sends the corresponding number of servers to the data middle platform system.

[0061] In another implementation scenario, the end to be configured includes the business corresponding to the target object to be processed, and the variable value also includes the number of target objects corresponding to the business. At this time, the step of using the script to obtain the variable value to be matched in the HBase table specifically includes: using the script to obtain the number of target objects corresponding to the business.

[0062] Specifically, the end to be configured includes services corresponding to various types of target objects that need to be processed. Therefore, the variable value also includes the number of target objects corresponding to the services. The number of target objects corresponding to the services that need to be processed can be obtained using a script.

[0063] In a specific implementation scenario, the number of target objects corresponding to the business to be processed that is directly input from the script using the object is obtained.

[0064] In another specific implementation scenario, a query tool is integrated into the script. After obtaining the query instruction input by the user on the query tool, the query tool on the script is used to obtain the number of target objects corresponding to the business to be processed based on the query instruction.

[0065] In summary, this application does not limit how to use scripts to obtain the number of target objects corresponding to the business to be processed.

[0066] S204: compress the configured HBase table and the configuration information to obtain a package to be configured, decompress the package to be configured to the corresponding end to be configured, and deploy the configured HBase table on the end to be configured based on the configuration information.

[0067] Specifically, the configured HBase table and configuration information are compressed to obtain the package to be configured. The content of the package to be configured includes the name of the HBase table, description type and other field information, and the configuration information consisting of the type to be matched, the target relationship and the variable value to be matched. In actual use, different packages to be configured can be quickly decompressed to the corresponding end to be configured according to business needs, and the configured HBase table can be deployed on the end to be configured based on the configuration information, which has strong use value in the actual deployment process.

[0068] Optionally, the package to be configured may be a zip compressed package or a rar compressed package, and this application does not impose any specific limitation on this.

[0069] See also Figure 4 , Figure 4 4 is a structural diagram of an embodiment of an electronic device of the present application, wherein the electronic device 40 comprises a memory 401 and a processor 402 coupled to each other, wherein the memory 401 stores program data (not shown), and the processor 402 calls the program data to implement the method in any of the above embodiments, and the description of the relevant contents can be found in the detailed description of the above method embodiments, which will not be repeated here. Specifically, the electronic device 40 includes but is not limited to: a desktop computer, a laptop computer, a tablet computer, a server, etc., which are not limited here. In addition, the processor 402 can also be referred to as a CPU (Center Processing Unit). The processor 402 may be an integrated circuit chip with signal processing capabilities. The processor 402 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 402 may be implemented by an integrated circuit chip.

[0070] See also Figure 5 , Figure 5 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 50 stores program data 500. When the program data 500 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0071] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0072] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0073] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., and other media that can store program codes.

[0074] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for configuring an HBase table, characterized in that: include: Obtaining an association relationship between a pre-partition value and a variable value of an HBase table, and obtaining a type of a target object to be stored in the HBase table; wherein the variable value is at least related to a database storing the HBase table, the type of the target object includes at least one, each type of the target object is provided with at least one association relationship, and the type of the target object is associated with a scenario of the end to be configured; Based on the association relationship and the type of the target object, generate a script corresponding to the HBase table; The script is used to obtain the to-be-matched type of the target object and the to-be-matched variable value of the HBase table, to obtain the target pre-partition value configured by the script for the HBase table, and to obtain the configured HBase table.

2. The method according to claim 1, characterized in that The obtaining of the association between the pre-partition value of the HBase table and the variable value includes: Determine the association relationship between the pre-partition value of the HBase table preset by the object and the variable value; Obtain a relationship source code that matches the association relationship.

3. The method according to claim 1, characterized in that The pre-partition value of the HBase table corresponds to a preset number; The obtaining of the association between the pre-partition value of the HBase table and the variable value includes: Based on the preset number, obtaining an association relationship between the pre-partition value of the HBase table and the variable value; Generate a relationship source code that matches the association relationship.

4. The method according to claim 2 or 3, characterized in that: The step of generating a script corresponding to the HBase table based on the association relationship and the type of the target object includes: Obtaining data to be compiled based on the relationship source code corresponding to the association relationship and the type of the target object; The data to be compiled is compiled to generate a script corresponding to the HBase table; wherein the script includes all association relationships corresponding to the type of the target object.

5. The method according to claim 4, characterized in that The method of using the script to obtain the to-be-matched type of the target object and the to-be-matched variable value of the HBase table, obtaining the target pre-partition value configured by the script for the HBase table, and obtaining the configured HBase table includes: Using the script to obtain the to-be-matched type of the target object, taking all the association relationships corresponding to the to-be-matched type as candidate relationships, and obtaining the target relationship obtained from the candidate relationships; Using the script to obtain the variable value to be matched of the HBase table; wherein the type to be matched, the target relationship and the variable value to be matched constitute the configuration information of the HBase table; The target pre-partition value configured by the script for the HBase table based on the configuration information is obtained to obtain the configured HBase table; wherein the configured HBase table is used to be configured to the end to be configured.

6. The method according to claim 5, characterized in that The using the script to obtain the variable value to be matched in the HBase table includes: In response to obtaining the number of servers corresponding to the database input from the script, using the number of servers as a variable value to be matched in the HBase table; In response to obtaining the preset configuration value, the script is used to send a data acquisition instruction to the end to be configured, and the number of the servers sent by the end to be configured in response to the data acquisition instruction is obtained.

7. The method according to claim 5, characterized in that The end to be configured includes the service corresponding to the target object to be processed, and the variable value also includes the number of the target objects corresponding to the service; The using the script to obtain the variable value to be matched in the HBase table includes: The script is used to obtain the number of the target objects corresponding to the business.

8. The method according to claim 5, characterized in that The method of using the script to obtain the to-be-matched type of the target object and the to-be-matched variable value of the HBase table, obtaining the target pre-partition value configured by the script for the HBase table, and obtaining the configured HBase table further includes: The configured HBase table and the configuration information are compressed to obtain a package to be configured, the package to be configured is decompressed to the corresponding end to be configured, and the configured HBase table is deployed on the end to be configured based on the configuration information.

9. An electronic device, characterized in that: The method comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the method described in any one of claims 1 to 8.

Images