Method for realizing HBase dynamic column based on reflection

Abstract

The application discloses a method for realizing HBase dynamic column based on reflection, which comprises the following steps: S1, dynamic column data writing / deleting process; the request body is enhanced, the corresponding information is obtained through reflection, the request object is obtained by assembling the rowKey, each column under the corresponding information cluster is assigned a value, and whether the dynamic value is empty is judged when the value is assigned; if not, the dynamic column is assembled and formed; S2, dynamic column data query process; specified columns and all specified columns are queried, each column value is grouped according to the last label, the return value entity object is assembled, and the client is returned; the technical key points are that the application supports dynamic expansion, does not need to modify the code logic, splices the dynamic value with business meaning behind the static column to form a dynamic column, thereby the data of a certain business type can be processed separately, and the technical problem that the code logic needs to be modified is solved; the operation and maintenance cost is low, and the old API is compatible while no special operation and maintenance personnel is needed.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of HBase dynamic column, and specifically relates to a method for realizing HBase dynamic column based on reflection. BACKGROUND

[0002] At present, HBase is an open source implementation referring to the Google BigTable model, is developed in Java, is non-relational (NoSQL), is KV key value (is also column (cluster) storage from the physical storage), is persistent, is built on the Hadoop distributed file system (HDFS) (and can also be built on other file systems conforming to the interface), is consistent by default (CP), is highly reliable (HBase realizes redundant storage and high reliability of data through the replication mechanism of the underlying HDFS), has high performance (HBase has approximately optimal write performance (can make the I / O utilization rate reach saturation) and relatively outstanding read performance), is an open source distributed database, and supports storage of millions of columns.

[0003] The existing package label marking service needs to use HBase dynamic column technology; one waybill number is associated with one or more labels, such as the current university piece, city piece, problem piece, time limit piece and the like. Hbase supports millions of columns, and has high storage and query performance; but other labels will be added or deleted in the later period, and the code needs to be modified to adapt to the change of the column, secondary development is needed, and it cannot be directly used. The existing solution to the problem is to modify the Hbase native client. Before writing data in the hbase client, the request entity is parsed by using reflection to obtain the hbase entity object elements, and the tableName, family, qualifiers and the like can be obtained. The above-mentioned java reflection mechanism is to know all the attributes and methods of a class in a running state, and to call any method of an object. The function of dynamically obtaining information and dynamically calling the method of the object is called the reflection mechanism of the java language. The Qualifier is combined with the dynamic value of the column to form a new column value, and a new client request is assembled.

[0004] However, the present inventors found at least the following technical problems in the process of implementing the technical solutions in the embodiments of the present application:

[0005] 1. The native client does not support dynamic expansion.

[0006] 2. Dynamically modifying HBase tables requires modifications to the code logic;

[0007] 3. Existing technologies cannot adapt to the rapid iteration of business. HBase supports storing millions of columns, but using dynamic columns in applications will generate a lot of redundant code. Summary of the Invention

[0008] Technical problems to be solved:

[0009] To address the shortcomings of existing technologies, this invention provides a method for implementing dynamic columns in HBase based on reflection, solving the problems mentioned in the background technology. This invention modifies the native client API to support dynamic columns, eliminating the need for operation and maintenance support and simplifying maintenance. By using the method of this invention, the usability, scalability, and adaptability of existing software system functions can be effectively improved.

[0010] Technical solution:

[0011] To achieve the above objectives, the present invention provides the following technical solution:

[0012] A method for implementing dynamic columns in HBase based on reflection includes the following steps:

[0013] S1, Dynamic column data write / delete process;

[0014] The request body is enhanced by retrieving the corresponding information through reflection, assembling the rowKey to obtain the request object, assigning values ​​to each column under the column family in the corresponding information, and checking whether the dynamic value is empty during the assignment. If it is not empty, it is assembled into a dynamic column.

[0015] The rowKey mentioned above represents a business code, such as a courier tracking number. Under one business code, there may be multiple business scenarios, such as university shipments, delivery and split shipments, time-sensitive shipments, etc. Each business scenario will have its own dynamic value and several dynamic columns to store its own data.

[0016] S2, Dynamic Column Data Query Process;

[0017] The function queries a specified column and all specified columns, and assembles the returned value entity object by grouping each column value according to the last label, and returns it to the client.

[0018] To address the specific technical issue that HBase supports storing millions of columns, but the use of dynamic columns in applications generates a large amount of redundant code, this application uses business scenario encoding as dynamic values. By using column names + dynamic values, it can be implemented with a small amount of code, and no code needs to be modified when business scenarios are expanded.

[0019] In view of the technical problem that the code logic needs to be modified for dynamically modifying the hbase table, the dynamic value with business meaning is spliced behind the static column to form a dynamic column, so that the data of a certain business type can be processed separately

[0020] Preferably, in the S1, the corresponding information comprises an hbase table name and a column.

[0021] Preferably, in the S1, when writing, the Hbase template enhances the request body, obtains the corresponding hbase table name, column cluster and column information through reflection, assembles the request object by adding the rowKey, assigns values to each column under the column cluster, and judges whether the dynamic value is empty during the assignment,

[0022] If not empty, the dynamic column is assembled and written into the hbase;

[0023] If empty, the data is stored in the static column.

[0024] Preferably, in the S1, when deleting, the Hbase template enhances the request body, obtains the corresponding hbase table name, column cluster and column information through reflection, assembles the request object by adding the rowKey, assigns values to each column under the column cluster, and judges whether the dynamic value is empty during the assignment,

[0025] If not empty, the dynamic column is assembled and deleted from the hbase;

[0026] If empty, the data is stored in the static column.

[0027] Preferably, in the S2, the request body is enhanced when querying the specified column, the corresponding hbase table name, column cluster and column information are obtained through reflection, the request object is assembled by adding the rowKey, each column under the column cluster is assigned, and it is judged whether the dynamic value is empty during the assignment, if not empty, the dynamic column is assembled;

[0028] According to the table name, column cluster and column, the rowKey queries all field information of the hbase, queries the hbase, assembles the return value entity object, and returns to the client.

[0029] Preferably, in the S2, the request body is enhanced when querying all columns, the corresponding hbase table name and column cluster information are obtained through reflection; the request object is assembled by adding the rowKey;

[0030] According to the table name, column cluster and rowKey, all field information of the hbase is queried, each column value is grouped according to the last label, the return value entity object is assembled, and returned to the client.

[0031] Beneficial effects:

[0032] Firstly, the application supports dynamic expansion, does not need to modify code logic, splices a dynamic value with business meaning to the back of a static column to form a dynamic column, thereby can separately process data of a certain business type to solve the technical problem that code logic needs to be modified.

[0033] Secondly, the application has low operation and maintenance cost and does not need special operation and maintenance personnel to maintain.

[0034] Thirdly, the application is compatible with an old API, and the original client API is modified to support a dynamic column, which is different from the existing scheme in that:

[0035] 1. Software function expansion;

[0036] 2. The existing scheme describes a static column, when HBase needs to store data of multiple different business scenarios with one rowkey and the business scenarios are to be increased subsequently, based on the existing scheme, the code needs to be modified, and the application does not need to modify the code, but can generate a dynamic column to store data of a new business scenario. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a whole process schematic diagram of the application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application. In addition, in the following description, the "up", "down", "left", "right" and the like are consistent with the up, down, left and right of the drawings, and "first", "second" and the like in the following description are for distinguishing, and have no other special meanings.

[0039] Embodiment 1:

[0040] As shown in the figure, a method for realizing a dynamic column of HBase based on reflection includes the following steps: Figure 1

[0041] S1, dynamic column data writing / deletion process;

[0042] ​The request body is enhanced, the corresponding information is obtained through reflection, the rowKey on the assembly obtains the request object, each column under the column cluster in the corresponding information is assigned a value, and it is judged whether the dynamic value is empty during assignment. If not, the dynamic column is assembled; the rowKey above represents a business code, such as a courier waybill number. There can be multiple business scenarios under one business code, such as college pieces, to pieces, and time-effective pieces. Each business scenario will have its own dynamic value and several dynamic columns for storing its own data. The corresponding information includes column cluster, hbase table name and column.

[0043] When writing,

[0044] The Hbase template enhances the request body, obtains the corresponding hbase table name, column cluster and column information through reflection, assembles the rowKey to obtain the request object, assigns a value to each column under the column cluster, and judges whether the dynamic value is empty during assignment,

[0045] If not, the dynamic column is assembled and written into hbase;

[0046] If empty, the data is stored in the static column.

[0047] When deleting,

[0048] The Hbase template enhances the request body, obtains the corresponding hbase table name, column cluster and column information through reflection, assembles the rowKey to obtain the request object, assigns a value to each column under the column cluster, and judges whether the dynamic value is empty during assignment,

[0049] If not, the dynamic column is assembled and deleted from hbase;

[0050] If empty, the data is stored in the static column.

[0051] S2, dynamic column data query process;

[0052] Query specified columns and all columns, and group and return value entity objects according to the last label of each column value, and return them to the client.

[0053] In view of the technical problem that HBase supports storing millions of columns, but using dynamic columns in applications will generate a large amount of redundant code, the application takes business scenario coding as a dynamic value, and through the column name + dynamic value method, a small amount of code can be achieved, and no code needs to be changed when the business scenario is expanded.

[0054] Query specified columns,

[0055] The request body is enhanced, the corresponding hbase table name, column cluster and column information are obtained through reflection, the request object is assembled on the rowKey, each column under the column cluster is assigned, and it is judged whether the dynamic value is empty when assigning, if not empty, the dynamic column is assembled;

[0056] According to the table name, column cluster and rowKey, all field information of hbase is queried, hbase is queried, a return value entity object is assembled, and is returned to the client.

[0057] All columns are queried,

[0058] The request body is enhanced, the corresponding hbase table name and column cluster information are obtained through reflection, and the request object is assembled on the rowKey;

[0059] According to the table name, column cluster and rowKey, all field information of hbase is queried, each column value is grouped according to the last label, a return value entity object is assembled, and is returned to the client.

[0060] The application supports dynamic expansion, does not need to modify the code logic, splices the dynamic value with business meaning to the back of the static column, forms a dynamic column, and thus can process data of a certain business type alone; The application has low operation and maintenance cost, and does not need special operation and maintenance personnel to maintain;

[0061] The application is compatible with the old API, the original client API is modified, and the dynamic column is supported, which is different from the existing scheme.

[0062] 1. Software function expansion;

[0063] 2. The existing scheme describes a static column, when hbase needs to store multiple different business scene data with a rowkey, and the business scene will be increased subsequently, based on the existing scheme, the code needs to be modified, and the application does not need to modify the code, and can generate a dynamic column for storing new business scene data.

[0064] Finally, it should be noted that: obviously, the above embodiments are only examples for clearly illustrating the application, and not the limitation of the implementation mode. For ordinary skilled in the art, on the basis of the above description, other different forms of changes or changes can be made. Here, all the implementation modes are not enumerated. The obvious changes or changes derived therefrom are still within the protection scope of the application.

Claims

1. A method for implementing HBase dynamic column based on reflection, characterized in that, The method comprises the following steps: S1, dynamic column data writing / deleting process; In the S1, the corresponding information comprises a hbase table name and a column. In the S1, when writing, the Hbase template enhances the request body, obtains the corresponding hbase table name, column cluster and column information through reflection, assembles the rowKey to obtain the request object, assigns values to each column under the column cluster, and judges whether the dynamic value is empty during the assignment, if not empty, the dynamic column is assembled and written into the hbase; 2. The method for implementing HBase dynamic column based on reflection according to claim 1, wherein: if empty, the data is stored in the static column.

3. The method for implementing HBase dynamic column based on reflection according to claim 2, characterized in that: In the S1, when deleting, the Hbase template enhances the request body, obtains the corresponding hbase table name, column cluster and column information through reflection, assembles the rowKey to obtain the request object, assigns values to each column under the column cluster, and judges whether the dynamic value is empty during the assignment, if not empty, the dynamic column is assembled and deleted from the hbase; if empty, the data is stored in the static column.

4. The method for implementing HBase dynamic column based on reflection according to claim 2, characterized in that: In the S2, when querying the specified column, the request body is enhanced, the corresponding hbase table name, column cluster and column information are obtained through reflection, the rowKey is assembled to obtain the request object, values are assigned to each column under the column cluster, and it is judged whether the dynamic value is empty during the assignment, if not empty, the dynamic column is assembled; all field information of the hbase is queried according to the table name, column cluster and rowKey, the hbase is queried, the return value entity object is assembled and returned to the client.

5. The method for implementing HBase dynamic column based on reflection according to claim 2, characterized in that: In the S2, when querying all columns, the request body is enhanced, the corresponding hbase table name and column cluster information are obtained through reflection; the rowKey is assembled to obtain the request object; all field information of the hbase is queried according to the table name, column cluster and rowKey, the return value entity object is assembled and returned to the client.

6. The method for implementing HBase dynamic column based on reflection according to claim 2, characterized in that: ​ ​

Images