Data processing method and device, computer equipment and storage medium
Patent Information
- Application Number
- CA3140854
- Authority / Receiving Office
- CA · CA
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-12-01
- Filing Date
- 2021-12-01
- Publication Date
- 2026-08-11
- Estimated Expiration
- 2041-12-01
Abstract
Description
DATA PROCESSING METHOD AND DEVICE, COMPUTER EQUIPMENT AND STORAGE MEDIUM Technical field
[0001] The present invention relates to the field of data processing technologies, in particular, to a method, a device, a computer apparatus and storage medium for data processing. Background
[0002] With the rapid development and prevalence of big data, nowadays enterprises have their own profile systems. For example, the system to tag each individual (such as members), is further used for profile analysis from multiple dimensions for individuals based on tags, greatly improving data usage efficiency. As an example, membership tags provide member analysis for marketing data support. However, the tags are obtained via sparse data sources from data warehouses and summarized into one or a few tables, for downstream systems applications of these highly integrated data. Due to the practical factors of "sparse functions" and "discrete data resource" of the described profile systems, the integration of generated member tags into a table with a large field volume can be complicated and uncontrollable to ensure processing efficiency.
[0003] The currently available relational database tables are stored in the means of row storage or column storage. However, both storage methods read data updates in rows, especially for Hive that fits for massive volume analysis but is not data-updating database friendly. For a table with a large field volume, taking example of a general profile data in the big data business, a table involving thousands of fields significantly complicates the processes.
[0004] Therefore, a new data processing method is in demand to solve the forementioned problem. Summary
[0005] To solve the current technical problems, a method, a device, a computer apparatus and storage medium for data processing are provided in embodiments of the present invention, to overcome the problems in the current technologies.
[0006] To solve the forementioned one or more technical problems, the technical proposals in the present invention include:
[0007] from the first perspective, a data processing method, comprising:
[0008] parsing a primary data table to identify a primary field and a secondary field, and acquiring a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table;
[0009] generating a primary key value based on the described primary field value, generating a secondary key value based on the described secondary field (value), and generating a data value based on the described secondary field value; and
[0010] generating a secondary data table according to the described primary and secondary key values and the described data value and saving the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values.
[0011] In some embodiments, the described method further includes:
[0012] receiving process-pending data, to generate a primary data table based on the described process-pending data and primary rules, wherein the described primary data table contains a primary field and a corresponding primary field value, and a secondary field and a corresponding secondary field value.
[0013] In some embodiments, the described method further includes:
[0014] receiving and parsing the process-pending request, to acquire the type of a data table associated with the described data processing request, wherein the described data table includes a bivariate table and / pr a key-value pair table;
[0015] determining a target data table based on the described data table type, wherein the described target data table contains a primary data table and a secondary data table; and
[0016] processing the data in the described target data table according to the described data processing request.
[0017] In some embodiments, wherein the described data processing request includes a data read request, the described processing of the data in the described target data table according to the described data processing request further includes:
[0018] based on the described data read request, acquiring the target data from the described target data table, and returning the described target data to the data requesting end.
[0019] In some embodiments, wherein the described data processing request further includes a data update request, the described processing of the data in the described target data table according to the described data processing request further includes:
[0020] based on the described data update request, updating the data in the described target data table.
[0021] In some embodiments, the described primary field includes a major key of the described primary data table.
[0022] From the second perspective, a data processing device is provided, comprising:
[0023] a data parsing module, configured to parse a primary data table for identifying a primary field and a secondary field, and acquire a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table;
[0024] a primary processing module, configured to generate a primary key value based on the described primary field value, generate a secondary key value based on the described secondary field (value), and generate a data value based on the described secondary field value; and
[0025] a table generation module, configured to generate a secondary data table according to the described primary and secondary key values and the described data value, and save the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values.
[0026] In some embodiments, wherein the described device further comprises a secondary processing module, the described secondary processing module comprises:
[0027] a request receiving unit, configured to receive and parse data processing requests, for acquire the type of a data table associated with the described data processing request, wherein the described data table includes a bivariate table and / pr a key-value pair table;
[0028] a table determination unit, configured to determine a target data table based on the described data table type, wherein the described target data table contains a primary data table and a secondary data table; and
[0029] a data processing unit, configured to process the data in the described target data table according to the described data processing request.
[0030] From the third perspective, a computer apparatus is provided, comprising a memory unit, a processor, and computer programs stored in the memory unit executable on the processor, wherein the following procedures are performed when the described processor executes the described computer programs:
[0031] parsing a primary data table to identify a primary field and a secondary field, and acquiring a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table;
[0032] generating a primary key value based on the described primary field value, generating a secondary key value based on the described secondary field (value), and generating a data value based on the described secondary field value; and
[0033] generating a secondary data table according to the described primary and secondary key values and the described data value and saving the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values.
[0034] From the fourth perspective, a readable computer storage medium with computer programs stored is provided, wherein the following procedures are performed when the described computer programs are executed on the described processor:
[0035] parsing a primary data table to identify a primary field and a secondary field, and acquiring a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table;
[0036] generating a primary key value based on the described primary field value, generating a secondary key value based on the described secondary field (value), and generating a data value based on the described secondary field value; and
[0037] generating a secondary data table according to the described primary and secondary key values and the described data value and saving the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values.
[0038] The benefits provided by the present invention include that:
[0039] The method, device, computer apparatus, and storage medium for data processing disclosed the present invention, permits parsing a primary data table to identify a primary field and a secondary field, and acquiring a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table; generating a primary key value based on the described primary field value, generating a secondary key value based on the described secondary field (value), and generating a data value based on the described secondary field value; and generating a secondary data table according to the described primary and secondary key values and the described data value and saving the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values. By saving data according to key-value pairs in physical layers, data in each column are allowed to update independently based on key-value pairs, wherein individual tasks can inset data into tables independently and parallelly, to reduce the coupling amongst tasks. Brief descriptions of the drawings
[0040] For better explanation of the technical proposal of embodiments in the present invention, the accompanying drawings are briefly introduced in the following. Obviously, the following drawings represent only a portion of embodiments of the present invention. Those skilled in the art are able to create other drawings according to the accompanying drawings without making creative efforts.
[0041] Fig.1 is a structural diagram of the data processing system provided in an illustrative embodiment of the present invention.
[0042] Fig.2 is a flow diagram of the data processing method provided in an illustrative embodiment of the present invention.
[0043] Fig.3 is a structural diagram of the data processing device provided in an illustrative embodiment of the present invention.
[0044] Fig.4 is an internal structure diagram of the computer apparatus provided in an illustrative embodiment of the present invention.
[0045] In order to make the objective, the technical scheme, and the advantages of the present invention clearer, the present invention will be explained further in detail precisely below with references to the accompany drawings. Obviously, the embodiments described below are only a portion of embodiments of the present invention and cannot represent all possible embodiments. Based on the embodiments in the present invention, the other applications by those skilled in the art without any creative works are falling within the scope of the present invention.
[0046] Embodiment one
[0047] As discussed in the background, taking the general member tag processing system as an example, the most widely used method is to adopt technologies such as Hive or HBASE, wherein the offline member tags are calculated by Hive capable of massive data computation volume. However, based on the current available functions of Hive, a data table with one or more columns is first generated based on the base data source, followed by layered integration and summarization into a massive-field table with the major dimension as members. In the described process, many calculation tasks are involved, with complicated layers and diversified computations. The majorities involve one column corresponding to one or more data sources. Many tags require converting the column-stored data into row-stored data, wherein all tags for one member are integrated into a single record in a row. Overall, the currently available techniques cannot offer a relatively ideal technique to solve the problems of multi-source data integration and processing.
[0048] To solve the forementioned problems, embodiments of the present invention creatively propose a data processing method, allowing one table to read and write data in rows or in columns (as the results by row-column conversion based on row-column data). The method combines Hive database technologies, in particular applicable with relational database, wherein the table data storage methods by row-column conversion and raw data management methods are revealed in the logic layer. For example, for member tag processing for data warehouses by Hive, the data management adopting the described method allows to read and write data in rows or in columns of the corresponding data.
[0049] Fig. 1 is a structural diagram of the data processing system provided in an illustrative embodiment of the present invention. Referring to Fig. 1, the system displays the relationship between the bivariate data and physical files. The physical data storage of the described data processing system adopts the key-value pair method, wherein the keys include row keys (primary keys) and column keys (secondary keys). In the logic layer, two drives compatible with physical layers are implemented, wherein one of the drives is a row drive with the data logic listed in the following Table 1. Similar to general relational bivariate tables, the data is read in physical layers following row logics. The other drive is a column drive with the data logic listed in the Table 2. The data is read and write according to the column formats after the row-column conversion. The two drives use a same physical data file.
[0050] Table 1, relational bivariate table [Image disponible dans le document PDF, Image available in the PDF document] [Image disponible dans le document PDF, Image available in the PDF document]
[0051] Table 2, key-value pair stored data table [Image disponible dans le document PDF, Image available in the PDF document]
[0052] In particular, the forementioned proposal can be achieved via the following procedures:
[0053] Step 1, based on the storage-pending data, generating a primary data table, wherein the primary data table includes but not limited to relational bivariate table.
[0054] In detail, taking member profiles as an example, member profiles need to generate multiple tags for each member. In practical services, a system is supposed to generate thousands tags for members, to be integrated into a massive-field table. Taking Hive database as an example, after receiving the storage-pending data, the "column storage" can be assigned to generate an internal table (as the primary data table) as shown in the Table 2 above. The primary data table includes at least a primary field, primary field values corresponding to the primary field, a secondary field, secondary field values corresponding to the secondary field. In particular, the primary field is the major key of the primary data table. The number of the secondary fields can be one or multiples that is not restricted herein.
[0055] Step 2, generating the secondary data table based on the primary data table, wherein the secondary data table stores data in the key-value pair format, including the primary key value, the secondary key value, and data values.
[0056] In particular, taking Hive for data warehouses as an example, due to sparse data source, data from thousand data sources is supposed to be summarized in one table. For the personnel of this data warehouse, the great data processing volume is involved. Furthermore, the Hive warehouses do not support level updates or only permit to update a few fields. In the current technologies, multiple tables are only able to be linked, resulting two major drawbacks that 1) each field update requires code modification; and 2) more data sources requires more linkages and consequent larger computation volumes. Aiming at the forementioned problems, in the present embodiment, the "column stored" external table with the first generated physically stored files are used to generate a column format table (known as the secondary data table). Therefore, the two tables (the primary data table and the secondary data table) use the same data file. To clarify, the primary data table generation can be used to provide data externally. The secondary data table can be designed to be partitioned by "column keys", then, with compatibility of Hive techniques of data update with independent partitions, the data in individual columns can be updated independently by the key-value pair format. Each task can insert data into tables independently and parallelly, to reduce coupling between tasks and provide data externally based on the result of column-to-row conversion.
[0057] Step 3, receiving data update request, and determining a corresponding port for data update based on the corresponding data table type.
[0058] In detail, in the present embodiment, the ports are provided in advance, such as being provided based on Hive for read and write drive packages of rows and columns, individually. In other word, in the logic layer, two drives compatible with physical layers are implemented, wherein one of the drives is a row drive. Similar to general relational bivariate tables, the data is read in physical layers following row logics. The other drive is a column drive. The data is read and write according to the column formats after the row-column conversion. After the following date update request being received, the received data update request is parsed to obtain data table type corresponding to the data update request. Where if the data source is a data set in the row format, the primary data table can be used to initiate data update. Where if the data source is a data set in the column format, the secondary data table is used for data update. To clarify, in the present embodiment, for reading data sets of rows or columns in a table, the corresponding tables can also be used to initiate the searching and reading.
[0059] Embodiment two
[0060] Fig.2 is a flow diagram of the data processing method provided in an illustrative embodiment of the present invention. Referring to Fig. 2, the described method comprises the following procedures:
[0061] S1, parsing a primary data table to identify a primary field and a secondary field, and acquiring a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table.
[0062] In detail, in the present embodiment, the primary data table includes but not limited to a bivariate table, such as a relational bivariate table. In particular, the primary field is the major key of the primary data table, and the number of the secondary field can be one or more. Based on the primary data table, the row-based logics are provided for data read and write in the physical layers.
[0063] S2, generating a primary key value based on the described primary field value, generating a secondary key value based on the described secondary field (value), and generating a data value based on the described secondary field value.
[0064] In detail, in the present embodiment, in order to provide post row-column conversion column-based format for data read and write, a secondary data table generated from the primary data table for storing data based on the key-value pair format is further required. In practical applications, the primary field value of the primary data table is identified as the primary key value (also as the row key) and the secondary field as the secondary key value (as the column key), wherein the secondary field value is identified as the corresponding data value.
[0065] S3, generating a secondary data table according to the described primary and secondary key values and the described data value and saving the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values.
[0066] In detail, after generating the secondary data table, wherein the secondary data table is stored in the described relational databased, the primary data table and the secondary data table are using the same data file. Based on the secondary data table, the column-based logics are provided for data read and write in the physical layers.
[0067] As a preferred application, in the present embodiment, the described method further includes:
[0068] receiving process-pending data, to generate a primary data table based on the described process-pending data and primary rules, wherein the described primary data table contains a primary field and a corresponding primary field value, and a secondary field and a corresponding secondary field value.
[0069] As a preferred application, in the present embodiment, the described method further includes:
[0070] receiving and parsing the process-pending request, to acquire the type of a data table associated with the described data processing request, wherein the described data table includes a bivariate table and / pr a key-value pair table;
[0071] determining a target data table based on the described data table type, wherein the described target data table contains a primary data table and a secondary data table; and
[0072] processing the data in the described target data table according to the described data processing request.
[0073] As a preferred application, in the present embodiment wherein the described data processing request includes a data read request, the described processing of the data in the described target data table according to the described data processing request further includes:
[0074] based on the described data read request, acquiring the target data from the described target data table, and returning the described target data to the data requesting end.
[0075] As a preferred application, in the present embodiment, wherein the described data processing request further includes a data update request, the described processing of the data in the described target data table according to the described data processing request further includes:
[0076] based on the described data update request, updating the data in the described target data table.
[0077] As a preferred application, in the present embodiment, the described primary field includes a major key of the described primary data table.
[0078] Fig.3 is a structural diagram of the data processing device provided in an illustrative embodiment of the present invention. The described device comprises:
[0079] a data parsing module, configured to parse a primary data table for identifying a primary field and a secondary field, and acquire a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table;
[0080] a primary processing module, configured to generate a primary key value based on the described primary field value, generate a secondary key value based on the described secondary field (value), and generate a data value based on the described secondary field value; and
[0081] a table generation module, configured to generate a secondary data table according to the described primary and secondary key values and the described data value, and save the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values.
[0082] As a preferred application, in the present embodiment, wherein the described device further comprises a secondary processing module, the described secondary processing module comprises:
[0083] a request receiving unit, configured to receive and parse data processing requests, for acquire the type of a data table associated with the described data processing request, wherein the described data table includes a bivariate table and / pr a key-value pair table;
[0084] a table determination unit, configured to determine a target data table based on the described data table type, wherein the described target data table contains a primary data table and a secondary data table; and
[0085] a data processing unit, configured to process the data in the described target data table according to the described data processing request.
[0086] As a preferred application, in the present embodiment, the described table determination unit is further configured to
[0087] receive process-pending data, for generating a primary data table based on the described process-pending data and primary rules, wherein the described primary data table contains a primary field and a corresponding primary field value, and a secondary field and a corresponding secondary field value
[0088] As a preferred application, in the present embodiment, the described data processing unit is particularly configured to:
[0089] acquire the target data from the described target data table based on the described data read request, and return the described target data to the data requesting end.
[0090] As a preferred application, in the present embodiment, the described data processing unit is particularly configured to:
[0091] update the data in the described target data table based on the described data update request.
[0092] As a preferred application, in the present embodiment, the described primary field includes a major key of the described primary data table.
[0093] Fig.4 is an internal structure diagram of the computer apparatus provided in an illustrative embodiment of the present invention. Referring to Fig. 4, The computer apparatus comprises a processor, a memory unit, a network connection port, and a database connected by system bus control. The memory unit of the computer apparatus includes a nonvolatile storage medium and an internal memory. The operating system, computer programs, and databases are stored in the nonvolatile storage medium. The internal memory provides the operation environment for the execution of the operating system and the computer programs stored in the nonvolatile storage medium. The database of the computer apparatus is configured to store the message execution results. The network connection port of the computer apparatus is configured for communication with the external terminals via network connection. The execution of the computer apparatus by the processor permits the method of an optimized method for data executing.
[0094] It is comprehensible for those skilled in the art that the structure shown in Fig. 4 represents only a portion of structure associated with the applications of the present invention. The computer apparatus associated with the applications of the present invention are not restricted or limited by the structure. An exact computer apparatus may include more components or less components than that is shown in the drawings, possibly with combinations of some components or different component layouts.
[0095] As a preferred application, in the present embodiment, a computer apparatus is provided, comprising a memory unit, a processor, and computer programs stored in the memory unit executable on the processor, wherein the following procedures are performed when the described processor executes the described computer programs:
[0096] parsing a primary data table to identify a primary field and a secondary field, and acquiring a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table;
[0097] generating a primary key value based on the described primary field value, generating a secondary key value based on the described secondary field (value), and generating a data value based on the described secondary field value; and
[0098] generating a secondary data table according to the described primary and secondary key values and the described data value and saving the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values.
[0099] As a preferred application, in the present embodiment, following procedures are further performed when the described processor executes the described computer programs:
[0100] receiving process-pending data, for generating a primary data table based on the described process-pending data and primary rules, wherein the described primary data table contains a primary field and a corresponding primary field value, and a secondary field and a corresponding secondary field value
[0101] As a preferred application, in the present embodiment, following procedures are further performed when the described processor executes the described computer programs:
[0102] receiving and parsing the process-pending request, to acquire the type of a data table associated with the described data processing request, wherein the described data table includes a bivariate table and / pr a key-value pair table;
[0103] determining a target data table based on the described data table type, wherein the described target data table contains a primary data table and a secondary data table; and
[0104] processing the data in the described target data table according to the described data processing request.
[0105] As a preferred application, in the present embodiment, following procedures are further performed when the described processor executes the described computer programs:
[0106] based on the described data read request, acquiring the target data from the described target data table, and returning the described target data to the data requesting end.
[0107] As a preferred application, in the present embodiment, following procedures are further performed when the described processor executes the described computer programs:
[0108] based on the described data update request, updating the data in the described target data table.
[0109] As a preferred application, a readable computer storage medium with computer programs stored is provided, wherein the following procedures are performed when the described computer programs are executed on the described processor:
[0110] parsing a primary data table to identify a primary field and a secondary field, and acquiring a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table;
[0111] generating a primary key value based on the described primary field value, generating a secondary key value based on the described secondary field (value), and generating a data value based on the described secondary field value; and
[0112] generating a secondary data table according to the described primary and secondary key values and the described data value and saving the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values.
[0113] As a preferred application, in the present embodiment, following procedures are further performed when the described computer programs are executed on the described processor:
[0114] receiving process-pending data, for generating a primary data table based on the described process-pending data and primary rules, wherein the described primary data table contains a primary field and a corresponding primary field value, and a secondary field and a corresponding secondary field value
[0115] As a preferred application, in the present embodiment, following procedures are further performed when the described computer programs are executed on the described processor:
[0116] receiving and parsing the process-pending request, to acquire the type of a data table associated with the described data processing request, wherein the described data table includes a bivariate table and / pr a key-value pair table;
[0117] determining a target data table based on the described data table type, wherein the described target data table contains a primary data table and a secondary data table; and
[0118] processing the data in the described target data table according to the described data processing request.
[0119] As a preferred application, in the present embodiment, following procedures are further performed when the described computer programs are executed on the described processor:
[0120] based on the described data read request, acquiring the target data from the described target data table, and returning the described target data to the data requesting end.
[0121] As a preferred application, in the present embodiment, following procedures are further performed when the described computer programs are executed on the described processor:
[0122] based on the described data update request, updating the data in the described target data table.
[0123] To summarize, the benefits provided by the present invention include that
[0124] The method, device, computer apparatus, and storage medium for data processing disclosed the present invention, permits parsing a primary data table to identify a primary field and a secondary field, and acquiring a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table; generating a primary key value based on the described primary field value, generating a secondary key value based on the described secondary field (value), and generating a data value based on the described secondary field value; and generating a secondary data table according to the described primary and secondary key values and the described data value and saving the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values. By saving data according to key-value pairs in physical layers, data in each column are allowed to update independently based on key-value pairs, wherein individual tasks can inset data into tables independently and parallelly, to reduce the coupling amongst tasks.
[0125] To clarify, when the data processing service is invoked in the data processing device in the forementioned embodiments, the described functional module configurations are used for illustration only. In practical applications, the described functions can be assigned to different functional modules according to practical demands, wherein the internal structural configuration of the device is divided into different functional modules to perform all or a portion of the described functions. Besides, the forementioned data processing device in the embodiment adopts the same concepts in the described data processing method embodiments. The described device is based on the implementation of the data processing method, whereas the detailed procedures can be referred to the method embodiments and are not explained in further detail.
[0126] All or portions of the forementioned procedures are comprehensible for those skilled in the art, and may be achieved by the computer program configured for sending commands to the related hardware. The computer programs can be stored in computer readable memory units, wherein procedures of the forementioned data processing method are performed when the described computer programs are executed. The storage medium may include ROM / RAM, diskettes, disc, memory cards, etc.
[0127] The forementioned contents are a portion of preferred embodiments of the present invention with detailed explanation and descriptions, and shall not limit the applications of the present invention. Therefore, within concepts of the present invention, alterations and modifications may be performed that shall fall within the scope of the present invention.
Claims
<pat:ClaimStatement>CLAIMS< / pat:ClaimStatement> <pat:Claims com:id="claims"> <pat:Claim com:id="CLM-00001"> <pat:ClaimNumber>1< / pat:ClaimNumber> <pat:ClaimText>1. A data processing method, the method comprising: parsing a primary data table to identify a primary field and a secondary field, and acquiring a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table; generating a primary key value based on the described primary field value, generating a secondary key value based on the described secondary field (value), and generating a data value based on the described secondary field value; and generating a secondary data table according to the described primary and secondary key values and the described data value and saving the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00002"> <pat:ClaimNumber>2< / pat:ClaimNumber> <pat:ClaimText>2. The method of claim 1, further comprising receiving a data update request, and determining a corresponding port for data update based on the corresponding data table type. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00003"> <pat:ClaimNumber>3< / pat:ClaimNumber> <pat:ClaimText>3. The method of claim 2, wherein the correspond port is provided to the method in advance < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00004"> <pat:ClaimNumber>4< / pat:ClaimNumber> <pat:ClaimText>4. The method of any one of claims 2 or 3, wherein after the data update request is received, the received data update request is parsed to obtain data table type corresponding to the data update request. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00005"> <pat:ClaimNumber>5< / pat:ClaimNumber> <pat:ClaimText>5. The method of claim 4, wherein the data source is a data set in a format and the primary data table is configured for a data update. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00006"> <pat:ClaimNumber>6< / pat:ClaimNumber> <pat:ClaimText>6. The method of claim 4, wherein if the data source is a data set in a column format, the secondary data table is configured for a data update. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00007"> <pat:ClaimNumber>7< / pat:ClaimNumber> <pat:ClaimText>7. The method of claim 1, wherein after generating the secondary data table, the secondary data table is stored in the described relational databased, and the primary data table and the secondary data table comprise the same data file. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00008"> <pat:ClaimNumber>8< / pat:ClaimNumber> <pat:ClaimText>8. The method of claim 1, the method further comprising: receiving process-pending data, to generate a primary data table based on the described process-pending data and primary rules, wherein the described primary data table contains a primary field and a corresponding primary field value, and a secondary field and a corresponding secondary field value. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00009"> <pat:ClaimNumber>9< / pat:ClaimNumber> <pat:ClaimText>9. The method of any one of claims 1 or 2, the method further comprising: receiving and parsing a process-pending request, to acquire the type of a data table associated with a described data processing request, wherein the described data table includes a bivariate table and / or a key-value pair table; determining a target data table based on the described data table type, wherein the described target data table contains a primary data table and a secondary data table; and processing the data in the described target data table according to the described data processing request. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00010"> <pat:ClaimNumber>10< / pat:ClaimNumber> <pat:ClaimText>10. The method of claim 9, wherein the described data processing request includes a data read request, and the described processing of the data in the described target data table according to the described data processing request further comprises: based on the described data read request, acquiring the target data from the described target data table, and returning the described target data to the data requesting end. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00011"> <pat:ClaimNumber>11< / pat:ClaimNumber> <pat:ClaimText>11. The method of claim 9, wherein the described data processing request further includes a data update request, the described processing of the data in the described target data table according to the described data processing request further comprises: based on the described data update request, updating the data in the described target data table. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00012"> <pat:ClaimNumber>12< / pat:ClaimNumber> <pat:ClaimText>12. The method of any one of claims 1 or 2, wherein the described primary field includes a major key of the described primary data table. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00013"> <pat:ClaimNumber>13< / pat:ClaimNumber> <pat:ClaimText>13. A data processing device, the device comprising: a data parsing module, configured to parse a primary data table for identifying a primary field and a secondary field, and acquire a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table; a primary processing module, configured to generate a primary key value based on the described primary field value, generate a secondary key value based on the described secondary field (value), and generate a data value based on the described secondary field value; and a table generation module, configured to generate a secondary data table according to the described primary and secondary key values and the described data value, and save the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00014"> <pat:ClaimNumber>14< / pat:ClaimNumber> <pat:ClaimText>14. The device of claim 13, wherein the described device further comprises a secondary processing module, the described secondary processing module comprises: a request receiving unit, configured to receive and parse data processing requests, for acquire the type of a data table associated with a described data processing request, wherein the described data table includes a bivariate table and / pr a key-value pair table; a table determination unit, configured to determine a target data table based on the described data table type, wherein the described target data table contains a primary data table and a secondary data table; and a data processing unit, configured to process the data in the described target data table according to the described data processing request. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00015"> <pat:ClaimNumber>15< / pat:ClaimNumber> <pat:ClaimText>15. The device of claim 14, wherein the table determination unit is further configured to receive process-pending data, for generating a primary data table based on the described process- pending data and primary rules, wherein the described primary data table contains a primary field and a corresponding primary field value, and a secondary field and a corresponding secondary field value. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00016"> <pat:ClaimNumber>16< / pat:ClaimNumber> <pat:ClaimText>16. The device of any one of claims 14 or 15, wherein data processing unit is further configured to acquire the target data from the described target data table based on the described data read request and return the described target data to the data requesting end. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00017"> <pat:ClaimNumber>17< / pat:ClaimNumber> <pat:ClaimText>17. The device of any one of claims 14 to 16, wherein the data processing unit is further configured to update the data in the described target data table based on the described data update request. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00018"> <pat:ClaimNumber>18< / pat:ClaimNumber> <pat:ClaimText>18. The device of any one of claims 14 to 17, the primary field includes a major key of the described primary data table. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00019"> <pat:ClaimNumber>19< / pat:ClaimNumber> <pat:ClaimText>19. A computer apparatus, comprising a memory unit, a processor, and computer readable instructions stored in the memory unit executable on the processor, wherein the methods of any one of claims <semantics>1−12<annotation encoding="application / x-tex">1 - 12< / annotation>< / semantics> are performed when the described processor executes the described computer readable instructions. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00020"> <pat:ClaimNumber>20< / pat:ClaimNumber> <pat:ClaimText>20. A readable computer storage medium, wherein the readable computer storage medium has computer readable instructions stored thereon, wherein the methods of any one of claims 1 – 12 are performed when the described computer readable instructions are executed on a processor. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00021"> <pat:ClaimNumber>21< / pat:ClaimNumber> <pat:ClaimText>21. A data processing system, the system comprising: a data parsing module, configured to parse a primary data table for identifying a primary field and a secondary field, and acquire a primary field value for the described primary field and a secondary field value for the described secondary field, wherein the described primary data table includes a bivariate table; a primary processing module, configured to generate a primary key value based on the described primary field value, generate a secondary key value based on the described secondary field (value), and generate a data value based on the described secondary field value; and a table generation module, configured to generate a secondary data table according to the described primary and secondary key values and the described data value, and save the described secondary data table into a relational database to search for the described data value based on the described primary and secondary key values. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00022"> <pat:ClaimNumber>22< / pat:ClaimNumber> <pat:ClaimText>22. The system of claim 21, wherein the system further comprises a secondary processing module, the described secondary processing module comprises: a request receiving unit, configured to receive and parse data processing requests, for acquire the type of a data table associated with a described data processing request, wherein the described data table includes a bivariate table and / pr a key-value pair table; a table determination unit, configured to determine a target data table based on the described data table type, wherein the described target data table contains a primary data table and a secondary data table; and a data processing unit, configured to process the data in the described target data table according to the described data processing request. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00023"> <pat:ClaimNumber>23< / pat:ClaimNumber> <pat:ClaimText>23. The system of claim 22, wherein the table determination unit is further configured to receive process-pending data, for generating a primary data table based on the described process- pending data and primary rules, wherein the described primary data table contains a primary field and a corresponding primary field value, and a secondary field and a corresponding secondary field value. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00024"> <pat:ClaimNumber>24< / pat:ClaimNumber> <pat:ClaimText>24. The system of any one of claims 22 or 23, wherein data processing unit is further configured to acquire the target data from the described target data table based on the described data read request and return the described target data to the data requesting end. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00025"> <pat:ClaimNumber>25< / pat:ClaimNumber> <pat:ClaimText>25. The system of any one of claims 22 to 24, wherein the data processing unit is further configured to update the data in the described target data table based on the described data update request. < / pat:ClaimText> < / pat:Claim> <pat:Claim com:id="CLM-00026"> <pat:ClaimNumber>26< / pat:ClaimNumber> <pat:ClaimText>26. The system of any one of claims 22 to 25, the primary field includes a major key of the described primary data table. < / pat:ClaimText> < / pat:Claim> < / pat:Claims>