Data processing system

Abstract

The invention provides a data processing method which is used for a data processing system to process uploaded data. Wherein the data processing system comprises an associated database and a non-associated database, and the uploaded data comprises a data content and a data table name. The data processing system determines a data type of the uploaded data according to the data content; determining a first data table of a first database according to the data type, the data content and the data table name of the uploaded data; and storing the uploaded data in the first data table of the first database. Wherein the first database is the associated database or the non-associated database.

Description

Technical Field

[0001] The present application relates to a data processing system, and in particular to a data processing system that can automatically perform database selection, data structure adjustment, and data migration according to data content. Background Art

[0002] With the digitization of information and the popularization of the Internet, there is a need to access databases through application programming interfaces (APIs) in various fields. In order to meet the diversification of data applications, the types of databases have also flourished. Today, mainstream databases can be mainly divided into relational databases and non-relational databases (NoSQL). Relational databases such as Oracle, MySQL, and SQL Server usually store data through multiple data tables. The data in the data tables and the data tables include pre-defined and rigorous structured relationships. Non-relational databases such as MangoDB and Redis are different from traditional relational databases. They do not need to use a fixed data table format to store data, and can define data models at will, including better flexibility and scalability. The technology of accessing databases through APIs allows users to face multiple database types and diverse database technologies without having to have database-related professional knowledge. They can access databases based on the definition of the API.

[0003] However, generally speaking, before uploading data, users still need to confirm their needs with database technicians. Database technicians will select a database that meets the needs, plan the corresponding data relationship architecture, and establish the corresponding data table based on the actual needs of the user. In addition, they must provide a customized API for accessing data. After the database and data table are set up and a certain amount of data is stored, if there is a subsequent need for database or data table structure changes, it is still necessary to rely on database technicians to transfer the database and adjust the data structure. With the increasing amount of information, database technicians will face considerable manpower burdens and database change costs. This problem is particularly serious when using a relational database. However, changing data content in response to changes in demand has always been an unavoidable problem in this field. Therefore, it is urgent to establish a method that can automatically identify and automatically adjust the database and data table according to the data content uploaded by the user to save manpower and reduce costs. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of this application is to provide a data processing system that can automatically determine the content of data, so as to improve the shortcomings of the conventional technology.

[0005] In a first aspect, the present application provides a data processing system for processing uploaded data, wherein the uploaded data includes data content and a data table name, and the data processing system includes: a relational database; a non-relational database; a data processing unit coupled to the relational database and the non-relational database, used to execute a program code; and a storage unit coupled to the data processing unit, used to store the program code to instruct the processing unit to execute a data processing method, the data processing method including: determining a data type of the uploaded data according to the data content; determining a first data table of a first database according to the data type, the data content and the data table name of the uploaded data; and storing the uploaded data in the first data table of the first database; wherein the first database is the relational database or the non-relational database.

[0006] In an implementation manner of the first aspect, the data type is divided into a structured data type and a non-structured data type.

[0007] In an implementation of the first aspect, the step of determining the data type of the uploaded data according to the data content includes determining the data type of the uploaded data to be the structured data type when the data content of the uploaded data satisfies a first judgment condition, otherwise determining the data type of the uploaded data to be the unstructured data type, wherein the first judgment condition includes: the data content includes at least one field; the data type of the corresponding data of each field of the at least one field is consistent; and the data type of each field of the at least one field is one of a floating point number, an integer, a date and time, and a character string.

[0008] In an implementation of the first aspect, the data processing system further receives a database specified parameter specified by a user, and determines the data type according to the database specified parameter, wherein the step of determining the data type of the uploaded data according to the data content further includes: in response to the database specified parameter indicating the use of the relational database and the data content of the uploaded data satisfies the first judgment condition, determining the data type of the uploaded data to be the structured data type; in response to the database specified parameter indicating the use of the relational database and the data type of the uploaded data does not satisfy the first judgment condition, the data processing system returns an error message to the user; and in response to the database specified parameter indicating the use of the non-relational database, determining the data type of the uploaded data to be the non-structured data type.

[0009] In an implementation of the first aspect, the step of determining the first data table of the first database according to the data type, the data content and the data table name of the uploaded data includes: determining whether a second data table including the data table name is stored in the data processing system, wherein the second data table is stored in a second database of the data processing system; in response to the second data table not existing, determining the first database according to the data type of the uploaded data, and establishing the first data table including the data table name in the first database; and in response to the second data table already existing, determining the first database and the first data table according to the uploaded data and the second data table.

[0010] In an implementation of the first aspect, in response to the second data table not existing, the step of determining the first database according to the data type of the uploaded data includes: in response to the data type of the uploaded data being a structured data type, determining the first database to be the relational database; and in response to the data type of the uploaded data being a non-structured data type, determining the first database to be the non-relational database.

[0011] In an implementation of the first aspect, in response to the second data table already existing, the step of determining the first database and the first data table according to the uploaded data and the second data table includes: in response to the data type of the uploaded data being the structured data type and the second database being the relational database, determining the first database and the first data table according to the data content of the uploaded data and the second data table; in response to the data type of the uploaded data being the structured data type and the second database being the non-relational database, determining the first database to be the non-relational database and determining the first data table to be the second data table; in response to the data type of the uploaded data being the non-structured data type and the second database being the relational database, determining the first database to be the non-relational database and performing a database migration; and in response to the data type of the uploaded data being the non-structured data type and the second database being the non-relational database, determining the first database to be the non-relational database and determining the first data table to be the second data table.

[0012] In an implementation of the first aspect, in response to the data type of the uploaded data being the structured data type and the second database being the relational database, the step of determining the first database and the first data table according to the data content of the uploaded data and the second data table includes: in response to all fields of the data content of the uploaded data and their data types being consistent with the second data table, determining the first database to be the relational database and determining the first data table to be the second data table; and in response to all fields of the second data table being consistent with a subset of all fields of the data content of the uploaded data and the data content of the uploaded data including newly added fields, determining the first database to be the relational database and performing a data table migration.

[0013] In an implementation of the first aspect, the step of transferring the data table includes: exporting the data of the second data table as temporary data; deleting the second data table; establishing the first data table including the data table name in the relational database based on the data content of the uploaded data and the data content of the second data table; and importing the temporary data into the first data table, wherein the data of the newly added field is set to a null value.

[0014] In an implementation of the first aspect, in response to the data type of the uploaded data being the non-structured data type and the second database being the relational database, determining that the first database is the non-relational database of the data processing system, and performing the steps of migrating the database include: exporting the data of the second data table as temporary data; deleting the second data table; establishing the first data table including the data table name in the non-relational database; and importing the temporary data into the first data table. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Shown is a schematic diagram of a user accessing a database via an API according to an embodiment of the present application.

[0016] Figure 2 Shown is a schematic diagram of a data processing system described in an embodiment of the present application.

[0017] Figure 3 Shown is a schematic diagram of a data processing flow described in an embodiment of the present application.

[0018] Figure 4 Shown is a schematic diagram of a data type determination process described in an embodiment of the present application.

[0019] Figure 5 Shown is a schematic diagram of a process described in an embodiment of the present application.

[0020] Figure 6 Shown is a schematic diagram of a process described in an embodiment of the present application.

[0021] Figure 7 Shown is a schematic diagram of a database migration process described in an embodiment of the present application.

[0022] Figure 8 Shown is a schematic diagram of a data table migration process described in an embodiment of the present application.

[0023] Component number description

[0024] 100 API

[0025] 110 users

[0026] 120 Database

[0027] 130 Data

[0028] 20 Data Processing System

[0029] 200 API

[0030] 210 Relational Database

[0031] 212 Non-relational database

[0032] 220 Data Processing Unit

[0033] 230 storage units

[0034] 232 Program Code

[0035] 240 Upload data

[0036] 30 Data processing flow

[0037] 300~308 Step 40 Data type determination process

[0038] 400~416 Steps 50 and 60 Process

[0039] 500~51、600~618 Step 70 Database Migration Process

[0040] 700~710 Step 80 Data table migration process

[0041] Steps 800-810 DETAILED DESCRIPTION

[0042] Certain words are used in the specification and subsequent patent claims to refer to specific components. It should be understood by those skilled in the art that hardware manufacturers may use different terms to refer to the same component. This specification and subsequent patent claims do not distinguish components by differences in name, but by differences in function. The term "including" used throughout the specification and subsequent patent claims is an open-ended term and should be interpreted as "including but not limited to".

[0043] Please refer to Figure 1, which is a diagram of how a user accesses a database through an API. Figure 1 As shown, user 110 sends a request to access database 120 through API 100 and obtains a response from database 120. In this process, user 110 only needs to carry data 130 that needs to be stored in database 120 in the request according to the specification of API 100, without knowing the type of database 120 or having professional knowledge related to database 120. However, in general, API 100 and database 120 need to be planned in detail by database technicians according to user needs and implemented in a form that is easy to access, and it is difficult to change once the system is put into use. Data 130 can be a structured table, or it can be various multimedia data, or it can be a data set including any structure, and in response to changes in actual needs, the structure and content of data 130 may need to be updated at any time. In this case, database technicians need to consider the old data structure stored in the database 120 and the requirements of the new data 130, and update the data table structure, data table association, etc. in the database 120 and the content of the API 100 accordingly. Under specific requirements, data migration between different types of databases is also required. This update process will consume considerable manpower and material resources.

[0044] In response to the above problems, please refer to FIG. 2, which is a schematic diagram of a data processing system 20 of an embodiment of the present application. As shown in FIG. 2, the data processing system 20 includes a relational database 210, a non-relational database 212, a data processing unit 220, and a storage device 230. The relational database 210 can be any relational database such as Oracle, MySQL, SQL Server, etc., and is not limited thereto. The non-relational database 212 can be any non-relational database such as MangoDB, Redis, etc., and is not limited thereto. The data processing unit 220 can be a general-purpose processor, a microprocessor, an application-specific integrated circuit (ASIC), etc., or a combination thereof. The storage unit 230 can be any data storage device for storing a program code 232, and reading and executing the program code 232 through the data processing unit 220. For example, the storage unit 230 can be a read-only memory (ROM), a flash memory, a random access memory (RAM), a hard disk, an optical data storage device, and a non-volatile storage unit, but is not limited thereto.

[0045] As shown in FIG. 2 , the data processing system 20 provides an API 200 for the user 110 to access an uploaded data 240, and the uploaded data 240 must at least include a data table name and its data content. The data processing system 20 executes a data processing method according to the uploaded data 240 uploaded by the user 110, and thereby determines the database and data table suitable for storing the uploaded data 240, thereby achieving the purpose of automatically storing data. As shown in FIG. 3 , the data processing method of the embodiment of the present application can be summarized as a data processing process 30 and compiled into a program code 232, which includes the following steps:

[0046] Step 300: Start.

[0047] Step 302 : Determine a data type of the uploaded data 240 according to the data content of the uploaded data 240 .

[0048] Step 304: Determine a first database and a first data table according to the data type, data content and data table name of the uploaded data 240.

[0049] Step 306: Store the data content of the uploaded data 240 in the first data table of the first database.

[0050] Step 308: End.

[0051] According to the process 30, in step 302, the data processing system 20 first determines the data type of the uploaded data 240 according to the data content. Then, in step 304, the data processing system 20 searches the relational database 210 and the non-relational database 212 according to the data table name of the uploaded data 240 to determine whether relevant data has been stored in the data processing system 20, and accordingly determines the first database and the first data table for finally storing the uploaded data 240. Finally, in step 306, the data processing system 20 stores the data content of the uploaded data 240 in the first data table of the first database determined in step 304.

[0052] In detail, in step 302, the data processing system 20 first determines the data type of the uploaded data 240 according to the structure of the data content. The data type of the uploaded data 240 can be divided into two types: structured data type and unstructured data type. Data of structured data type is suitable for storage in the relational database 210, while data of unstructured data type is suitable for storage in the unrelational database 212. The embodiment of the present application further determines the final storage method of the uploaded data 240 based on this information. In one embodiment, the data processing system 20 can receive a database specified parameter specified by the user 110 while receiving the uploaded data 240. The database specified parameter can specify whether the data is stored in the relational database 210 or the unrelational database 212. Please refer to Figure 4, which is a data type judgment process 40 for performing data type judgment in step 302, which includes the following steps:

[0053] Step 400: Start.

[0054] Step 402: Determine whether the database specified parameter is true. If so, execute step 406; if not, execute step 404.

[0055] Step 404: Determine whether the data content of the uploaded data 240 satisfies a first determination condition. If yes, execute step 410; if no, execute step 414.

[0056] Step 406: Determine whether the database designation parameter indicates to use a relational database or a non-relational database. If a relational database is used, execute step 408; if a non-relational database is used, execute step 414.

[0057] Step 408: Determine whether the data content of the uploaded data 240 satisfies a first determination condition. If yes, execute step 410; if no, execute step 412.

[0058] Step 410: Determine that the data type of the uploaded data 240 is a structured data type.

[0059] Step 412: Return an error message of "data structure does not match".

[0060] Step 414: Determine that the data type of the uploaded data 240 is a non-structured data type.

[0061] Step 416: End.

[0062] According to the process 40, the data processing system 20 can determine the data structure of the uploaded data 240 stored in the data processing system 20 according to the data content of the uploaded data 240 and the database specified parameters. The data processing system 20 first determines whether the user 110 has set the database specified parameters (step 402). If the user 110 has not uploaded the database specified parameters, the data processing system 20 automatically determines the data type of the uploaded data 240 according to whether the data content of the uploaded data 240 meets a first judgment condition (step 404). When the first judgment condition is met, the data processing system 20 determines that the data type of the uploaded data 240 is a structured data type (step 410); when the first judgment condition is not met, the data processing system 20 determines that the data type of the uploaded data 240 is a non-structured data type (step 414). On the other hand, when the user 110 has uploaded the database specified parameters (the database specified parameter value is true), the data type of the uploaded data 240 needs to be further determined according to the database specified parameters and the data content of the uploaded data 240 (step 406). If the database designation parameter indicates that a relational database is used, it is necessary to determine whether the data content of the uploaded data 240 meets the first determination condition (step 408). When the data content of the uploaded data 240 meets the first determination condition, the data processing system 20 determines that the data type of the uploaded data 240 is a structured data type (step 410); when the data content of the uploaded data 240 does not meet the first determination condition, the data processing system 20 needs to return an error message to the user 110 and prompt that the specified database type does not conform to the data structure (step 412). If the database designation parameter indicates that a non-relational database is used, the data processing system 20 can directly determine that the data type of the uploaded data 240 is a non-structured data type (step 414).

[0063] In the process 40, the data processing system 20 determines the data format and data type of the uploaded data 240 according to whether the data content of the uploaded data 240 satisfies the first judgment condition. In one embodiment, the first judgment condition can include the data content structure of the uploaded data 240, which is composed of a field (column) name and a row (row) representing a data value group, and all data types (data type) of the data corresponding to each field must be consistent, and the data type of each field is one of a floating point number (including an integer) (Float), a date and time (Datetime) and a string (String). When the above conditions are met, the data processing system 20 can determine that the data content of the uploaded data 240 is a structured data type, otherwise it is a non-structured data type. Compared with the non-structured data type, the structured data type includes strict format restrictions and can be stored in the relational database 210 or the non-relational database 212. Compared with the structured data type, the non-structured data type includes a looser structure and can only be stored in the non-relational database 212. When the user 110 has not set the database specified parameters, the data processing system 20 can directly determine the data type of the uploaded data 240 according to the first judgment condition. When the user 110 has set the database specified parameters, it is necessary to consider whether the specified database type and the data content of the uploaded data 240 meet the first judgment condition. If the data content of the uploaded data 240 cannot meet the first judgment condition, it cannot be stored in the relational database 210 with a more rigorous data structure. Therefore, if the user 110 specifies to use the relational database 210 under this condition, the data processing system 20 needs to return an error message and end the process.

[0064] Regarding the method of determining the data type in the first judgment condition, existing software or packages can be used, and are not limited to this. For example, when using Python to determine the data type, the "dateutil.parser.parse" package can be used to check the date and time type, the "float()" function can be used to check the floating point type, and the "isinstance(input,str)" function can be used to check the string type. When using R language to determine the data type, "as.Date" can be used to check the date and time type, "float.TryParse(input,out valid_float)" can be used to check the floating point type, and "is.character()" can be used to check the string type. People with general knowledge in this field can use appropriate tools to determine the data type according to actual needs, and are not limited to this.

[0065] According to the process 30, in step 304, the data processing system 20 determines the first database and the first data table for storing the uploaded data 240 according to the data type, data content and data table name of the uploaded data 240. In this step, the data processing system 20 must first search the relational database 210 and the non-relational database 212 according to the data table name of the uploaded data 240 to determine whether relevant data has been stored in the data processing system 20, and accordingly determines the first database and the first data table for finally storing the uploaded data 240.

[0066] Please refer to FIG. 5, which illustrates a process 50 for determining a first database and a first data table for actually storing the uploaded data 240 according to the data type of the uploaded data 240 when the data processing system 20 fails to find a data table with a data table name matching that of the uploaded data 240 in the relational database 210 and the non-relational database 212. In this case, the data processing system 20 does not store data related to the uploaded data 240, and can automatically determine the first database and the first data table according to the data type of the uploaded data 240 determined by the data type determination process 40 in step 302. The process 50 includes the following steps:

[0067] Step 500: Start.

[0068] Step 502: Determine the data type of the uploaded data 240. If it is a structured data type, execute step 504; if it is a non-structured data type, execute step 506.

[0069] Step 504 : Determine that the first database is the relational database 210 .

[0070] Step 506 : Determine that the first database is the non-relational database 212 .

[0071] Step 508: Create a first data table including the data table name of the uploaded data 240 in the first database.

[0072] Step 510: End.

[0073] According to the process 50, the data processing system 20 needs to determine the first database according to the data type of the uploaded data 240 in step 302 (step 502). When the data type of the uploaded data 240 is a structured data type, the first database for storing the uploaded data 240 is determined to be the relational database 210 (step 504). When the data type of the uploaded data 240 is a non-structured data type, the first database for storing the uploaded data 240 is determined to be the non-relational database 212 (step 506). Finally, the data processing system 20 needs to establish a first data table according to the data table name of the uploaded data 240 in the first database determined in steps 504 and 506 to store the data content of the uploaded data 240.

[0074] Please refer to FIG. 6, which illustrates a process 60 for determining a first database and a first data table for actually storing the uploaded data 240 when the data processing system 20 searches for a data table with a data table name that matches the uploaded data 240 in the relational database 210 and the non-relational database 212. Assume that the processing system 20 finds a second data table with a data table name that matches the uploaded data 240 in a second database. In this case, the data processing system 20 needs to consider the compatibility of the second database, the second data table, and the uploaded data 240 to determine the first database and the first data table. The process 60 includes the following steps:

[0075] Step 600: Start.

[0076] Step 602: Determine the data type of the uploaded data 240. If it is a structured data type, execute step 604; if it is a non-structured data type, execute step 606.

[0077] Step 604: Determine whether the second database is a relational database 210 or a non-relational database 212. If it is a relational database 210, execute step 608; if it is a non-relational database 212, execute step 610.

[0078] Step 606: Determine whether the second database is a relational database 210 or a non-relational database 212. If it is a relational database 210, execute step 612; if it is a non-relational database 212, execute step 610.

[0079] Step 608: Determine whether the data content of the uploaded data 240 is compatible with the second data table. If yes, execute step 616; if not, execute step 614.

[0080] Step 610: Determine that the first database is the non-relational database 212 and the first data table is the second data table.

[0081] Step 612: Determine that the first database is the non-relational database 212, and perform a database migration process.

[0082] Step 614: Determine that the first database is the relational database 210, and perform a data table migration process.

[0083] Step 616: Determine that the first database is the relational database 210 and the first data table is the second data table.

[0084] Step 618: End.

[0085] According to the process 60, the data processing system 20 needs to judge and determine the first database and the first data table according to the data type of the uploaded data 240 and the type of the second database (steps 602-606). When the second database is a non-relational database 212, regardless of whether the data type of the uploaded data 240 is structured data, the uploaded data must be stored in the non-relational database 212. In this case, the data processing system 20 determines that the first database is the non-relational database 212 and the first data table is the second data table (step 610). In other words, the uploaded data 240 can be directly stored in the second data table in the existing second database.

[0086] Furthermore, when the second database is a relational database 210, the data processing system 20 needs to determine whether the uploaded data 240 is suitable for being stored in the relational database 210 together with the data content of the second data table according to the data type of the uploaded data 240. For example, when the second database is a relational database 210 and the data type of the uploaded data 240 is a non-structured data type, the relational database 210 will not be able to store the non-structured data in the uploaded data 240. Accordingly, the data processing system 20 needs to first migrate the data of the second data table from the relational database 210 to the non-relational database 212, and then store the uploaded data 240. Therefore, the data processing system 20 determines that the first database is the non-relational database 212, and performs a database migration process (step 612).

[0087] On the other hand, when the second database is a relational database 210 and the data type of the uploaded data 240 is a structured data type, the uploaded data 240 can be stored in the relational database 210. Since the relational database has a strict database schema definition for the storage method of data, the data processing system 20 needs to first check whether the data content of the second data table and the uploaded data 240 is compatible, and the uploaded data 240 cannot be directly stored in the second data table (step 608). If all fields and data types of the data content of the uploaded data 240 are compatible or consistent with the second data table, the data processing system 20 can directly store the data content of the uploaded data 240 in the second data table. Therefore, the data processing system 20 determines that the first database is a relational database 210, and the first data table is the second data table (step 616). If compared with the second data table, the data content of the uploaded data 240 includes all the fields of the second data table and has a new field, the uploaded data 240 cannot be directly stored in the second data table. In this case, the data processing system 20 needs to expand the data table fields first, and then store the uploaded data 240. Therefore, the data processing system 20 determines that the first database is the relational database 210, and performs a data table migration process (step 614).

[0088] For the database migration process in step 612, please refer to FIG. 7. FIG. 7 is a schematic diagram of a database migration process 70 according to an embodiment of the present application. The database migration process 70 includes the following steps:

[0089] Step 700: Start.

[0090] Step 702: Export the data in the second data table as temporary data.

[0091] Step 704: Delete the second data table.

[0092] Step 706 : Create a first data table including the data table name of the uploaded data 240 in the non-relational database 212 .

[0093] Step 708: Import the temporary data into the first data table.

[0094] Step 710: End.

[0095] According to the database migration process 70, after the data processing system 20 exports the data content of the second data table and temporarily stores it as a temporary data (step 702), the second data table can be deleted (step 704). Then, the data processing system 20 can create a first data table including the data table name of the uploaded data 240 in the non-relational database 212 (step 706). Finally, the data processing system 20 imports the temporary data into the newly created first data table (step 708). In this way, the cross-database data migration of the second data table can be completed, and the first data table compatible with the uploaded data 240 can be created in the non-relational database 212.

[0096] In one embodiment, the data processing system 20 can simultaneously receive a database migration parameter specified by the user 110 while receiving the uploaded data 240. When the database migration parameter value is true, regardless of whether the uploaded data 240 is a non-structured data type, the data processing system 20 can execute the database migration process 70 to transfer the second data table data stored in the relational database 210 to the newly created first data table in the non-relational database 212. In the case that the user 110 does not set the database migration parameter, the data processing system 20 can make a determination according to the process 60 to determine the data processing method.

[0097] Please refer to FIG8 for the data table migration process in step 614. FIG8 is a schematic diagram of a data table migration process 80 according to an embodiment of the present application. The data table migration process 80 includes the following steps:

[0098] Step 800: Start.

[0099] Step 802: Export the data in the second data table as temporary data.

[0100] Step 804: Delete the second data table.

[0101] Step 806 : Create a first data table including the data table name of the uploaded data 240 in the relational database 210 according to the data content of the uploaded data 240 and the data content of the second data table.

[0102] Step 808: Import the temporarily stored data into the first data table, wherein the data of the newly added field is set to a null value.

[0103] Step 810: End.

[0104] According to the data table migration process 80, after the data processing system 20 exports the data content of the second data table and temporarily stores it as a temporary data (step 802), the second data table can be deleted (step 804). Next, the data processing system 20 can establish a first data table including the data table name of the uploaded data 240 in the relational database 210 according to the data structure of the uploaded data 240 and the temporary data (step 806). Finally, the data processing system 20 imports the temporary data into the newly created first data table (step 808). In this way, the data migration of the second data table can be completed, and a first data table compatible with the uploaded data 240 can be established.

[0105] Accordingly, according to processes 50, 60, database transfer process 70 and data table transfer process 80 in step 304, the data processing system 20 can automatically determine the data type of the uploaded data 240 and determine the appropriate database and data table based on the existing data in the data processing system 20, thereby reducing the human and material burdens caused by changes in the data structure.

[0106] Finally, in step 306 , the data processing system 20 can store the uploaded data 240 according to the first database and the first data table determined in step 304 .

[0107] In summary, the present application provides a data processing system and a data processing method, so that users do not need to repeatedly confirm their needs with database technicians before uploading data, and when there is any change in the uploaded data, there is no need to spend huge manpower costs and time to adjust the database architecture, migrate data, and modify the API. Therefore, the present application can improve the shortcomings of the conventional technology and meet the needs of practical applications.

[0108] The above description is only a preferred embodiment of the present application. All equivalent changes and modifications made within the scope of the claims of the present application shall fall within the scope of the present application.

Claims

1. A data processing system, characterized in that: For processing uploaded data, wherein the uploaded data includes data content and a data table name, the data processing system includes: A relational database; A non-relational database; a data processing unit, coupled to the relational database and the non-relational database, for executing a program code; and a storage unit, coupled to the data processing unit, for storing the program code to instruct the processing unit to execute a data processing method, the data processing method comprising: Determining a data type of the uploaded data according to the data content; Determining a first data table of a first database according to the data type, the data content and the data table name of the uploaded data; and storing the uploaded data in the first data table of the first database; The first database is the relational database or the non-relational database.

2. The data processing system according to claim 1, characterized in that The data type is divided into a structured data type and a non-structured data type.

3. The data processing method according to claim 2, characterized in that: The step of determining the data type of the uploaded data according to the data content includes determining the data type of the uploaded data as the structured data type when the data content of the uploaded data satisfies a first judgment condition, and otherwise determining the data type of the uploaded data as the non-structured data type, wherein the first judgment condition includes: The data content includes at least one field; The data type of the corresponding data of each field of the at least one field is consistent; and The data type of each field of the at least one field is a floating point number, an integer, a date and time, and a character string.

4. The data processing system according to claim 3, characterized in that: The data processing system further receives a database specified parameter specified by a user, and determines the data type according to the database specified parameter, wherein the step of determining the data type of the uploaded data according to the data content further includes: In response to the database designation parameter indicating the use of the relational database and the data content of the uploaded data satisfying the first judgment condition, determining the data type of the uploaded data to be the structured data type; In response to the database designation parameter indicating the use of the relational database and the data type of the uploaded data not satisfying the first determination condition, the data processing system returns an error message to the user; and In response to the database specifying parameter indicating the use of the unrelated database, the data type of the uploaded data is determined to be the unstructured data type.

5. The data processing system according to claim 2, characterized in that: The step of determining the first data table of the first database according to the data type, the data content and the data table name of the uploaded data comprises: Determining whether a second data table including the data table name is stored in the data processing system, wherein the second data table is stored in a second database of the data processing system; In response to the second data table not existing, determining the first database according to the data type of the uploaded data, and establishing the first data table including the data table name in the first database; and In response to the second data table already existing, the first database and the first data table are determined according to the uploaded data and the second data table.

6. The data processing system according to claim 5, characterized in that: In response to the second data table not existing, the step of determining the first database according to the data type of the uploaded data comprises: In response to the data type of the uploaded data being a structured data type, determining the first database to be the relational database; and In response to the data type of the uploaded data being a non-structured data type, determining the first database to be the non-relational database.

7. The data processing system according to claim 5, characterized in that: In response to the second data table already existing, the step of determining the first database and the first data table according to the uploaded data and the second data table includes: In response to the data type of the uploaded data being the structured data type and the second database being the relational database, determining the first database and the first data table according to the data content of the uploaded data and the second data table; In response to the data type of the uploaded data being the structured data type and the second database being the non-relational database, determining the first database to be the non-relational database, and determining the first data table to be the second data table; In response to the data type of the uploaded data being the non-structured data type and the second database being the relational database, determining the first database to be the non-relational database, and performing a database migration; and In response to the data type of the uploaded data being the non-structured data type and the second database being the non-relational database, it is determined that the first database is the non-relational database, and the first data table is determined to be the second data table.

8. The data processing system according to claim 7, characterized in that: In response to the data type of the uploaded data being the structured data type and the second database being the relational database, the step of determining the first database and the first data table according to the data content of the uploaded data and the second data table includes: In response to all fields and data types of the data content of the uploaded data being consistent with those of the second data table, determining the first database to be the relational database and determining the first data table to be the second data table; and In response to all fields of the second data table being consistent with a subset of all fields of the data content of the uploaded data and the data content of the uploaded data including a newly added field, determining the first database to be the relational database and performing a data table migration.

9. The data processing system according to claim 8, characterized in that: The steps for transferring a data table include: Exporting the data of the second data table as temporary data; Deleting the second data table; According to the data content of the uploaded data and a data content of the second data table, establishing the first data table including the data table name in the relational database; and The temporary data is imported into the first data table, wherein the data of the newly added field is set to a null value.

10. The data processing system according to claim 7, characterized in that: In response to the data type of the uploaded data being the non-structured data type and the second database being the relational database, determining that the first database is the non-relational database of the data processing system and performing the database migration step include: Exporting the data of the second data table as temporary data; Deleting the second data table; Creating the first data table including the data table name in the non-relational database; and Import the temporary data into the first data table.

Images