A data online service processing method and apparatus

Abstract

The present invention provides a method and apparatus for processing data online services, which relates to the technical field of big data. The method includes: receiving an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier; querying corresponding online service information in an online service table according to the service query identifier and the user identifier, where the online service information includes target database access information, a target data table, and a query statement; where the online service information in the online service table is pre-generated; obtaining a query result according to the target database access information, the target data table, and the query statement, and returning the query result to the user terminal. The apparatus is used to execute the above method. The data online service processing method and apparatus provided by the embodiments of the present invention improve the data query efficiency.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and particularly relates to a method and device for processing data online services. Background Art

[0002] With the booming development of big data technology, the scale and form of data have shown explosive and diversified development. To meet the processing requirements of massive data, data architectures based on the open-source Hadoop system have been favored by many enterprises.

[0003] In the prior art, a data lake can be built based on the data architecture of the Hadoop system. For the data tables obtained after batch processing of the data in the data lake, users usually need to synchronize the relevant data to the database inside the relevant business system for online access. When performing data synchronization, users need to be familiar with the database system, the data table structure, and the relevant query language, resulting in a complex online data query process and low query efficiency. Summary of the Invention

[0004] In view of the problems in the prior art, embodiments of the present invention provide a method and device for processing data online services, which can at least partially solve the problems existing in the prior art.

[0005] On the one hand, the present invention proposes a method for processing data online services, including:

[0006] Receiving an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier;

[0007] Querying corresponding online service information in an online service table according to the service query identifier and the user identifier, where the online service information includes target database access information, a target data table, and a query statement; wherein, the online service information in the online service table is pre-generated;

[0008] Obtaining a query result according to the target database access information, the target data table, and the query statement, and returning it to the user terminal.

[0009] On the other hand, the present invention provides a device for processing data online services, including:

[0010] A first receiving module, configured to receive an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier;

[0011] A query module, configured to query corresponding online service information in an online service table according to the service query identifier and the user identifier, where the online service information includes target database access information, a target data table, and a query statement; wherein, the online service information in the online service table is pre-generated;

[0012] A return module, configured to obtain a query result according to the target database access information, the target data table, and the query statement, and return the query result to the user terminal.

[0013] In another aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the data online service processing method described in any one of the above embodiments are implemented.

[0014] In yet another aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data online service processing method described in any one of the above embodiments are implemented.

[0015] The data online service processing method and device provided by the embodiments of the present invention can receive an online service request sent by a user terminal. The online service request includes a service query identifier and a user identifier. The corresponding online service information is queried in the online service table according to the service query identifier and the user identifier. The online service information includes target database access information, a target data table, and a query statement. According to the target database access information, the target data table, and the query statement, a query result is obtained and returned to the user terminal. The user does not need to be familiar with the database system, the data table structure, and the related query language, nor does the user need to know the details of the data query in the database. Only by providing the parameters of the data online service processing can data query be performed, which simplifies the process of data online query and improves the data query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0017] Figure 1 is a schematic flowchart of the data online service processing method provided by the first embodiment of the present invention.

[0018] Figure 2 is a schematic flowchart of the data online service processing method provided by the second embodiment of the present invention.

[0019] Figure 3 is a schematic flowchart of the data online service processing method provided by the third embodiment of the present invention.

[0020] Figure 4It is a schematic structural diagram of a distributed data online service system provided by the fourth embodiment of the present invention.

[0021] Figure 5 It is a schematic structural diagram of a data online service processing device provided by the fifth embodiment of the present invention.

[0022] Figure 6 It is a schematic structural diagram of a data online service processing device provided by the sixth embodiment of the present invention.

[0023] Figure 7 It is a schematic structural diagram of a data online service processing device provided by the seventh embodiment of the present invention.

[0024] Figure 8 It is a schematic structural diagram of a data online service processing device provided by the eighth embodiment of the present invention.

[0025] Figure 9 It is a schematic structural diagram of a data online service processing device provided by the ninth embodiment of the present invention.

[0026] Figure 10 It is a schematic physical structure diagram of an electronic device provided by the tenth embodiment of the present invention. Detailed implementation manners

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer and more understandable, the following further elaborates on the embodiments of the present invention with reference to the accompanying drawings. Herein, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but do not limit the present invention. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined arbitrarily with each other.

[0028] In the technical solutions of this application, the acquisition, storage, use, processing, etc. of data all comply with the relevant regulations of national laws and regulations.

[0029] The execution subjects of the data online service processing method provided by the embodiments of the present invention include, but are not limited to, servers and distributed data online service systems. The distributed data online service system adopts a distributed service architecture and includes multiple servers. Hereinafter, taking the distributed data online service system as the execution subject as an example, the specific implementation process of the data online service processing method provided by the embodiments of the present invention is described.

[0030] Figure 1 It is a schematic flowchart of the data online service processing method provided by the first embodiment of the present invention. As Figure 1 shown, the data online service processing method provided by the embodiments of the present invention includes:

[0031] S101. Receive an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier;

[0032] Specifically, when data query is required, the user terminal sends an online service request to the distributed data online service system. The online service request includes a service query identifier and a user identifier. Among them, the user identifier corresponds to a user one by one, and the service query identifier corresponds to online service information. The user terminal includes, but is not limited to, a desktop computer, a laptop computer, etc.

[0033] For example, when a user needs to perform a data query, the user can manually trigger the user terminal to send an online service request to the distributed data online service system. If data query needs to be performed regularly, the data query period can be set so that the user terminal sends an online service request to the distributed data online service system regularly.

[0034] S102. Query the corresponding online service information in the online service table according to the service query identifier and the user identifier. The online service information includes target database access information, a target data table, and a query statement. Among them, the online service information in the online service table is pre-generated;

[0035] Specifically, an online service table is established in advance in the distributed data online service system, and the generated online service information will be stored in the online service table. The distributed data online service system will query the corresponding online service information in the online service table according to the service query identifier and the user identifier. The online service information includes target database access information, a target data table, a query statement, etc., which are set according to actual needs and are not limited in the embodiments of the present invention. Among them, the online service information in the online service table is pre-generated. The target database access information is used to access the target database, the target data table is used to query data, and the query statement is used to perform data query in the target data table of the target database. It can be understood that if the corresponding online service information cannot be queried according to the service query identifier and the user identifier, a prompt message indicating that the online service information does not exist can be returned to the user terminal.

[0036] S103. Obtain a query result according to the target database access information, the target data table, and the query statement, and return it to the user terminal.

[0037] Specifically, the distributed data online service system accesses the target database according to the target database access information, performs data query in the target data table through the query statement, obtains the query result, and then returns the query result to the user terminal.

[0038] For example, if the query statement is an SQL statement, the distributed data online service system will convert the query data corresponding to the SQL statement into a Json-formatted string line by line, and return the converted data as the query result to the user terminal. The user terminal can display the query result in the form of a graphical table for the user to view.

[0039] The data online service processing method provided by the embodiments of the present invention can receive an online service request sent by a user terminal. The online service request includes a service query identifier and a user identifier, query the corresponding online service information in the online service table according to the service query identifier and the user identifier. The online service information includes target database access information, target data tables, and query statements. According to the target database access information, target data tables, and query statements, obtain the query result and return it to the user terminal. There is no need for the user to be familiar with the database system, data table structure, and related query languages, nor to know the details of data query in the database. Only by providing the parameters for data online service processing can data query be performed, which simplifies the process of data online query and improves the data query efficiency. In addition, since the query statement is encapsulated, the user usage threshold is reduced.

[0040] Figure 2 It is a schematic flowchart of the data online service processing method provided by the second embodiment of the present invention. As Figure 2 shown, on the basis of the above embodiments, further, the steps of generating the online service information in the online service table include:

[0041] S201. Receive a data service request sent by a user terminal. The data service request includes a user identifier, a source data table identifier, source database access information, and query information;

[0042] Specifically, in order to be able to perform data query quickly in the follow-up, it is necessary to generate online service information first. The user can send a data service request to the distributed data online service system through the user terminal. The data server request includes a user identifier, a source data table identifier, source database access information, and query information.

[0043] Among them, the source data table identifier may include one data table identifier or two or more data table identifiers. The data table identifier corresponds to the data table one by one. The data table identifier may adopt the data table name or code, which is set according to actual needs and is not limited in the embodiments of the present invention. The source database access information is used to access the source database, that is, the database to which the data table corresponding to the source data table identifier belongs. The source database access information may include information such as the database name, username, and login password of the source database, which is set according to actual needs and is not limited in the embodiments of the present invention. The query information is used to generate a query statement and may include information such as query fields and query conditions, which is set according to actual needs and is not limited in the embodiments of the present invention. The source database includes but is not limited to MongoDB database, Redis database, HIve database based on Hadoop data architecture, etc.

[0044] S202. Obtain the table structure information corresponding to the source data table identifier according to the source database access information and the source data table identifier, and construct the target data table corresponding to the data service request in the target database according to the table structure information corresponding to the source data table identifier;

[0045] Specifically, the distributed data online service system may access the corresponding source database according to the source database access information, and obtain the table structure information of each data table corresponding to the source data table identifier according to the source data table identifier. The distributed data online service system may construct the target data table corresponding to the data service request in the target database according to the table structure information corresponding to the source data table identifier. Each data table in the target data table corresponding to the data service request corresponds to each data table corresponding to the source data table identifier one by one. Among them, the target database may adopt a Gaussian database, a MySQL database, etc., which is selected according to actual needs and is not limited in the embodiments of the present invention.

[0046] For example, if the source data table identifier includes a data table identifier a, then the table structure information of the data table A corresponding to the data table identifier a will be obtained from the source database, and then a data table B will be established in the target database. The table structure information of the data table B corresponds to the table structure information of the data table A. If the database types of the source database and the target database are different, it will result in differences in field type definitions and naming. When establishing the data table B, the field types included in the table structure information of the data table A will be converted into the corresponding field types in the target database. Among them, the table structure information may include fields, field types, primary keys, foreign keys, indexes, etc., which is set according to actual needs and is not limited in the embodiments of the present invention.

[0047] For example, if the source data table identifier includes two data table identifiers: data table identifier c and data table identifier d, then the table structure information of data table C corresponding to data table identifier c and the table structure information of data table D corresponding to data table identifier d will be obtained from the source database, and then data table E and data table F will be created in the target database. The table structure information of data table E corresponds to the table structure information of data table C, and the table structure information of data table F corresponds to the table structure information of data table D.

[0048] S203. Generate a query statement according to the query information and the target data table corresponding to the data service request;

[0049] Specifically, the distributed data online service system generates a query statement according to the query information and the target data table corresponding to the data service request. The query statement is used to query data in the target data table of the target database.

[0050] For example, if the source data table identifier only includes one data table identifier, the query statement uses the SQL statement, and the query information includes query fields and query conditions. The query statement can be constructed in the following format:

[0051] Select query fields from target table where query conditions

[0052] Among them, the query fields are obtained from the query information, the target table is the target data table established in step S202, and the query conditions are obtained from the query information.

[0053] For example, if the source data table identifier includes two data table identifiers c and d, the query statement uses the SQL statement, the query information includes query fields and query conditions, and the query conditions include query condition 1 and query condition 2. The query statement can be constructed in the following format:

[0054] Select query fields

[0055] from target table 1 join target table 2 On query condition 1

[0056] where query condition 2

[0057] Among them, the query fields are obtained from the query information, target table 1 is the target data table corresponding to data table identifier c established in step S202, target table 2 is the target data table corresponding to data table identifier d established in step S202, query condition 1 and query condition 2 are obtained from the query information, and query condition 1 is the association condition between target table 1 and target table 2.

[0058] S204. Generate online service information based on the user identifier, the service identifier corresponding to the data service request, the source database access information, the source data table identifier, the target database access information, the target data table, and the query statement, and store the online service information into the online service table; wherein, the service identifier corresponding to the data service request is used as the service query identifier.

[0059] Specifically, the distributed data online service system will use the user identifier, the service identifier corresponding to the data service request, the source database access information, the source data table identifier, the target database access information, the target data table, and the query statement as the online service information and store it into the online service table. Among them, the service identifier corresponding to the data service request uniquely corresponds to the data service request. The service identifier corresponding to the data service request can be used as the service query identifier. According to the user identifier and the service query identifier, an online service information can be uniquely determined. Associating the online service information with the user identifier can restrict the access authority of the online service information and improve the security of data access. The service identifier corresponding to the data service request is pre-generated, and the service identifier corresponds to the data service request one by one. The online service table is pre-generated and used to store the online service information.

[0060] Based on the above embodiments, further, the data service request further includes an online service path; correspondingly, the method further includes:

[0061] Generate online service information based on the user identifier, the service identifier corresponding to the data service request, the online service path, the source database access information, the source data table identifier, the target database access information, the target data table, and the query statement, and store the online service information into the online service table; wherein, the online service path is used as the service query identifier.

[0062] Specifically, the data service request further includes an online service path, and the online service path is set by the user for easy memory and use by the user. The distributed data online service system can use the user identifier, the service identifier corresponding to the data service request, the online service path, the source database access information, the source data table identifier, the target database access information, the target data table, and the query statement as the online service information and store it into the online service table. The online service path uniquely corresponds to the data service request, and the online service path can be used as the service query identifier.

[0063] Based on the above embodiments, further, the data online service processing method provided by the embodiments of the present invention further includes:

[0064] Create a corresponding data synchronization task according to the online service information and add it to the synchronization task list.

[0065] Specifically, since the information in the data tables in the source database may change, a corresponding data synchronization task can be created for the online service information, so that the data in the data table corresponding to the source data table identifier is synchronized to the corresponding target data table. The data synchronization task corresponding to the online service information may include the service identifier corresponding to the online service information, the source database access information, the source data table identifier, the target database access information, the target data table, and information such as the synchronization time range and synchronization fields, which are set according to actual needs and are not limited in the embodiments of the present invention.

[0066] Among them, since the time range of the data stored in the data tables in the source database can be very large, such as the data in the past 5 years, when the user only needs the data in the past 1 year, the synchronization time range can be set to the past 1 year. On the one hand, it can improve the data query efficiency, and on the other hand, it can save the storage resources of the target database. The synchronization time range can also be set by default, such as 3 months before the current time.

[0067] Among them, since when querying data based on the online service information, the data corresponding to the query fields in the query statement is queried, so for the target data table, only the data corresponding to the query fields in the query statement needs to be stored. When synchronizing the data in the data table corresponding to the source data table identifier to the corresponding target data table, only the data corresponding to the query fields included in the query statement included in the corresponding online service information can be synchronized, thereby saving the storage resources of the target database, reducing the resource consumption for executing the data synchronization task, and improving the efficiency of data synchronization.

[0068] Figure 3 is a schematic flowchart of the data online service processing method provided by the third embodiment of the present invention. As Figure 3 shown, on the basis of the above embodiments, further, the data online service processing method provided by the embodiments of the present invention further includes:

[0069] S301. Obtain the source database access information, source data table identifier, and synchronization time range included in each data synchronization task from the synchronization task list;

[0070] Specifically, the distributed data online service system can obtain each synchronization task from the synchronization task list and obtain the source database access information, source data table identifier, and synchronization time range included in each synchronization task.

[0071] S302. According to the source database access information and source data table identifier included in each data synchronization task, obtain the modification timestamp corresponding to the source data table identifier of each data synchronization task, and obtain the data tables to be synchronized based on the current time, the synchronization time range included in each data synchronization task, and the modification timestamp corresponding to the source data table identifier of each data synchronization task;

[0072] Specifically, for each data synchronization task, the distributed data online service system can access the source database according to the source database access information and source data table identifier included in the data synchronization task to obtain the modification timestamp corresponding to the source data table identifier of the data synchronization task. If the source data table identifier includes one data table identifier, then obtain one modification timestamp as the modification timestamp corresponding to the source data table identifier; if the source data table identifier includes two or more data table identifiers, then each data table identifier corresponds to one modification timestamp, and obtain the modification timestamps corresponding to each data table identifier as the modification timestamp corresponding to the source data table identifier. The last modification time of the corresponding data table can be obtained through the modification timestamp.

[0073] For each data synchronization task, the distributed data online service system can obtain the synchronization threshold time according to the current time and the synchronization time range included in the data synchronization task. If the time corresponding to one of the modification timestamps in the modification timestamps corresponding to the source data table identifier is later than or equal to the synchronization threshold time, then the data table in the source database corresponding to this modification timestamp needs to be synchronized and will be used as one of the data tables to be synchronized; if the time corresponding to one of the modification timestamps in the modification timestamps corresponding to the source data table identifier is earlier than the synchronization threshold time, then the data table in the source database corresponding to this modification timestamp will not be used as a data table to be synchronized. The distributed data online service system can obtain all the data tables in the source database that need to be synchronized as the data tables to be synchronized.

[0074] For example, if the synchronization time range is 1 month before the current time, then subtract one month from the current time to obtain the synchronization threshold time.

[0075] S303. Execute the data synchronization tasks in the synchronization task list, and synchronize the data in the data tables to be synchronized to the corresponding target data tables.

[0076] Specifically, after obtaining the data tables to be synchronized, the distributed data online service system may execute the data synchronization tasks in the synchronization task list to synchronize the data to be synchronized in each data table in the data tables to be synchronized to the corresponding target data tables. Among them, when performing data synchronization for each data table in the data tables to be synchronized, the data synchronization may be performed in sequence according to the order of the time partitions in the data table. If the data format of the data table in the source database is inconsistent with the data format of the target data table in the target database, the data format may be first converted to process the data in the data tables to be synchronized into a format recognizable by the target database, and then the converted data is stored in the target data table.

[0077] Based on the above embodiments, further, the querying of the corresponding online service information in the online service table according to the service query identifier and the user identifier includes:

[0078] If the service query identifier included in an online service information in the online information service table is the same as the service query identifier included in the online service request, and the user identifier included in the online service information is the same as the user identifier included in the online service request, it is determined that the corresponding online service information is queried in the online service table.

[0079] Specifically, the distributed data online service system will query the corresponding online service information in the online service table according to the service query identifier and the user identifier included in the online service request. If there is an online service information in the online service table whose service query identifier is the same as the service query identifier included in the online service request, and the user identifier included in this online service information is the same as the user identifier included in the online service request, then it can be determined that the corresponding online service information is queried in the online service table.

[0080] Based on the above embodiments, further, the obtaining of the query result according to the target database access information, the target data table, and the query statement includes:

[0081] Access the target database according to the target database access information;

[0082] Execute the query statement to perform data query on the target data table in the target database to obtain the query result.

[0083] Specifically, the distributed data online service system can log in to the target database according to the target database access information, so as to query the data tables in the target database. Among them, the target database access information may include the IP address for accessing the target database, port number, username, password, etc., which are set according to actual needs and are not limited in the embodiments of the present invention.

[0084] The distributed data online service system can execute the query statement to query the data of the target data table, and use the obtained query data as the query result.

[0085] The following uses a specific embodiment to illustrate the implementation process of the data online service processing method provided by the embodiments of the present invention.

[0086] The source database is a Hive database, and the target database is a GaussDB database. The Hive database stores a large amount of business data, including the data of data tables X and Y. User A needs to regularly perform associated data queries from data tables X and Y, involving 10 query fields. The query language uses Structured Query Language (SQL for short).

[0087] For the convenience of subsequent queries, User A needs to generate online service information in the distributed data online service system. User A sends a data service request to the distributed data online service system through the user terminal. The data service request includes the user identifier z of User A, the data table identifier x of data table X, the data table identifier y of data table Y, the access information of the Hive database, the query information Q, and the online service path xy. The query information Q includes the above 10 query fields and query conditions. The query conditions include the associated query conditions and filtering conditions for data tables X and Y.

[0088] After the distributed data online service system receives the above data service request, it will generate a unique service identifier for the above data service request. Access the Hive database according to the access information of the Hive database, obtain the table structure information of data table X according to data table identifier x, and obtain the table structure information of data table Y according to data table identifier y. Create data table X' corresponding to data table X in GaussDB according to the table structure information of data table X, and create data table Y' corresponding to data table Y in GaussDB according to the table structure information of data table Y. Since the field types of the Hive database and GaussDB are not completely consistent, the corresponding relationship between the field types in the Hive database and the field types of GaussDB can be established in advance, and the above corresponding relationship of field types is used for field type conversion when creating data tables X' and Y'. When creating data tables X' and Y', data table X' only includes the query fields that appear in data table X among the 10 query fields in query information Q, and data table Y' only includes the query fields that appear in data table Y among the 10 query fields in query information Q.

[0089] The distributed data online service system generates the following query statement P according to query information Q, data tables X' and Y':

[0090] Select 10 query fields

[0091] from data table X' join data table Y' On the association query condition of data tables X and Y

[0092] where filtering condition

[0093] The distributed data online service system stores the user identifier z, the service identifier corresponding to the above data service request, the online service path xy, the access information of the Hive database, the data table identifier x, the data table identifier y, the access information of the Gauss database, the data table identifier x' of data table X', the data table identifier y' of data table Y', and the query statement P into the online service table as the online service information zxy. The distributed data online service system also generates a data synchronization task corresponding to the online service information zxy, and adds the data synchronization task corresponding to the online service information zxy to the synchronization task list. The data synchronization task corresponding to the online service information zxy includes the service identifier corresponding to the above data service request, the access information of the Hive database, the data table identifier x, the data table identifier y, the access information of the Gauss database, the data table identifier x' of data table X', the data table identifier y' of data table Y', the above 10 query fields, and the synchronization time range. The synchronization time range T is 1 month before the current time.

[0094] Every other day, the distributed data online service system retrieves the data synchronization task corresponding to the online service information zxy from the synchronization task list, and obtains the access information of the Hive database, the data table identifier x, the data table identifier y, and the synchronization time range T from it.

[0095] Based on the access information of the Hive database, the distributed data online service system accesses the Hive database, and obtains the modification timestamp of data table X and the modification timestamp of data table Y from the Hive database. The time corresponding to the modification timestamp of data table X is t1, and the time corresponding to the modification timestamp of data table Y is t2. Based on the current time and the synchronization time range T, the obtained synchronization threshold time is t. Compare the time t1 and t2 with the synchronization threshold time t respectively. If the time t1 is earlier than the synchronization threshold time t and the time t2 is later than the synchronization threshold time t, then data table Y is the data table to be synchronized, and the data of data table X does not need to be synchronized this time.

[0096] The distributed data online service system executes the data synchronization task corresponding to the online service information zxy, retrieves the data of the above 10 query fields existing in data table Y in the previous 1 month before the current time from the Hive database, and synchronizes it to data table Y' in GaussDB. During the data synchronization process, the data that needs to be synchronized in data table Y will be converted into a format that GaussDB can recognize, and then the CopyManager tool of GaussDB Kernel will be called to achieve high-speed data import.

[0097] After the above data service request is completed, when it is time to perform an associated data query on data table X and data table Y, the user terminal can send an online service request to the distributed data online service system. The online service request includes the user identifier z and the online service path xy.

[0098] The distributed data online service system queries and obtains the online service information zxy from the online service table according to the user identifier z and the online service path xy, and obtains the access information of the Gauss database, the data table identifier x', the data table identifier y', and the query statement P from the online service information zxy.

[0099] The distributed data online service system accesses the Gauss database according to the access information of the Gauss database, performs a data query in data tables X' and Y' through the query statement P, obtains the query result, and then returns the query result to the user terminal for user A to view.

[0100] The data online service processing method provided by the embodiments of the present invention has the following advantages:

[0101] (1) The present invention encapsulates the processing details of data synchronization and migration of big data databases, is transparent to users, and significantly optimizes the operation process.

[0102] (2) The present invention provides a unified service call interface to access data, which reduces the user's usage threshold. Users do not need to have a foundation in query language grammar. They only need to specify the data table and conditions to be queried according to their needs, and the system quickly returns the query results to the users, improving the data query efficiency.

[0103] Figure 4 It is a schematic structural diagram of the distributed data online service system provided by the fourth embodiment of the present invention. As Figure 4 shown, the distributed data online service system provided by the embodiment of the present invention includes a service construction module 401, a data monitoring module 402, a data synchronization module 403, and an online service module 404, where:

[0104] The service construction module 401 is used to receive a data service request sent by a user terminal, obtain the table structure information corresponding to the source data table identifier according to the source database access information and the source data table identifier, construct the target data table corresponding to the data service request in the target database according to the table structure information corresponding to the source data table identifier, generate a query statement according to the query information and the target data table corresponding to the data service request, generate online service information and store it in the online service table, and generate a corresponding data synchronization task according to the online service information and add it to the synchronization task list.

[0105] The data monitoring module 402 is used to obtain the source database access information, the source data table identifier, and the synchronization time range included in each data synchronization task from the synchronization task list; obtain the modification timestamp corresponding to the source data table identifier of each data synchronization task according to the source database access information and the source data table identifier included in each data synchronization task, and obtain the data table to be synchronized according to the current time, the synchronization time range included in each data synchronization task, and the modification timestamp corresponding to the source data table identifier of each data synchronization task.

[0106] The data synchronization module 403 is used to execute the data synchronization tasks in the synchronization task list and synchronize the data in the data table to be synchronized to the corresponding target data table.

[0107] The online service module 404 is used to receive an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier; query the corresponding online service information in the online service table according to the service query identifier and the user identifier, and the online service information includes target database access information, a target data table, and a query statement; where the online service information in the online service table is pre-generated; obtain a query result according to the target database access information, the target data table, and the query statement and return it to the user terminal.

[0108] Figure 5 It is a schematic structural diagram of a data online service processing device provided by the fifth embodiment of the present invention. As Figure 5 shown, the data online service processing device provided by the embodiment of the present invention includes a first receiving module 501, a query module 502, and a return module 503, where:

[0109] The first receiving module 501 is configured to receive an online service request sent by a user terminal, and the online service request includes a service query identifier and a user identifier; the query module 502 is configured to query corresponding online service information in an online service table according to the service query identifier and the user identifier, and the online service information includes target database access information, a target data table, and a query statement; wherein, the online service information in the online service table is pre-generated; the return module 503 is configured to obtain a query result according to the target database access information, the target data table, and the query statement, and return it to the user terminal.

[0110] Specifically, when data query is required, the user terminal sends an online service request to the first receiving module 501, and the online service request includes a service query identifier and a user identifier. Wherein, the user identifier corresponds to a user one by one, and the service query identifier corresponds to online service information. The user terminal includes, but is not limited to, a desktop computer, a laptop computer, etc.

[0111] An online service table is established in advance, and the generated online service information will be stored in the online service table. The query module 502 will query corresponding online service information in the online service table according to the service query identifier and the user identifier. The online service information includes target database access information, a target data table, a query statement, etc., which are set according to actual needs and are not limited in the embodiment of the present invention. Wherein, the online service information in the online service table is pre-generated. The target database access information is used to access the target database, the target data table is used to query data, and the query statement is used to query data in the target data table of the target database. It can be understood that if the corresponding online service information cannot be queried according to the service query identifier and the user identifier, a prompt message indicating that the online service information does not exist can be returned to the user terminal.

[0112] The return module 503 accesses the target database according to the target database access information, performs data query in the target data table through the query statement, obtains a query result, and then returns the query result to the user terminal.

[0113] The data online service processing device provided by the embodiment of the present invention can receive an online service request sent by a user terminal. The online service request includes a service query identifier and a user identifier. The corresponding online service information is queried in the online service table according to the service query identifier and the user identifier. The online service information includes target database access information, a target data table, and a query statement. According to the target database access information, the target data table, and the query statement, a query result is obtained and returned to the user terminal, which simplifies the process of data online query and improves the data query efficiency. In addition, since the query statement is encapsulated, the user usage threshold is reduced.

[0114] Figure 6 is a schematic structural diagram of the data online service processing device provided by the sixth embodiment of the present invention. As Figure 6 shown, on the basis of the above embodiments, further, the data online service processing device provided by the embodiment of the present invention further includes a second receiving module 504, a constructing module 505, a first generating module 506, and a second generating module 507, where:

[0115] The second receiving module 504 is configured to receive a data service request sent by a user terminal. The data service request includes a user identifier, a source data table identifier, source database access information, and query information. The constructing module 505 is configured to obtain the table structure information corresponding to the source data table identifier according to the source database access information and the source data table identifier, and construct the target data table corresponding to the data service request in the target database according to the table structure information corresponding to the source data table identifier. The first generating module 506 is configured to generate a query statement according to the query information and the target data table corresponding to the data service request. The second generating module 507 is configured to generate online service information according to the user identifier, the service identifier corresponding to the data service request, the source database access information, the source data table identifier, the target database access information, the target data table, and the query statement, and store the online service information into the online service table; where the service identifier corresponding to the data service request is used as the service query identifier.

[0116] Figure 7 is a schematic structural diagram of the data online service processing device provided by the seventh embodiment of the present invention. As Figure 7 shown, on the basis of the above embodiments, further, the data service request further includes an online service path; correspondingly, the data online service processing device further includes a third generating module 508, where:

[0117] The third generation module 508 is used to generate online service information according to the user identification, the service identification corresponding to the data service request, the online service path, the source database access information, the source data table identification, the target database access information, the target data table, and the query statement, and store the online service information into the online service table; wherein, the online service path serves as the service query identification.

[0118] Figure 8 FIG. 4 is a schematic structural diagram of a data online service processing device provided by the eighth embodiment of the present invention. As Figure 8 shown, on the basis of the above embodiments, further, the data online service processing device provided by the embodiment of the present invention further includes a fourth generation module 509, wherein:

[0119] The fourth generation module 509 is used to generate corresponding data synchronization tasks according to the online service information and add them to the synchronization task list.

[0120] Figure 9 FIG. 5 is a schematic structural diagram of a data online service processing device provided by the ninth embodiment of the present invention. As Figure 9 shown, on the basis of the above embodiments, further, the data online service processing device provided by the embodiment of the present invention further includes an acquisition module 510, an obtaining module 511, and an execution module 512, wherein:

[0121] The acquisition module 510 is used to acquire the source database access information, the source data table identification, and the synchronization time range included in each data synchronization task from the synchronization task list; the obtaining module 511 is used to obtain the modification timestamp corresponding to the source data table identification of each data synchronization task according to the source database access information and the source data table identification included in each data synchronization task, and obtain the data table to be synchronized according to the current time, the synchronization time range included in each data synchronization task, and the modification timestamp corresponding to the source data table identification of each data synchronization task; the execution module 512 is used to execute the data synchronization tasks in the synchronization task list and synchronize the data in the data table to be synchronized to the corresponding target data table.

[0122] On the basis of the above embodiments, further, the query module 502 is specifically used for:

[0123] If the service query identification included in an online service information in the online information service table is the same as the service query identification included in the online service request, and the user identification included in the online service information is the same as the user identification included in the online service request, then it is determined to query the corresponding online service information in the online service table.

[0124] On the basis of the above embodiments, further, the return module 503 is specifically used for:

[0125] Access the target database according to the target database access information; execute the query statement to perform a data query on the target data table in the target database to obtain the query result.

[0126] The embodiments of the device provided by the embodiments of the present invention can specifically be used to execute the processing flows of the above method embodiments, and their functions will not be elaborated here. Reference can be made to the detailed descriptions of the above method embodiments.

[0127] It should be noted that the data online service processing method and device provided by the embodiments of the present invention can be used in the financial field, and can also be used in any technical field other than the financial field. The embodiments of the present invention do not limit the application fields of the data online service processing method and device.

[0128] The user information in the embodiments of this application is obtained through legal and compliant channels, and the acquisition, storage, use, processing, etc. of the user information have obtained the authorization and consent of the customer.

[0129] Figure 10 It is a schematic physical structure diagram of an electronic device provided by an embodiment of the present invention. As Figure 10 shown, the electronic device may include: a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004. Among them, the processor 1001, the communication interface 1002, and the memory 1003 complete mutual communication through the communication bus 1004. The processor 1001 can call the logical instructions in the memory 1003 to execute the following method: receive an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier; query corresponding online service information in the online service table according to the service query identifier and the user identifier, where the online service information includes target database access information, a target data table, and a query statement; where the online service information in the online service table is pre-generated; obtain a query result according to the target database access information, the target data table, and the query statement, and return it to the user terminal.

[0130] In addition, when the logical instructions in the above-mentioned memory 1003 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0131] This embodiment discloses a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided in the above-mentioned method embodiments, for example, including: receiving an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier; querying corresponding online service information in an online service table according to the service query identifier and the user identifier, where the online service information includes target database access information, a target data table, and a query statement; where the online service information in the online service table is pre-generated; obtaining a query result according to the target database access information, the target data table, and the query statement and returning it to the user terminal.

[0132] This embodiment provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. The computer program causes the computer to execute the methods provided in the above-mentioned method embodiments, for example, including: receiving an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier; querying corresponding online service information in an online service table according to the service query identifier and the user identifier, where the online service information includes target database access information, a target data table, and a query statement; where the online service information in the online service table is pre-generated; obtaining a query result according to the target database access information, the target data table, and the query statement and returning it to the user terminal.

[0133] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0134] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of blocks.

[0135] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realize the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of blocks.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of blocks.

[0137] In the description of this specification, the descriptions with reference to terms such as "one embodiment", "a specific embodiment", "some embodiments", "for example", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0138] The above-described specific embodiments have further elaborated on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing data online services, characterized in that, Including: Receiving an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier; Querying corresponding online service information in an online service table according to the service query identifier and the user identifier, where the online service information includes target database access information, a target data table, and a query statement; wherein, the online service information in the online service table is pre-generated; Obtaining a query result according to the target database access information, the target data table, and the query statement, and returning it to the user terminal; Wherein, the step of generating the online service information in the online service table includes: Receiving a data service request sent by a user terminal, where the data service request includes a user identifier, a source data table identifier, source database access information, and query information; Obtaining table structure information corresponding to the source data table identifier according to the source database access information and the source data table identifier, and constructing a target data table corresponding to the data service request in a target database according to the table structure information corresponding to the source data table identifier; Generating a query statement according to the query information and the target data table corresponding to the data service request; Generating online service information according to the user identifier, the service identifier corresponding to the data service request, the source database access information, the source data table identifier, the target database access information, the target data table, and the query statement, and storing it in the online service table; wherein, the service identifier corresponding to the data service request is used as the service query identifier.

2. The method for processing data online services according to claim 1, characterized in that, The data service request further includes an online service path; correspondingly, the method further includes: Generating online service information according to the user identifier, the service identifier corresponding to the data service request, the online service path, the source database access information, the source data table identifier, the target database access information, the target data table, and the query statement, and storing it in the online service table; wherein, the online service path is used as the service query identifier.

3. The method for processing data online services according to claim 1, characterized in that, Also including: Generating a corresponding data synchronization task according to the online service information, and adding it to a synchronization task list.

4. The method for processing data online services according to claim 3, characterized in that, Also including: Obtaining the source database access information, the source data table identifier, and the synchronization time range included in each data synchronization task from the synchronization task list; Obtaining a modification timestamp corresponding to the source data table identifier of each data synchronization task according to the source database access information and the source data table identifier included in each data synchronization task, and obtaining a data table to be synchronized according to the current time, the synchronization time range included in each data synchronization task, and the modification timestamp corresponding to the source data table identifier of each data synchronization task; Executing the data synchronization tasks in the synchronization task list, and synchronizing the data in the data table to be synchronized to the corresponding target data table.

5. The method for processing data online services according to claim 1, characterized in that, The querying corresponding online service information in the online service table according to the service query identifier and the user identifier includes: If the service query identifier included in an online service information in the online service table is the same as the service query identifier included in the online service request, and the user identifier included in the online service information is the same as the user identifier included in the online service request, then it is determined to query the corresponding online service information in the online service table.

6. The method for processing data online services according to any one of claims 1 to 5, characterized in that, The obtaining the query result according to the target database access information, the target data table, and the query statement includes: Access the target database according to the target database access information; Execute the query statement to perform data query on the target data table in the target database to obtain the query result.

7. A device for processing data online services, characterized in that, Includes: A first receiving module, configured to receive an online service request sent by a user terminal, where the online service request includes a service query identifier and a user identifier; A query module, configured to query corresponding online service information in the online service table according to the service query identifier and the user identifier, where the online service information includes target database access information, a target data table, and a query statement; wherein, the online service information in the online service table is pre-generated; A return module, configured to obtain a query result according to the target database access information, the target data table, and the query statement and return it to the user terminal; The apparatus further includes a second receiving module, a construction module, a first generation module, and a second generation module; The second receiving module is configured to receive a data service request sent by a user terminal, where the data service request includes a user identifier, a source data table identifier, source database access information, and query information; The construction module is configured to obtain table structure information corresponding to the source data table identifier according to the source database access information and the source data table identifier, and construct a target data table corresponding to the data service request in the target database according to the table structure information corresponding to the source data table identifier; The first generation module is configured to generate a query statement according to the query information and the target data table corresponding to the data service request; The second generation module is configured to generate online service information according to the user identifier, the service identifier corresponding to the data service request, the source database access information, the source data table identifier, the target database access information, the target data table, and the query statement, and store the online service information in the online service table; wherein, the service identifier corresponding to the data service request is used as the service query identifier.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images