Data query method, data query device, computer equipment and medium

Abstract

The present invention provides a data query method, a data query device, a computer device and a medium. The data query method comprises: receiving a query statistics command. Parsing the query statistics command to determine a first number of data result sets to be queried and counted. According to historical query records, determining the second number of second result sets in the first number of data result sets and the corresponding data validity period, and obtaining second data information, wherein the second result set is a data result set with query results. Determine the data result set that does not have query results in the first number of data result sets as a third result set, and obtain a third number of third result sets. Send the second data information and the third number of third result sets to a data query party to respond to the query statistics command. The present invention helps to reduce the number of queries for data result sets, reduce time costs and computing acquisition costs, and help improve query efficiency.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing, and in particular to a data query method, a data query device, a computer device and a medium. Background Art

[0002] With the promotion of Internet applications and the rise of cloud computing technology, traditional data query methods based on single and closed data sources no longer meet the business needs of users. Usually, a business requirement requires querying data from multiple data sources to ultimately obtain the results required by the user. How to quickly and accurately obtain valid data from multiple different data sources has become a problem that currently plagues business system development. Summary of the invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the defect of low data query efficiency in the prior art, thereby providing a data query method, a data query device, a computer device and a medium.

[0004] According to a first aspect, the present invention provides a data query method, the method comprising:

[0005] Receive query statistics command;

[0006] Parsing the query statistics command to determine a first number of data result sets to be queried and counted;

[0007] Determine, according to the historical query records, a second number of second result sets in the first number of data result sets and corresponding data validity periods, and obtain second data information, wherein the second result set is a data result set having query results;

[0008] Determine the data result sets that do not have query results in the first number of data result sets as third result sets, to obtain a third number of third result sets;

[0009] The second data information and the third number of third result sets are sent to the data query party in response to the query statistics command.

[0010] In this way, based on the historical query results, the second result set with query results and the corresponding data validity period in the data result set to be queried and counted can be quickly determined. When performing subsequent queries, the user only needs to query the third result set, which helps to reduce the number of queries in the data result set, reduce time cost and computing acquisition cost, and help improve query efficiency.

[0011] In combination with the first aspect, in a first embodiment of the first aspect, sending the second data information and the third number of third result sets to the data query party includes:

[0012] The data validity period of each second result set is compared with the current system time respectively. If all data validity periods are greater than the current system time, the second data information and the third result set of the third quantity are sent to the data query party.

[0013] In combination with the first embodiment of the first aspect, in a second embodiment of the first aspect, the method further includes:

[0014] If there is at least one second result set whose data validity period is less than or equal to the current system time, determine a fourth number of fourth result sets from the second data information, the fourth result set being a second result set whose data validity period is less than or equal to the current system time;

[0015] updating the query result of each fourth result set in a fourth number of fourth result sets to obtain fourth data information;

[0016] The fourth data information is sent to the data query party.

[0017] In combination with the second embodiment of the first aspect, in the third embodiment of the first aspect, updating the query result of each fourth result set in the fourth number of fourth result sets to obtain the fourth data information includes:

[0018] According to the historical query records, a fourth number of query command sets are acquired, wherein different query command sets correspond to different fourth result sets;

[0019] sending the fourth number of query command sets to multiple data sources to re-query each fourth result set;

[0020] receiving the updated query result of each fourth result set, and determining the query time of each fourth result set;

[0021] Based on the query time of each fourth result set, updating the data validity period of each fourth result set;

[0022] The fourth data information is obtained according to each updated query result of the fourth result set and the corresponding data validity period.

[0023] In combination with the third embodiment of the first aspect, in a fourth embodiment of the first aspect, determining the data validity period of each second result set according to the historical query record includes:

[0024] According to the historical query record, obtaining the historical query time of querying the current second result set, wherein the current second result set is one of the second number of data result sets;

[0025] Determine the sum of the historical query time and the specified update duration as the effective duration of the current second result set;

[0026] The difference between the validity period and the current system time is determined as the validity period of the data of the current second result set.

[0027] In combination with the first aspect, in a fifth embodiment of the first aspect, the method further includes:

[0028] Obtaining, from a preset configuration file, a fifth number of target data set names and a target query command set corresponding to each target data set name;

[0029] According to the target query command set corresponding to each target data set name, multiple data sources are accessed respectively to obtain the query result of the target data result set corresponding to each target data set name;

[0030] Receive the query result of each target data result set, determine the query time of each target data result set, and obtain the query record of each target data result set;

[0031] The query records of each target data result set are saved respectively to obtain historical query records, wherein the historical query records include the second data information.

[0032] In this method, by querying the fifth number of target data result sets in advance according to needs, historical query records are obtained, which helps to establish the association relationship between the target and the existing result set. Subsequently, when query statistics are performed based on the received query statistics commands, it helps to reduce the time and computing resource overhead required for the operation, and effectively improve the overall operation efficiency and query speed.

[0033] In combination with the fifth embodiment of the first aspect, in a sixth embodiment of the first aspect, the method further includes:

[0034] If the data validity period of the current target data result set is less than the specified time threshold, the query record of the current target data result set is refreshed;

[0035] The current target data result set is one of the fifth number of target data result sets.

[0036] According to a second aspect, the present invention further provides a data query method, the method comprising:

[0037] Receive a data query request to obtain a query result of each data result set in a first number of data result sets to be queried and counted;

[0038] Parsing the data query request to obtain second data information and a third number of third result sets, wherein the second data information includes a second number of second result sets and a data validity period of each second result set, the second result set is a data result set having query results in the first number of data result sets, and the third number of third result sets is a data result set not having query results in the first number of data result sets;

[0039] Sending data query requests to multiple data sources to obtain query results of each third result set;

[0040] The query result of each third result set is received and combined with the second data information to obtain a total query result of the first number of data result sets.

[0041] In this way, without changing the existing architecture, the data result set with query results can be used to meet the user's needs for data query statistical analysis, thereby effectively reducing the time and computing power consumed by calculations, reducing overall losses, and thus improving the query speed and computing efficiency of multiple data sources.

[0042] In combination with the second aspect, in a first embodiment of the second aspect, the method further includes:

[0043] Receive fourth data information.

[0044] According to a third aspect, the present invention provides a data query device, comprising:

[0045] A first receiving unit receives a statistical query command;

[0046] A parsing unit, used for parsing the query statistics command to determine a first number of data result sets to be queried and counted;

[0047] A first determining unit, configured to determine, according to the historical query records, a second number of second result sets in the first number of data result sets and corresponding data validity periods, to obtain second data information, wherein the second result set is a data result set having query results;

[0048] A second determining unit, configured to determine the data result sets that do not have query results in the first number of data result sets as third result sets, to obtain a third number of third result sets;

[0049] The first sending unit is used to send the second data information and the third number of third result sets to the data query party in response to the query statistics command.

[0050] In combination with the third aspect, in a first embodiment of the third aspect, the first sending unit includes:

[0051] The first sending subunit is used to compare the data validity period of each second result set with the current system time respectively. If all data validity periods are greater than the current system time, the second data information and the third number of third result sets are sent to the data query party.

[0052] In combination with the first embodiment of the third aspect, in a second embodiment of the third aspect, the device further includes:

[0053] a third determining unit, configured to determine a fourth number of fourth result sets from the second data information if there is at least one second result set whose data validity period is less than or equal to the current system time, the fourth result set being a second result set whose data validity period is less than or equal to the current system time;

[0054] A first updating unit is used to update the query result of each fourth result set in a fourth number of fourth result sets to obtain fourth data information;

[0055] The second sending unit is used to send the fourth data information to the data query party.

[0056] In combination with the second embodiment of the third aspect, in a third embodiment of the third aspect, the first updating unit includes:

[0057] A first acquisition unit, configured to acquire a fourth number of query command sets according to the historical query records, wherein different query command sets correspond to different fourth result sets;

[0058] a third sending unit, configured to send the fourth number of query command sets to a plurality of data sources, so as to re-query each fourth result set;

[0059] A second receiving unit, configured to receive each updated query result of the fourth result set and determine a query time of each fourth result set;

[0060] A first updating subunit, configured to update a data validity period of each fourth result set based on a query time of each fourth result set;

[0061] The second acquisition unit is used to obtain fourth data information according to each updated query result of the fourth result set and the corresponding data validity period.

[0062] In combination with the third embodiment of the third aspect, in a fourth embodiment of the third aspect, the first determining unit includes:

[0063] A third acquisition unit is used to acquire a historical query time of querying a current second result set according to the historical query record, wherein the current second result set is one of the second number of data result sets;

[0064] A fourth determining unit, configured to determine the sum of the historical query time and the specified update duration as the effective duration of the current second result set;

[0065] The fifth determining unit is used to determine the difference between the validity period and the current system time as the validity period of the data of the current second result set.

[0066] In combination with the third aspect, in a fifth embodiment of the third aspect, the device further includes:

[0067] a fourth acquisition unit, configured to acquire, from a preset configuration file, a fifth number of target data set names and a target query command set corresponding to each target data set name;

[0068] A first access unit is used to access multiple data sources respectively according to a target query command set corresponding to each target data set name, so as to obtain a query result of a target data result set corresponding to each target data set name;

[0069] A second receiving unit is used to receive the query result of each target data result set, determine the query time of each target data result set, and obtain the query record of each target data result set;

[0070] The storage unit is used to save the query record of each target data result set respectively to obtain a historical query record, wherein the historical query record includes the second data information.

[0071] In combination with the fifth embodiment of the third aspect, in a sixth embodiment of the third aspect, the device further includes:

[0072] A refresh unit, configured to refresh the query record of the current target data result set if the data validity period of the current target data result set is less than a specified time threshold;

[0073] The current target data result set is one of the fifth number of target data result sets.

[0074] According to a fourth aspect, the present invention further provides a data query device, the device comprising:

[0075] A request receiving unit, configured to receive a data query request to obtain a query result of each data result set in a first number of data result sets to be queried and counted;

[0076] a request parsing unit, configured to parse the data query request to obtain second data information and a third number of third result sets, wherein the second data information includes a second number of second result sets and a data validity period of each second result set, the second result set is a data result set having query results in the first number of data result sets, and the third number of third result sets is a data result set not having query results in the first number of data result sets;

[0077] A request sending unit, used to send a data query request to multiple data sources to obtain a query result of each third result set;

[0078] The first data receiving unit is used to receive the query result of each third result set and merge it with the second data information to obtain the total query result of the first number of data result sets.

[0079] In combination with the fourth aspect, in a first embodiment of the fourth aspect, the device further includes:

[0080] The second data receiving unit is used to receive fourth data information.

[0081] According to the fifth aspect, an embodiment of the present invention also provides a computer device, comprising a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the data query method of any one of the first aspect and its optional embodiments or the data query method of any one of the second aspect and its optional embodiments by executing the computer instructions.

[0082] According to the sixth aspect, an embodiment of the present invention also provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the data query method of any one of the first aspect and its optional embodiments or the data query method of any one of the second aspect and its optional embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0084] Figure 1 It is a flow chart of a data query method proposed according to an exemplary embodiment.

[0085] Figure 2It is a flowchart of another method for acquiring historical query records proposed according to an exemplary embodiment.

[0086] Figure 3 It is a structural diagram of a data pre-analysis engine proposed according to an exemplary embodiment.

[0087] Figure 4 is a flowchart of yet another data query method proposed according to an exemplary embodiment.

[0088] Figure 5 is a flowchart of yet another data query method proposed according to an exemplary embodiment.

[0089] Figure 6 It is a structural block diagram of a data query device proposed according to an exemplary embodiment.

[0090] Figure 7 It is a structural block diagram of another data query device proposed according to an exemplary embodiment.

[0091] Figure 8 It is a schematic diagram of the hardware structure of a computer device proposed according to an exemplary embodiment. DETAILED DESCRIPTION

[0092] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be pointed out that the words "first", "second" and the like used in the present invention do not indicate any order, quantity or importance.

[0093] In related technologies, data query can be performed through a pre-built big data platform architecture. However, due to the large underlying platform architecture and complex hardware deployment structure, when querying data, it is necessary to go through complex syntax analysis, data object location, extraction, sorting, merging and other steps to obtain the data required by the user, which affects the efficiency of data query, and in the query process, it consumes a lot of time and computing costs.

[0094] In order to solve the above problems, a data query method is provided in an embodiment of the present invention, which is used in a computer device. It should be noted that its execution subject can be a data query device, which can be implemented as part or all of the computer device through software, hardware, or a combination of software and hardware. The computer device can be a terminal, a client, or a server. The server can be a single server or a server cluster composed of multiple servers. The terminal in the embodiment of the present application can be a smart phone, a personal computer, a tablet computer, a wearable device, and other intelligent hardware devices such as an intelligent robot. In the following method embodiments, the execution subject is a computer device as an example for explanation.

[0095] The computer device in this embodiment is suitable for use scenarios where multiple data result sets need to be queried from multiple data sources. The data source is a large relational database management system (RDBMS) that supports standard SQL statements, and can build its own table structure and data objects (such as stored procedures, functions, triggers, etc.) according to actual business needs. It can return a data result set in response to a received request. The data source can be a database of an existing system, or a data query interface or data set provided by a third party.

[0096] According to the data query method provided by the present invention, by parsing the received query statistics command, a first number of data result sets to be counted and queried are determined. By comparing the historical query records, a second number of second result sets with query results in the first number of data result sets are determined, and then the second data information is obtained according to the data validity period of each second result set in the historical query records. The third number of third result sets and the second data information that do not have query results in the first number of data result sets are sent to the sender of the query statistics command, so that the sender can quickly obtain the query results of each second result set according to the historical query records when querying statistics, and then subsequently perform real-time queries on each third result set in the third number of third result sets, without repeatedly obtaining the query results of each second result set, thereby helping to reduce the time required for calculation and the computing resource overhead, and effectively improve the overall computing efficiency and query speed.

[0097] Figure 1 FIG. 1 is a flow chart of a data query method proposed according to an exemplary embodiment. Figure 1 As shown, the data query method includes the following steps S101 to S105.

[0098] In step S101, a statistics query command is received.

[0099] In the embodiment of the present invention, the query statistics command is a command for querying whether the first number of data result sets required by the user for query statistics include data result sets with query results. The query statistics command may be sent by the application server.

[0100] In step S102, the query statistics command is parsed to determine a first number of data result sets to be queried and counted.

[0101] In an embodiment of the present invention, the query statistics command is parsed by an access process to determine a first number of data result sets to be queried and counted, so as to determine whether there is a data result set with query results in the first number of data result sets, that is, to determine whether there is a second result set in the first number of data result sets.

[0102] In step S103, according to the historical query records, the second number of second result sets in the first number of data result sets and the corresponding data validity period are determined to obtain second data information.

[0103] In an embodiment of the present invention, in the historical query record, the query record of each queried data result set is recorded, and the query result of each queried data result set and the corresponding data validity period are marked, and then compared with the first number of data result sets, the second number of second result sets in the first number of data result sets and the data validity period of each second result set can be determined, thereby obtaining the second data information.

[0104] In step S104, data result sets in the first number of data result sets that do not have query results are determined as third result sets, to obtain a third number of third result sets.

[0105] In an embodiment of the present invention, if no query result is found in the historical query record, it indicates that the data result set has not been queried, and multiple data sources need to be accessed in real time to obtain the corresponding query result. Therefore, the data result set that does not have a query result in the first number of data result sets is determined as the third result set, and the third number of third result sets is obtained by comparing with the historical query record.

[0106] In step S105, the second data information and the third number of third result sets are sent to the sender of the query statistics command to respond to the query statistics command.

[0107] In an embodiment of the present invention, the second data information and the third number of third result sets are sent to the sender of the query statistics command, so that the sender can directly obtain the query results of each second result set in the second number of second result sets, and it is clear that the third number of third result sets need to be obtained through real-time query, which helps to reduce the number of queries for data result sets, reduce time costs and computing acquisition costs, and thus help improve query efficiency.

[0108] In one embodiment, in order to meet the user's demand for the timeliness of the data result set, the data validity period of each second result set is compared with the current system time. If all data validity periods are greater than the current system time, the second data information and the third number of third result sets are sent to the sender of the query statistics command to ensure the validity of the query results.

[0109] Through the above embodiment, the second result set with query results and the corresponding data validity period in the data result set to be queried and counted can be quickly determined based on historical query results. When performing subsequent queries, the user only needs to query the third result set, which helps to reduce the number of queries for the data result set, reduce time cost and computing acquisition cost, and is conducive to improving query efficiency.

[0110] In one embodiment, in the process of sending the second data information and the third result set of the third quantity to the sender of the query statistics command, the data validity period of the query results of some second result sets may be exceeded due to factors such as system delay or network delay, making the query results of the part of the second result set invalid. Therefore, in order to avoid the occurrence of this situation, the data validity period of each second result set is compared with the current system time again. If there is at least one second result set whose data validity period is less than or equal to the current system time, the fourth quantity of the fourth result set is determined from the second data information. The fourth result set is the second result set whose data validity period is less than the specified time threshold. Re-query each fourth result set in the fourth result set of the fourth quantity, update the query results of each fourth result set in the fourth result set of the fourth quantity, and then obtain the fourth data information, and send the fourth data information to the sender of the query statistics command. So that the query results in the fourth result set are valid data.

[0111] In another embodiment, the fourth data information can be obtained in the following manner: in the historical query record, there is also a record of a query command set for querying each second result set. Therefore, through the historical query record, the query command set that has queried each fourth result set can be obtained, and the query command set of the fourth number is obtained. The query command set of the fourth number is sent to multiple data sources to re-query each fourth result set, thereby ensuring the accuracy of the re-query. When receiving the updated query result of each fourth result set, the query time of each fourth result set is determined respectively, so as to determine the data validity period of the corresponding fourth result set according to the query time, and then the fourth data information is obtained according to the updated query result of each fourth result set and the corresponding data validity period.

[0112] In a real-time scenario, the command statement for obtaining the data result set may include one or more of the following commands: [insert into row_set1(vir_name,max_time)as select vir_name,max(act_time)from tbl1 group by vir_name;], [insert into row_set1(vir_name,max_lvl)asselect vir_name,max(act_lvl)from tbl2 group by vir_name;], [select vir_name,max(max_time),max(max_lvl)from row_set1 group by vir_name;].

[0113] In another embodiment, the historical query record records the historical query time of obtaining the query result of each second result set. To facilitate the description of the determination process of the data validity period of each second result set, one of the second result sets in the second number of data result sets is used as an example for description. To facilitate the description, one of the second result sets in the second number of data result sets is replaced by the current second result set in the description.

[0114] According to the historical query records, obtain the historical query time of the current second result set. The sum of the historical query time and the specified update duration is determined as the effective duration of the current second result set. Among them, the specified update duration can be understood as the duration of the periodic refresh of the query results. The difference between the effective duration and the current system time is determined as the data validity period of the current second result set. For example: if the historical query time of the current second result set is 12:30 and the specified update duration is 30 minutes, the effective duration of the current second result set is 12:30-13:00. If the current system time is 12:45, the data validity period of the current second result set is 13:00-12:45=15 minutes.

[0115] The following embodiments will specifically illustrate the method for obtaining historical query records.

[0116] Figure 2 FIG. 1 is a flowchart of another method for obtaining historical query records according to an exemplary embodiment. Figure 2 As shown, the method for obtaining historical query records includes the following steps.

[0117] In step S201, a fifth number of target data set names and a target query command set corresponding to each target data set name are obtained from a preset configuration file.

[0118] In the embodiment of the present invention, the target data result set can be understood as a data result set that needs to be queried in advance to determine the query result. In a preset configuration file, the target data set name to be queried in advance and the target query command set corresponding to each target data set name can be pre-determined, and then the query can be performed according to the target query command set corresponding to each target data set name to obtain the query result of the target data result set corresponding to each target data set name.

[0119] In one implementation scenario, the configuration file includes a data definition dictionary library, which can specify the relevant content to be obtained, the type of relevant content, and the function of each content when querying the target data result set. Among them, the relevant content may include: the unique identification number (Identity Document, ID) of the target data result set, the target data set name (dataset_name) of the data result set, the specified update time (flash_time), the attribute information of the target data result set (dataset_attr), and the query command set (cmd_list) required to query the data result set to be queried. In one example, the relevant content can be set according to actual needs and is only used as an example in this application.

[0120] In one implementation scenario, the data definition dictionary library includes multiple data definition dictionaries, and different data definition dictionaries are used to determine and obtain different target data result sets. The data definition dictionary can be as shown in Table 1, and then the configuration file obtains the fifth number of target data set names and the target query command set corresponding to each target data set name according to the content recorded in the data definition dictionary library.

[0121] Table 1

[0122]

[0123] In step S202, multiple data sources are accessed respectively according to the target query command set corresponding to each target data set name to obtain the query result of the target data result set corresponding to each target data set name.

[0124] In the embodiment of the present invention, the data source is a database server or data file used to provide a data source for an application. According to the target query command set corresponding to each target data set name, multiple data sources are accessed respectively to obtain the required query data from the multiple data sources and obtain the query result of the corresponding target data result set.

[0125] In step S203, the query result of each target data result set is received, and the query time of each target data result set is determined to obtain the query record of each target data result set.

[0126] In step S204, the query records of each target data result set are saved respectively to obtain historical query records.

[0127] In the embodiment of the present invention, the historical query record includes second data information.

[0128] In one implementation scenario, according to the data definition dictionary library shown in Table 1, the query records of the target data result set can be obtained as shown in Table 2.

[0129] Table 2

[0130]

[0131] Through the above embodiment, by querying the fifth number of target data result sets in advance according to needs, historical query records are obtained, which helps to establish an association relationship between the target and the existing result set, and then when query statistics are subsequently performed according to the received query statistics command, it helps to reduce the time and computing resource overhead required for the operation, and effectively improve the overall operation efficiency and query speed.

[0132] In one embodiment, if the data validity period of the current target data result set is less than the specified time threshold, the query record of the current target data result set is refreshed to ensure the data validity of the current target data result set. The current target data result set is one of the target data result sets in the fifth number of target data result sets, and the specified time threshold can be understood as the shortest time interval for determining that the query result is valid data.

[0133] In one implementation scenario, the data definition dictionary library can be stored in a pre-built data pre-analysis engine, and then when query statistics are performed according to the query statistics command, the query statistics can be performed through the data pre-analysis engine. Figure 3 shown. Figure 3 The structure diagram of a data pre-analysis engine proposed according to an exemplary embodiment is shown in the figure. As shown in the figure, the data pre-analysis engine includes a disk, a memory, a data source access module, a syntax check module, a timer module and a result set positioning module.

[0134] The disk is used to store the data definition dictionary library to ensure that data is not lost when the power is off or the system is restarted.

[0135] The memory can be understood as a result set cache, which is used to store the data result sets that need to be shared, that is, the fifth number of target data result sets included in the historical storage record. Since the data result sets are time-sensitive and do not need to be kept for a long time, storing them in the memory helps to improve the efficiency of data storage and access, and thus helps to quickly obtain the second data information.

[0136] The data source access module is used to access multiple data sources according to the contents recorded in the data definition dictionary library to obtain the query records of each target data result set, and name the query records that have been successfully accessed using the corresponding target data set name in the data definition dictionary library. If the access fails (including: unable to access or unable to establish connections with multiple data sources), the records related to the target data result set in the data definition dictionary library are deleted, and the event is recorded in the system operation log.

[0137] The syntax verification module is used to verify the query records obtained so that each query record in the historical query records is a compliant record. For example, during the verification process, if the query record read meets the syntax specification, the query record is characterized as a compliant record and is retained. If the query record read does not meet the syntax specification, the query record is characterized as an uncompliant record and is deleted.

[0138] The timer module is used to refresh the query results of each data result set in the memory according to the specified update time to ensure the validity of each data result set. The query results include the target data set name, data validity period and attribute information of the current target data result set.

[0139] The result set locating module is used to determine a second number of second result sets according to the parsing results after receiving the query statistics command.

[0140] In another implementation scenario, the data pre-analysis engine can be deployed in an independent hardware server that supports Ethernet (more than 1000MB), has a permanent storage device (hard disk more than 2TB) and memory storage (more than 32GB), and can install an operating system.

[0141] Based on the same inventive concept, the present invention also provides another data query method, which is applicable to the scenario in which data needs to be obtained from multiple different data sources and combined for statistical analysis in an Internet cloud computing environment.

[0142] The data query method is applied to a data query party. The data query party may be an application server or a business framework platform. Both the application server and the business framework platform may support a Model-View-Controller (MVC) framework for data query. The application server or the business framework platform may establish a network connection with the data pre-analysis engine by modifying a configuration file.

[0143] In one implementation scenario, the content recorded in Table 3 can be supplemented into the configuration file of the application server or the business framework platform, thereby realizing the network connection with the data pre-analysis engine.

[0144] Table 3

[0145]

[0146] Figure 4 FIG. 1 is a flow chart of another data query method proposed according to an exemplary embodiment. Figure 4 As shown, the data query method includes the following steps S401 to S404.

[0147] In step S401, a data query request is received to obtain a query result of each data result set in a first number of data result sets to be queried and counted.

[0148] In the embodiment of the present invention, the query statistics command may be an application server or a business framework platform.

[0149] In step S402, the data query request is parsed to obtain second data information and a third number of third result sets.

[0150] In the embodiment of the present invention, the following can be used Figure 1-Figure 3 The second data information and the third number of third result sets obtained by any one of the data query methods provided in the present invention are as follows. The second data information includes the second number of second result sets and the data validity period of each second result set, the second result set is the data result set with query results in the first number of data result sets, and the third number of third result sets is the data result set without query results in the first number of data result sets.

[0151] In step S403, a data query request is sent to multiple data sources to obtain a query result of each third result set.

[0152] In step S404, the query results of each third result set are received and merged with the second data information to obtain the total query results of the first number of data result sets.

[0153] Through the above embodiments, it is possible to utilize the data result set with query results without changing the existing architecture to meet the user's needs for data query statistical analysis, thereby effectively reducing the time and computing power consumed in calculations and reducing overall losses, thereby improving the query speed and computing efficiency of multiple data sources.

[0154] In one embodiment, the data query method further includes receiving fourth data information to update the data validity period of the fourth number of fourth result sets in the second data information, thereby ensuring that the total query results of the first number of data result sets are all valid data.

[0155] In one implementation scenario, combined with Figure 3 , the flowchart of data query can be as follows Figure 5 shown. Figure 5 This is a flow chart of another data query method proposed according to an exemplary embodiment, wherein an application server, a business framework platform and multiple data sources share resources of the same server.

[0156] The application server generates an access process by loading the class name corresponding to the specified data pre-analysis engine to access the data pre-analysis engine. The access process is used to read the local configuration file of the application server, establish a network connection with the data pre-analysis engine, and obtain historical query records. The historical query records include query records of each target data result set in the fifth number of target data result sets.

[0157] The application server sends the query command statistics to the access process. The access process parses it and obtains the first number of data result sets to be queried and the second result sets with query results in the first number of data result sets, that is, the first number of data result sets and the second number of second result sets are obtained, and the data set name of each second result set is marked to obtain the corresponding query result from the historical access record. The access process sends the first number of data result sets, the second number of second result sets, and the data set name of each second result set to the data pre-analysis engine.

[0158] After receiving the first number of data result sets, the second number of second result sets, and the data set name of each second result set, the data pre-analysis engine compares the data set name of each second result set with the target data name of each target data result set in the historical query record to determine whether the historical query record contains the query record of each second result set in the second number of second result sets. If the historical query record does not contain the query record of each second result set in the second number of second result sets, an error message is issued and the engine exits. Based on the first number of data result sets and the second number of second result sets, a third result set is obtained in the first number of data result sets, which requires real-time access to multiple data sources to obtain the corresponding query results, that is, a third number of third result sets is obtained.

[0159] The data pre-analysis engine sends a request to the disk to obtain query records of each second result set in the second number of second result sets.

[0160] The disk feeds back the query records of each second result set to the data pre-analysis engine, so that the data pre-analysis engine obtains the second data information.

[0161] The data pre-analysis engine verifies the data validity period of each second result set according to the second data information. If all data validity periods are greater than the current system time, the second data information and the third number of third result sets are sent to the business framework platform. If the data validity period of the fourth number of fourth result sets in the second data information is less than or equal to the current system time due to system delay or network delay, all fourth result sets are acquired to obtain the fourth number of fourth result sets. According to the historical query records, the fourth number of query command sets are acquired, and the fourth number of query command sets are sent to multiple data sources to re-query each fourth result set. The data pre-analysis engine determines the query time of each fourth result set based on the updated query results of each fourth result set received, and then updates the data validity period of each fourth result set, thereby obtaining the fourth data information based on the updated query results and updated data validity period of each fourth result set. The fourth data information is sent to the business framework platform to update the second data information.

[0162] The business framework platform parses the data query request sent by the data pre-analysis engine to obtain the second data information and the third number of third result sets, and sends the data query request to multiple data sources to obtain the query results of each third result set.

[0163] The business framework platform combines the received third number of query results with the second data information to obtain the total query results of the first number of data result sets, and sends the first number of query results to the application server in the form of a response packet to complete the query statistics.

[0164] In another embodiment, the application server, the business framework platform, multiple data sources and the data pre-analysis engine may share resources of the same server, or may be established in different servers respectively.

[0165] Based on the same inventive concept, the present invention also provides a data query device.

[0166] Figure 6 is a structural block diagram of a data query device proposed according to an exemplary embodiment. Figure 6 As shown, the data query device includes a first receiving unit 601 , a parsing unit 602 , a first determining unit 603 , a second determining unit 604 and a first sending unit 605 .

[0167] The first receiving unit 601 receives a statistics query command;

[0168] A parsing unit 602 is used to parse the query statistics command to determine a first number of data result sets to be queried and counted;

[0169] A first determining unit 603 is used to determine, according to the historical query records, a second number of second result sets in the first number of data result sets and corresponding data validity periods, to obtain second data information, wherein the second result set is a data result set having query results;

[0170] A second determining unit 604 is configured to determine the data result sets that do not have the query results in the first number of data result sets as third result sets, to obtain a third number of third result sets;

[0171] The first sending unit 605 is used to send the second data information and the third number of third result sets to the sender of the query statistics command to respond to the query statistics command.

[0172] In one embodiment, the first sending unit 605 includes: a first sending sub-unit, used to compare the data validity period of each second result set with the current system time respectively, and if all data validity periods are greater than the current system time, the second data information and the third number of third result sets are sent to the sender of the query statistics command.

[0173] In another embodiment, the device further includes: a third determining unit, configured to determine a fourth number of fourth result sets from the second data information if there is at least one second result set whose data validity period is less than a specified time threshold, the fourth result set being a second result set whose data validity period is less than the specified time threshold. A first updating unit, configured to update the query result of each fourth result set in the fourth number of fourth result sets to obtain fourth data information. A second sending unit, configured to send the fourth data information to a sender of the query statistics command.

[0174] In another embodiment, the first update unit includes: a first acquisition unit, used to acquire a fourth number of query command sets based on historical query records, wherein different query command sets correspond to different fourth result sets. A third sending unit, used to send the fourth number of query command sets to multiple data sources to re-query each fourth result set. A second receiving unit, used to receive the updated query results of each fourth result set and determine the query time of each fourth result set. A first update subunit, used to update the data validity period of each fourth result set based on the query time of each fourth result set. A second acquisition unit, used to obtain fourth data information based on the updated query results of each fourth result set and the corresponding data validity period.

[0175] In another embodiment, the first determination unit 603 includes: a third acquisition unit, configured to acquire the historical query time of querying the current second result set according to the historical query record, wherein the current second result set is one of the second result sets in the second number of data result sets. A fourth determination unit, configured to determine the sum of the historical query time and the specified update duration as the effective duration of the current second result set. A fifth determination unit, configured to determine the difference between the effective duration and the current system time as the data effective period of the current second result set.

[0176] In another embodiment, the device further includes: a fourth acquisition unit, used to acquire a fifth number of target data set names and a target query command set corresponding to each target data set name from a preset configuration file, so as to acquire a fifth number of target data result sets, wherein the fifth number of target data result sets includes a second number of second result sets. A first access unit, used to access multiple data sources respectively according to the target query command set corresponding to each target data set name, so as to acquire the query result of the target data result set corresponding to each target data set name. A second receiving unit, used to receive the query result of each target data result set, and determine the query time of each target data result set, so as to obtain the query record of each target data result set. A storage unit, used to save the query record of each target data result set respectively, so as to obtain the historical query record, and the historical query record includes the second data information.

[0177] In another embodiment, the device further comprises: a refreshing unit, configured to refresh the query record of the current target data result set if the data validity period of the current target data result set is less than a specified time threshold, wherein the current target data result set is one of the target data result sets of the fifth number.

[0178] Figure 7 FIG. 1 is a structural block diagram of another data query device proposed according to an exemplary embodiment. Figure 7 As shown, the data query device includes a request receiving unit 701, a request parsing unit 702, a request sending unit 703 and a first data receiving unit 704.

[0179] A request receiving unit 701 is used to receive a data query request to obtain a query result of each data result set in a first number of data result sets to be queried and counted;

[0180] The request parsing unit 702 is used to parse the data query request to obtain second data information and a third number of third result sets, wherein the second data information includes a second number of second result sets and a data validity period of each second result set, the second result set is a data result set having query results in the first number of data result sets, and the third number of third result sets is a data result set not having query results in the first number of data result sets;

[0181] A request sending unit 703, configured to send a data query request to multiple data sources to obtain a query result of each third result set;

[0182] The first data receiving unit 704 is used to receive the query result of each third result set and merge it with the second data information to obtain the total query results of the first number of data result sets.

[0183] In one embodiment, the device further includes: a second data receiving unit, configured to receive fourth data information.

[0184] The specific limitations and beneficial effects of the above-mentioned data query device can be found in the above-mentioned limitations on the data query method, which will not be repeated here. The above-mentioned modules can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0185] Figure 8 FIG. 1 is a schematic diagram of a hardware structure of a computer device according to an exemplary embodiment. Figure 8 As shown, the device includes one or more processors 810 and a memory 820, and the memory 820 includes a persistent memory, a volatile memory, and a hard disk. Figure 8 A processor 810 is taken as an example. The device may also include: an input device 830 and an output device 840.

[0186] The processor 810, the memory 820, the input device 830 and the output device 840 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0187] The processor 810 may be a central processing unit (CPU). The processor 810 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. A general-purpose processor may be a microprocessor or the processor may be any conventional processor.

[0188] The memory 820 is a non-transient computer-readable storage medium, including a persistent memory, a volatile memory, and a hard disk, and can be used to store non-transient software programs, non-transient computer executable programs, and modules, such as program instructions / modules corresponding to the business management method in the embodiment of the present application. The processor 810 executes various functional applications and data processing of the server by running the non-transient software programs, instructions, and modules stored in the memory 820, that is, implementing any of the above-mentioned data query methods.

[0189] The memory 820 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data required for use, etc. In addition, the memory 820 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 820 may optionally include a memory remotely arranged relative to the processor 810, and these remote memories may be connected to the data processing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0190] The input device 830 can receive input digital or character information and generate key signal input related to user settings and function control. The output device 840 can include display devices such as a display screen.

[0191] One or more modules are stored in the memory 820, and when executed by one or more processors 810, the execution is as follows: Figure 1-Figure 7 The method shown.

[0192] The above-mentioned product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to Figure 1-Figure 5 Related description of the illustrated embodiment.

[0193] The embodiment of the present invention also provides a non-transitory computer storage medium, which stores computer executable instructions, and the computer executable instructions can execute the authentication method in any of the above method embodiments. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0194] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A data query method, characterized in that: The method comprises: Receive query statistics command; Parsing the query statistics command to determine a first number of data result sets to be queried and counted; Determine, according to the historical query records, a second number of second result sets in the first number of data result sets and corresponding data validity periods, and obtain second data information, wherein the second result set is a data result set having query results; Determine the data result sets that do not have query results in the first number of data result sets as third result sets, to obtain a third number of third result sets; Sending the second data information and the third number of third result sets to the data query party in response to the query statistics command; The sending the second data information and the third number of third result sets to the data querying party includes: The data validity period of each second result set is compared with the current system time respectively. If all data validity periods are greater than the current system time, the second data information and the third result set of the third quantity are sent to the data query party.

2. The method according to claim 1, characterized in that The method further comprises: If there is at least one second result set whose data validity period is less than or equal to the current system time, determine a fourth number of fourth result sets from the second data information, the fourth result set being a second result set whose data validity period is less than or equal to the current system time; updating the query result of each fourth result set in a fourth number of fourth result sets to obtain fourth data information; The fourth data information is sent to the data query party.

3. The method according to claim 2, characterized in that The updating of the query result of each fourth result set in the fourth number of fourth result sets to obtain fourth data information includes: According to the historical query records, a fourth number of query command sets are acquired, wherein different query command sets correspond to different fourth result sets; sending the fourth number of query command sets to multiple data sources to re-query each fourth result set; receiving the updated query result of each fourth result set, and determining the query time of each fourth result set; Based on the query time of each fourth result set, updating the data validity period of each fourth result set; The fourth data information is obtained according to each updated query result of the fourth result set and the corresponding data validity period.

4. The method according to claim 3, characterized in that According to the historical query records, determine the validity period of each second result set, including: According to the historical query record, obtaining the historical query time of querying the current second result set, wherein the current second result set is one of the second number of data result sets; Determine the sum of the historical query time and the specified update duration as the effective duration of the current second result set; The difference between the validity period and the current system time is determined as the validity period of the data of the current second result set.

5. The method according to claim 1, characterized in that The method further comprises: Obtaining, from a preset configuration file, a fifth number of target data set names and a target query command set corresponding to each target data set name; According to the target query command set corresponding to each target data set name, multiple data sources are accessed respectively to obtain the query result of the target data result set corresponding to each target data set name; Receive the query result of each target data result set, determine the query time of each target data result set, and obtain the query record of each target data result set; The query records of each target data result set are saved respectively to obtain historical query records, wherein the historical query records include the second data information.

6. The method according to claim 5, characterized in that The method further comprises: If the data validity period of the current target data result set is less than the specified time threshold, the query record of the current target data result set is refreshed; The current target data result set is one of the fifth number of target data result sets.

7. A data query method, characterized in that: The method comprises: Receive a data query request to obtain a query result of each data result set in a first number of data result sets to be queried and counted; Parsing the data query request to obtain second data information and a third number of third result sets, wherein the second data information includes a second number of second result sets and a data validity period of each second result set, the second result set is a data result set having query results in the first number of data result sets, and the third number of third result sets is a data result set not having query results in the first number of data result sets; Sending data query requests to multiple data sources to obtain query results of each third result set; receiving the query result of each third result set and combining it with the second data information to obtain the total query result of the first number of data result sets; Receiving the query result of each third result set and merging it with the second data information includes: The data validity period of each second result set is compared with the current system time respectively. If all data validity periods are greater than the current system time, the second data information and the third number of third result sets are received.

8. The method according to claim 7, characterized in that According to the historical query records, determine the validity period of each second result set, including: According to the historical query record, obtaining the historical query time of querying the current second result set, wherein the current second result set is one of the second number of data result sets; Determine the sum of the historical query time and the specified update duration as the effective duration of the current second result set; The difference between the validity period and the current system time is determined as the validity period of the data of the current second result set.

9. The method according to claim 7, characterized in that: The method further comprises: Obtaining, from a preset configuration file, a fifth number of target data set names and a target query command set corresponding to each target data set name; According to the target query command set corresponding to each target data set name, multiple data sources are accessed respectively to obtain the query result of the target data result set corresponding to each target data set name; Receive the query result of each target data result set, determine the query time of each target data result set, and obtain the query record of each target data result set; The query records of each target data result set are saved respectively to obtain historical query records, wherein the historical query records include the second data information.

10. The method according to claim 7, characterized in that The method further comprises: Receive fourth data information.

11. A data query device, characterized in that: The device comprises: A first receiving unit receives a statistical query command; A parsing unit, used for parsing the query statistics command to determine a first number of data result sets to be queried and counted; A first determining unit, configured to determine, according to the historical query records, a second number of second result sets in the first number of data result sets and corresponding data validity periods, to obtain second data information, wherein the second result set is a data result set having query results; A second determining unit, configured to determine the data result sets that do not have query results in the first number of data result sets as third result sets, to obtain a third number of third result sets; A first sending unit, configured to send the second data information and the third number of third result sets to a data query party in response to the query statistics command; The first sending unit includes: The first sending subunit is used to compare the data validity period of each second result set with the current system time respectively. If all data validity periods are greater than the current system time, the second data information and the third number of third result sets are sent to the data query party.

12. The device according to claim 11, characterized in that The device also includes: a third determining unit, configured to determine a fourth number of fourth result sets from the second data information if there is at least one second result set whose data validity period is less than or equal to the current system time, the fourth result set being a second result set whose data validity period is less than or equal to the current system time; A first updating unit is used to update the query result of each fourth result set in a fourth number of fourth result sets to obtain fourth data information; The second sending unit is used to send the fourth data information to the data query party.

13. The device according to claim 12, characterized in that The first updating unit comprises: A first acquisition unit, configured to acquire a fourth number of query command sets according to the historical query records, wherein different query command sets correspond to different fourth result sets; a third sending unit, configured to send the fourth number of query command sets to a plurality of data sources, so as to re-query each fourth result set; A second receiving unit, configured to receive each updated query result of the fourth result set and determine a query time of each fourth result set; A first updating subunit, configured to update a data validity period of each fourth result set based on a query time of each fourth result set; The second acquisition unit is used to obtain fourth data information according to each updated query result of the fourth result set and the corresponding data validity period.

14. The device according to claim 13, characterized in that The first determining unit includes: A third acquisition unit is used to acquire a historical query time of querying a current second result set according to the historical query record, wherein the current second result set is one of the second number of data result sets; A fourth determining unit, configured to determine the sum of the historical query time and the specified update duration as the effective duration of the current second result set; The fifth determining unit is used to determine the difference between the validity period and the current system time as the validity period of the data of the current second result set.

15. The device according to claim 11, characterized in that The device also includes: a fourth acquisition unit, configured to acquire, from a preset configuration file, a fifth number of target data set names and a target query command set corresponding to each target data set name; A first access unit is used to access multiple data sources respectively according to a target query command set corresponding to each target data set name, so as to obtain a query result of a target data result set corresponding to each target data set name; A second receiving unit is used to receive the query result of each target data result set, determine the query time of each target data result set, and obtain the query record of each target data result set; The storage unit is used to save the query record of each target data result set respectively to obtain a historical query record, wherein the historical query record includes the second data information.

16. The device according to claim 15, characterized in that The device also includes: A refresh unit, configured to refresh the query record of the current target data result set if the data validity period of the current target data result set is less than a specified time threshold; The current target data result set is one of the fifth number of target data result sets.

17. A data query device, characterized in that: The device comprises: A request receiving unit, configured to receive a data query request to obtain a query result of each data result set in a first number of data result sets to be queried and counted; a request parsing unit, configured to parse the data query request to obtain second data information and a third number of third result sets, wherein the second data information includes a second number of second result sets and a data validity period of each second result set, the second result set is a data result set having query results in the first number of data result sets, and the third number of third result sets is a data result set not having query results in the first number of data result sets; A request sending unit, used to send a data query request to multiple data sources to obtain a query result of each third result set; A first data receiving unit is used to receive the query result of each third result set and merge it with the second data information to obtain the total query result of the first number of data result sets; Receiving the query result of each third result set and merging it with the second data information includes: The data validity period of each second result set is compared with the current system time respectively. If all data validity periods are greater than the current system time, the second data information and the third number of third result sets are received.

18. The device according to claim 17, characterized in that The request parsing unit includes: A third acquisition unit is used to acquire a historical query time of querying a current second result set according to the historical query record, wherein the current second result set is one of the second number of data result sets; A fourth determining unit, configured to determine the sum of the historical query time and the specified update duration as the effective duration of the current second result set; The fifth determining unit is used to determine the difference between the validity period and the current system time as the validity period of the data of the current second result set.

19. The device according to claim 17, characterized in that The device also includes: a fourth acquisition unit, configured to acquire, from a preset configuration file, a fifth number of target data set names and a target query command set corresponding to each target data set name; A first access unit is used to access multiple data sources respectively according to a target query command set corresponding to each target data set name, so as to obtain a query result of a target data result set corresponding to each target data set name; A second receiving unit is used to receive the query result of each target data result set, determine the query time of each target data result set, and obtain the query record of each target data result set; The storage unit is used to save the query record of each target data result set respectively to obtain a historical query record, wherein the historical query record includes the second data information.

20. The device according to claim 17, characterized in that The device also includes: The second data receiving unit is used to receive fourth data information.

21. A computer device, characterized in that: It includes a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data query method according to any one of claims 1 to 6 or the data query method according to claims 7 to 10 by executing the computer instructions.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the data query method according to any one of claims 1 to 6 or the data query method according to claims 7 to 10.

Images