A data query method and device, computer equipment and a storage medium

Abstract

The present disclosure provides a data query method and device, computer equipment and a storage medium, comprising: receiving a query request carrying a target query field; determining an analysis data set matched with the target query field, wherein the analysis data set is a first data set containing initial data, or a second data set obtained by pre-aggregation processing of the initial data; determining target data corresponding to the target query field from the analysis data set.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and in particular, to a data query method and device, computer device and storage medium. BACKGROUND

[0002] When performing data query, a general user needs to manually input query conditions or manually select query conditions, and then initiates a query request based on the query conditions. After receiving the query request, the query system can perform data query according to the query conditions.

[0003] Then when the query needs to be performed in massive data, the input query conditions are relatively complex, and in this case, the query speed is slow. SUMMARY

[0004] The present disclosure provides at least a data query method and device, computer device and storage medium.

[0005] In a first aspect, the present disclosure provides a data query method, comprising:

[0006] receiving a query request carrying a target query field;

[0007] determining an analysis data set matched with the target query field, wherein the analysis data set is a first data set containing initial data, or a second data set obtained by pre-aggregation processing of the initial data;

[0008] determining target data corresponding to the target query field from the analysis data set.

[0009] In a possible implementation, the method further comprises pre-aggregation processing the initial data according to the following method to obtain the second data set:

[0010] determining at least one high-frequency query field;

[0011] respectively pre-aggregation processing the initial data according to each high-frequency query field to obtain at least one second data set.

[0012] In a possible implementation, the high-frequency query field includes a high-frequency query indicator and a high-frequency query dimension.

[0013] For any high-frequency query field, the method further comprises pre-aggregation processing the initial data according to the high-frequency query field according to the following method:

[0014] determining candidate initial data satisfying the high-frequency query dimension of the high-frequency query field;

[0015] Based on the candidate initial data, determine the index value corresponding to the high-frequency query index of the high-frequency query field, and the index value corresponding to the high-frequency query index constitutes a second data set corresponding to the high-frequency query field.

[0016] In a possible implementation, the pre-aggregation processing of the initial data according to each high-frequency query field to obtain at least one second data set comprises:

[0017] For any high-frequency query field, according to the aggregation period corresponding to the high-frequency query field, the initial data generated between the previous aggregation and the current time is pre-aggregated on the basis of the previous aggregated data set, to obtain the second data set.

[0018] In a possible implementation, the method further comprises determining the analysis data set matched with the target query field, comprising:

[0019] Matching the target query field with the high-frequency query field;

[0020] In the case of successful matching, the second data set corresponding to the matched high-frequency query field is taken as the analysis data set matched with the target query field;

[0021] In the case of failed matching, the first data set is taken as the analysis data set matched with the target query field.

[0022] In a possible implementation, the method further comprises determining the initial data according to the following method:

[0023] Obtaining source data from a data source;

[0024] Preprocessing the source data based on the processing conditions corresponding to each data index to obtain the initial data.

[0025] In a possible implementation, the method further comprises pre-aggregating the initial data according to the following method to obtain the second data set:

[0026] Obtaining the historical query record of the current user terminal;

[0027] Based on the historical query record, determining the aggregation field matched with the user terminal;

[0028] Pre-aggregating the initial data according to the aggregation field to obtain the second data set.

[0029] In a second aspect, the embodiments of the present disclosure further provide a data query device, comprising:

[0030] The receiving module is configured to receive a query request carrying a target query field;

[0031] The first determining module is configured to determine an analysis data set matching the target query field, wherein the analysis data set is a first data set containing initial data or a second data set obtained by pre-aggregation of the initial data;

[0032] The second determining module is configured to determine target data corresponding to the target query field from the analysis data set.

[0033] In a possible implementation, the first determining module is further configured to obtain the second data set by pre-aggregation of the initial data according to the following method:

[0034] determine at least one high-frequency query field;

[0035] pre-aggregate the initial data according to each high-frequency query field to obtain at least one second data set.

[0036] In a possible implementation, the high-frequency query field includes a high-frequency query dimension and a high-frequency query indicator.

[0037] For any high-frequency query field, the first determining module is further configured to pre-aggregate the initial data according to the high-frequency query field according to the following method:

[0038] determine candidate initial data satisfying the high-frequency query dimension of the high-frequency query field;

[0039] determine an indicator value corresponding to the high-frequency query indicator of the high-frequency query field based on the candidate initial data, and the indicator value corresponding to the high-frequency query indicator constitutes a second data set corresponding to the high-frequency query field.

[0040] In a possible implementation, when the first determining module pre-aggregates the initial data according to each high-frequency query field to obtain at least one second data set, the first determining module is configured to:

[0041] For any high-frequency query field, pre-aggregate initial data generated between the previous aggregation and the current time according to the high-frequency query field on the basis of the previous aggregated data set according to an aggregation period corresponding to the high-frequency query field to obtain the second data set.

[0042] In a possible implementation, when the second determining module determines an analysis data set matching the target query field, the second determining module is configured to:

[0043] match the target query field with the high-frequency query field;

[0044] In the case of successful matching, the second data set corresponding to the high-frequency query field of successful matching is taken as the analysis data set matched with the target query field.

[0045] In the case of failed matching, the first data set is taken as the analysis data set matched with the target query field.

[0046] In a possible implementation, the receiving module is further configured to determine the initial data according to the following method:

[0047] obtaining source data from a data source;

[0048] performing preprocessing on the source data based on processing conditions corresponding to each data index to obtain the initial data.

[0049] In a possible implementation, the first determining module is further configured to perform pre-aggregation processing on the initial data according to the following method to obtain the second data set:

[0050] obtaining historical query records of a current user terminal;

[0051] determining an aggregation field matched with the user terminal based on the historical query records;

[0052] performing pre-aggregation processing on the initial data according to the aggregation field to obtain the second data set.

[0053] In a third aspect, the embodiments of the present disclosure further provide a computer device, including a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the first aspect or any possible implementation of the first aspect.

[0054] In a fourth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the first aspect or any possible implementation of the first aspect.

[0055] The data query method, device, computer device and storage medium provided by the embodiments of the present disclosure can determine an analysis data set matched with a target query field in a query request after receiving the query request, and then determine target data corresponding to the target query field from the analysis data set. Here, the analysis data set matched with the target query field can be a first data set containing initial data, or a second data set obtained by pre-aggregation of the initial data. In this way, since the analysis data set can be the first data set containing initial data, the first data set can satisfy queries of multiple query fields. Since the second data set is pre-aggregated from the initial data, when the target data set matched with the target query field is the second data set, the target data meeting the target query field can be quickly queried, and the query speed is improved.

[0056] In order to make the above objectives, characteristics and advantages of the present disclosure more apparent, clear and easy to understand, the following will specifically describe the preferred embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. The drawings herein are incorporated into the specification and form a part of the specification, which illustrate the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be considered as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0058] Figure 1 A flow chart of a data query method provided by the embodiments of the present disclosure is shown;

[0059] Figure 2 A flow chart of a data query method provided by the embodiments of the present disclosure is shown;

[0060] Figure 3 An architectural schematic diagram of a data query device provided by the embodiments of the present disclosure is shown;

[0061] Figure 4 A structural schematic diagram of a computer device provided by the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0062] To make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the following will be combined with the accompanying drawings to make a clear and complete description of the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure and are not all the embodiments. The components of the embodiments of the present disclosure generally described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0063] With the increasing amount of data, the user often queries data from multiple dimensions when performing a query. When querying in massive data, each piece of data is generally traversed to select data meeting the query dimension set by the user.

[0064] For example, if the database queried contains 100 million pieces of data, when performing a query, each piece of data needs to be traversed to determine whether it meets the query dimension of the user, and finally data meeting the query dimension set by the user is queried. When the amount of data is large, the query speed of the piece-by-piece traversal method is slow.

[0065] Based on the above research, the present disclosure provides a data query method, device, computer device and storage medium, which can determine an analysis data set matching a target query field in a query request after receiving the query request, and then determine target data corresponding to the target query field from the analysis data set. Here, the analysis data set matching the target query field can be a first data set containing initial data, or a second data set obtained by pre-aggregation of the initial data. In this way, since the analysis data set can be the first data set containing initial data, the first data set can meet the query of multiple query fields; since the second data set is pre-aggregated from the initial data, when the target data set matching the target query field is the second data set, the target data meeting the target query field can be quickly queried, improving the query speed.

[0066] It should be noted that: similar reference numerals and letters indicate similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0067] The term "and / or", merely describes an associated relationship, which means that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of the plurality or any combination of at least two of the plurality, for example, including at least one of A, B, and C, which means including any one or more elements selected from the set consisting of A, B, and C.

[0068] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type of personal information involved in the present disclosure, the use range, the use scenario, etc. should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0069] For example, in response to receiving the active request of the user, the prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the software or hardware such as the electronic device, the application program, the server or the storage medium, etc. performing the operation of the technical solutions of the present disclosure according to the prompt information.

[0070] As an optional but non-limiting implementation manner, in response to receiving the active request of the user, the prompt information can be sent to the user in the form of a pop-up window, and the prompt information can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry selection controls for the user to select "agree" or "disagree" to provide the personal information to the electronic device.

[0071] It can be understood that the above notification and obtaining of the authorization of the user are only illustrative, and do not limit the implementation manners of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manners of the present disclosure.

[0072] For the convenience of understanding the present embodiment, first, a data query method disclosed by the present embodiment is introduced in detail, and the execution subject of the data query method provided by the present embodiment is generally a computer device with certain computing power, which includes, for example, a terminal device or a server or other processing device. The terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the data query method can be realized by calling the computer readable instructions stored in the memory by the processor.

[0073] Referring to Figure 1As shown, a flowchart of a data query method provided by an embodiment of the present disclosure is shown, and the method comprises steps 101-103, wherein:

[0074] Step 101: receiving a query request carrying a target query field.

[0075] Step 102: determining an analysis data set matching the target query field, wherein the analysis data set is a first data set containing initial data, or a second data set obtained by pre-aggregation processing of the initial data.

[0076] Step 103: determining target data corresponding to the target query field from the analysis data set.

[0077] The following is a detailed description of the above steps.

[0078] For step 101,

[0079] In one possible scenario, the method provided by the present disclosure can be applied to a server, and the receiving of the query request carrying the target query field can be receiving a query request carrying a target query field sent by a user terminal.

[0080] Here, the target query field can include a target query indicator and a target query dimension, and the specific target query indicator and target query dimension can be set by a user.

[0081] The target query dimension can include attribute information of a user corresponding to the data to be screened, such as a region, or can include attribute information of the data to be screened, such as a generation time. Here, the attribute information of the user corresponding to the data to be screened can refer to attribute information of a user generating the data, and the attribute information of the user can be obtained after legal authorization.

[0082] The target query indicator can refer to a measurement standard for measuring business or product performance, such as the number of clicks, the number of completions, the number of shares, etc.

[0083] In one possible implementation, the target query field can include multiple target query indicators and multiple target query dimensions, and thus in this case, each target query indicator together with multiple target query dimensions can constitute a query field, and the target query field includes multiple query fields.

[0084] For example, if the target query field includes target query indicators "click times" and "complete play times", and the target query dimensions include "time dimension - last 24 hours" and "region dimension - A region", the target query field includes two query fields, i.e., click times + last 24 hours + A region and complete play times + last 24 hours + A region.

[0085] When the target query field includes multiple target query indicators, in one possible implementation, since the initial data is aggregated based on a single query indicator when pre-aggregation is performed, the target query field can be split into multiple query fields based on the multiple target query indicators, and the corresponding analysis data sets are determined respectively; in another possible implementation, the initial data can be aggregated based on multiple query indicators when pre-aggregation is performed, and thus the target query field can be directly used for query, i.e., the analysis data set matched with the target query field is directly determined. The specific matching process will be described below, which is not described in detail here.

[0086] For steps 102 and 103,

[0087] The initial data can refer to data processed by data indicators. Specifically, the initial data can be obtained by first obtaining source data from a data source, and then pre-processing the source data based on processing conditions corresponding to each data indicator.

[0088] The processing conditions corresponding to different data indicators can be different. For example, for the data indicator "play times", the corresponding processing condition can be that a play time of more than 3 seconds is considered as one play. Therefore, for the data indicator "play times", when the source data is pre-processed, the play duration in the source data can be determined, and in the case where the play duration is more than 3 seconds, the play times of the source data can be marked as 1, otherwise, the play times of the source data can be marked as 0.

[0089] In one possible implementation, the second data set can be obtained by pre-aggregating the initial data according to certain fields. For example, at least one high-frequency query field can be determined first, and then the initial data is pre-aggregated according to each high-frequency query field to obtain at least one second data set.

[0090] Specifically, the high-frequency query field can be determined based on query fields carried in historical query requests of multiple users. For example, a query field with a query frequency exceeding a preset frequency and / or a query field with a query times exceeding a preset times can be used as the high-frequency query field.

[0091] In actual applications, the high-frequency query field can be updated continuously, for example, can be updated periodically. After the high-frequency query field is updated, in one possible manner, the second data set obtained based on the high-frequency query field before the update can be deleted, and the second data set obtained based on the high-frequency query field after the update can be obtained through re-pre-aggregation processing. Alternatively, the second data set obtained based on the high-frequency query field before the update can be retained, and the initial data can be re-pre-aggregated based on the high-frequency query field after the update, and the data set obtained through the pre-aggregation processing can also be the second data set.

[0092] It should be noted that the second data set can be a collective term of multiple data sets, rather than a certain data set. The specific pre-aggregation process of the second data set will be described below, and will not be described here.

[0093] In one possible implementation, when determining the analysis data set matched with the target query field, the target query field can be matched with each query field in the aggregation of the second data set, and the analysis data set matched with the target query field can be determined based on the matching result.

[0094] Specifically, taking the second data set obtained through the pre-aggregation processing based on the high-frequency query field as an example, when determining the analysis data set matched with the target query field, the target query field can be matched with the high-frequency query field. In the case of successful matching, the second data set corresponding to the high-frequency query field matched successfully can be taken as the analysis data set matched with the target query field. In the case of failed matching, the first data set can be taken as the analysis data set matched with the target query field.

[0095] Here, the target query field includes a target query index and a target query dimension, and the high-frequency query field includes a high-frequency query index and a high-frequency query dimension.

[0096] In one possible implementation, the second data set is pre-aggregated based on a high-frequency query field composed of a high-frequency query index and at least one high-frequency query dimension. In this case, when the target query field is matched with the high-frequency query field, it can be determined whether there is a high-frequency query index that contains a high-frequency query index identical to the target query index and contains a high-frequency query dimension identical to the target query dimension. If there is, it is determined that the matching is successful.

[0097] For example, if the high-frequency query field includes several dimensions and indexes as shown in the following table:

[0098] Table 1

[0099]

[0100]

[0101] If the target query field is composed of dimension 2 + dimension 3 + dimension 4, and the second high frequency query field includes the target query field, and the high frequency query dimensions in the second high frequency query field are identical to the target query dimensions in the target query field, then the second high frequency query field can be determined as the query field matched with the target query field.

[0102] In another possible implementation, the second data set can be pre-aggregated based on a plurality of high frequency query dimensions and at least one high frequency query dimension when pre-aggregation is performed. In this case, when the target query field is matched with the high frequency query field, it can be determined whether there is a high frequency query field including the target query dimension and having the same high frequency query dimensions as the target query dimensions. If there is, then it is determined that the matching is successful.

[0103] For example, if the high frequency query field includes several dimensions and dimensions as shown in the following table:

[0104] Table 2

[0105]

[0106] If the target query field is composed of dimension 2 + dimension 3 + dimension 4, and the second high frequency query field includes the target query field, and the high frequency query dimensions in the second high frequency query field are identical to the target query dimensions in the target query field, then the second high frequency query field can be determined as the query field matched with the target query field.

[0107] Here, no matter how many high frequency query dimensions the second data set is pre-aggregated with, when the target query field is matched with the high frequency query field, for any high frequency query field, if the target query dimension in the target query field needs to be included in the high frequency query field, and the high frequency query dimensions in the high frequency query field are identical to the target query dimensions in the target query field, then the high frequency query field can be determined as matched with the target query field. If the target query dimension in the target query field is not included in the high frequency query field, or the high frequency query dimensions in the high frequency query field are not identical to the target query dimensions in the target query field, then the high frequency query field can be determined as not matched with the target query field.

[0108] In a possible implementation, after determining the analysis data set matching the target query field, the target data corresponding to the target query field can be determined from the analysis data set.

[0109] Specifically, the second data set can include the index values of the query indicators corresponding to the pre-aggregation processing after the pre-aggregation processing of the second data set. For example, if the query indicators in the aggregation field (which can be the high-frequency query field) of the second data set are indicator A and indicator B, the second data set can include the index value of indicator A and the index value of indicator B.

[0110] Correspondingly, after taking the second data set as the analysis data set, the index value of the target query indicator in the target query field can be directly taken as the target data.

[0111] If the first data set is taken as the analysis data set, when determining the target data corresponding to the target query field from the analysis data set, each initial data in the first data set can be traversed, and then the initial data meeting the target query field is selected as the target data.

[0112] In the above method provided by the present disclosure, after receiving the query request, the analysis data set matching the target query field in the query request can be determined, and then the target data corresponding to the target query field can be determined from the analysis data set. Here, the analysis data set matching the target query field can be the first data set containing the initial data, or the second data set obtained by pre-aggregation processing of the initial data. In this way, since the analysis data set can be the first data set containing the initial data, the first data set can meet the query of various query fields. Since the second data set is pre-aggregated with the initial data, when the target data set matching the target query field is the second data set, the target data meeting the target query field can be quickly queried, and the query speed is improved.

[0113] The above pre-aggregation processing process will be introduced below.

[0114] In a possible implementation, when pre-aggregating the initial data, the initial data can be pre-aggregated based on the high-frequency query field according to the above embodiments.

[0115] Specifically, for any high-frequency query field, the initial data can be pre-aggregated according to the high-frequency query field according to the following steps:

[0116] Step A1, determining candidate initial data satisfying the high-frequency query dimension of the high-frequency query field.

[0117] Step A2, based on the candidate initial data, determining the index value corresponding to the high-frequency query index of the high-frequency query field, the index value corresponding to the high-frequency query index constitutes the second data set corresponding to the high-frequency query field.

[0118] Specifically, in the high-frequency query field, the dimension value under each high-frequency query dimension can be included, and the candidate initial data satisfying the high-frequency query dimension can be initial data with the same attribute information as the dimension value.

[0119] When determining the index value corresponding to the high-frequency query index of the high-frequency query field based on the candidate initial data, the candidate initial data can be statistically processed according to the statistical method corresponding to the high-frequency query index to determine the index value corresponding to the high-frequency query index.

[0120] The statistical method corresponding to different high-frequency query indexes can be different. For example, if the high-frequency query index is the number of plays, the number of candidate initial data corresponding to the number of plays of 1 in the candidate initial data can be determined as the index value of the number of plays (when processing the source data, the source data that can be counted as the number of plays can be marked as 1, and the source data that cannot be counted as the number of plays can be marked as 0); if the high-frequency query index is the total play time, the play time of each candidate initial data in the candidate initial data can be added, and the sum after addition is taken as the index value of the total play time.

[0121] In a possible implementation, the pre-aggregation process can be periodically performed. When the initial data is pre-aggregated according to each high-frequency field to obtain at least one second data set, for any high-frequency query field, the initial data generated between the previous aggregation and the current time can be pre-aggregated according to the high-frequency query field on the basis of the previous aggregated data set according to the aggregation period corresponding to the high-frequency query field, to obtain the second data set.

[0122] For example, if the high-frequency query index in the high-frequency query field is the number of clicks, the high-frequency query dimension is "A region", the value of the high-frequency query index "number of clicks" in the second data set after the Kth pre-aggregation is M, and the value of the high-frequency query index "number of clicks" after the K+1th pre-aggregation is N, the value of the high-frequency query index "number of clicks" in the second data set can be updated to M+N.

[0123] It should be noted that, when the initial data generated between the previous aggregation and the current time is pre-aggregated according to the high-frequency query field, the calculation method can be different for different high-frequency query indexes. For example, the value of the high-frequency query index "number of clicks" in the second data set can be updated by addition, or other calculation methods can be used for other high-frequency query indexes. The specific calculation method can be set according to actual conditions, and the present disclosure is not limited in this regard.

[0124] It should be noted that each high-frequency query field can include different high-frequency query dimensions and dimension values included in each high-frequency query dimension. Alternatively, each high-frequency query field can include the same high-frequency query dimension and different dimension values included in the high-frequency query dimension.

[0125] For example, the high-frequency query field A includes dimension 1, dimension 2 and dimension 3, and dimension value 1 under dimension 1, dimension value 2 under dimension 2, and dimension value 3 under dimension 3. The high-frequency query field B can include the same high-frequency query dimensions as the high-frequency query field A, i.e., the high-frequency query field B also includes dimension 1, dimension 2 and dimension 3, but the dimension values under each dimension of the high-frequency query field B are not completely the same as the dimension values under each dimension of the high-frequency query field A (they can be partially the same). For example, the high-frequency query field B can include dimension value 1 under dimension 1, dimension value 4 under dimension 2, and dimension value 5 under dimension 3.

[0126] In another possible implementation, when the initial data is pre-aggregated, the historical query conditions of different user terminals can also be combined for personalized pre-aggregation.

[0127] Specifically, the historical query records of the current user terminal can be obtained, and then based on the historical query records, the aggregation field matched with the user terminal is determined, and the initial data is pre-aggregated according to the aggregation field to obtain the second data set.

[0128] Here, when determining the aggregation field matching the user terminal based on the historical query records, for example, the high-frequency query field of the user terminal can be determined, and then the high-frequency query field of the user terminal is taken as the aggregation field. Here, the high-frequency query field of the user terminal is different from the high-frequency query field in the above embodiment. The high-frequency query field in the above embodiment can be understood as a high-frequency query field determined in combination with the historical query records of all user terminals. The high-frequency query field of the user terminal here can refer to a high-frequency query field only for the current user terminal.

[0129] Alternatively, when performing personalized aggregation, the weights of each preset query index and query dimension can be determined based on the historical query situation of the user terminal, and then the aggregation field is determined based on the weights of each query index and query dimension.

[0130] Specifically, when determining the weights of each preset query index and query dimension based on the historical query situation of the user terminal, for example, the query times of each query index and query dimension can be combined. For a query index or query dimension with a higher query time, a higher weight can be set. For a query index or query dimension with a lower query time, a lower weight can be set. Then, when the aggregation field is determined based on the weights of each query index and query dimension, the query indexes and query dimensions with weights greater than a preset value can be arranged and combined to obtain the aggregation field.

[0131] Since the query speed of the target data is faster when the second data set is taken as the analysis data set, after the user terminal initiates a query request, the user terminal can be recommended to display to recommend the user to query data from the second data set.

[0132] For example, the user terminal can display a recommended query field, which is a query field used when the initial data is pre-aggregated. After triggering any recommended query field, the user terminal can directly take the second data set corresponding to the recommended query field as the analysis data set.

[0133] In another possible implementation, the recommended query field can be related to the query service of the user terminal, and the recommended query field corresponding to each query service can be pre-set. When the query service of the user terminal is a query multimedia content delivery effect service, the recommended query field can be pre-set as a query field related to the multimedia content delivery effect.

[0134] In this way, the user can be prompted to select the second data set as the analysis data set to help the user perform fast query and improve query efficiency.

[0135] The above overall process will be introduced in combination with specific drawings. Referring to Figure 2 An overall flowchart of a data query method provided by an embodiment of the present disclosure is shown, including the following steps (the following steps do not necessarily be executed in the order described) :

[0136] First, source data is acquired from a data source, and then the source data is processed (i.e., preprocessed) to obtain processed data. On one hand, the data can be imported into a second data set after pre-aggregation processing, and on the other hand, the processed data can be imported into a first data set. A user can initiate a query service on a user analysis page, and a query service control module can dynamically select the first data set or the second data set as an analysis data set, and acquire target data from the analysis data set.

[0137] The specific description process is referred to the description of the above embodiment, which will not be repeated here.

[0138] Those skilled in the art can understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0139] Based on the same inventive concept, the present disclosure also provides a data query device corresponding to the data query method. Since the principle of solving problems by the device in the present disclosure is similar to the above-mentioned data query method of the present disclosure, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be repeated.

[0140] Referring to Figure 3 An architecture schematic diagram of a data query device provided by an embodiment of the present disclosure is shown, which includes a receiving module 301, a first determination module 302, and a second determination module 303. The device includes:

[0141] The receiving module 301 is configured to receive a query request carrying a target query field.

[0142] The first determination module 302 is configured to determine an analysis data set matched with the target query field, wherein the analysis data set is a first data set containing initial data, or a second data set obtained by pre-aggregation processing of the initial data.

[0143] The second determination module 303 is configured to determine target data corresponding to the target query field from the analysis data set.

[0144] In a possible implementation, the first determination module 302 is further configured to pre-aggregate the initial data to obtain the second data set according to the following method:

[0145] determining at least one high-frequency query field;

[0146] pre-aggregating the initial data according to each high-frequency query field to obtain at least one second data set.

[0147] In a possible implementation, the high-frequency query field includes a high-frequency query dimension and a high-frequency query indicator.

[0148] For any high-frequency query field, the first determining module 302 is further configured to pre-aggregate the initial data according to the high-frequency query field according to the following method:

[0149] determining candidate initial data that satisfies the high-frequency query dimension of the high-frequency query field;

[0150] based on the candidate initial data, determining an indicator value corresponding to the high-frequency query indicator of the high-frequency query field, and the indicator value corresponding to the high-frequency query indicator constitutes a second data set corresponding to the high-frequency query field.

[0151] In a possible implementation, when the first determining module 302 pre-aggregates the initial data according to each high-frequency query field to obtain at least one second data set, the first determining module 302 is configured to:

[0152] For any high-frequency query field, pre-aggregating the initial data generated between the previous aggregation and the current time according to the high-frequency query field on the basis of the previous aggregated data set according to an aggregation period corresponding to the high-frequency query field, to obtain the second data set.

[0153] In a possible implementation, when the second determining module 303 determines the analysis data set matched with the target query field, the second determining module 303 is configured to:

[0154] matching the target query field with the high-frequency query field;

[0155] in a case of successful matching, taking the second data set corresponding to the high-frequency query field matched successfully as the analysis data set matched with the target query field;

[0156] in a case of failed matching, taking the first data set as the analysis data set matched with the target query field.

[0157] In a possible implementation, the receiving module 301 is further configured to determine the initial data according to the following method:

[0158] obtaining source data from a data source;

[0159] Preprocess the source data based on the processing condition corresponding to each data index to obtain the initial data.

[0160] In a possible implementation, the first determining module 302 is further configured to perform pre-aggregation processing on the initial data according to the following method to obtain the second data set:

[0161] Obtain the historical query record of the current user terminal;

[0162] Determine the aggregation field matched with the user terminal based on the historical query record;

[0163] Perform pre-aggregation processing on the initial data according to the aggregation field to obtain the second data set.

[0164] The description of the processing procedure of each module in the apparatus and the interaction procedure between the modules can refer to the related description in the method embodiments, which will not be repeated here.

[0165] Based on the same technical concept, the embodiments of the present disclosure further provide a computer device. Referring to FIG. 4, Figure 4 The structure schematic diagram of the computer device 400 provided by the embodiments of the present disclosure includes a processor 401, a memory 402, and a bus 404. The memory 402 is used to store execution instructions, including an internal memory 4021 and an external memory 4022. The internal memory 4021 is also called an internal memory, which is used to temporarily store operation data in the processor 401 and exchange data with the external memory 4022 such as a hard disk. The processor 401 exchanges data with the external memory 4022 through the internal memory 4021. When the computer device 400 is running, the processor 401 and the memory 402 communicate through the bus 404, so that the processor 401 executes the following instructions:

[0166] Receive a query request carrying a target query field;

[0167] Determine an analysis data set matched with the target query field, wherein the analysis data set is a first data set containing initial data, or a second data set obtained by performing pre-aggregation processing on the initial data;

[0168] Determine target data corresponding to the target query field from the analysis data set.

[0169] In a possible implementation, the instructions executed by the processor 401 further include performing pre-aggregation processing on the initial data according to the following method to obtain the second data set:

[0170] Determine at least one high-frequency query field;

[0171] The initial data is pre-aggregated according to each high-frequency query field to obtain at least one second data set.

[0172] In a possible implementation, the high-frequency query field includes a high-frequency query dimension and a high-frequency query index.

[0173] For any high-frequency query field, the method further includes pre-aggregating the initial data according to the high-frequency query field according to the following method:

[0174] determining candidate initial data that satisfies the high-frequency query dimension of the high-frequency query field;

[0175] Based on the candidate initial data, determining the index value corresponding to the high-frequency query index of the high-frequency query field, and the index value corresponding to the high-frequency query index constitutes a second data set corresponding to the high-frequency query field.

[0176] In a possible implementation, the processor 401 executes instructions, and the pre-aggregation of the initial data according to each high-frequency query field to obtain at least one second data set includes:

[0177] For any high-frequency query field, the initial data generated between the previous aggregation and the current time is pre-aggregated according to the high-frequency query field on the basis of the previous aggregated data set according to the aggregation period corresponding to the high-frequency query field, to obtain the second data set.

[0178] In a possible implementation, the processor 401 executes instructions, and the determination of the analysis data set matched with the target query field includes:

[0179] Matching the target query field with the high-frequency query field;

[0180] In the case of successful matching, the second data set corresponding to the matched high-frequency query field is taken as the analysis data set matched with the target query field;

[0181] In the case of unsuccessful matching, the first data set is taken as the analysis data set matched with the target query field.

[0182] In a possible implementation, the processor 401 executes instructions, and the method further includes determining the initial data according to the following method:

[0183] Obtaining source data from a data source;

[0184] Preprocessing the source data based on the processing conditions corresponding to each data index to obtain the initial data.

[0185] In a possible implementation, the processor 401 executes instructions, and the method further includes performing pre-aggregation processing on the initial data according to the following method to obtain the second data set:

[0186] obtaining a historical query record of the current user terminal;

[0187] determining an aggregation field matched with the user terminal based on the historical query record;

[0188] performing pre-aggregation processing on the initial data according to the aggregation field to obtain the second data set.

[0189] The embodiment of the present disclosure further provides a computer readable storage medium, which stores a computer program. The computer program is run by a processor to perform the steps of the data query method described in the method embodiment. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0190] The embodiment of the present disclosure further provides a computer program product, which carries a program code. The program code includes instructions for performing the steps of the data query method described in the method embodiment. For details, refer to the method embodiment, which will not be repeated here.

[0191] The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0192] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. In several embodiments provided by the present disclosure, it should be understood that the disclosed system, device and method can be implemented by other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms.

[0193] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0194] In addition, each functional unit in various embodiments of the present disclosure can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0195] If the functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the part of the prior art or the part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present disclosure. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various program code storage media.

[0196] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and not to limit them, the protection scope of the present disclosure is not limited thereto, although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: any person skilled in the art in the technical range disclosed by the present disclosure, still can modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and all should be covered in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A data query method, characterized in that, include: Receive query requests that carry the target query fields; One of the first dataset containing initial data and the second dataset obtained by pre-aggregating the initial data according to at least one high-frequency query field is determined as the analysis dataset that matches the target query field; Determine the target data corresponding to the target query field from the analyzed dataset. The step of determining one of a first dataset containing initial data and a second dataset obtained by pre-aggregating the initial data according to at least one high-frequency query field as the analysis dataset for matching the target query field includes: Match the target query field with the at least one high-frequency query field; If a match is successful, the second dataset corresponding to the high-frequency query field that was successfully matched is used as the analysis dataset that matches the target query field, thereby directly determining the target data from the second dataset; In the event of a failed match, the first dataset is used as the analysis dataset that matches the target query field. The high-frequency query fields include query fields whose query count exceeds a preset number of times and / or query fields whose query frequency exceeds a preset frequency.

2. The method according to claim 1, characterized in that, The method further includes pre-aggregating the initial data according to the at least one high-frequency query field to obtain the second dataset: Determine the at least one frequently queried field; The initial data is pre-aggregated according to each high-frequency query field to obtain at least one second dataset.

3. The method according to claim 2, characterized in that, The high-frequency query fields include high-frequency query metrics and high-frequency query dimensions; For any frequently queried field, the method further includes pre-aggregating the initial data according to the frequently queried field using the following method: Identify candidate initial data for the high-frequency query dimensions that satisfy the high-frequency query field; Based on the candidate initial data, the indicator values ​​corresponding to the high-frequency query indicators of the high-frequency query field are determined, and the indicator values ​​corresponding to the high-frequency query indicators constitute the second dataset corresponding to the high-frequency query field.

4. The method according to claim 2, characterized in that, The step of pre-aggregating the initial data according to each high-frequency query field to obtain at least one second dataset includes: For any high-frequency query field, based on the aggregation period corresponding to that high-frequency query field, the initial data generated between the previous aggregation and the current time is pre-aggregated according to that high-frequency query field to obtain the second dataset.

5. The method according to claim 1, characterized in that, The method further includes determining the initial data according to the following method: Retrieve source data from the data source; The source data is preprocessed based on the processing conditions corresponding to each data indicator to obtain the initial data.

6. The method according to claim 1, characterized in that, The method further includes pre-aggregating the initial data according to the at least one high-frequency query field to obtain the second dataset: Retrieve the current user's historical query records; Based on the historical query records, determine the high-frequency aggregation fields that match the user's client. The second dataset is obtained by pre-aggregating the initial data according to the high-frequency aggregation field.

7. A data query device, characterized in that, include: The receiving module is used to receive query requests carrying the target query fields; The first determining module is used to determine one of a first dataset containing initial data and a second dataset obtained by pre-aggregating the initial data according to at least one high-frequency query field as an analysis dataset that matches the target query field; The second determining module is used to determine the target data corresponding to the target query field from the analysis dataset. The step of determining one of a first dataset containing initial data and a second dataset obtained by pre-aggregating the initial data according to at least one high-frequency query field as the analysis dataset for matching the target query field includes: Match the target query field with the at least one high-frequency query field; If a match is successful, the second dataset corresponding to the high-frequency query field that was successfully matched is used as the analysis dataset that matches the target query field, thereby directly determining the target data from the second dataset; In the event of a failed match, the first dataset is used as the analysis dataset that matches the target query field. The high-frequency query fields include query fields whose query count exceeds a preset number of times and / or query fields whose query frequency exceeds a preset frequency.

8. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the data query method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the data query method as described in any one of claims 1 to 6.

Images