Data query method supporting complex filtering conditions and related equipment
By dividing the query fields in the HR system into calculation classes and stock classes, and configuring the corresponding number-take framework, the problem that traditional systems are difficult to deal with complex filtering conditions is solved, and the support and efficiency of complex queries are improved.
Patent Information
- Application Number
- CN202411998606.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-05-02
AI Technical Summary
Traditional HR systems are difficult to support data queries under complex filtering conditions, especially when users need to use calculation-class fields as filtering conditions, they cannot directly obtain the required values from the stock table.
By dividing the query field into a calculation class field and a stock class field, and configuring a corresponding number-take framework for each type, the value of the calculation class field is calculated through the preset inventory class field and value formula, and the value of the inventory class field is directly obtained from the inventory table.
It realizes support for complex filtering conditions, can effectively obtain target data corresponding to the data query scheme, and improves the efficiency and flexibility of data query.
Smart Images
Figure CN119917533A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of data query, and in particular to a data query method and related equipment that support complex filtering conditions. Background Art
[0002] In the field of human resources (HR) management, data query and management are important parts of daily work. As the scale of enterprises expands, the amount of data that HR needs to process increases day by day, which requires the HR system to be able to not only store large amounts of data, but also to be able to quickly retrieve and analyze this data.
[0003] In traditional HR systems, users are allowed to select query fields from stock fields, and configure filter conditions for query fields based on fixed values or other stock fields other than the query field itself. For example, if the query field is the number of days on the job, the filter condition corresponding to the query field can be that the number of days on the job is not less than half a year, or that the number of days on the job is greater than the number of days the department was established. In this way, during the data query and filtering process, the HR system will directly obtain the values of each field required for comparison from the stock table, filter them, and finally obtain the data required by the user.
[0004] However, in actual applications, users may need the query field itself as a field in the corresponding filter condition. For example, an employee's performance score is higher than the average performance score of his department, where the average performance score of the department needs to be calculated based on the performance scores of each employee in the department and cannot be directly obtained from the stock table. It can be seen that the above complex filter conditions cannot adapt to the data query method of the traditional HR system. Summary of the invention
[0005] The embodiments of the present application provide a data query method and related devices that support complex filtering conditions, which are used to support data query under complex filtering conditions.
[0006] A first aspect of an embodiment of the present application provides a data query method supporting complex filtering conditions, including:
[0007] Acquire a data query scheme, the data query scheme including multiple query fields and filtering conditions corresponding to each of the query fields, the multiple query fields including calculation fields and stock fields, the filtering condition corresponding to at least one of the query fields including the calculation field, the query field belonging to the calculation field having a corresponding value formula, the value formula including a preset stock field, and the query field belonging to the stock field being derived from any stock table;
[0008] Determine a data retrieval framework for a calculation field and a data retrieval framework for an inventory field, wherein the data retrieval framework is used to obtain the value of a corresponding query field;
[0009] According to the data acquisition framework of the calculation type field, the data acquisition framework of the stock type field, the value acquisition formula, the stock table and the filtering condition corresponding to each of the query fields, the target data corresponding to the data query solution is obtained.
[0010] In a specific implementation, the filtering condition corresponding to each query field includes a first type of condition and / or a second type of condition, the first type of condition corresponding to the query field does not include a field belonging to a different stock table from the query field, the second type of condition corresponding to the query field includes a field belonging to a different stock table from the query field, and the acquiring of target data corresponding to the data query scheme according to the data acquisition framework of the calculation field, the data acquisition framework of the stock field, the value acquisition formula, the stock table, and the filtering condition corresponding to each query field includes:
[0011] If any of the query fields is a calculation field, a first value of the query field is obtained according to the first type of condition corresponding to the query field, the value formula corresponding to the query field, and the value acquisition framework of the calculation field;
[0012] If any of the query fields belongs to a stock field, obtaining a first value of the query field according to the first type of condition corresponding to the query field, the stock table to which the query field belongs, and the data acquisition framework of the stock field;
[0013] According to the corresponding second type of conditions, target data corresponding to the data query solution is filtered from the first value of each query field.
[0014] In a specific implementation, if any of the query fields is a calculation field, obtaining target data corresponding to the data query solution according to the data acquisition framework of the calculation field, the value acquisition formula, and the filtering condition corresponding to the query field includes:
[0015] Analyze the value formula corresponding to the query field to obtain a preset inventory field;
[0016] Substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain a second value of the query field;
[0017] Based on the filtering condition corresponding to the query field, the target data corresponding to the data query solution is screened out from the second value of the query field.
[0018] In a specific implementation, before substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain the second value of the query field, the method further includes:
[0019] If the preset stock class field is the same as any of the query fields, the value of the target field is used as the value of the preset stock class field;
[0020] If the preset stock class field is different from the multiple query fields, the value of the preset stock class field is obtained from the stock table to which the preset stock class field belongs.
[0021] In a specific implementation, if the value of the query field is an array, substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain the value of the query field includes:
[0022] Substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain the original value of the query field;
[0023] Each element in the original value is concatenated into a string, and the concatenated string is used as the value of the calculation class field.
[0024] In a specific implementation, if any of the query fields is a stock type field, then according to the data acquisition framework of the stock type field, the stock table from which the query field comes, and the filtering condition corresponding to the query field, the target data corresponding to the data query scheme is acquired, including:
[0025] Acquire a second value of the query field from the stock table of the query field source;
[0026] Based on the filtering condition corresponding to the query field, the target data corresponding to the data query solution is screened out from the second value of the query field.
[0027] A second aspect of an embodiment of the present application provides a computer device, including:
[0028] an acquisition unit, configured to acquire a data query scheme, wherein the data query scheme includes a plurality of query fields and a filtering condition corresponding to each of the query fields, wherein the plurality of query fields include a calculation field and a stock field, wherein the filtering condition corresponding to at least one of the query fields includes the calculation field, and the query field belonging to the calculation field has a corresponding value formula, wherein the value formula includes a preset stock field, and the query field belonging to the stock field is derived from any stock table;
[0029] A determination unit, used to determine a data acquisition framework for a calculation type field and a data acquisition framework for a stock type field, wherein the data acquisition framework is used to obtain a value of a corresponding type query field;
[0030] The acquisition unit is also used to acquire target data corresponding to the data query solution based on the data acquisition framework of the calculation type field, the data acquisition framework of the stock type field, the value acquisition formula, the stock table and the filtering conditions corresponding to each of the query fields.
[0031] In a specific implementation, the filtering conditions corresponding to each of the query fields include first-category conditions and / or second-category conditions, the first-category conditions corresponding to the query fields do not include fields that belong to different stock tables from the query fields, and the second-category conditions corresponding to the query fields include fields that belong to different stock tables from the query fields. The acquisition unit is specifically used to, if any of the query fields belongs to a calculation field, acquire the first value of the query field according to the first-category conditions corresponding to the query field, the value-taking formula corresponding to the query field, and the data-taking framework of the calculation field; if any of the query fields belongs to a stock field, acquire the first value of the query field according to the first-category conditions corresponding to the query field, the stock table to which the query field belongs, and the data-taking framework of the stock field; and filter the target data corresponding to the data query scheme from the first value of each of the query fields according to the corresponding second-category conditions.
[0032] In a specific implementation, if any of the query fields is a calculation type field, the acquisition unit is specifically used to parse the value formula corresponding to the query field to obtain a preset stock type field; substitute the value of the preset stock type field into the value formula corresponding to the query field to obtain a second value of the query field; and based on the filtering condition corresponding to the query field, filter the target data corresponding to the data query scheme from the second value of the query field.
[0033] In a specific implementation, before substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain the second value of the query field,
[0034] The determining unit is further configured to use the value of the target field as the value of the preset stock class field if the preset stock class field is the same as any of the query fields;
[0035] The determining unit is further configured to obtain a value of the preset stock class field from the stock table to which the preset stock class field belongs if the preset stock class field is different from the multiple query fields.
[0036] In a specific implementation, if the value of the query field is an array, the acquisition unit is specifically used to substitute the value of the preset inventory type field into the value formula corresponding to the query field to obtain the original value of the query field; concatenate each element in the original value into a string, and use the concatenated string as the value of the calculation type field.
[0037] In a specific implementation, if any of the query fields is a stock field, the acquisition unit is specifically used to obtain a second value of the query field from the stock table from which the query field comes; based on the filtering condition corresponding to the query field, the target data corresponding to the data query scheme is filtered out from the second value of the query field.
[0038] A third aspect of an embodiment of the present application provides a computer device, including:
[0039] CPU, memory and input / output interface;
[0040] The memory is a short-term storage memory or a persistent storage memory;
[0041] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method described in the first aspect.
[0042] A fourth aspect of the embodiments of the present application provides a computer program product comprising instructions, and when the computer program product is run on a computer, the computer is caused to execute the method described in the first aspect.
[0043] A fifth aspect of an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the method described in the first aspect.
[0044] It can be seen from the above technical scheme that the embodiment of the present application has the following advantages: all query fields are divided into calculation fields and stock fields, wherein the value of the calculation field needs to be calculated by the preset stock field and the corresponding value formula, while the value of the stock field can be directly obtained from the stock table to which it belongs. Specifically, for the different value acquisition methods of the above two fields, the embodiment of the present application configures the corresponding data acquisition framework for them respectively to obtain their corresponding values. In this way, after obtaining the value of each query field, the target data corresponding to the data query scheme can be filtered out according to the corresponding filtering conditions. It should be noted that in the embodiment of the present application, the filtering condition corresponding to at least one stock field includes a calculation field. The value of the calculation field is obtained through the configured calculation field. The value of the corresponding stock field can be filtered based on the filtering condition containing the calculation field under the original query method, thereby realizing data query under complex filtering conditions. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 A system architecture diagram of the data query method disclosed in the embodiment of the present application;
[0046] Figure 2 A schematic diagram of a flow chart of a data query method disclosed in an embodiment of the present application;
[0047] Figure 3 An example diagram of obtaining target data from a stock table disclosed in an embodiment of the present application;
[0048] Figure 4 Another schematic diagram of the data query method disclosed in the embodiment of the present application;
[0049] Figure 5 An example diagram of the array type values disclosed in the embodiment of the present application;
[0050] Figure 6 Another structural schematic diagram of a computer device disclosed in an embodiment of the present application;
[0051] Figure 7 Another structural schematic diagram of the computer device disclosed in the embodiment of the present application. DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0053] For traditional HR systems, they generally only support the configuration of query solutions and the display of target data. The step of obtaining target data that meets the filtering conditions from the stock table is usually completed by calling the existing data acquisition engine. The data acquisition logic of the existing data acquisition engine is to directly obtain the value of the query field from the stock table, and compare the value of each query field based on a simple numerical filtering condition, and filter out the target data. It can be seen that if the filtering condition of a query field in the embodiment of the present application contains a calculated field whose value cannot be directly obtained from the corresponding stock table, the data acquisition engine cannot directly obtain the value of the calculated field used for comparison in the filtering condition, and then cannot compare it with the value of each query field based on this, and filter out the target data. Among them, if the filtering condition of a query field in the embodiment of the present application contains a calculated field, or a query field itself is a calculated field, the data acquisition engine cannot complete the filtering or value acquisition of such query fields under the existing business logic.
[0054] In order to solve the above technical problems, the embodiments of the present application provide a data query method and related devices that support complex filtering conditions, which are used to support data query under complex filtering conditions.
[0055] In order to better implement the data query method of the embodiment of the present application, the embodiment of the present application provides the following Figure 1 The data query framework shown includes a request end and a query end, wherein the request end can be a different business end, which is used to send a configured data query plan, and the query end is used to query the target data that matches the corresponding data query plan according to the data acquisition framework corresponding to different query fields, and feed it back to the request end. In addition, in the case of limited computing resources, the request end can also determine the value of the calculation class field according to the data acquisition framework of the calculation class field sent by the query end, the value acquisition formula corresponding to the calculation class field, and the value of the stock class field preset in the value acquisition formula, and feed back the determined value of the calculation class field to the query end for further filtering and querying.
[0056] In some specific implementations, if there are a large number of data query solutions sent by different request ends, the query end can be split into two parts: a middle platform and a data retrieval engine. The middle platform is used to provide and store the data retrieval framework, receive data query solutions sent from different request ends, and assign the data query solutions sent from different request ends to query data retrieval processing, while the data retrieval engine is used to query the target data that matches the corresponding data query solution and feed it back to the request end.
[0057] See also Figure 2Based on the aforementioned data query framework, the embodiment of the present application provides a data query method supporting complex filtering conditions, which is applied to the query end and / or the request end, and includes the following steps:
[0058] 201. Obtain a data query plan, the data query plan including multiple query fields and filtering conditions corresponding to each query field, the multiple query fields including calculation fields and stock fields belonging to any stock table, the filtering conditions corresponding to at least one query field including calculation fields, the query field belonging to the calculation field having a corresponding value-taking formula, the value-taking formula including a preset stock field, the query field belonging to the stock field originating from any stock table.
[0059] A complete data query solution includes at least query fields and filter conditions corresponding to the query fields. In the embodiment of the present application, in order to realize queries based on complex filter conditions, the query fields are divided into two categories, one of which is a calculation field and the other is an inventory field. It should be noted that the value of the inventory field can be directly obtained from the corresponding inventory table, that is, the inventory field comes from any inventory table; and the value of the calculation field is determined by the corresponding value formula, and the value formula includes the preset inventory field, and the value formula is based on the logical operation of the preset inventory field to obtain the logical expression required for the value of the calculation field. Among them, the logical operation includes but is not limited to various functions and mathematical and / or logical calculations such as addition, subtraction, multiplication and division. For example, the value formula is the value of the calculation field = the value of the preset inventory field 1 + the value of the preset inventory field 2.
[0060] In other words, the value of the calculation class field needs to be calculated through the value of at least one stock class field. However, the preset stock class field included in the value formula does not necessarily belong to the query field of the data query scheme of the embodiment of the present application. In addition, if the multiple query fields included in the data query scheme include multiple calculation class fields, such as calculation class field 1 and calculation class field 2, then the value formula of calculation class field 1 can also include other calculation class fields besides itself, that is, calculation class field 2.
[0061] 202. Determine a data acquisition framework for a calculation type field and a data acquisition framework for an inventory type field, wherein the data acquisition framework is used to obtain a value of a corresponding type query field.
[0062] On the basis of the foregoing embodiments, the embodiments of the present application abstract data acquisition frameworks for query fields of different types, and obtain the value of each query field based on the corresponding data acquisition framework, wherein the data acquisition framework for calculation type fields is used to obtain the value of calculation type fields, and the data acquisition framework for stock type fields is used to obtain the value of stock type fields, and query fields of the same type share a set of query frameworks, which include all the steps required for acquiring data for fields of the corresponding type, thereby expanding the original data acquisition logic.
[0063] It is understandable that in addition to the simple data acquisition process for obtaining the value of the corresponding query field in the data acquisition framework, a second process can be added to the data acquisition process. It is understandable that for different technical implementation solutions in the specification,
[0064] The query field is filtered using the filter conditions corresponding to the query field, and the values retained after filtering are the target data corresponding to the data query solution.
[0065] 203. According to the data acquisition framework of the calculation type field, the data acquisition framework of the stock type field, the value acquisition formula, the stock table and the filtering condition corresponding to each query field, the target data corresponding to the data query solution is obtained.
[0066] Since different types of query fields have corresponding data retrieval frameworks, and the calculation of the value of the calculation field depends on the corresponding value formula, and the value of the stock field depends on the stock table, and both the calculation field and the stock field have corresponding filtering conditions, the embodiment of the present application obtains the target data corresponding to the data query solution based on the actual situation.
[0067] It should be noted that the embodiment of the present application can query and obtain at least one set of target data, each set of target data contains a value of all query fields. If the query fields in the data query solution include the age field and the name field, the filter condition of the age field is to take a number less than 30 years old, and the filter condition of the name field is all names, then Figure 3 The target data of the age field and name field can be found in the age table and name table, as shown in Figure 3 All the values in a column corresponding to each field in the multiple sets of target data shown. Figure 3Each row of data shown in the multiple groups of target data tables is a group of target data. If the stock-type fields included in the multiple query fields and the stock-type fields preset in the value-taking formula all belong to the same business table, the target data whose values of each query field in the business table meet the corresponding filtering conditions can be selected, and multiple values of each query field can be combined to form multiple groups of target data. In addition, if the stock-type fields included in the multiple query fields of the embodiment of the present application and the stock-type fields preset in the value-taking formula are from multiple business tables, the association relationships between the business entries corresponding to each field also need to be combined.
[0068] In the embodiment of the present application, all query fields are divided into calculation fields and stock fields, wherein the value of the calculation field needs to be calculated by the preset stock field and the corresponding value formula, while the value of the stock field can be directly obtained from the stock table to which it belongs. Specifically, for the different value acquisition methods of the above two fields, the embodiment of the present application configures corresponding data acquisition frameworks for them respectively to obtain their corresponding values. In this way, after obtaining the value of each query field, the target data corresponding to the data query scheme can be filtered out according to the corresponding filtering conditions. It should be noted that in the embodiment of the present application, the filtering condition corresponding to at least one stock field includes a calculation field. The value of the calculation field is obtained through the configured data acquisition framework of the calculation field, and the value of the corresponding stock field can be filtered based on the filtering condition containing the calculation field under the original query method, thereby realizing data query under complex filtering conditions.
[0069] In some specific implementations, if the query field is a calculation type field, then in an embodiment of the present application, the target data corresponding to the data query scheme under the query field can be obtained according to the data acquisition framework, the value formula of the calculation type field and the filtering condition corresponding to the query field. The steps specifically include the following steps: parsing the preset stock type field from the value formula corresponding to the query field; substituting the value of the preset stock type field into the value formula corresponding to the query field to obtain a second value of the query field; and based on the filtering condition corresponding to the query field, filtering the target data corresponding to the data query scheme from the second value of the query field.
[0070] Specifically, the configuration of the data extraction formula corresponding to the calculation type field depends on the various calculation type fields and / or stock type fields that have been constructed, and the various calculation type fields are also calculated through the corresponding stock type fields, which is equivalent to indirectly including the stock type fields. In other words, by parsing the preset stock type fields directly or indirectly included in the data extraction formula, the embodiment of the present application can obtain each field used in the process of configuring the data extraction formula. Among them, the calculation type field and its corresponding data extraction formula can be pre-configured, or configured in real time when configuring the data query plan, which is not limited here.
[0071] After obtaining the value of the preset inventory field, directly substitute it into the value formula corresponding to the query field to obtain the value of the query field, that is, the second value. Finally, according to the filtering condition corresponding to the query field, the target data corresponding to the data query solution can be screened from multiple second values of the query field.
[0072] Generally, the embodiments of the present application can filter the target data based on the corresponding filtering conditions after obtaining all the values of each query field. Therefore, in order to ensure the overall data query efficiency, after obtaining each preset stock class field contained in the calculation class field, if it is found that the preset stock class field is the same as any query field, there is no need to re-obtain or calculate the value of the query field, but the value of any of the aforementioned query fields can be directly used as the value of the preset stock class field; if it is found that the preset stock class field is different from each query field, it is necessary to obtain the value of the preset stock class field from the corresponding stock table. Any of the aforementioned query fields can be a calculation class field or a stock class field, which is not limited here.
[0073] In some other specific implementations, if the query field is a stock type field, then in an embodiment of the present application, the target data corresponding to the data query scheme under the query field can be obtained based on the data acquisition framework of the stock type field, the stock table to which the query field belongs, and the filtering condition corresponding to the query field. The steps specifically include the following steps: obtaining a second value of the query field from the stock table from which the query field comes; and based on the filtering condition corresponding to the query field, filtering the target data corresponding to the data query scheme from the second value of the query field.
[0074] Specifically, it is different from the complex data acquisition logic of the aforementioned calculation-type fields. The stock-type fields of the embodiment of the present application are from the stock table. Therefore, for each stock-type field, after determining the stock table to which the stock-type field belongs, the value of the stock-type field can be directly obtained from the stock table to which the stock-type field belongs.
[0075] In some specific implementations, in order to reduce the occupancy of computing resources on the query end, the value of the inventory calculation class field can also be calculated by the request end, specifically sending the preset value of the inventory class field and the corresponding value formula to the request end; receiving the value of the calculation class field sent by the request end, the value of the calculation class field is determined by the request end based on the preset value of the inventory class field and the corresponding value formula.
[0076] In general, such as Figure 1In the embodiment shown, the embodiment of the present application can be run on the request end and the query end. Among them, the request end is used to configure and send a data query plan to the query end, and can receive the value of the preset stock class field and the corresponding value formula sent by the query end, and substitute the value into the value formula with reference to the above embodiment to obtain the value of the stock class field, which will not be repeated here. Finally, the calculated value of the corresponding calculation class field is returned to the query end for the query end to filter the target data.
[0077] Furthermore, in actual applications, the filtering conditions corresponding to each query field can be further divided into first-category conditions and / or second-category conditions, wherein the first-category conditions corresponding to the query field do not include fields that belong to different stock tables than the query field, and the second-category conditions corresponding to the query field include fields that belong to different stock tables than the query field. Based on this, the aforementioned step 203 specifically includes: if any query field belongs to a calculation field, then according to the first-category conditions corresponding to the query field, the value-taking formula corresponding to the query field, and the data-taking framework of the calculation field, the first value of the query field is obtained; if any query field belongs to a stock field, then according to the first-category conditions corresponding to the query field, the stock table to which the query field belongs, and the data-taking framework of the stock field, the first value of the query field is obtained; according to the corresponding second-category conditions, the target data corresponding to the data query scheme is filtered from the first value of each query field.
[0078] Specifically, the first type of condition corresponding to the query field can be used in the process of obtaining the value of the stock class field from the stock table, because it does not depend on the stock class field of other stock tables or any calculation class field, while the second type of condition corresponding to the query field is different, because it depends on the stock class field or calculation class field of other stock tables. Therefore, in the process of obtaining the value of each query field, regardless of whether the query field belongs to the calculation class field or the stock class field, all values are not directly obtained or calculated, but combined with the corresponding first type of condition, at least part of all values of the query field are obtained as the first value of the query field. Then, according to the corresponding second type of condition, the target data corresponding to the data query scheme is filtered from the first value of each query field. In this way, when filtering the target data corresponding to the data query scheme from the first value of each query field, the amount of data required to be traversed is significantly reduced, which can effectively improve the efficiency of data query.
[0079] Furthermore, if the query end or the data retrieval engine of the query end does not support filtering and comparison of array type values, then the aforementioned step of "substituting the value of the preset stock class field into the value formula corresponding to the query field to obtain the value of the query field" can be implemented in the following way: substituting the value of the preset stock class field into the value formula corresponding to the query field to obtain the original value of the query field; concatenating each element in the original value into a string, and using the concatenated string as the value of the calculation class field.
[0080] Specifically, without affecting the values of each element in the array, this one-dimensional array can be serialized into a string. And this string is used as the value of the corresponding calculation field. For example, the value of the array type is serialized into a string in json format, and it is calculated and compared in the form of a string in the query end or the query end's data retrieval engine. For specific array type values, please refer to Figure 5 The value of the personnel array field in the illustrated embodiment.
[0081] The above describes various embodiments of the present application. Figure 4 , in a specific business scenario, the data query solution of the embodiment of the present application is explained.
[0082] First, the data query system of the embodiment of the present application includes a request end, a middle platform, and a data acquisition engine, wherein the request end is used to initiate a data query request, the middle platform is used to obtain a data query solution according to the data query request, and the data acquisition engine is used to execute this data query solution. In addition, the data acquisition frameworks of different types of query fields can be encapsulated in different types of methods and obtained through the microservice interface provided by the middle platform.
[0083] After receiving the data query request, the middle station will execute the following steps 1-4:
[0084] 1. Build a data source: determine the stock-type fields in multiple query fields and the stock table to which the preset stock-type fields belong.
[0085] 2. Obtain the data acquisition framework corresponding to different types of query fields: This is the encapsulation method proposed in the embodiment of the present application. Here, the middle station can obtain the corresponding method according to the type of the query field. This corresponding method is configured with a custom map function for the corresponding type of query field as the data acquisition framework for the corresponding type of query field. When it is recognized that the data query plan sent by the query end contains a query field as a calculation type field, the corresponding method is called to obtain the map function corresponding to the calculation type field, and the processing of the corresponding type of calculation type field is completed based on this map function.
[0086] In actual applications, the map function corresponds to the implementation of an abstract class, that is, the implementation of a calculation method, which contains the method of obtaining the value of the calculation class field, or the method of parsing the calculation class field. The input parameters of this calculation method during operation are the values of various calculation class fields and / or inventory class fields that have been constructed (on which the calculation class field depends), and the output is the value of the corresponding calculation class field. In addition, the implementation of this abstract class also includes the following global parameters, such as global header data and related attributes of the calculation class field, such as field name and data type.
[0087] 3. Analysis of filtering and sorting conditions: Here, the corresponding filtering and sorting rules are set according to the filtering and sorting conditions configured for the current query field.
[0088] 4. Send to the data acquisition engine: The current step is for the middle platform to build the complete query process code of the current data query solution, and the constructed code is run by the data acquisition engine.
[0089] After the middle station completes the above steps 1-4, the data acquisition engine performs the following steps 5-6 based on the centroid of the received data query plan:
[0090] 5. Multi-threaded and multi-operator data output: refers to the difference in data output, which can be single-instance multi-threaded data acquisition or multi-instance deployment of multi-threaded data acquisition and calculation.
[0091] 6. Obtain target data: After running the query code constructed in step 4, the data acquisition engine obtains and outputs the target data.
[0092] Furthermore, in actual applications, the microservice interface provided by the middle platform is used to standardize the input and output parameters of the downstream implementation, restrict the downstream implementation through the interface, and allow the middle platform to call the microservice interface of the downstream business party in a universal manner in the form of an interface to obtain the downstream custom map function.
[0093] In addition, in some specific implementations, the request end, the middle station, and the data acquisition engine need to add map function classes respectively to realize the processing of different types of query fields in different stages. The definition and description of the map function class in each device of the specific embodiment of the present application are as follows.
[0094] Map function class of the data acquisition engine: This class is a custom map function class exposed by the data acquisition engine. In actual applications, the data acquisition engine will first obtain data from each inventory table according to each query field, and continuously store the obtained data in the query data set until the data acquisition is completed. In order to ensure the data acquisition performance of the data acquisition engine, the data acquisition engine is only used to obtain the values of each query field, and the values of the calculation class fields that need to be obtained through the corresponding calculation method can be handed over to the downstream middle station and the downstream request end for processing. For example, when encountering a query field that belongs to a calculation class field, the data acquisition engine will obtain the values of various calculation class fields and / or inventory class fields that have been built on which this calculation class field depends from the query data set, and build a calculation data set based on this, and send this calculation data set to the middle station, and receive the result data set including the values of the calculation class fields returned by the middle station through the corresponding interface, and add them to the query data set. Among them, the addition of the calculation class field header in the query data set and the addition of the values of the calculation class fields can be realized through the map function class of the data acquisition engine. The interface of the map function in the data acquisition engine provides the getRowMeta method, which is used to obtain the column field header data after data conversion; the map function method is to obtain the object of each row data after conversion.
[0095] The map function class of the middle station: This class is an abstract base class repackaged in the embodiment of the present application, which is used to provide the downstream request end with the implementation of customized data conversion logic, and at the same time restricts the downstream request end to only add columns to the data, but not to modify other column data. The getPluginFieldValue method takes the current row data as input, and returns the value of the new column data calculated based on the current row data. The value returned is an Object object, so it is compatible with different types of newly added column result values, such as the values of newly added data types such as array types.
[0096] In the base class provided by the middle platform, the modification logic of the data set header and the filling logic of the data set are encapsulated, and the calculation logic of the data column is exposed downward. Therefore, when the downstream requests daunt to implement, it only needs to focus on the calculation and return of the data column, and the middle platform base class is responsible for filling the returned data into the column of the calculated data set to obtain the result data set.
[0097] The map function class on the request side: the specific business mapFunction implemented based on the abstract class encapsulated by the middle platform. The downstream business party only needs to focus on the calculation logic implementation of the current newly added column value.
[0098] Based on the above configuration, we can finally configure the corresponding calculated fields on the page, or automatically call the corresponding map function for calculation, so as to display the calculated values of the calculated fields on the page in the form of column data.
[0099] The embodiment of the present application provides an encapsulated microservice interface by being compatible with the idea of MapReduce and streaming computing, allowing the upstream and downstream to inherit and obtain the methods in the corresponding map function through the interface, and also has good performance and memory performance in the case of extremely large data volumes. At the same time, it is flexibly compatible with the value calculation of the requester's custom request data type. Finally, the implementation of the calculation class field expands the filtering method of data query, and supports the filtering form of comparing the inventory class field with the calculation class field.
[0100] See also Figure 6 , an embodiment of the present application provides a computer device, including:
[0101] An acquisition unit 601 is used to acquire a data query scheme, wherein the data query scheme includes multiple query fields and a filtering condition corresponding to each query field, wherein the multiple query fields include a calculation field and a stock field, wherein the filtering condition corresponding to at least one query field includes a calculation field, wherein the query field belonging to the calculation field has a corresponding value formula, wherein the value formula includes a preset stock field, and the query field belonging to the stock field is derived from any stock table;
[0102] A determination unit 602 is used to determine a data acquisition framework for a calculation type field and a data acquisition framework for a stock type field, wherein the data acquisition framework is used to obtain a value of a corresponding type query field;
[0103] The acquisition unit 601 is also used to acquire target data corresponding to the data query solution according to the data acquisition framework of the calculation type field, the data acquisition framework of the stock type field, the value acquisition formula, the stock table and the filtering conditions corresponding to each query field.
[0104] In a specific implementation, the filtering conditions corresponding to each query field include first-category conditions and / or second-category conditions. The first-category conditions corresponding to the query field do not include fields that belong to different stock tables from the query field, and the second-category conditions corresponding to the query field include fields that belong to different stock tables from the query field. The acquisition unit 601 is specifically used to obtain the first value of the query field according to the first-category conditions corresponding to the query field, the value-taking formula corresponding to the query field, and the data-taking framework of the calculation field if any query field belongs to a calculation field; if any query field belongs to a stock field, obtain the first value of the query field according to the first-category conditions corresponding to the query field, the stock table to which the query field belongs, and the data-taking framework of the stock field; and filter the target data corresponding to the data query solution from the first value of each query field according to the corresponding second-category conditions.
[0105] In a specific implementation, if any query field is a calculation type field, the acquisition unit 601 is specifically used to parse the preset stock type field from the value formula corresponding to the query field; substitute the value of the preset stock type field into the value formula corresponding to the query field to obtain the second value of the query field; based on the filtering condition corresponding to the query field, filter the target data corresponding to the data query solution in the second value of the query field.
[0106] In a specific implementation, before substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain the second value of the query field,
[0107] The determination unit 602 is further configured to use the value of the target field as the value of the preset stock class field if the preset stock class field is the same as any query field;
[0108] The determining unit 602 is further configured to obtain a value of the preset stock class field from the stock table to which the preset stock class field belongs if the preset stock class field is different from the multiple query fields.
[0109] In a specific implementation, if the value of the query field is an array, the acquisition unit 601 is specifically used to substitute the value of the preset inventory field into the value formula corresponding to the query field to obtain the original value of the query field; concatenate each element in the original value into a string, and use the concatenated string as the value of the calculation field.
[0110] In a specific implementation, if any query field is a stock field, the acquisition unit 601 is specifically used to obtain a second value of the query field from the stock table of the query field source; based on the filtering condition corresponding to the query field, the target data corresponding to the data query solution is filtered out from the second value of the query field.
[0111] Figure 7 It is a schematic diagram of a computer device structure provided in an embodiment of the present application. The computer device 700 may include one or more central processing units (CPU) 701 and a memory 705. The memory 705 stores one or more application programs or data.
[0112] The memory 705 may be a volatile storage or a persistent storage. The program stored in the memory 705 may include one or more modules, each of which may include a series of instruction operations in the computer device. Furthermore, the central processing unit 701 may be configured to communicate with the memory 705 and execute a series of instruction operations in the memory 705 on the computer device 700.
[0113] The computer device 700 may also include one or more power supplies 702, one or more wired or wireless network interfaces 703, one or more input and output interfaces 704, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.
[0114] The CPU 701 can execute the aforementioned Figures 1 to 6 The operations performed by the query end and / or the request end in the illustrated embodiment will not be described in detail here.
[0115] It should be noted that, although the steps in the flowcharts involved in the various embodiments are drawn in sequence as indicated by the arrows, unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.
[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0117] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0120] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.
[0121] An embodiment of the present application also provides a computer program product including instructions. When the computer program product is executed on a computer, the computer is enabled to execute the data query method supporting complex filtering conditions as described above.
Claims
1. A data query method supporting complex filtering conditions, characterized in that: include: Acquire a data query scheme, the data query scheme including multiple query fields and filtering conditions corresponding to each of the query fields, the multiple query fields including calculation fields and stock fields, the filtering condition corresponding to at least one of the query fields including the calculation field, the query field belonging to the calculation field having a corresponding value formula, the value formula including a preset stock field, and the query field belonging to the stock field being derived from any stock table; Determine a data retrieval framework for a calculation field and a data retrieval framework for an inventory field, wherein the data retrieval framework is used to obtain the value of a corresponding query field; According to the data acquisition framework of the calculation type field, the data acquisition framework of the stock type field, the value acquisition formula, the stock table and the filtering condition corresponding to each of the query fields, the target data corresponding to the data query solution is obtained.
2. The method according to claim 1, characterized in that The filtering condition corresponding to each query field includes a first condition and / or a second condition, the first condition corresponding to the query field does not include a field belonging to a different stock table from the query field, the second condition corresponding to the query field includes a field belonging to a different stock table from the query field, and the target data corresponding to the data query scheme is acquired according to the data acquisition framework of the calculation field, the data acquisition framework of the stock field, the value acquisition formula, the stock table, and the filtering condition corresponding to each query field, including: If any of the query fields is a calculation field, a first value of the query field is obtained according to the first type of condition corresponding to the query field, the value formula corresponding to the query field, and the value acquisition framework of the calculation field; If any of the query fields belongs to a stock field, obtaining a first value of the query field according to the first type of condition corresponding to the query field, the stock table to which the query field belongs, and the data acquisition framework of the stock field; According to the corresponding second type of conditions, target data corresponding to the data query solution is filtered from the first value of each query field.
3. The method according to claim 1, characterized in that If any of the query fields is a calculation field, then according to the data acquisition framework of the calculation field, the value acquisition formula and the filtering condition corresponding to the query field, the target data corresponding to the data query solution is acquired, including: Analyze the value formula corresponding to the query field to obtain a preset inventory field; Substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain a second value of the query field; Based on the filtering condition corresponding to the query field, the target data corresponding to the data query solution is screened out from the second value of the query field.
4. The method according to claim 3, characterized in that Before substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain the second value of the query field, the method further includes: If the preset stock class field is the same as any of the query fields, the value of the target field is used as the value of the preset stock class field; If the preset stock class field is different from the multiple query fields, the value of the preset stock class field is obtained from the stock table to which the preset stock class field belongs.
5. The method according to claim 3, characterized in that: If the value of the query field is an array, substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain the value of the query field includes: Substituting the value of the preset inventory field into the value formula corresponding to the query field to obtain the original value of the query field; Each element in the original value is concatenated into a string, and the concatenated string is used as the value of the calculation class field.
6. The method according to any one of claims 1, 3 to 5, characterized in that: If any of the query fields is a stock field, then according to the data acquisition framework of the stock field, the stock table from which the query field comes, and the filtering condition corresponding to the query field, the target data corresponding to the data query scheme is obtained, including: Acquire a second value of the query field from the stock table of the query field source; Based on the filtering condition corresponding to the query field, the target data corresponding to the data query solution is screened out from the second value of the query field.
7. A computer device, characterized in that: include: an acquisition unit, configured to acquire a data query scheme, wherein the data query scheme includes a plurality of query fields and a filtering condition corresponding to each of the query fields, wherein the plurality of query fields include a calculation field and a stock field, wherein the filtering condition corresponding to at least one of the query fields includes the calculation field, and the query field belonging to the calculation field has a corresponding value formula, wherein the value formula includes a preset stock field, and the query field belonging to the stock field is derived from any stock table; A determination unit, used to determine a data acquisition framework for a calculation type field and a data acquisition framework for a stock type field, wherein the data acquisition framework is used to obtain a value of a corresponding type query field; The acquisition unit is also used to acquire target data corresponding to the data query solution based on the data acquisition framework of the calculation type field, the data acquisition framework of the stock type field, the value acquisition formula, the stock table and the filtering conditions corresponding to each of the query fields.
8. A computer device, characterized in that: include: CPU, memory and input / output interface; The memory is a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method according to any one of claims 1 to 6.
9. A computer program product comprising instructions, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.
10. A computer storage medium, characterized in that: The computer storage medium stores instructions, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.