Data retrieval processing method, device and electronic device

Abstract

The present application discloses a data retrieval processing method, device and electronic device. The present application pre-constructs a pre-aggregate view and stores it in a pre-aggregate cache. The pre-aggregate view includes a retrieval directory set and a first retrieval result corresponding to each retrieval directory. The retrieval directory further includes: index information containing form attributes with index features; on this basis, after obtaining a data retrieval request for an application form, the target attributes carried in the obtained request are matched with the form attributes in the retrieval directory in the pre-aggregate cache, and finally a second retrieval result of the data retrieval request is generated based on the matching result. The present application scheme can avoid the retrieval of the data source of the form data when the requested form data exists in the pre-aggregate view, thereby realizing automatic and rapid retrieval of the form data according to the form attributes.

Description

Technical Field

[0001] The present application belongs to the technical field of data retrieval, and in particular, relates to a data retrieval processing method, device and electronic equipment. Background Art

[0002] Low-Code Development Platform (LCDP) is a development platform that can quickly generate applications without coding (0 code) or with a small amount of coding. Its power lies in allowing end users to develop their own applications using easy-to-understand visual tools instead of the traditional way of writing code.

[0003] The applicant's research found that the performance of low-code platform applications when retrieving large-scale data (such as retrieving hundreds of millions of data) is extremely poor, which greatly affects the user experience. Summary of the invention

[0004] To this end, this application discloses the following technical solutions:

[0005] A data retrieval processing method, the method comprising:

[0006] Obtaining a data retrieval request for an application form; the data retrieval request includes a target attribute used as a retrieval basis;

[0007] Matching the target attribute with the form attribute in the search directory in the pre-aggregation cache area to obtain a matching result; wherein the pre-aggregation cache area stores a pre-built pre-aggregation view, the pre-aggregation view includes a pre-generated search directory set and first search results corresponding to each search directory in the search directory set, the search directory includes: index information including the form attribute with index characteristics;

[0008] Based on the matching result, a second search result of the data search request is generated.

[0009] Optionally, generating a second search result of the data search request based on the matching result includes:

[0010] If the match is successful, generating a second search result of the data search request based on the first search result corresponding to the search directory to which the form attribute matching the target attribute belongs in the pre-aggregation cache area;

[0011] If the match fails, the data source of the form data of the application form is searched according to the target attribute to generate a second search result of the data search request.

[0012] Optionally, when the application form is published, the pre-aggregated view is constructed and stored in the pre-aggregated cache area;

[0013] The form attributes in the search directory are: when the application form is published, the corresponding category form attributes are obtained by identifying the attributes with index characteristics of the application form and classifying the attributes with index characteristics;

[0014] The first search result corresponding to the search directory includes: the result obtained by searching the data source and aggregating the search results of each search based on different form attributes in the same search directory and different search paths corresponding to different form attributes in the data source of the form data of the application form; the index information in the search directory includes: each form attribute with index characteristics of the same category and the data search paths corresponding to each form attribute with index characteristics.

[0015] Optionally, generating a second search result of the data search request based on a first search result corresponding to a search directory to which a form attribute matching the target attribute belongs in the pre-aggregation cache area includes:

[0016] The first search result is used as the second search result, or the first search result is searched according to the target attribute to generate a second search result of the data search request.

[0017] Optionally, the first search result corresponding to the search catalog included in the pre-aggregated view is: when the application form is published, the search result obtained by searching the data source of the form data of the application form, which matches the form attribute with the index feature of the application form and the target data range corresponding to the form attribute with the index feature;

[0018] Wherein, the target data range is the form data range statically required by the form attribute with index characteristics when the application form is published;

[0019] The matching of the target attribute with the form attribute in the search directory in the pre-aggregation cache area includes:

[0020] The target attribute and the current data range corresponding to the target attribute are matched with the form attribute in the search directory in the pre-aggregation buffer area and the target data range corresponding to the form attribute.

[0021] Optionally, the first search result corresponding to the search directory includes: a result obtained by searching a single data file in the data source;

[0022] When constructing the pre-aggregated view, the following steps are also included:

[0023] Determine a form attribute in the form attributes of the application form that has an index feature and can be used to perform a joint search on multiple data files in the data source, and obtain a joint search form attribute;

[0024] Based on the determined joint search form attributes, and according to the characteristics of the data definition language and / or data manipulation language of the data source of the application form, a joint search is performed on multiple data files of the data source to obtain a joint search result;

[0025] Adding a joint search catalog to the pre-aggregated view, and recording corresponding joint search results for the joint search catalog in the pre-aggregated view;

[0026] The joint search catalog includes index information including attributes of the joint search form.

[0027] Optionally, the method further includes:

[0028] Deleting expired data in the pre-aggregation cache area according to a predetermined data expiration policy;

[0029] And / or, updating data from a data source of form data of the application form to the pre-aggregation buffer area through a predetermined data update strategy.

[0030] Optionally, the form attributes of the application form include business data parameters bound to each form element of the application form;

[0031] By setting the business data parameters as interface parameters of the dynamic data interface and configuring the corresponding dynamic data interface for the form element, the binding of the form element with the corresponding business data parameters is achieved.

[0032] A data retrieval processing device, the device comprising:

[0033] An acquisition module, used for acquiring a data retrieval request for an application form; the data retrieval request includes a target attribute used as a retrieval basis;

[0034] A matching module, configured to match the target attribute with a form attribute in a search directory in a pre-aggregation cache; wherein the pre-aggregation cache stores a pre-built pre-aggregation view, the pre-aggregation view includes a pre-generated search directory set and first search results corresponding to each search directory in the search directory set, the search directory includes: index information including form attributes having index features;

[0035] A generating module is used to generate a second search result of the data search request based on the matching result.

[0036] An electronic device, comprising:

[0037] Memory, used to store computer instruction sets;

[0038] The processor is used to implement the data retrieval processing method as described in any one of the above items by executing an instruction set stored in the memory.

[0039] It can be seen from the above scheme that the data retrieval processing method, device and electronic device disclosed in the present application pre-constructs a pre-aggregate view and stores it in the pre-aggregate cache area. The pre-aggregate view includes a retrieval directory set and a first retrieval result corresponding to each retrieval directory. The retrieval directory further includes: index information containing form attributes with index features; on this basis, after obtaining a data retrieval request for an application form, the target attributes carried in the obtained request are matched with the form attributes in the retrieval directory in the pre-aggregate cache area, and finally a second retrieval result of the data retrieval request is generated based on the matching result. The present application scheme, by pre-constructing a pre-aggregate view and using the pre-aggregate view stored in the pre-aggregate cache area as a retrieval basis, can avoid the retrieval of the data source of the form data when the requested form data exists in the pre-aggregate view, thereby realizing automatic and rapid retrieval of form data based on form attributes, and effectively improving the retrieval performance of low-code platform applications when retrieving large-scale data. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0041] Figure 1 This is a processing logic diagram for realizing rapid retrieval of form data according to form attributes provided by this application;

[0042] Figure 2 This is a flowchart for implementing the optimization of pre-aggregated views through multi-table joint retrieval provided by this application;

[0043] Figure 3 It is a flow chart of the data retrieval and processing method provided by this application;

[0044] Figure 4 It is another flow chart of the data retrieval and processing method provided by the present application;

[0045] Figure 5 It is a structural schematic diagram of the data retrieval and processing device provided by the present application;

[0046] Figure 6It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The present application discloses a data retrieval and processing method, device and electronic device. The method and device can be applied to a server on the server side for providing form data retrieval services. By automatically realizing rapid retrieval of form data based on the form attributes of the application form, the problem of extremely poor performance of low-code platform applications when retrieving large-scale data (such as retrieving hundreds of millions of data) can be overcome.

[0049] Optionally, in an embodiment of the present application, the application form is an application form built based on a low-code development platform. The low-code development platform can reduce the amount of development code and development difficulty of developers by providing visual programming components and software full life cycle management capabilities.

[0050] Application forms are used to realize the interactive transmission of information between the user side and the server side. Through application forms, information such as user search requests and user orders can be collected from the user side, and then the collected information is processed by the server side and fed back to the user side. Application forms include but are not limited to basic form elements such as text fields, buttons, check boxes, drop-down boxes, lists / drop-down menus, and more complex form objects built based on basic form elements, such as employee information list components with page turning and query functions built based on input boxes and drop-down boxes.

[0051] The retrieval processing method, device and electronic device disclosed in the embodiments of the present application are to pre-build a pre-aggregate view and store the pre-aggregate view in a pre-aggregate cache area to support automatic rapid retrieval of form data according to form attributes of an application form. The following first describes the implementation process of building a pre-aggregate view and storing it in a pre-aggregate cache area.

[0052] The process of building pre-aggregated views and storing them in the pre-aggregated cache can be performed by a pre-built pre-aggregated engine, in conjunction with Figure 1 , the process specifically includes:

[0053] 11) The user completes the interactive design of the application form on the low-code development platform and binds the dynamic data interface (application programming interface, API). Through this process, the attribute configuration of the form elements, dynamic interface API, etc. is determined.

[0054] Specifically, users can use operations such as dragging, stacking, and combining on the low-code development platform to layout and associate form elements on the page, configure properties for the designed form elements, and configure the dynamic data interfaces and interface properties that need to be associated.

[0055] For example, configure the length, width, font, and allowed input language (Chinese / English) for a form element, and configure the dynamic data interface that needs to be connected (such as a dynamic data list, dynamic search box, data perspective chart, etc.), as well as the address, parameters and other attribute information of the dynamic data interface that needs to be connected.

[0056] At the same time, by setting business data parameters (such as basic employee information, department information of each organizational department of the company, etc.) as interface parameters of the dynamic data interface and configuring the corresponding dynamic data interface for the form elements, the binding of form elements with the corresponding business data parameters is realized, and the business data parameters bound by the interface parameters (such as employee name, gender, position, etc.) are used as form attributes.

[0057] That is, users design the interaction of application forms on the low-code development platform and bind the form attributes determined by the dynamic data interface, which include not only the basic attributes of the form elements themselves, such as length, width, font, allowed input languages, etc., but also the business data parameters bound to each form element of the application form, such as employee gender, organizational department, position, etc.

[0058] 12) The user selects a suitable database engine according to his / her needs, and publishes the application to which the application form belongs based on the selected database engine.

[0059] Specifically, after the user completes the design of the low-code platform application based on the interactive design of the application form and the dynamic data interface binding, he or she can select database engines such as Clickhouse, Hive, Spark, PostgreSQL, MySQL, etc. according to his or her own needs, and execute the release processing of the low-code application through the selected database engine.

[0060] 13) During the low-code application publishing process, the system performs a scanning and screening process on the form attribute configuration of the low-code application, and uses a classification algorithm to create an index feature model based on the scanning and screening results.

[0061] During the low-code platform application publishing process, the server-side system scans the application form attributes of the published low-code platform application, such as the length, width, font, allowed input language, employee gender, department, position, and other attributes corresponding to the bound business data of the form elements of the scanned application form. The scanned attribute information of each form is screened, and the attributes with index characteristics are screened. Among them, the attributes with index characteristics refer to the attributes that can be used to retrieve data from the form data (that is, the business data associated with the form), such as employee gender, department, position, etc. In this embodiment, these attributes are referred to as attributes with index characteristics.

[0062] Afterwards, the attributes with index features are further classified according to the association between the attributes based on a predetermined classification algorithm. The classification algorithm used may be, but is not limited to, classification algorithms such as linear regression. For example, when querying "employees with programming skills" based on the attribute "programming skills" and querying "male employees with programming skills" based on the combined attribute of "programming skills" + "male gender", both need to query the same data model (e.g., the same data table in a form database). According to this association, the attribute "programming skills" and the combined attribute of "programming skills" + "male gender" are classified into one category based on the classification algorithm.

[0063] An index feature model is created based on the classification result, and the created index feature model includes: each form attribute with index features of the same category and the data retrieval paths corresponding to each form attribute with index features in the data source of the form data. The data source of the form data may be a database of the form data, and the data retrieval paths corresponding to the form attributes in the data source of the form data may include retrieval path information of data tables, data files, etc. corresponding to the form data indicated by the form attributes (such as employee data indicated by the "gender" attribute) in the form database.

[0064] In this embodiment, each form attribute with index feature of each category and the data search path corresponding to each form attribute with index feature are called search directory, and each search directory forms a search directory set. That is, in this embodiment, the index feature model is essentially a search directory set.

[0065] 14) Create pre-aggregate views based on the index feature model of the low-code platform application and the selected database engine and store them in the pre-aggregate cache area.

[0066] Each type of attribute with index features in the index feature model (ie, the attributes in each search directory) corresponds to a set of data search paths.

[0067] When the low-code platform application is released, this embodiment simultaneously searches the form data source such as the form database based on different form attributes in the same search directory and different search paths corresponding to different form attributes in the form data source of the application form, using the data search implementation process corresponding to the selected database engine.

[0068] A predetermined classification algorithm (such as a decision tree) is used to aggregate the form attributes and search results corresponding to each search to create a pre-aggregated view, wherein the created pre-aggregated view includes a search directory set and the first search results corresponding to each search directory in the search directory set, and each attribute with search characteristics in the search directory, the search path and the corresponding first search result are organized into a view form obtained by aggregation based on a predetermined classification algorithm such as a decision tree.

[0069] Or, to put it another way, the pre-aggregated view includes a pre-generated search directory set and first search results corresponding to each search directory in the search directory set, wherein the search directory in the search directory set includes: index information containing form attributes with index characteristics; the form attributes in the search directory are specifically: corresponding category form attributes obtained by identifying the attributes with index characteristics of the application form and classifying the attributes with index characteristics when the application form is published; the first search result corresponding to the search directory includes: the result obtained by searching the data source and aggregating the search results of each search based on different form attributes in the same search directory and different search paths corresponding to different form attributes in the data source of the form data of the application form (for example, the search results for "employees with programming skills" and the search results for "male employees with programming skills" are merged into one search result, namely, the search results corresponding to "employees with programming skills" to save cache); the index information in the search directory includes: each form attribute with index characteristics of the same category and the data search paths corresponding to each form attribute with index characteristics.

[0070] After the pre-aggregation view is constructed, it is stored in the pre-aggregation cache area for use when retrieving form data, thereby speeding up the process of form data retrieval.

[0071] Further, preferably, in an embodiment of the present application, the first search result included in the pre-aggregate view and corresponding to the search directory is: when the application form is published, the data source of the form data of the application form is searched, and the search result matches the target data range corresponding to the form attribute with index characteristics of the application form and the form attribute with index characteristics. The target data range is: the form data range statically required by the form attribute with index characteristics when the application form is published. By storing only the search results matching the form data range statically required by the form attribute with index characteristics when the application form is published in the pre-aggregate cache, it is achieved to reduce the amount of data stored in the pre-aggregate cache as much as possible while speeding up the form data retrieval process.

[0072] The following are examples:

[0073] Assume that when querying the form data of "employee data" based on the form attribute of "employee gender", there is page turning, and the upper limit of each page is 100 employee data. Then the cache area only stores the employee data in the range of 1-100 data that can be determined through analysis when the form is published (that is, in the initial state when the low-code application is published, the data range statically required by the application form is 1-100, and when the user turns the page, the data range requirement is further dynamically changed to 101-200), and other page data are not cached.

[0074] In the pre-aggregated view constructed by the processing of the above steps 11)-14), the first search result corresponding to the search directory is specifically the result obtained by searching a single data file (such as a single data table) in the data source of the application form (such as the form database of the application form).

[0075] In the embodiment of the present application, the following processing may optionally be included:

[0076] 15) Optimize pre-aggregated views based on joint retrieval of multiple data files (e.g., multiple data tables) of the application form data source.

[0077] Among them, when the application is released, the pre-aggregation engine can further optimize the pre-aggregation view by performing a multi-table joint search on the application form data source and combining the application form data source such as the database's DDL (Data Definition Language) and DML (Data Manipulation Language) features.

[0078] For details, see Figure 2 , the process of optimizing the pre-aggregated view can be further implemented as follows:

[0079] Step 201: determine the form attributes of the application form that have index features and can be used to perform joint retrieval on multiple data files in the data source of the application form, and obtain the joint retrieval form attributes;

[0080] For example, when searching for employee information based on the attribute "organization / department" to which the employee belongs, you need to first query the "organization / department-employee" data table to find out which employees are in a department (such as the employee numbers of each employee in the organization), and then further query the employee's basic information data table to obtain detailed information about the employees in the organization, such as name, age, home address, position, etc. In this example, the attribute "department" can be used as a joint search form attribute.

[0081] Step 202: Based on the determined joint search form attribute, and according to the characteristics of the data definition language and / or data manipulation language of the data source of the application form, a joint search is performed on multiple data files of the data source of the application form to obtain a joint search result;

[0082] Specifically, according to the DDL and DML features of the data source of the application form, a DML statement composed of organization-related fields and employee-related fields for joint retrieval of the organization information table and the employee basic information table can be created. For example, according to the DML features of the database PostgreSQL, a DML statement composed of org-related fields and user-related fields for joint retrieval of the organization information table and the employee basic information table is created: "select a.org_id, a.org_name, u.userid, u.username from org a left join user u where a.org_id = ?", and a joint retrieval is performed on multiple data tables of the application form database based on the constructed DML statement.

[0083] Step 203: Add a joint search catalog to the pre-aggregate view, and record corresponding joint search results for the joint search catalog in the pre-aggregate view.

[0084] Afterwards, for the joint search results, a corresponding joint search directory is established, and the established joint search directory includes: index information containing the attributes of the joint search form. For example, for the above example, an index of the org_id field is established, and the established joint search directory and its corresponding joint search results are associated and stored in the pre-aggregation cache area to optimize the pre-aggregation view in the pre-aggregation cache area, so that the pre-aggregation view includes not only the search results and search directories of a single data file in the application form data source when the application is released, but also the joint search results and joint search directories of multiple data files.

[0085] Based on the pre-aggregated views stored in the pre-aggregated cache, see Figure 3 The data retrieval processing method flow chart provided herein includes at least the following steps:

[0086] Step 301: Obtain a data retrieval request for an application form; the obtained data retrieval request includes a target attribute used as a retrieval basis.

[0087] In the embodiment of the present application, rapid retrieval of form data is achieved based on form attributes.

[0088] After the low-code platform application is published and the application instance is started, refer to Figure 1 The user can initiate a retrieval / query request for form data by operating the application form. The initiated data retrieval request includes a target attribute used as a retrieval basis, and the target attribute included can specifically be a relevant business data parameter of the application form to be queried.

[0089] For example, a user enters the target attributes "employee gender" and "employee organization / department" into the search input box as the search basis to initiate a search for employee information. In response, the server can receive the data search request initiated by the user including "employee gender" and "employee organization / department", and use "employee gender" and "employee organization / department" as the search basis.

[0090] Step 302: Match the target attribute carried in the data retrieval request with the form attribute in the retrieval directory in the pre-aggregation cache area to obtain a matching result.

[0091] Among them, the pre-aggregation cache area caches pre-built pre-aggregation views. According to the above description of the pre-aggregation view, the pre-aggregation view includes a pre-generated search directory set and the first search results corresponding to each search directory in the search directory set. Each search directory in the search directory set includes: index information containing form attributes with index features. Specifically, the pre-aggregation view includes a single data table search result obtained by performing a single data table query on form data and a joint search result obtained by joint querying multiple data tables when the low-code platform application is released, and also includes the search directory corresponding to each query result.

[0092] After obtaining the data retrieval request submitted by the user, this embodiment first matches the target attribute carried in the data retrieval request as the retrieval basis (such as employee gender, organizational department to which the employee belongs, etc.) with the form attributes in the retrieval directory in the pre-aggregation cache area.

[0093] Furthermore, in the case where the first search result corresponding to the search directory in the pre-aggregate view is a search result obtained by searching the form data source of the application form when the application form is published, and the search result matches the form attributes with index characteristics of the application form and the form data range statically required by the application form, the target attribute carried in the data retrieval request and the current data range required corresponding to the target attribute are matched with the form attributes in the search directory in the pre-aggregate cache area and the data range corresponding to the form attributes (the data range statically required when the application form is published).

[0094] Step 303: Based on the above matching results, generate a second search result of the data search request.

[0095] On the basis of performing the above matching operation, a second search result of the data search request is generated based on the matching result.

[0096] Specifically, if the match is successful, that is, there is a form attribute consistent with the target attribute in the retrieval directory set of the pre-aggregate view, or there is a form attribute consistent with the target attribute and the data range corresponding to the form attribute in the pre-aggregate cache area is consistent with the current data range required by the target attribute, then the second retrieval result of the data retrieval request is generated directly based on the first retrieval result corresponding to the retrieval directory to which the form attribute matching the target attribute in the pre-aggregate cache area belongs.

[0097] Among them, based on the above-mentioned first retrieval result in the pre-aggregation cache area, generating a second retrieval result of the data retrieval request further includes: using the first retrieval result as the second retrieval result of the data retrieval request, or, performing a secondary retrieval on the first retrieval result in the pre-aggregation cache area according to the target attribute in the data retrieval request, and using the secondary retrieval result as the second retrieval result of the data retrieval request.

[0098] For example, in the case where the search results of "employees with programming skills" and the search results of "male employees with programming skills" in the pre-aggregation cache are merged into one search result (i.e., the search result corresponding to "employees with programming skills"), if the target attribute in the data retrieval request is "programming skills", the merged search result (first search result) can be directly used as the second search result of the data retrieval request. If the target attributes in the data retrieval request include "programming skills" and "male gender", the merged search result in the pre-aggregation cache is searched a second time using the target attributes of "programming skills" and "male gender", and the second search result of the data retrieval request is obtained through the secondary search.

[0099] In the case of a successful match, the second search result of the data retrieval request can be obtained directly based on the pre-aggregation view in the pre-aggregation cache area, without searching the form data source of the application form such as the form database, thereby speeding up the data retrieval process of the application form.

[0100] On the contrary, if the match fails, the form data source (such as form database) of the application form is searched according to the target attribute carried in the data search request, and the second search result of the data search request is generated by searching the data source of the form data.

[0101] It can be seen from the above scheme that the method of this embodiment pre-builds a pre-aggregate view and stores it in the pre-aggregate cache area. The pre-aggregate view includes a search directory set and a first search result corresponding to each search directory. The search directory further includes: index information containing form attributes with index features; on this basis, after obtaining a data retrieval request for an application form, the target attributes carried in the obtained request are matched with the form attributes in the search directory in the pre-aggregate cache area, and finally a second search result of the data retrieval request is generated based on the matching result. The present application scheme, by pre-building a pre-aggregate view and using the pre-aggregate view stored in the pre-aggregate cache area as a retrieval basis, can avoid the retrieval of the data source of the form data when the requested form data exists in the pre-aggregate view, thereby realizing automatic and rapid retrieval of form data based on form attributes, and effectively improving the retrieval performance of low-code platform applications when retrieving large-scale data.

[0102] In one embodiment, see Figure 4 The data retrieval processing method flow chart provided, the data retrieval processing method disclosed in the embodiment of the present application may also include the following processing:

[0103] Step 304: Deleting expired data in the pre-aggregation cache area according to a predetermined data expiration policy; and / or updating data from a data source of form data of the application form to the pre-aggregation cache area according to a predetermined data update policy.

[0104] Optionally, the data expiration policy may be set as follows: based on a preset time length, periodically detecting whether a deletion event occurs in the form data corresponding to the data stored in the pre-aggregation buffer in the data source of the form data; if a deletion event occurs (causing the corresponding data stored in the pre-aggregation buffer to expire), synchronously deleting the expired data corresponding to the deleted data in the pre-aggregation buffer;

[0105] The data update strategy can be set accordingly as follows: based on a preset time length, periodically detect whether new events occur in the form data source corresponding to the data stored in the pre-aggregation buffer area; if new events occur, the new data is synchronously updated to the pre-aggregation buffer area.

[0106] However, the present invention is not limited to this. A callback function may be pre-registered with the data source of the form data, and a data expiration policy or a data update policy may be set such that upon receiving a callback notification triggered by a data deletion or new addition event at the data source of the form data, a corresponding expired data deletion operation or data addition operation is performed in the pre-aggregation cache area.

[0107] On this basis, through the corresponding policy that is set, the expired data in the pre-aggregation cache is deleted or the data is updated from the form data source of the application form to the pre-aggregation cache. For example, due to the resignation of an employee, the employee data in the form database is deleted, which causes the employee data cached in the pre-aggregation cache to expire. In this case, the employee information in the pre-aggregation cache can be deleted according to the data expiration policy; at the same time, if the data of the new employee is added to the form database due to the entry of a new employee, the data information of the new employee can be synchronously added to the pre-aggregation cache according to the data update policy.

[0108] This embodiment can maintain the consistency of the data in the pre-aggregation cache area with that in the application form data source by deleting the relevant expired data in the pre-aggregation cache area or adding data to the pre-aggregation cache area according to the above-mentioned strategy, ensure the validity of the data in the pre-aggregation cache area, and thus provide effective support for accelerating the retrieval process of form data.

[0109] Corresponding to the above method, the present application embodiment also discloses a data retrieval processing device, the composition structure of which is as follows: Figure 5 As shown, including at least:

[0110] The acquisition module 501 is used to acquire a data retrieval request for the application form; the data retrieval request includes a target attribute used as a retrieval basis;

[0111] A matching module 502 is used to match the target attribute with the form attribute in the search directory in the pre-aggregation cache area; wherein the pre-aggregation cache area stores a pre-built pre-aggregation view, the pre-aggregation view includes a pre-generated search directory set and first search results corresponding to each search directory in the search directory set, the search directory includes: index information including the form attribute with index characteristics;

[0112] The generating module 503 is used to generate a second search result of the data search request based on the matching result.

[0113] In one implementation, the generating module 503 is specifically configured to:

[0114] If the match is successful, generating a second search result of the data search request based on the first search result corresponding to the search directory to which the form attribute matching the target attribute belongs in the pre-aggregation cache area;

[0115] If the match fails, the data source of the form data of the application form is searched according to the target attribute to generate a second search result of the data search request.

[0116] In one embodiment, when the application form is published, the pre-aggregated view is constructed and stored in a pre-aggregated cache area;

[0117] The form attributes in the search directory are: when the application form is published, the corresponding category form attributes are obtained by identifying the attributes with index characteristics of the application form and classifying the attributes with index characteristics;

[0118] The first search result corresponding to the search directory includes: the result obtained by searching the data source and aggregating the search results of each search based on different form attributes in the same search directory and different search paths corresponding to different form attributes in the data source of the form data of the application form; the index information in the search directory includes: each form attribute with index characteristics of the same category and the data search paths corresponding to each form attribute with index characteristics.

[0119] In one embodiment, the generating module 503, when generating the second search result of the data search request based on the first search result corresponding to the search directory to which the form attribute matching the target attribute belongs in the pre-aggregation cache area, is specifically used to:

[0120] The first search result is used as the second search result, or the first search result is searched according to the target attribute to generate a second search result of the data search request.

[0121] In one embodiment, the first search result corresponding to the search catalog included in the pre-aggregated view is: when the application form is published, the search result obtained by searching the data source of the form data of the application form, which matches the form attribute with the index feature of the application form and the target data range corresponding to the form attribute with the index feature;

[0122] Wherein, the target data range is the form data range statically required by the form attribute with index characteristics when the application form is published;

[0123] The matching module 502 is specifically used to match the target attribute and the current data range corresponding to the target attribute with the form attribute in the search directory in the pre-aggregation buffer area and the target data range corresponding to the form attribute.

[0124] In one embodiment, the first search result corresponding to the search directory includes: a result obtained by searching a single data file in the data source;

[0125] When constructing the pre-aggregated view, the following steps are also included:

[0126] Determine a form attribute in the form attributes of the application form that has an index feature and can be used to perform a joint search on multiple data files in the data source, and obtain a joint search form attribute;

[0127] Based on the determined joint search form attributes, and according to the characteristics of the data definition language and / or data manipulation language of the data source of the application form, a joint search is performed on multiple data files of the data source to obtain a joint search result;

[0128] Adding a joint search catalog to the pre-aggregated view, and recording corresponding joint search results for the joint search catalog in the pre-aggregated view;

[0129] The joint search catalog includes index information including attributes of the joint search form.

[0130] In one embodiment, the above device further comprises:

[0131] The updating module is used to: delete expired data in the pre-aggregation cache area according to a predetermined data expiration strategy; and / or update data from a data source of form data of an application form to the pre-aggregation cache area according to a predetermined data update strategy.

[0132] In one embodiment, the form attributes of the application form include business data parameters bound to each form element of the application form;

[0133] By setting the business data parameters as interface parameters of the dynamic data interface and configuring the corresponding dynamic data interface for the form element, the binding of the form element with the corresponding business data parameters is achieved.

[0134] As for the data retrieval and processing device disclosed in the embodiments of the present application, since it corresponds to the data retrieval and processing method disclosed in the above method embodiments, the description is relatively simple. For relevant similarities, please refer to the description of the corresponding method embodiments above, which will not be described in detail here.

[0135] The embodiment of the present application also discloses an electronic device, which may be, but is not limited to, a server in a server end for providing a form data retrieval service. The composition structure of the electronic device is as follows: Figure 6 As shown, specifically including:

[0136] Memory 601, used for storing computer instruction sets;

[0137] A set of computer instructions can be implemented in the form of a computer program.

[0138] The processor 602 is used to implement the data retrieval processing method disclosed in any of the above method embodiments by executing a computer instruction set.

[0139] The processor 602 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices.

[0140] In addition, the electronic device may also include components such as a communication interface and a communication bus. The memory, the processor and the communication interface communicate with each other through the communication bus.

[0141] The communication interface is used for communication between electronic devices and other devices. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0142] In summary, the data retrieval and processing method, device, and electronic device disclosed in the embodiments of the present application have at least the following technical advantages over conventional technologies:

[0143] 21) With the same data scale, it consumes less storage space, automatically reduces the query burden, speeds up the data retrieval process of the application form, improves its retrieval efficiency, and can obtain a better user experience;

[0144] 22) Pre-aggregate creation is achieved through classification algorithms (such as decision trees), and the pre-aggregate creation process is closely integrated with the database engine (such as Clickhouse, Hive, Spark, Postgresql, MySQL, etc.) to fully utilize the expansion capabilities of existing databases.

[0145] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0146] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0147] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0148] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0149] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data retrieval processing method, the method comprising: Get data retrieval request for application form; The data retrieval request includes a target attribute used as a retrieval basis; The target attribute is matched with the form attribute in the search directory in the pre-aggregation cache area to obtain a matching result; wherein the pre-aggregation cache area stores a pre-built pre-aggregation view, the pre-aggregation view includes a pre-generated search directory set and first search results corresponding to each search directory in the search directory set, the search directory includes: index information containing form attributes with index characteristics; wherein the form attributes in the search directory are: corresponding category form attributes obtained by identifying the attributes with index characteristics of the application form and classifying the attributes with index characteristics when the application form is published; the first search result corresponding to the search directory includes: the result obtained by searching the data source and aggregating the search results of each search based on different form attributes in the same search directory and different search paths corresponding to different form attributes in the data source of the form data of the application form; Based on the matching result, generating a second search result of the data search request; The generating a second search result of the data retrieval request based on the matching result includes: if the match is successful, generating a second search result of the data retrieval request based on a first search result corresponding to a search directory to which a form attribute matching the target attribute in the pre-aggregation cache belongs.

2. The method according to claim 1, wherein generating a second search result of the data search request based on the matching result further comprises: If the match fails, the data source of the form data of the application form is searched according to the target attribute to generate a second search result of the data search request.

3. The method according to claim 2, wherein: When the application form is published, the pre-aggregated view is constructed and stored in the pre-aggregated buffer area; The index information in the search directory includes: each form attribute with index characteristics of the same category and the data search paths corresponding to each form attribute with index characteristics.

4. The method according to claim 3, generating the second search result of the data search request based on the first search result corresponding to the search directory to which the form attribute matching the target attribute belongs in the pre-aggregation cache area, comprises: The first search result is used as the second search result, or the first search result is searched according to the target attribute to generate a second search result of the data search request.

5. The method according to claim 3, wherein the first search result corresponding to the search catalog included in the pre-aggregated view is: when the application form is published, the search result obtained by searching the data source of the form data of the application form, which matches the form attribute with index characteristics of the application form and the target data range corresponding to the form attribute with index characteristics; in, The target data range is the form data range required by the static form attribute with index characteristics when the application form is published; The matching of the target attribute with the form attribute in the search directory in the pre-aggregation cache area includes: The target attribute and the current data range corresponding to the target attribute are matched with the form attribute in the search directory in the pre-aggregation buffer area and the target data range corresponding to the form attribute.

6. According to the method of claim 3, the first search result corresponding to the search directory includes: a result of searching a single data file in the data source; When constructing the pre-aggregated view, the following steps are also included: Determine a form attribute in the form attributes of the application form that has an index feature and can be used to perform a joint search on multiple data files in the data source, and obtain a joint search form attribute; Based on the determined joint search form attributes, and according to the characteristics of the data definition language and / or data manipulation language of the data source of the application form, a joint search is performed on multiple data files of the data source to obtain a joint search result; Adding a joint search catalog to the pre-aggregated view, and recording corresponding joint search results for the joint search catalog in the pre-aggregated view; The joint search catalog includes index information including attributes of the joint search form.

7. The method according to claim 1, further comprising: Deleting expired data in the pre-aggregation cache area according to a predetermined data expiration policy; And / or, updating data from a data source of form data of the application form to the pre-aggregation buffer area through a predetermined data update strategy.

8. According to the method of claim 1, the form attributes of the application form include business data parameters bound to each form element of the application form; By setting the business data parameters as interface parameters of the dynamic data interface and configuring the corresponding dynamic data interface for the form element, the binding of the form element with the corresponding business data parameters is achieved.

9. A data retrieval processing device, the device comprising: The acquisition module is used to obtain data retrieval requests for application forms; The data retrieval request includes a target attribute used as a retrieval basis; A matching module is used to match the target attribute with the form attribute in the search directory in the pre-aggregation buffer; wherein the pre-aggregation buffer stores a pre-built pre-aggregation view, the pre-aggregation view includes a pre-generated search directory set and first search results corresponding to each search directory in the search directory set, the search directory includes: index information including form attributes with index characteristics; wherein the form attributes in the search directory are: corresponding category form attributes obtained by identifying the attributes with index characteristics of the application form and classifying the attributes with index characteristics when the application form is published; the first search result corresponding to the search directory includes: the result obtained by searching the data source and aggregating the search results of each search based on different form attributes in the same search directory and different search paths corresponding to different form attributes in the data source of the form data of the application form; A generation module is used to generate a second search result of the data retrieval request based on the matching result; the generating the second search result of the data retrieval request based on the matching result includes: if the match is successful, based on the first search result corresponding to the search directory to which the form attribute matching the target attribute in the pre-aggregation cache area belongs, generating the second search result of the data retrieval request.

10. An electronic device, comprising: Memory, used to store computer instruction sets; A processor, used to implement the data retrieval processing method as described in any one of claims 1 to 8 by executing an instruction set stored in a memory.

Images