Data query management method, system and equipment and readable storage medium
By configuring processing policies in the source database, including dictionary encoding conversion and field desensitization, the flexibility and security issues of data management and query in the prior art are solved, and efficient and secure data integration and query management are achieved.
Patent Information
- Application Number
- CN202411153306.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-12-06
AI Technical Summary
In the case of abnormal situations such as system crashes or power outages, data management and query have high time cost for frequently modifying configuration files and SQL mapping files, unable to adapt to one-to-many association queries, and lack of confidentiality measures, resulting in poor data security. question.
By configuring processing strategies based on the source database, including dictionary encoding conversion, time format standardization, and field desensitization strategies, write data to the search engine library and provide editable filtering and sorting items in the user terminal interface. , realize safe and reliable management and query of data.
Improves flexibility and adaptability of data integration, reduces manual configuration complexity, accelerates data access speed, ensures data consistency and security, simplifies the data mapping configuration process, and improves system response speed and data Availability.
Smart Images

Figure CN119089044A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of communication networks, and in particular relates to a data query management method, system, device and readable storage medium. Background Art
[0002] Nowadays, with the popularization of mobile Internet and the development of network technologies such as 5G, there are more and more scenarios for exchanging, calling, summarizing and querying information and data in various fields. In order to ensure that data will not be lost or damaged in abnormal situations such as system crash or power outage, and can still be used normally after the system is restored, the existing technology is usually implemented through file systems, databases or other data storage technologies.
[0003] In order to meet the requirements of data persistence, the existing technology often adopts fully automatic or semi-automatic ORM frameworks, such as MyBatis or Hibernate, etc. However, the existing technology inevitably has the problem of frequent modification of configuration files and SQL mapping files, high time and labor costs for redeployment, inability to adapt to one-to-many associated query services, lack of confidentiality measures, and poor security.
[0004] Therefore, in view of the above technical problems, it is necessary to provide an improved data management and data query method.
[0005] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention
[0006] The object of the present invention is to provide a data query management method, system, device and readable storage medium, which can quickly respond to user requests and realize safe and reliable data management and query in multi-source data.
[0007] In order to achieve the above object, a technical solution provided by a specific embodiment of the present invention is as follows:
[0008] In a first aspect, the present invention provides a data query management method, which comprises:
[0009] Based on the table data and corresponding table structure recording the original data in the source database, configure a processing strategy for the original data, process the original data based on the processing strategy, and write the processed data into the search engine library; the processing strategy includes: dictionary encoding conversion strategy, time format standardization strategy and field desensitization strategy;
[0010] Setting editable function items and the field range to which the function items apply, and rendering the function items to the user terminal interface based on the switch status of the function items;
[0011] In response to the user's request, the matching data information is fed back to the user terminal interface.
[0012] In one or more embodiments of the present invention, the processing of the original data based on the processing strategy and writing the processed data into the search engine library includes:
[0013] Based on the table data and table structure in the source database, a mapping relationship between the original data and each preset table item in the search engine library is established;
[0014] The time data format of the original data is synchronized based on the time format standardization strategy, and the data labels and code values of the original data are converted based on the dictionary encoding conversion strategy;
[0015] Based on the table structure of the original data, the converted original data is stored in a preset standard database;
[0016] Extract data from the standard database, desensitize the data based on the field desensitization strategy, generate a description value of the desensitized data based on the dictionary encoding conversion strategy, and write the description value and code value of the desensitized data into the table items corresponding to the search engine library respectively.
[0017] In one or more embodiments of the present invention, extracting data from a standard database, desensitizing the data based on a desensitization strategy, and writing the desensitized data into a table item corresponding to the search engine library includes:
[0018] Obtain the corresponding data in the standard information library;
[0019] Based on the mapping relationship between the original data and each preset table item in the search engine library, determine the writing position of the corresponding data in the standard information library in the search engine library;
[0020] The corresponding data in the standard information library is desensitized, and based on a preset entity model, the desensitized data is encapsulated and written into the corresponding table item of the search engine library.
[0021] In one or more embodiments of the present invention, the dictionary-based encoding conversion converts the code value of the original data, including:
[0022] Matching each code value of the original data with the fields in the dictionary;
[0023] If the code value of the original data does not match the code value in the dictionary, the code value is set to a null value;
[0024] If the code value of the original data matches the code value in the dictionary, conversion is performed based on the mapping relationship of the code value.
[0025] In one or more embodiments of the present invention, the method further comprises:
[0026] Set editable filter items and sort items, and define the data range in the search engine library to which each filter item and sort item applies;
[0027] Based on the filtering items and / or sorting items that are in an enabled state, the data within the corresponding application scope in the search engine library is pre-filtered / pre-sorted.
[0028] In one or more embodiments of the present invention, the method further comprises:
[0029] Splitting the description values of the data in the search engine library;
[0030] A mapping relationship between each phrase after splitting and the description value before splitting is established to form a search table.
[0031] In one or more embodiments of the present invention, in response to a user's request, feeding back the matched data information to the user terminal interface includes:
[0032] Generate editable filter items and sort items in the user terminal interface;
[0033] Based on the fields input by the user, traverse the search table to obtain one or more description values associated with the input fields;
[0034] Based on the user's selection of the filter items and the sorting items, the associated description values are retrieved from the corresponding preprocessed and / or presorted data, and displayed on the user terminal interface according to the order after preprocessing.
[0035] In one or more embodiments of the present invention, the method further comprises:
[0036] If the user selects a displayed description value in the user terminal interface, the standard database or the original database is indexed based on the data information corresponding to the description value;
[0037] Based on the dictionary encoding conversion strategy, time format standardization strategy and field desensitization strategy, the corresponding indexed data is converted, and the converted code value is translated into a description value and displayed on the user terminal interface.
[0038] In a second aspect, the present invention provides a data query management system, comprising:
[0039] A configuration module, configured to configure a processing strategy for the original data based on the table data and table structure of the source database, wherein the processing strategy includes: a dictionary encoding conversion strategy, a time format standardization strategy, and a field desensitization strategy;
[0040] A writing module, used to process the original data based on the processing strategy and write the processed data into the search engine library;
[0041] A preprocessing module, used to set editable function items and the field range to which the function items are applied, and render the function items to the user terminal interface based on the switch state of the function items;
[0042] The interactive module is used to respond to the user's request and feed back the matching data information to the user terminal interface.
[0043] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the data query management method by executing the computer instructions.
[0044] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data query management method.
[0045] Compared with the prior art, the data query management method provided by the present invention can efficiently adapt to different data sources in different environments based on standardized interfaces, making the data integration process more flexible, reducing the complexity of manual configuration, and accelerating the speed of data access. Policy configuration based on the table structure of the source database can ensure the consistency and accuracy of data in the efficient processing process, while simplifying the process of data mapping configuration. Various processing strategies ensure the security of sensitive information while realizing data cleaning, time format standardization and dictionary conversion, further improving the availability of data. Editable function items can realize real-time modification and re-editing of functions, and there is no need to modify the back-end business code at the same time when correcting errors and modifying configurations of function items. Preprocessing of data enables the system to respond to user requests more quickly and give corresponding feedback. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0047] Figure 1 This is a schematic diagram of a data query management scenario in one embodiment of the present invention;
[0048] Figure 2 It is a flow chart of a data query management method in one embodiment of the present invention;
[0049] Figure 3 It is a structural block diagram of a data query management system in one embodiment of the present invention;
[0050] Figure 4 It is a structural block diagram of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION
[0051] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.
[0052] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.
[0053] In order to facilitate understanding of the technical solutions of the present application, the technical terms that may appear in the present invention are first explained in detail below.
[0054] ORM (Object-Relationl Mapping): Object-relational mapping is a programming technique that aims to handle database operations by using high-level, object-oriented syntax instead of traditional SQL queries. The ORM framework is responsible for mapping objects to database tables, allowing developers to perform data operations by operating objects instead of directly operating the database.
[0055] SQL (Structured Query Language): It is a database language with multiple functions such as data manipulation and data definition. This language is interactive and can provide great convenience for users. Database management systems should make full use of SQL language to improve the work quality and efficiency of computer application systems. SQL language can not only be used independently in terminals, but also can be used as a sub-language to provide effective assistance for other program designs. In this program application, SQL can optimize program functions together with other program languages, thereby providing users with more comprehensive information.
[0056] In the existing technical solutions, there are methods for data query based on business needs, for example, using the MyBatis framework, based on the fine-grained SQL control capabilities it provides, developers can write native SQL statements while maintaining the convenience of object-relational mapping. Through the SqlSession (SQL session) created by the SqlSessionFactory (SQL session factory), developers can execute predefined SQL statements, process transactions, and map result sets.
[0057] However, since the configuration of MyBatis is relatively static, it is difficult to cope with changes and customization requirements between multiple environments and systems. In addition, MyBatis does not provide field mapping and data desensitization functions, which require additional development work to implement. When the business requires a lot of table data and the relationship between tables is complex, it is very time-consuming. In the face of scenarios where data sources and table structures change due to different environments and systems, MyBatis needs to frequently modify configuration files and SQL mapping files, which greatly increases the time and labor costs of redeployment.
[0058] The above-mentioned defects of the MyBatis framework are also the common defects of fully automatic or semi-automatic ORM frameworks in the prior art. The inventor of the present invention has found the main shortcomings of the prior art, and based on the shortcomings of the prior art, has proposed a new technical implementation idea: by establishing a standardized interface, a connection with one or more source databases is achieved, and after the connection, the performance data and table item structure in the source database can be called. In the face of heterogeneous data sources, a customized mapping relationship configuration can be used to achieve the purpose of permanent adaptation of the data source. At the same time, different from the prior art that sets the mapping relationship in advance, which may cause some data to have no corresponding mapping items or some preset mapping items to be redundant, the present invention is based on the table data and table structure of the corresponding data source. The targeted setting of data processing strategies increases the efficiency of the data processing flow.
[0059] Furthermore, this solution achieves the desensitization of private data, and the desensitization of data is set before the data is written into the search engine library and when the user terminal accesses the source database and / or the standard database. The privacy of sensitive data is protected from multiple dimensions, and on the other hand, it does not affect the user's query operation.
[0060] In addition, the filter items in the prior art are locked on the user terminal page, which greatly limits the user's operability. In order to better meet the needs of today's users, the embodiments of the present invention implement configurable filter items. The configurable filter items include the user's modification of the filter items on the interface side and the arbitrary selection of each filter item; and also include the configuration of the filter items by the backend operator.
[0061] Since it takes a certain amount of time for the user to select a function, it also takes time to execute the function unit in response to the user's operation. In actual operation, the user side will wait when processing the request, and the waiting time will inevitably show an increasing trend with the increase in the amount of data synchronized in the search engine library. With the gradual increase of database data today, in order to ensure the comprehensiveness of the query function, the data synchronized to the search engine library is gradually increasing. The present invention provides an implementation method, based on the open state of the filter item, the data is pre-processed in advance, and after receiving the corresponding request instruction from the user, the query function can be directly implemented on this basis to speed up the processing efficiency.
[0062] Please refer to Figure 1 , which is a schematic diagram of an application scenario of the data query management method provided by the present invention under an implementation mode, and the scenario specifically includes: a resource management module 101, a search engine 102 and a user terminal 103.
[0063] It should be noted that communication connections are provided between the resource management module 101, the search engine 102 and the user terminal 103. The communication network derived from the communication connection may include various connection types, including but not limited to: wired connection, wireless connection or optical fiber cable connection, etc. At the same time, the communication network may be a local area network, a metropolitan area network, a wide area network or any combination of the three.
[0064] The resource management module 101 is used to realize the transmission, conversion, translation and standardization of data across systems, etc. The search engine 102 is configured with a search engine library, which can be set based on the data in the search engine library, and can search the data in the search engine library based on the fields input by the user. The user terminal 103 is used to present a search page for the user to input search content, select function items, etc.
[0065] It should be noted that the user terminal 103 is installed with a computer software program that matches the data query management method provided by the present method; the user terminal 103 may include but is not limited to a desktop computer (PC), a desktop computer, a smart phone, a handheld computer, a tablet computer, a personal digital assistant (PDA), and other portable electronic devices or wearable electronic devices, and the embodiments of the present invention do not limit the above contents.
[0066] It should also be noted that the data management query method of the embodiment of the present invention can be applied to the data management query system of the embodiment of the present invention. The data management query system can be configured in a terminal. The terminal can include but is not limited to a PC (Personal Computer), a PDA (tablet computer), a smart phone, a smart wearable device, and the like.
[0067] Please refer to Figure 2FIG. 1 is a flow chart of a data query management method according to an embodiment of the present invention. The data query management method specifically comprises the following steps:
[0068] S201: Based on the table data and corresponding table structure recording the original data in the source database, configure a processing strategy for the original data.
[0069] It is understandable that different data sources store data in different ways, including but not limited to: using different table structures to store data, storing data in different formats, etc. Different data formats and different table structures make it impossible to directly configure data from different sources in the search engine library of the present invention, and it is necessary to pre-process the data based on the processing strategy.
[0070] At the same time, if the corresponding processing strategy is directly configured based on the existing technology, some content in the strategy will be redundant and invalid, and some original data will not be included in the corresponding processing strategy. It is very easy to cause the processing strategy to be poorly compatible with the source database, slow data processing efficiency, and high error rate. Therefore, the present invention adopts a solution to configure the strategy based on the table data and table structure that record the original data in the source database.
[0071] It should be noted that the processing strategies include, but are not limited to: dictionary encoding conversion strategies, time format standardization strategies, and field desensitization strategies. The embodiments of the present invention do not limit the types of processing strategy configurations. By adopting predefined or customized field desensitization strategies, the system can automatically execute data protection measures to improve the security of some privacy data; by binding dictionary mapping strategies, the conversion of code values and / or descriptive values and / or label values can be automatically completed; through the configuration of time format standardization, the time data is converted to the same format, thereby ensuring the consistency and readability of the data.
[0072] Specifically, the dictionary mapping strategy includes: label mapping strategy, code value mapping strategy and description value mapping strategy. Among them, the label is used to distinguish the categories of each data, such as name, age, education, etc.; it can be understood that in the database, the data is not stored in the form of text, but in the form of code or parameter. The code value refers to the parameter or code of the data content under the corresponding label. The description value is converted from the corresponding code value and can be a value in the form of text that the user can understand, which is used to display on the user terminal interface.
[0073] The source database sets the corresponding time data when storing the original data. Since the time data formats of different data sources are different, the original data needs to be standardized based on the preset time format.
[0074] Field desensitization strategy, that is, data deprivation. Under a given desensitization strategy, sensitive data can be reversibly or irreversibly converted to prevent sensitive data from being directly exposed in the user terminal interface, causing leakage of core data in the source database. The field desensitization strategy involved in the present invention may include but is not limited to truncation, hiding, data replacement, symmetric encryption, etc. The embodiments of the present invention are not limited to this.
[0075] S202: Processing the original data based on the processing strategy, and writing the processed data into a search engine library;
[0076] As mentioned above, different data sources have different data storage formats, so it is necessary to process the data retrieved from the source database based on a preset processing strategy. In an exemplary embodiment, the original data is processed based on the processing strategy, and the processed data is written into the search engine library, including: based on the table data and table structure in the source database, a mapping relationship between the original data and each preset table item in the search engine library is established; based on the time format standardization strategy, the time data of the original data is formatted and synchronized, and based on the dictionary encoding conversion strategy, the data label and code value of the original data are converted; based on the table structure of the original data, the converted original data code value is stored in a preset standard database; data in the standard database is extracted, data is desensitized based on the field desensitization strategy, and a description value of the desensitized data is generated based on the dictionary encoding conversion strategy, and the description value and code value of the desensitized data are respectively written into the corresponding table item of the search engine library.
[0077] It should be noted that the standard database is used to store the data after dictionary mapping and time synchronization in the same table structure as the source database. When users query data details, they do not need to access the source database, but can directly access the standard database that stores the converted original data. On the other hand, the setting of the standard database does not require the user to open the source database. For example, when the user is inconvenient to open the corresponding source database or the user has the ability to clean the data, the user can process the data to form a standard database.
[0078] In order to obtain detailed data before desensitization without accessing the source database in certain circumstances, the data in the standard database is preferably not desensitized. However, in another embodiment, the data confidentiality level in the source database is extremely high. In order to avoid leakage of sensitive information during the process of calling data, the data in the source database can also be directly desensitized before being stored in the standard database.
[0079] It should also be noted that, in order to implement the above dictionary mapping strategy, in one embodiment of the present invention, different dictionaries can be set based on data labels. That is, different labels are associated with different dictionaries. When data dictionary mapping is required, the dictionary corresponding to the label index of the data to be processed is firstly indexed, and then the data under the label is subjected to dictionary mapping based on the mapping relationship in the dictionary.
[0080] In fact, the essence of dictionary mapping is to clean the data in the source database. The cleaned data can be formed into a unified format to facilitate subsequent processing. In one embodiment of the present invention, in order to avoid abnormal values in the cleaned data, that is, to prevent incomplete data cleaning, the code value of the original data is converted based on the dictionary encoding conversion, which may include: matching each code value of the original data with the field in the dictionary; if the code value of the original data does not match the code value in the dictionary, setting the code value to a null value; if the code value of the original data matches the code value in the dictionary, converting based on the mapping relationship of the code value.
[0081] It should be noted that, in actual use scenarios, the difficulty of writing code is much greater than the difficulty of writing SQL. At the same time, in order to facilitate the subsequent dictionary mapping work, the conversion of data labels should be configured at the top of all conversion work. In an exemplary embodiment, the dictionary mapping strategy for label conversion can be written into an API interface that calls the original data based on the structured query language. Based on this interface, the label value of the received original data is converted.
[0082] In one embodiment of the present invention, the steps for generating an API interface specifically include: writing business SQL, converting business fields into field aliases required by the product; encapsulating the parameters required in SQL into parameters required by the interface; displaying the input and output parameters of the encapsulated API; publishing and calling the API interface to automatically call the data in the source database and convert the label values. The parameters required in SQL include but are not limited to: greater than, less than, range values, etc. The parameter settings in the script need to be passed when the interface is called, that is, the interface parameters need to correspond to the parameters in SQL. Therefore, it is necessary to encapsulate the parameters required in SQL into the parameters required by the interface.
[0083] As mentioned above, the time format standardization strategy aims to standardize the time data of data in different data sources. The above time data in different data sources may include but is not limited to: timestamps, time strings, etc. For example, the data timestamp in a source database is a millisecond timestamp. In fact, the data that is finally written into the search engine library, that is, the data that is finally presented to the user, does not need to be accurate to milliseconds. Therefore, according to specific business needs, the above data timestamp can be converted into the format expected to be presented in the user terminal interface, such as year-month-day, etc.
[0084] It is understandable that in order to meet the needs of user-side query and other functions, it is necessary to write part or all of the processed data in the source database into the search engine library. To write data, it is necessary to first determine the data to be written and the location where the data is written. The data to be written can be configured based on the needs of different users for the product, and the location of the write needs to be based on the mapping relationship between the table data and table structure in the source database and the preset table items in the search engine library, traverse the location of the data to be written in the source database, and then determine the location of the write to the search engine library.
[0085] In an exemplary embodiment, data in a standard database is extracted, data is desensitized based on a desensitizing strategy, and the desensitized data is written into a table item corresponding to a search engine library, including: obtaining corresponding data in a standard information library; determining a write position of corresponding data in the standard information library in the search engine library based on a mapping relationship between original data and preset table items in the search engine library; desensitizing the corresponding data in the standard information library, and based on a preset entity model, encapsulating the desensitized data and writing it into a table item corresponding to the search engine library.
[0086] In the original database or standard database, the data that needs to be written to the search engine library may span multiple tables in the database. Therefore, the embodiment of the present invention sets a unified entity model for recording the data information that needs to be captured and specifies the data storage format.
[0087] Furthermore, since the capacity of the search engine library is capped, the original data stored in the source database is usually large in volume, and writing all of it will consume a lot of time and occupy the capacity of the engine library. Therefore, in actual application scenarios, the technical means can be adaptively selected based on specific business needs and device status.
[0088] In an exemplary embodiment, the data that is expected to be displayed to the user can be selected from the processed data based on the user's needs, and the data can be written to the corresponding position in the search engine library. For example, based on the user's business needs, the physical parameters of teenagers aged 18-24 need to be retrieved. Then the corresponding database can be called, and the height, weight and other information of teenagers in the age group of 18-24 can be written into the search engine library. This saves writing time and meets the needs of users.
[0089] In another exemplary embodiment, the processed data can be divided into query data and full data, the query data only displays part of the key information of the corresponding data, and the full data is all the data of the corresponding data. Based on the needs of the user, the query data of the data expected to be displayed is selected and written into the search engine library. For example, based on the business needs of the user, it is necessary to retrieve civil dispute cases that occurred in the jurisdiction in the past ten years. Due to the large number of civil dispute cases in ten years, it is not convenient to write the full data into the search engine library. Therefore, only the time and judgment results of each case in 10 years can be written into the search engine library as query data. It is used for users to query data.
[0090] It should be noted that each item of original data in the source database usually has a unique identifier. Therefore, when the user needs to know the full amount of data of the query content, the system responds to the user operation, and can obtain a unique identifier based on the query data displayed on the user interface after the query function is implemented, and retrieve the corresponding data in the standard database or the source database through the interface, and display it in the user terminal interface after data processing. Among them, since the data format of the source database has not been converted and the calling process is cumbersome, it is preferred to call the data in the standard database. In another embodiment, if there is no unique identifier in the original data in the source database, a unique identifier is added to the original data when the original data is processed.
[0091] S203: Setting editable function items and a field range to which the function items are applied, and rendering the function items to a user terminal interface based on the switch state of the function items;
[0092] It should be noted that the data in the search engine library is the files for users to search. But in fact, as users' demand for search continues to increase, the capacity of the search engine library is also increasing. In order to ensure fast and accurate response to user query requests, different field ranges can be defined based on data labels, data types, etc.
[0093] For example, the database of the relevant government department is called and processed to generate an emergency data query device. The relevant data written into the search engine library can be divided into enterprise data, material data, plan data, etc. Different data types correspond to different data field ranges.
[0094] Furthermore, editable function items can be set to assist users in implementing functions such as querying and retrieving data. Based on the same purpose as the aforementioned data classification, it is to facilitate faster response to user operations and meet user needs. In one embodiment of the present invention, by setting the field range corresponding to the application of the function item, the implementation of the function item can be based on part of the data in the search engine library, rather than all the data. Among them, the function items may include but are not limited to: filtering items, sorting items, etc. The embodiment of the present invention does not limit the selection of function items.
[0095] In an exemplary embodiment, editable function items and field ranges to which the function items apply are set, and based on the switch state of the function items, the function items are rendered to the user terminal interface, including: setting editable filter items and sorting items, and defining the data range to which each filter item and sorting item applies in the search engine library; and pre-coding the data logic based on the filter items and / or sorting items that are in the turned-on state.
[0096] Among them, the filter item configuration is to set editable filter items and filter values. Based on the switch status of the filter items, different filter item selection ranges are presented to the user. Based on user requests, the data of the corresponding field range in the search engine library is filtered, and the matching data information is fed back to the user terminal interface.
[0097] Specifically, the interface obtains information and provides a page based on the enabled status of the filter items or sorting items. The page dynamically renders the filter items according to the order, type, and content of the filter items. Users filter according to actual needs. The interface matches related businesses based on user operations and returns business information according to user sorting rules.
[0098] Among them, the hard-coded filter item configuration strategy adopted by the prior art makes it impossible to modify the filter items on the user terminal interface, and the code can only be re-edited on a large scale based on the modified content. Different from the prior art, the present invention uses the storage form of a database table to display various functional items in the user terminal interface, so that the filter items, sorting items and other functions provided by the present invention can be modified in the user terminal interface. At the same time, the background customization of whether to enable the filter item operation makes it unnecessary to develop separate filter functions for different users. You only need to configure the filter items and choose whether to enable them according to user needs.
[0099] S204: In response to the user's request, the matching data information is fed back to the user terminal interface.
[0100] It is understandable that the user request includes the selection of function items and the data retrieval operation of the input field. The configured filter items can be displayed on the user terminal interface for the user to select only after being confirmed and enabled by the background. Based on the above operations, the system can be adaptively applied to customers with different needs after one configuration.
[0101] Data retrieval based on input fields, in an exemplary embodiment, includes: splitting the description values of the data in the search engine library; establishing a mapping relationship between each phrase after the split and the description value before the split, to form a retrieval table; based on the field input by the user, traversing the retrieval table to obtain one or more description values associated with the input field.
[0102] For example, a data description value is "Suzhou Fire Prevention and Control Plan". The search engine will split the description value into multiple phrases such as "Suzhou", "fire", "prevention and control", "plan", "emergency plan", and "fire situation". The split phrases include phrases directly split from the original description value and phrases generated by association based on user search records or other historical data. The phrases split from the same field establish a mapping relationship with the original field, thereby forming a search table.
[0103] If the user enters the search content as "Suzhou emergency plan", the search engine traverses the search table, obtains multiple relevant description values, and further indexes the complete data field corresponding to the above description values. It can be understood that the more phrases in the data field are identified as relevant, the more relevant the data is to the user's search content. Different from the existing technology, the search accuracy is higher when searching data based on this technical means.
[0104] It should be noted that data information for users to search has been written into the search engine library. In order to speed up the response to user requests, the embodiment of the present invention will immediately start preprocessing the data in the search engine library after setting the function item and confirming that the function item is turned on.
[0105] For example, a sorting item is preset to be sorted in reverse order by age. When the sorting item is set to be turned on, that is, when it is determined that the user is provided with the function item of sorting in reverse order by age, the search engine library immediately starts to sort the data in reverse order by age, forming an independent table item. When the user enters content for data retrieval, and it is determined that the user has selected the sorting item, the search is performed based on the above independent table item, and the searched content is displayed in the order of the data items in the independent table item, so that the function item processing can be pre-placed, and the purpose of completing user needs as quickly as possible can be achieved.
[0106] It is understandable that within the capacity of the database, the table items in the case of multiple filter items and sorting items can also be preprocessed to provide customers with more complicated data query work. The embodiment of the present invention does not limit the sorting combination of function items and the types of independent table items.
[0107] Please refer to Figure 3 As shown, based on the same inventive concept as the aforementioned data management query method, an embodiment of the present invention provides a data management query system 300 , which includes: a configuration module 301 , a preprocessing module 302 and an interaction module 303 .
[0108] Specifically, the configuration module 301 is used to configure the processing strategy for the original data based on the table data and table structure of the source database, process the original data based on the processing strategy, and write the processed data into the search engine library; the processing strategy includes: dictionary encoding conversion strategy, time format standardization strategy and field desensitization strategy; the preprocessing module 302 is used to set editable function items and the field range of the function item application, and preprocess the data within the corresponding field range in the search engine library based on the switch state of the function item; the interaction module 303 is used to respond to the user's request and feedback the matching data information to the user terminal interface.
[0109] It should be noted that the configuration module 301 is also used to establish a mapping relationship between the original data and each preset table item in the search engine library based on the table data and table structure in the source database; synchronize the format of the time data of the original data based on the time format standardization strategy, and convert the data label and code value of the original data based on the dictionary coding conversion strategy; based on the table structure of the original data, store the converted original data into a preset standard database; extract data from the standard database, desensitize the data based on the field desensitization strategy, generate a description value of the desensitized data based on the dictionary coding conversion strategy, and write the description value and code value of the desensitized data into the corresponding table items of the search engine library respectively.
[0110] The configuration module 301 is also used to obtain the corresponding data in the standard information library; determine the writing position of the corresponding data in the standard information library in the search engine library based on the mapping relationship between the original data and the preset table items in the search engine library; desensitize the corresponding data in the standard information library, and based on the preset entity model, encapsulate the desensitized data and write it into the corresponding table items of the search engine library.
[0111] The configuration module 301 is also used to match each code value of the original data with the fields in the dictionary; if the code value of the original data does not match the code value in the dictionary, the code value is set to a null value; if the code value of the original data matches the code value in the dictionary, conversion is performed based on the mapping relationship of the code value.
[0112] The preprocessing module 302 is also used to set editable filter items and sorting items, and define the data range in the search engine library to which each filter item and sorting item applies; based on the filter items and / or sorting items that are in the enabled state, pre-filter / pre-sort the data within the corresponding application range in the search engine library.
[0113] The pre-processing module 302 is also used to split the description values of the data in the search engine library; establish a mapping relationship between each word group after the split and the description value before the split, and form a search table.
[0114] The interactive module 303 is also used to generate editable filter items and sorting items on the user terminal interface; based on the fields input by the user, traverse the search table to obtain one or more description values associated with the input fields; based on the user's selection of filter items and sorting items, retrieve the associated description values in the corresponding preprocessed and / or pre-sorted data, and display them on the user terminal interface in the order of their preprocessing.
[0115] The interactive module 303 is also used to index the standard database or the original database based on the data information corresponding to the description value if the user selects the description value to be displayed on the user terminal interface; based on the dictionary coding conversion strategy, the time format standardization strategy and the field desensitization strategy, the corresponding indexed data is converted, and the description value corresponding to the converted code value is displayed on the user terminal interface.
[0116] Please refer to Figure 4 As shown, an embodiment of the present invention further provides an electronic device 400, which includes at least one processor 401, a memory 402 (such as a non-volatile memory), a memory 403, and a communication interface 404, and the at least one processor 401, the memory 402, the memory 403, and the communication interface 404 are connected together via a bus 405. At least one processor 401 is used to call at least one program instruction stored or encoded in the memory 402, so that the at least one processor 401 performs various operations and functions of the data query management method described in various embodiments of this specification.
[0117] In an embodiment of the present specification, the electronic device 400 may include, but is not limited to, a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile electronic device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable electronic device, a consumer electronic device, and the like.
[0118] An embodiment of the present invention further provides a computer-readable medium carrying computer-executable instructions. When the computer-executable instructions are executed by a processor, they can be used to implement various operations and functions of the data query management method described in various embodiments of this specification.
[0119] The computer-readable medium in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.
[0120] In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0121] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0122] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatuses, systems, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0123] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.
Claims
1. A data query management method, characterized in that: include: Based on the table data and corresponding table structure recording the original data in the source database, configure a processing strategy for the original data, the processing strategy including: a dictionary encoding conversion strategy, a time format standardization strategy, and a field desensitization strategy; Processing the original data based on the processing strategy and writing the processed data into the search engine library; Setting editable function items and the field range to which the function items apply, and rendering the function items to the user terminal interface based on the switch status of the function items; In response to the user's request, the matching data information is fed back to the user terminal interface.
2. The data query management method according to claim 1, characterized in that: The processing of the original data based on the processing strategy and writing the processed data into the search engine library includes: Based on the table data and table structure in the source database, a mapping relationship between the original data and each preset table item in the search engine library is established; Based on the time format standardization strategy, the time data format of the original data is synchronized, and based on the dictionary encoding conversion strategy, the data labels and code values of the original data are converted; Based on the table structure of the original data, the converted original data is stored in a preset standard database; Extract data from the standard database, desensitize the data based on the field desensitization strategy, generate a description value of the desensitized data based on the dictionary encoding conversion strategy, and write the description value and code value of the desensitized data into the table items corresponding to the search engine library respectively.
3. The data query management method according to claim 2, characterized in that: The extracting data from the standard database, desensitizing the data based on the desensitization strategy, and writing the desensitized data into the table item corresponding to the search engine library includes: Obtain the corresponding data in the standard information library; Based on the mapping relationship between the original data and each preset table item in the search engine library, determine the writing position of the corresponding data in the standard information library in the search engine library; The corresponding data in the standard information library is desensitized, and based on a preset entity model, the desensitized data is encapsulated and written into the corresponding table item of the search engine library.
4. The data query management method according to claim 2, characterized in that: The dictionary-based encoding conversion converts the code value of the original data, including: Matching each code value of the original data with the fields in the dictionary; If the code value of the original data does not match the code value in the dictionary, the code value is set to a null value; If the code value of the original data matches the code value in the dictionary, conversion is performed based on the mapping relationship of the code value.
5. The data query management method according to claim 1, characterized in that: The method further comprises: Set editable filter items and sort items, and define the data range in the search engine library to which each filter item and sort item applies; Based on the filtering items and / or sorting items that are in an enabled state, the data within the corresponding application scope in the search engine library is pre-filtered / pre-sorted.
6. The data query management method according to claim 5, characterized in that: The method further comprises: Splitting the description values of the data in the search engine library; A mapping relationship between each phrase after splitting and the description value before splitting is established to form a search table.
7. The data query management method according to claim 6, characterized in that: In response to the user's request, the matching data information is fed back to the user terminal interface, including: Generate editable filter items and sort items in the user terminal interface; Based on the fields input by the user, traverse the search table to obtain one or more description values associated with the input fields; Based on the user's selection of the filter items and the sorting items, the associated description values are retrieved from the corresponding preprocessed and / or presorted data, and displayed on the user terminal interface according to the order after preprocessing.
8. The data query management method according to claim 7, characterized in that: The method further comprises: If the user selects a displayed description value in the user terminal interface, the standard database or the original database is indexed based on the data information corresponding to the description value; Based on the dictionary encoding conversion strategy, time format standardization strategy and field desensitization strategy, the corresponding indexed data is converted, and the converted code value is translated into a description value and displayed on the user terminal interface.
9. A data query management system, characterized in that: include: A configuration module, for configuring a processing strategy for the original data based on the table data and table structure of the source database, wherein the processing strategy includes: a dictionary encoding conversion strategy, a time format standardization strategy, and a field desensitization strategy; A writing module, used to process the original data based on the processing strategy and write the processed data into the search engine library; A preprocessing module, used to set editable function items and the field range to which the function items are applied, and render the function items to the user terminal interface based on the switch state of the function items; The interactive module is used to respond to the user's request and feed back the matching data information to the user terminal interface.
10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data query management method according to any one of claims 1 to 8 by executing the computer instructions.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data query management method according to any one of claims 1 to 8.
