Data query method and device, computer equipment and storage medium

Abstract

The invention belongs to the technical field of data processing, and relates to a data query method, which comprises the following steps of: verifying a received query request based on a verification strategy; if the query request passes the verification, analyzing the query request to obtain a query keyword, and judging whether a query result matched with the query keyword exists in a cache library or not; if not, constructing a target query statement based on the query keyword; calling a target database to search the target query statement to obtain a search result; performing data extraction processing on the search result to obtain feedback data corresponding to the query keyword; wherein the target feedback data comprises context information and subsystem data corresponding to the query keyword; and sending the feedback data to the user. The invention further provides a data query device, computer equipment and a storage medium. In addition, the feedback data may be stored in the blockchain. Based on an automatic query processing mode, the processing efficiency and accuracy of query processing are effectively improved.

Description

Technical Field

[0001] The present application relates to the fields of data processing technology and financial technology, and in particular to data query methods, devices, computer equipment and storage media. Background Art

[0002] In the daily operation and management of the financial industry, accurate grasp and efficient use of data are the key to supporting business decisions, optimizing service processes, and preventing financial risks. However, in current practical operations, there are significant inefficiencies and inaccuracies in the detailed query process for the use of data in various business systems.

[0003] When it is necessary to query the details of the data used by each business system, it is usually necessary to rely on the group manager to send statistical requirements via email. After receiving the statistical requirements, the person in charge of each group subsystem needs to coordinate the team to manually check whether there is any relevant information involved in the statistics within the business scope. This process not only involves a large amount of repetitive work and requires additional human resources, but also because there may be information islands between groups, it is difficult to form a unified data view, further reducing work efficiency. After the investigation is completed, each group will feedback the data via email, and then a dedicated person will summarize and organize it. This manual summary method is not only time-consuming and labor-intensive, but also prone to human errors during the data integration process, such as data duplication, omissions, or inconsistent formats, which seriously affects the accuracy and reliability of the data. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a data query method, device, computer equipment and storage medium to solve the technical problems that the existing manual processing method for querying the details of the data used by each business system has low processing efficiency and cannot guarantee the accuracy of the queried data.

[0005] In order to solve the above technical problems, the present application provides a data query method, which adopts the following technical solution:

[0006] Determine whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword;

[0007] If yes, verify the query request based on a preset verification strategy;

[0008] If the query request passes the verification, the query request is parsed to obtain the query keyword, and it is determined whether there is a query result matching the query keyword in the preset cache library;

[0009] If there is no query result matching the query keyword in the cache library, constructing a corresponding target query statement based on the query keyword;

[0010] Calling a preset target database to search and process the target query statement to obtain corresponding search results;

[0011] Performing data extraction processing on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword;

[0012] The feedback data is sent to the user.

[0013] Furthermore, the step of verifying the query request based on a preset verification strategy specifically includes:

[0014] Obtaining request parameters of the query request;

[0015] Obtain the preset legality verification strategy and effectiveness verification strategy;

[0016] Performing a validity check on the request parameters based on the validity check strategy;

[0017] If the request parameter passes the validity check, performing validity check on the request parameter based on the validity check strategy;

[0018] If the request parameter passes the validity check, then it is determined that the query request passes the check; otherwise, it is determined that the query request fails the check.

[0019] Furthermore, the step of sending the feedback data to the user specifically includes:

[0020] Performing formatting processing on the feedback data to obtain corresponding first feedback data;

[0021] Performing deduplication processing on the first feedback data to obtain corresponding second feedback data;

[0022] performing aggregation analysis on the second feedback data to obtain corresponding third feedback data;

[0023] Constructing corresponding response data based on the third feedback data;

[0024] The response data is displayed in the preset interface.

[0025] Furthermore, before the step of determining whether there is a query result matching the query keyword in the preset cache library, the method further includes:

[0026] Determine whether the current time reaches the preset trigger time of the scheduled task;

[0027] If yes, pull the data within the preset time period from the preset message middleware;

[0028] Performing offline collision calculation on the data to obtain specified context information and specified subsystem data related to the specified keyword;

[0029] Performing format conversion on the specified context information and the specified subsystem data based on the specified keyword to obtain corresponding specified data;

[0030] The designated data is stored in the cache library.

[0031] Furthermore, the step of performing offline collision calculation on the data to obtain specified context information and specified subsystem data related to the specified keyword specifically includes:

[0032] Preprocessing the data to obtain corresponding processed data;

[0033] Get the preset offline collision calculation strategy;

[0034] Get the preset specified keyword;

[0035] Based on the offline calculation collision strategy, a preset offline calculation engine is called to perform offline calculation collision on the data to obtain specified context information and specified subsystem data related to the specified keyword.

[0036] Furthermore, before the step of calling a preset target database to search the target query statement to obtain corresponding search results, the method further includes:

[0037] Acquire sensitive information data reported by each group service based on a preset scanning thread, and put the sensitive information data into a preset blocking queue;

[0038] Obtaining the sensitive information data from the blocking queue based on a preset consumer thread, and putting the sensitive information data into a preset message middleware;

[0039] Calling a target entry program corresponding to the target database;

[0040] The target data in the message middleware is consumed based on the target storage program, and the target data is stored in the target database.

[0041] Furthermore, after the step of determining whether there is a query result matching the query keyword in the preset cache library, the method further includes:

[0042] If there is a query result matching the query keyword in the cache library, then obtaining the query result;

[0043] Convert the query result to a target format to obtain the corresponding query result.

[0044] The target query result is sent to the user.

[0045] In order to solve the above technical problems, the embodiment of the present application further provides a data query device, which adopts the following technical solution:

[0046] A first judgment module is used to judge whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword;

[0047] A verification module, used for verifying the query request based on a preset verification strategy;

[0048] A second judgment module is used to parse the query request to obtain the query keyword if the query request passes the verification, and to judge whether there is a query result matching the query keyword in a preset cache library;

[0049] A construction module, configured to construct a corresponding target query statement based on the query keyword if there is no query result matching the query keyword in the cache library;

[0050] A search module is used to call a preset target database to search the target query statement and obtain corresponding search results;

[0051] An extraction module, configured to perform data extraction processing on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword;

[0052] The first sending module is used to send the feedback data to the user.

[0053] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0054] Determine whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword;

[0055] If yes, verify the query request based on a preset verification strategy;

[0056] If the query request passes the verification, the query request is parsed to obtain the query keyword, and it is determined whether there is a query result matching the query keyword in the preset cache library;

[0057] If there is no query result matching the query keyword in the cache library, constructing a corresponding target query statement based on the query keyword;

[0058] Calling a preset target database to search and process the target query statement to obtain corresponding search results;

[0059] Performing data extraction processing on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword;

[0060] The feedback data is sent to the user.

[0061] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0062] Determine whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword;

[0063] If yes, verify the query request based on a preset verification strategy;

[0064] If the query request passes the verification, the query request is parsed to obtain the query keyword, and it is determined whether there is a query result matching the query keyword in the preset cache library;

[0065] If there is no query result matching the query keyword in the cache library, constructing a corresponding target query statement based on the query keyword;

[0066] Calling a preset target database to search and process the target query statement to obtain corresponding search results;

[0067] Performing data extraction processing on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword;

[0068] The feedback data is sent to the user.

[0069] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0070] The present application first determines whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword; if so, the query request is verified based on a preset verification strategy; if the query request passes the verification, the query request is parsed to obtain the query keyword, and it is determined whether there is a query result matching the query keyword in a preset cache library; if there is no query result matching the query keyword in the cache library, a corresponding target query statement is constructed based on the query keyword; then a preset target database is called to search and process the target query statement to obtain corresponding search results; subsequently, data extraction is performed on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword; and finally, the feedback data is sent to the user. After receiving a query request triggered by a user through a preset interface and detecting that the query request has passed verification, the present application will parse the query request to obtain a query keyword, and then first detect whether there is a query result matching the query keyword in the cache library. If not, the present application will intelligently construct a corresponding target query statement based on the query keyword, and then call the target query statement of the target database to perform search processing to obtain search results, and perform data extraction processing on the search results to obtain feedback data corresponding to the query keyword, and finally send the feedback data to the user. Different from the existing data query method based on manual processing, the present application uses an automated query processing method based on the cache library and the target database instead of manual screening and merging and aggregation methods, which effectively reduces repetitive work, reduces the risk of manual statistical errors, and improves the processing efficiency and accuracy of query processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0072] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0073] Figure 2 A flowchart of an embodiment of a data query method according to the present application;

[0074] Figure 3 is a structural schematic diagram of an embodiment of a data query device according to the present application;

[0075] Figure 4It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0076] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0077] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0078] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0079] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0080] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0081] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0082] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0083] It should be noted that the data query method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the data query device is generally arranged in the server / terminal device.

[0084] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0085] Continue to refer Figure 2 , shows a flow chart of an embodiment of the data query method according to the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The data query method provided in the embodiment of the present application can be applied to any scenario where data query is required, and the data query method can be applied to products in these scenarios, for example, data query in the financial insurance field. The data query method comprises the following steps:

[0086] Step S201, determining whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword.

[0087] In this embodiment, the data query method is executed on the electronic device (eg Figure 1The server / terminal device shown in the figure) can obtain the query request through a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future. The executor of this application is a data query system, or simply referred to as the system. This application can be applied to the query process for the details of the use of data by various business systems in the financial industry. The above-mentioned preset interface can be the front-end interface of the system. The user can enter the specified query keyword on the front-end page and trigger the corresponding query request.

[0088] Step S202: If yes, verify the query request based on a preset verification strategy.

[0089] In this embodiment, the specific implementation process of verifying the query request based on the preset verification strategy will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0090] Step S203: if the query request passes the verification, the query request is parsed to obtain the query keyword, and it is determined whether there is a query result matching the query keyword in the preset cache library.

[0091] In this embodiment, the corresponding query keyword can be extracted by parsing the query request. After receiving the query request, the system will first detect whether there is a query result matching the query keyword in the preset cache library. If there is a query result matching the query keyword in the cache library, it is directly obtained and returned to the user. If there is no query result matching the query keyword, you can choose to trigger a real-time query, or you can also inform the user that the result needs to wait for offline calculation and update.

[0092] Step S204: If there is no query result matching the query keyword in the cache library, a corresponding target query statement is constructed based on the query keyword.

[0093] In this embodiment, if it is detected that there is no query result matching the query keyword in the cache library, a target query statement corresponding to the target database will be generated according to the query keyword. The target query statement includes keyword matching (such as match query), range query (such as range query), Boolean query (such as bool query), etc.

[0094] Step S205: calling a preset target database to perform search processing on the target query statement to obtain corresponding search results.

[0095] In this embodiment, the target database may be an Elasticsearch cluster. After receiving a query request, the target database may be called to search for documents related to the query keyword in the index according to the constructed target query statement to obtain the search results. The index is where data is stored in the target database, and is usually constructed by inverted indexing based on the field values ​​of the document. In addition, the search process of the target database may involve distributed search, word segmentation, filtering, sorting and other operations.

[0096] Step S206, performing data extraction processing on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword.

[0097] In this embodiment, the generated search results can be parsed and processed to extract context information and subsystem data related to the query keyword, namely the feedback data. The feedback data may include specific field values ​​of the document, attributes of nested objects, etc.

[0098] Step S207: sending the feedback data to the user.

[0099] In this embodiment, the specific implementation process of sending the feedback data to the user will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0100] The present application first determines whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword; if so, the query request is verified based on a preset verification strategy; if the query request passes the verification, the query request is parsed to obtain the query keyword, and it is determined whether there is a query result matching the query keyword in a preset cache library; if there is no query result matching the query keyword in the cache library, a corresponding target query statement is constructed based on the query keyword; then a preset target database is called to search and process the target query statement to obtain corresponding search results; subsequently, data extraction is performed on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword; and finally, the feedback data is sent to the user. After receiving a query request triggered by a user through a preset interface and detecting that the query request has passed verification, the present application will parse the query request to obtain a query keyword, and then first detect whether there is a query result matching the query keyword in the cache library. If not, the present application will intelligently construct a corresponding target query statement based on the query keyword, and then call the target query statement of the target database to perform search processing to obtain search results, and perform data extraction processing on the search results to obtain feedback data corresponding to the query keyword, and finally send the feedback data to the user. Different from the existing data query method based on manual processing, the present application uses an automated query processing method based on the cache library and the target database instead of manual screening and merging and aggregation methods, which effectively reduces repetitive work, reduces the risk of manual statistical errors, and improves the processing efficiency and accuracy of query processing.

[0101] In some optional implementations, step S202 includes the following steps:

[0102] Get the request parameters of the query request.

[0103] In this embodiment, corresponding request parameters can be extracted from the URL query parameters (Query Parameters), request body, request headers (Headers) or path parameters (Path Parameters) of the query request, wherein the request parameters at least include the query keyword, and may also include information such as the query type, paging parameters, and sorting parameters.

[0104] Get the preset legality verification strategy and validity verification strategy.

[0105] In this embodiment, the policy content of the above-mentioned legality verification policy includes: non-empty verification and type verification. Specifically, non-empty verification includes: checking whether key parameters (such as keywords, query types) are empty or undefined. If empty or undefined, throw an exception or return an error response. Type verification includes: ensuring that the type of the parameter matches the expected type. For example, check whether the paging parameter is an integer and whether the sorting parameter is a valid sorting direction (such as ascending, descending).

[0106] The policy contents of the above validity verification strategy include: range verification, format verification and business logic verification. Specifically, range verification includes: for parameters of numeric type, checking whether they are within the valid range. For example, check whether the page number is greater than 0, and whether the number of records displayed per page is within a reasonable range. Format verification includes: for parameters in a specific format (such as date, email address, phone number, etc.), use regular expressions or special libraries for verification. Business logic verification includes: additional verification based on business logic. For example, check whether the query type is a type supported by the system, whether the keyword contains sensitive words, etc.

[0107] The request parameters are subjected to a validity check based on the validity check strategy.

[0108] In this embodiment, the request parameters may be subjected to a legality check according to the policy content of the legality check policy to obtain a corresponding legality check result, wherein the content of the legality check result includes: whether the request parameters pass the legality check or whether the request parameters fail the legality check.

[0109] If the request parameter passes the validity check, the request parameter is validated based on the validity check strategy.

[0110] In this embodiment, the request parameters may be validated according to the policy content of the validity validation policy to obtain a corresponding validity validation result, wherein the validity validation result includes: whether the request parameters pass the validity validation or whether the request parameters fail the validity validation.

[0111] If the request parameter passes the validity check, then it is determined that the query request passes the check; otherwise, it is determined that the query request fails the check.

[0112] In this embodiment, only when it is detected that the request parameters have passed both the legality check and the validity check, will it be determined that the query request has passed the check; otherwise, it is determined that the query request has not passed the check.

[0113] The present application obtains the request parameters of the query request; then obtains the preset legality verification strategy and validity verification strategy; subsequently performs a legality verification on the request parameters based on the legality verification strategy; if the request parameters pass the legality verification, then the validity of the request parameters is verified based on the validity verification strategy; and if the request parameters pass the validity verification, then the query request is determined to have passed the verification, otherwise, the query request is determined to have failed the verification. The present application obtains the request parameters of the query request, and then performs a legality verification and validity verification on the request parameters based on the legality verification strategy and the validity verification strategy, respectively. Only when it is detected that the request parameters have passed both the legality verification and the validity verification, will the query request be determined to have passed the verification, which effectively improves the accuracy of the verification processing of the query request, ensures the accuracy of the verification results obtained, and is conducive to improving the subsequent processing standardization and processing security of the query request.

[0114] In some optional implementations of this embodiment, step S207 includes the following steps:

[0115] Formatting is performed on the feedback data to obtain corresponding first feedback data.

[0116] In this embodiment, the feedback data may be formatted according to actual business requirements. For example, the date field may be formatted into an easy-to-read format, and the number field may be formatted into a format with thousands separators, etc., so as to obtain the corresponding first feedback data.

[0117] The first feedback data is deduplicated to obtain corresponding second feedback data.

[0118] In this embodiment, if there are duplicate documents in the first feedback data, a deduplication operation may be performed on the first feedback data to obtain corresponding second feedback data.

[0119] Aggregate and analyze the second feedback data to obtain corresponding third feedback data.

[0120] In this embodiment, the aggregation function of the Elasticsearch database can be used to perform aggregation analysis on the second feedback data, such as calculating the average value, maximum value, minimum value of a field, counting the number of different values, etc., to obtain the third feedback data.

[0121] Corresponding response data is constructed based on the third feedback data.

[0122] In this embodiment, the above response data is an HTTP response. The response body of the response data is usually a string in JSON format, including query results, paging information, total number of matches, etc. The response header may include status code, content type, cache control and other information.

[0123] The response data is displayed in the preset interface.

[0124] In this embodiment, a display area may be pre-developed in a preset interface, and then the response data may be filled into the display area in the preset interface, thereby completing the display processing of the response data.

[0125] The present application obtains the corresponding first feedback data by formatting the feedback data; then deduplicates the first feedback data to obtain the corresponding second feedback data; then aggregates and analyzes the second feedback data to obtain the corresponding third feedback data; subsequently, the corresponding response data is constructed based on the third feedback data, and the response data is displayed in the preset interface. The present application obtains the third feedback data by formatting, deduplicating, and aggregate analyzing the feedback data, thereby effectively ensuring the accuracy and diversity of the generated third feedback data. Subsequently, the corresponding response data is constructed based on the third feedback data, and the response data is displayed in the preset interface, so that the user can easily view the response data corresponding to the keyword to be queried, which is conducive to improving the user experience.

[0126] In some optional implementations, before step S203, the electronic device may further perform the following steps:

[0127] Determine whether the current time has reached the preset trigger time of the scheduled task.

[0128] In this embodiment, a scheduling framework (such as Quartz, Spring Task Scheduler, etc.) can be used to configure the scheduled task, and the triggering time interval of the task (such as every hour, every day, etc.) can be set to ensure that the scheduled task can be executed regularly.

[0129] If yes, pull the data within the preset time period from the preset message middleware.

[0130] In this embodiment, when a scheduled task is triggered, the API or client library of the message middleware is called to pull data within a preset time period. The data usually exists in the form of a message queue, and the data can be obtained by consuming the message queue. In addition, there is no specific limitation on the selection of the above-mentioned preset time period, for example, within the past three months can be used.

[0131] Offline collision calculation is performed on the data to obtain specified context information and specified subsystem data related to the specified keyword.

[0132] In this embodiment, the above-mentioned specific implementation process of performing offline calculation collision on the data to obtain the specified context information and specified subsystem data related to the specified keyword will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0133] The specified context information and the specified subsystem data are format-converted based on the specified keyword to obtain corresponding specified data.

[0134] In this embodiment, the results of the offline collision calculation (specified context information and specified subsystem data) are organized into a certain format, such as JSON, XML, etc., to ensure that the result format matches the requirements of subsequent queries and processing. In addition, the offline collision calculation results can be indexed by keywords for quick search.

[0135] The designated data is stored in the cache library.

[0136] In this embodiment, the cache library may specifically be a Redis cache library, and the specified data may be stored by using data structures such as hashes, lists, and sets in the Redis cache library.

[0137] This application determines whether the current time has reached the preset trigger time of the scheduled task; if so, pulls the data within the preset time period from the preset message middleware; then performs offline calculation collision on the data to obtain the specified context information and specified subsystem data related to the specified keyword; then converts the format of the specified context information and the specified subsystem data based on the specified keyword to obtain the corresponding specified data; and subsequently stores the specified data in the cache. Through this application, data can be regularly pulled from the message middleware and offline calculation collisions can be performed, and the results of the offline calculation collisions can be subsequently stored in the cache, thereby providing users with fast and accurate query responses. At the same time, the continuous operation of the scheduled offline analysis service and the update mechanism of the cache ensure the timeliness and accuracy of the data.

[0138] In some optional implementations of this embodiment, performing offline collision calculation on the data to obtain specified context information and specified subsystem data related to the specified keyword includes the following steps:

[0139] The data is preprocessed to obtain corresponding processed data.

[0140] In this embodiment, the above-mentioned preprocessing may include deduplication processing and formatting processing. Among them, deduplication can be performed using a unique identifier (such as a primary key, a timestamp, etc.) according to the business logic and data characteristics. Specifically, data structures such as hash tables and Bloom filters can be used for efficient deduplication. In addition, the formatting process can be completed by converting the data into a format suitable for subsequent processing and performing standardized data field names and data types.

[0141] Get the preset offline collision calculation strategy.

[0142] In this embodiment, the strategy content of the above-mentioned offline calculation collision strategy may include: keyword matching, context analysis, and subsystem data association processing. Specifically, the keyword matching processing includes: extracting keywords or identifiers in the data, and matching them using set operations (such as intersection, union, etc.). The context analysis processing includes: performing text analysis on the data and extracting context information (such as sentences, paragraphs, etc.). And using natural language processing (NLP) technology (such as TF-IDF, Word2Vec, etc.) for semantic analysis. Subsystem data association processing includes: associating data from different subsystems according to business logic and data structure. And using foreign keys, primary keys and other relationships for association queries.

[0143] Gets the specified keyword of the preset.

[0144] In this embodiment, the above-mentioned designated keywords may be key sensitive words set according to actual business needs, for example, sensitive words that appear frequently.

[0145] Based on the offline calculation collision strategy, a preset offline calculation engine is called to perform offline calculation collision on the data to obtain specified context information and specified subsystem data related to the specified keyword.

[0146] In this embodiment, according to the policy content of the above-mentioned offline calculation collision strategy, operations such as join, groupBy, and aggregate in the offline calculation engine can be used to perform collision calculation on the data, thereby obtaining the specified context information and the specified subsystem data related to the specified keyword. Specifically, the offline calculation engine can use the Spark engine, which is suitable for real-time and batch processing of big data, has higher performance, and supports more complex operations.

[0147] The present application obtains corresponding processed data by preprocessing the data; then obtains a preset offline calculation collision strategy; then obtains a preset designated keyword; and subsequently based on the offline calculation collision strategy, calls a preset offline calculation engine to perform offline calculation collision on the data to obtain designated context information and designated subsystem data related to the designated keyword. The present application obtains corresponding processed data by preprocessing the data; then obtains an offline calculation collision strategy and designated keywords; and then based on the offline calculation collision strategy, performs offline calculation collision on the data using an offline calculation engine, so that designated context information and designated subsystem data related to the designated keyword can be obtained quickly and accurately, thereby improving the processing efficiency and accuracy of offline calculation collisions, and ensuring the data accuracy of the designated context information and designated subsystem data related to the designated keyword.

[0148] In some optional implementations, before step S205, the electronic device may further perform the following steps:

[0149] Based on a preset scanning thread, sensitive information data reported by each group service is obtained, and the sensitive information data is placed into a preset blocking queue.

[0150] In this embodiment, the scanning thread is a pre-built thread responsible for scanning the sensitive information data reported by each group service under the target path. After the scanned sensitive information data is parsed, it will be placed in a blocking queue (BlockingQueue). Among them, the reported data needs to follow the agreed fixed format. If the data format is not uniform, the data can be sorted through the ETL (Extract, Transform, Load) component before reporting.

[0151] The sensitive information data is obtained from the blocking queue based on a preset consumer thread, and the sensitive information data is placed into a preset message middleware.

[0152] In this embodiment, the above-mentioned consumer thread is used to obtain data in batches from the blocking queue and process it into the message queue. The data will be stored in the specified TOPIC according to sensitive words of different data types, and the data quality table will be recorded synchronously for subsequent verification of data processing.

[0153] The target entry program corresponding to the target database is called.

[0154] In this embodiment, the target entry program is specifically an ES warehousing program.

[0155] The target data in the message middleware is consumed based on the target storage program, and the target data is stored in the target database.

[0156] In this embodiment, the target database is specifically an Elasticsearch database. The ES storage program consumes data in the message middleware and stores the data in the Elasticsearch database. The data in the Elasticsearch database can be exposed to the outside through the application layer packaging interface to provide services for users to query the original sensitive information reported by each group.

[0157] In addition, the system also has a Spark Streaming storage program. The Spark Streaming program consumes data in the message middleware and performs the following processing: storing massive data in the big data platform (Flume components can be used for log collection and transmission). Then, key sensitive words (high-frequency statistics, thresholds can be set) are used for collision analysis, and the results of the collision analysis are stored in the Redis cache library, and the interface exposure service is encapsulated. The customized report data is statistically calculated and the results are pushed to the relational database.

[0158] This application obtains sensitive information data reported by each group service based on a preset scanning thread, and puts the sensitive information data into a preset blocking queue; then obtains the sensitive information data from the blocking queue based on a preset consumption thread, and puts the sensitive information data into a preset message middleware; then calls the target entry program corresponding to the target database; subsequently consumes the target data in the message middleware based on the target storage program, and stores the target data in the target database. This application obtains sensitive information data reported by each group service based on the use of a scanning thread, and puts the sensitive information data into a blocking queue; then obtains sensitive information data from the blocking queue based on the use of a consumption thread, and puts the sensitive information data into the message middleware, and then consumes the target data in the message middleware based on the use of the target entry program, and stores the target data in the target database, thereby improving the construction efficiency and construction intelligence of the target database. It is beneficial to provide users with fast and accurate query responses based on the subsequent use of the target database, thereby improving the user experience.

[0159] In some optional implementations of this embodiment, after step S203, the electronic device may further perform the following steps:

[0160] If there is a query result matching the query keyword in the cache library, the query result is obtained.

[0161] In this embodiment, according to the query keyword included in the query request, the query result matching the query keyword can be queried from the cache library.

[0162] The query result is formatted based on a preset target format to obtain a corresponding target query result.

[0163] In this embodiment, the above target format may be a format suitable for front-end display. The extracted data may be converted into a format suitable for front-end display according to actual business needs. For example, the date field may be converted into a date format recognizable by the front-end, the numeric field may be formatted, etc., so as to obtain the corresponding target query result.

[0164] The target query result is sent to the user.

[0165] In this embodiment, the above processing procedure of sending the target query result to the user can refer to the above specific processing procedure of sending the feedback data to the user, and will not be described in detail here.

[0166] If the present application detects that there are query results matching the query keywords in the cache library, the query results are obtained; then the query results are formatted based on a preset target format to obtain a corresponding target query result; and the target query result is subsequently sent to the user. The present application can intelligently provide users with fast and accurate query responses based on the use of a cache library, effectively improving the efficiency of obtaining query results. Subsequently, the query data will be intelligently formatted based on the target format to obtain the target query result, and then the target query result will be sent to the user, which is conducive to improving the user experience.

[0167] In some optional implementations, the user information obtained is subject to the user's consent and complies with relevant laws and policies.

[0168] In addition, the non-Company software tools or components that appear in the embodiments of the present application are merely examples and do not represent actual use.

[0169] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0170] It should be emphasized that in order to further ensure the privacy and security of the above feedback data, the above feedback data can also be stored in a blockchain node.

[0171] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0172] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0173] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0174] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0175] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0176] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a data query device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0177] like Figure 3 As shown, the data query device 300 described in this embodiment includes: a first judgment module 301, a verification module 302, a second judgment module 303, a construction module 304, a search module 305, an extraction module 306 and a first sending module 307. Among them:

[0178] The first judgment module 301 is used to judge whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword;

[0179] A verification module 302, configured to verify the query request based on a preset verification strategy;

[0180] The second judgment module 303 is used to parse the query request to obtain the query keyword if the query request passes the verification, and to judge whether there is a query result matching the query keyword in the preset cache library;

[0181] A construction module 304 is used to construct a corresponding target query statement based on the query keyword if there is no query result matching the query keyword in the cache library;

[0182] Search module 305, used to call a preset target database to search the target query statement and obtain corresponding search results;

[0183] The extraction module 306 is used to perform data extraction processing on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword;

[0184] The first sending module 307 is configured to send the feedback data to the user.

[0185] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data query method in the aforementioned implementation mode, and are not described in detail here.

[0186] In some optional implementations of this embodiment, the verification module 302 includes:

[0187] A first acquisition submodule, used to acquire request parameters of the query request;

[0188] The second acquisition submodule is used to acquire a preset legality verification strategy and validity verification strategy;

[0189] A first verification submodule, used to perform a validity check on the request parameters based on the validity check strategy;

[0190] A second verification submodule, configured to perform validity verification on the request parameters based on the validity verification strategy if the request parameters pass the validity verification;

[0191] The determination submodule is used to determine that the query request has passed the verification if the request parameter passes the validity verification, and otherwise determine that the query request has not passed the verification.

[0192] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data query method in the aforementioned implementation mode, and are not described in detail here.

[0193] In some optional implementations of this embodiment, the first sending module 307 includes:

[0194] A first processing submodule, configured to perform formatting processing on the feedback data to obtain corresponding first feedback data;

[0195] A second processing submodule, configured to perform deduplication processing on the first feedback data to obtain corresponding second feedback data;

[0196] A third processing submodule, configured to perform aggregation analysis on the second feedback data to obtain corresponding third feedback data;

[0197] A construction submodule, configured to construct corresponding response data based on the third feedback data;

[0198] The display submodule is used to display the response data in the preset interface.

[0199] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data query method in the aforementioned implementation mode, and are not described in detail here.

[0200] In some optional implementations of this embodiment, the data query device further includes:

[0201] The third judgment module is used to judge whether the current time reaches the trigger time of the preset scheduled task;

[0202] A pull module is used to pull data within a preset time period from a preset message middleware;

[0203] A calculation module, used for performing offline calculation collision on the data to obtain specified context information and specified subsystem data related to the specified keyword;

[0204] A first conversion module, configured to perform format conversion on the specified context information and the specified subsystem data based on the specified keyword to obtain corresponding specified data;

[0205] The storage module is used to store the specified data in the cache library.

[0206] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data query method in the aforementioned implementation mode, and are not described in detail here.

[0207] In some optional implementations of this embodiment, the calculation module includes:

[0208] A preprocessing submodule, used to preprocess the data to obtain corresponding processed data;

[0209] The third acquisition submodule is used to obtain a preset offline calculation collision strategy;

[0210] The fourth acquisition submodule is used to acquire a preset designated keyword;

[0211] The calculation submodule is used to call a preset offline calculation engine to perform offline calculation collision on the data based on the offline calculation collision strategy to obtain specified context information and specified subsystem data related to the specified keyword.

[0212] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data query method in the aforementioned implementation mode, and are not described in detail here.

[0213] In some optional implementations of this embodiment, the data query device further includes:

[0214] A first acquisition module, used for acquiring sensitive information data reported by each group service based on a preset scanning thread, and putting the sensitive information data into a preset blocking queue;

[0215] An insert module is used to obtain the sensitive information data from the blocking queue based on a preset consumer thread, and insert the sensitive information data into a preset message middleware;

[0216] A calling module, used for calling a target entry program corresponding to the target database;

[0217] The storage module is used to consume the target data in the message middleware based on the target storage program and store the target data into the target database.

[0218] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data query method in the aforementioned implementation mode, and are not described in detail here.

[0219] In some optional implementations of this embodiment, the data query device further includes:

[0220] A second acquisition module is used to acquire the query result if there is a query result matching the query keyword in the cache library;

[0221] A second conversion module, used to convert the format of the query result based on a preset target format to obtain a corresponding target query result;

[0222] The second sending module is used to send the target query result to the user.

[0223] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the data query method in the aforementioned implementation mode, and are not described in detail here.

[0224] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0225] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0226] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0227] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the data query method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0228] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the data query method.

[0229] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0230] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0231] In an embodiment of the present application, after receiving a query request triggered by a user through a preset interface and detecting that the query request has passed the verification, the query request will be parsed to obtain a query keyword, and then it will be detected whether there is a query result matching the query keyword in the cache library. If not, a corresponding target query statement will be intelligently constructed based on the query keyword, and then the target query statement of the target database will be called to perform search processing to obtain search results, and data extraction processing will be performed on the search results to obtain feedback data corresponding to the query keyword, and finally the feedback data will be sent to the user. Different from the existing data query method based on manual processing, the present application uses an automated query processing method based on the cache library and the target database instead of manual screening and merging and aggregation methods, which effectively reduces repetitive work, reduces the risk of manual statistical errors, and improves the processing efficiency and accuracy of query processing.

[0232] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the data query method as described above.

[0233] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0234] In an embodiment of the present application, after receiving a query request triggered by a user through a preset interface and detecting that the query request has passed the verification, the query request will be parsed to obtain a query keyword, and then it will be detected whether there is a query result matching the query keyword in the cache library. If not, a corresponding target query statement will be intelligently constructed based on the query keyword, and then the target query statement of the target database will be called to perform search processing to obtain search results, and data extraction processing will be performed on the search results to obtain feedback data corresponding to the query keyword, and finally the feedback data will be sent to the user. Different from the existing data query method based on manual processing, the present application uses an automated query processing method based on the cache library and the target database instead of manual screening and merging and aggregation methods, which effectively reduces repetitive work, reduces the risk of manual statistical errors, and improves the processing efficiency and accuracy of query processing.

[0235] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0236] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A data query method, characterized in that: The steps include: Determine whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword; If yes, verify the query request based on a preset verification strategy; If the query request passes the verification, the query request is parsed to obtain the query keyword, and it is determined whether there is a query result matching the query keyword in the preset cache library; If there is no query result matching the query keyword in the cache library, constructing a corresponding target query statement based on the query keyword; Calling a preset target database to search and process the target query statement to obtain corresponding search results; Performing data extraction processing on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword; The feedback data is sent to the user.

2. The data query method according to claim 1, characterized in that: The step of verifying the query request based on a preset verification strategy specifically includes: Obtaining request parameters of the query request; Obtain the preset legality verification strategy and effectiveness verification strategy; Performing a validity check on the request parameters based on the validity check strategy; If the request parameter passes the validity check, performing validity check on the request parameter based on the validity check strategy; If the request parameter passes the validity check, then it is determined that the query request passes the check; otherwise, it is determined that the query request fails the check.

3. The data query method according to claim 1, characterized in that: The step of sending the feedback data to the user specifically includes: Performing formatting processing on the feedback data to obtain corresponding first feedback data; performing deduplication processing on the first feedback data to obtain corresponding second feedback data; Performing aggregation analysis on the second feedback data to obtain corresponding third feedback data; Constructing corresponding response data based on the third feedback data; The response data is displayed in the preset interface.

4. The data query method according to claim 1, characterized in that: Before the step of determining whether there is a query result matching the query keyword in the preset cache library, the method further includes: Determine whether the current time reaches the preset trigger time of the scheduled task; If yes, pull the data within the preset time period from the preset message middleware; Performing offline collision calculation on the data to obtain specified context information and specified subsystem data related to the specified keyword; Performing format conversion on the specified context information and the specified subsystem data based on the specified keyword to obtain corresponding specified data; The designated data is stored in the cache library.

5. The data query method according to claim 4, characterized in that: The step of performing offline calculation collision on the data to obtain specified context information and specified subsystem data related to the specified keyword specifically includes: Preprocessing the data to obtain corresponding processed data; Get the preset offline collision calculation strategy; Get the preset specified keyword; Based on the offline calculation collision strategy, a preset offline calculation engine is called to perform offline calculation collision on the data to obtain specified context information and specified subsystem data related to the specified keyword.

6. The data query method according to claim 1, characterized in that: Before the step of calling the preset target database to search the target query statement to obtain the corresponding search results, the method further includes: Acquire sensitive information data reported by each group service based on a preset scanning thread, and put the sensitive information data into a preset blocking queue; Obtaining the sensitive information data from the blocking queue based on a preset consumer thread, and putting the sensitive information data into a preset message middleware; Calling a target entry program corresponding to the target database; The target data in the message middleware is consumed based on the target storage program, and the target data is stored in the target database.

7. The data query method according to claim 1, characterized in that: After the step of determining whether there is a query result matching the query keyword in the preset cache library, the method further includes: If there is a query result matching the query keyword in the cache library, then obtaining the query result; Convert the query result to a target format to obtain the corresponding query result. The target query result is sent to the user.

8. A data query device, characterized in that: include: A first judgment module is used to judge whether a query request triggered by a user through a preset interface is received; wherein the query request carries a query keyword; A verification module, used for verifying the query request based on a preset verification strategy; A second judgment module is used to parse the query request to obtain the query keyword if the query request passes the verification, and to judge whether there is a query result matching the query keyword in a preset cache library; A construction module, configured to construct a corresponding target query statement based on the query keyword if there is no query result matching the query keyword in the cache library; A search module is used to call a preset target database to search the target query statement and obtain corresponding search results; An extraction module, configured to perform data extraction processing on the search results to obtain feedback data corresponding to the query keyword; wherein the target feedback data includes context information and subsystem data corresponding to the query keyword; The first sending module is used to send the feedback data to the user.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data query method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data query method according to any one of claims 1 to 7 are implemented.

Images