Bank historical data query system based on big data analysis

Abstract

The invention relates to a data query method and system, computer equipment and a readable storage medium. The method is applied to a data query system, the data query system comprises an interaction module, a plurality of query engines, a distributed storage module and an analysis module, and the method comprises the steps that after the interaction module obtains a query condition input by a user and at least one target query engine selected by the user from the plurality of query engines, the target query engine is stored in the distributed storage module; converting the query condition into a target query statement, and sending the target query statement to each target query engine; each target query engine queries from the distributed storage module based on a dynamic pruning strategy according to the target query statement to obtain an initial query result; and the analysis module processes the initial query result queried by each target query engine based on a vectorization acceleration calculation strategy to obtain a target query result. By adopting the method, the query speed can be increased.

Description

Technical Field

[0001] This application relates to the field of financial data query technology, and in particular to a data query method, system, computer device, readable storage medium, and program product. Background Technology

[0002] Financial institutions (such as banks), as typical data-intensive industries, generate hundreds of millions of structured and unstructured transaction data points daily and continuously accumulate massive amounts of historical business data. This data contains high-value business insights and risk control information, which frontline business personnel (such as tellers and account managers) and middle and back-office management personnel (such as auditors and risk analysts) must efficiently query and multi-dimensionally analyze to support key scenarios such as precision marketing, customer service, risk management, and compliance auditing.

[0003] However, traditional data query systems suffer from slow query response speeds when faced with massive amounts of data. Summary of the Invention

[0004] Therefore, it is necessary to provide a data query method, system, computer equipment, readable storage medium, and program product that can improve query speed to address the above-mentioned technical problems.

[0005] Firstly, this application provides a data query method applied to a data query system. The data query system includes an interaction module, multiple query engines, a distributed storage module, and an analysis module. The method includes: after the interaction module obtains the query conditions input by the user and at least one target query engine selected by the user from the multiple query engines, it converts the query conditions into target query statements and sends the target query statements to each target query engine; each target query engine retrieves initial query results from the distributed storage module based on the target query statements and a dynamic pruning strategy; the analysis module processes the initial query results retrieved by each target query engine based on a vectorized accelerated computing strategy to obtain the target query results.

[0006] In one embodiment, the distributed storage module stores transaction data of multiple accounts. The method further includes: an analysis module obtaining transaction data of multiple accounts from the distributed storage module, classifying each account into different levels based on the transaction data of each account, and obtaining the account level corresponding to each account; and storing the account level corresponding to each account in the distributed storage module for users to view.

[0007] In one embodiment, the method further includes: an analysis module identifying whether each account is an abnormal account based on the transaction data of each account; if there are abnormal accounts among multiple accounts, the account identifier of the abnormal account is sent to the interaction module; the interaction module displays the account identifier of the abnormal account to remind the user.

[0008] In one embodiment, the data query system further includes a security compliance module, and the method further includes: the security compliance module performing anonymization on the data to be stored in the distributed storage module, and storing the anonymized data in the distributed storage module; correspondingly, each target query engine obtains initial query results from the distributed storage module based on the target query statement and a dynamic pruning strategy, including: each target query engine sending a data acquisition request to the distributed storage module based on the target query statement and a dynamic pruning strategy; the distributed storage module responding to each data acquisition request sending the undecrypted initial query results to the security compliance module; the security compliance module decrypting the undecrypted initial query results to obtain the initial query results, and sending the initial query results to the corresponding target query engine.

[0009] In one embodiment, converting query conditions into target query statements includes: the interaction module sending the user's identity identifier to the security compliance module; the security compliance module determining the user's corresponding data query range identifier based on the identity identifier and sending the data query range identifier to the interaction module; and the interaction module converting the query conditions and data query range identifier into statements to obtain the target query statement.

[0010] In one embodiment, the distributed storage module includes a first database and a second database; the first database stores hot data partitioned according to time and business type; the second database stores compressed cold data stored in a columnar format.

[0011] Secondly, this application also provides a data query system, which includes an interaction module, multiple query engines, a distributed storage module, and an analysis module: the interaction module is used to execute the steps performed by the interaction module as described in any one of the first aspects; after being selected by the user, the query engine is used to execute the steps performed by the target query engine as described in any one of the first aspects; the distributed storage module is used to execute the steps performed by the distributed storage module as described in any one of the first aspects; and the analysis module is used to execute the steps performed by the analysis module as described in any one of the first aspects.

[0012] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any one of the first aspects above.

[0013] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0014] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0015] The aforementioned data query method, system, computer equipment, readable storage medium, and program product, in the data query system, after the interaction module receives the user's input query conditions and at least one target query engine selected by the user from multiple query engines, converts the query conditions into target query statements and sends the target query statements to each target query engine. Each target query engine, based on the target query statement and using a dynamic pruning strategy, retrieves initial query results from the distributed storage module. The analysis module, based on a vectorized accelerated computing strategy, processes the initial query results retrieved by each target query engine to obtain the target query results. Thus, on the one hand, the dynamic pruning strategy can skip irrelevant data partitions, greatly reducing the amount of data read; on the other hand, the vectorized accelerated computing strategy enables parallel computing, improving computing speed. Therefore, this data query system can significantly improve data query speed. Furthermore, different query engines have different advantages; users can select a target query engine according to their needs, allowing different query tasks to be executed using the query engine they are best suited for, further improving query speed. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a data query method in one embodiment;

[0018] Figure 2 This is a flowchart illustrating the data query method in another embodiment;

[0019] Figure 3 This is a block diagram of a data query system in one embodiment;

[0020] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0022] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0023] Traditional data query systems use relational databases, such as Oracle and MySQL. Relational databases rely on SQL (Structured Query Language) for queries, are adept at handling structured data, and support transaction consistency. However, when the data volume exceeds terabytes, query response time increases significantly (e.g., complex report queries may take several minutes). Limited by single-machine or master-slave architectures, vertical scaling (upgrading hardware) is costly, and horizontal scaling (database sharding) requires restructuring business logic, resulting in high scaling costs.

[0024] Traditional data query systems use data warehouses, such as Teradata and IBM Db2, which employ an MPP (Massively Parallel Processing) architecture. These support batch ETL (Extract, Transform, Load) and OLAP (Online Analytical Processing) analyses for historical data statistics. However, ETL processes suffer from high latency, making them unsuitable for real-time query requirements. Furthermore, the schema (the logical framework of a traditional data warehouse) design must be fixed in advance, making it difficult to adapt to rapid business changes, resulting in poor flexibility. Finally, maintenance requires a dedicated team, and hardware and license costs are extremely high, making operation and maintenance complex.

[0025] Traditional data query systems utilize big data technologies such as Hadoop MapReduce, which solve the problem of massive data storage through distributed storage (HDFS, Hadoop Distributed File System) and batch processing (MapReduce). However, MapReduce tasks are time-consuming to start, and even simple queries require minutes to respond, resulting in high query latency; they require writing Java code, making them difficult for business users to use directly, thus posing a high development barrier; and they only support offline batch processing, thus lacking real-time analysis capabilities.

[0026] Traditional data query systems use intelligent tools such as SAP BW and Cognos, which provide visual reports and dashboards and rely on predefined data models. However, these intelligent tools are difficult to integrate with other systems, require repeated data extraction, and thus suffer from data silos. Users cannot explore data independently and rely on professional teams to customize reports, resulting in weak interactivity.

[0027] The shortcomings of traditional data query systems can be attributed to several factors. First, they suffer from architectural limitations, as traditional data query systems employ centralized or semi-distributed architectures that fail to balance performance and scalability. Second, their technology stacks are fragmented, with storage, computation, and analysis belonging to different systems, resulting in low data flow efficiency. Third, there is a conflict between security and performance, as encryption and access control typically slow down query speeds.

[0028] While traditional data query systems can guarantee the ACID properties (Atomicity, Consistency, Isolation, Durability) of transactions in small to medium-sized data scenarios, they reveal significant shortcomings in performance (such as query speed), cost, and flexibility when querying petabytes of historical data from financial institutions.

[0029] Therefore, it is necessary to propose effective technical means to solve the above problems. The technical solution of this application and how it solves the above technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0030] In one exemplary embodiment, such as Figure 1 As shown, a data query method is provided, which is applied to a data query system. The data query system includes an interaction module, multiple query engines, a distributed storage module, and an analysis module. The method includes steps 101 to 103. Wherein:

[0031] Step 101: After obtaining the query conditions input by the user and at least one target query engine selected by the user from multiple query engines, the interaction module converts the query conditions into target query statements and sends the target query statements to each target query engine.

[0032] In one optional implementation, the interaction module displays an interactive interface, which includes a query field input area, a voice input control, and multiple query engine selection controls; the query field input area is used for the user to input query conditions; and the multiple query engine selection controls correspond one-to-one with multiple query engines.

[0033] The interaction module obtains the content entered by the user in the input area of ​​the query field to get the query conditions entered by the user, or the interaction module recognizes the user's voice after detecting the trigger operation of the voice input control to get the query conditions entered by the user.

[0034] After the interaction module detects the triggering operation of the query engine selection control, it uses the query engine corresponding to the query engine selection control as the target query engine.

[0035] It's understandable that users (such as bank staff) typically input rather colloquial query conditions, such as "compare total corporate loan amounts in 2022 and 2023," "show the top 10 loss-making clients of the Shenzhen branch last year," and "top 10 deposit clients of the Shenzhen branch in 2023." Therefore, these need to be converted into target query statements, such as SQL. The interactive module can use NLQ (Natural Language Query) technology to convert query conditions into target query statements.

[0036] Multiple query engines are available, including those that handle complex analytical queries (such as multi-table JOINs and window functions) and those that support full-text search and fuzzy queries (such as matching by the first letter of a customer's name in pinyin).

[0037] Step 102: Each target query engine retrieves the initial query results from the distributed storage module based on the target query statement and a dynamic pruning strategy.

[0038] The distributed storage module includes a first database, which stores data based on a partitioning strategy. The dynamic pruning strategy refers to using metadata to skip (prune) irrelevant partitions during query execution. For example, if the query condition is "Top 10 deposit customers of Shenzhen branch in 2023", then only partitions with a date of 2023 will be queried, ignoring partitions from other years.

[0039] In one implementation, for each target query engine, the target query engine parses the target query statement and generates an execution plan. For example, it determines which tables to access (such as transaction records), which columns are needed (such as customer name, transaction quantity, transaction amount, date, region, business type, etc.), and what operations to perform (such as filtering, grouping, aggregation, joining, sorting).

[0040] The target query engine includes multiple query nodes and a coordinator. The coordinator distributes multiple query tasks based on the execution plan to multiple query nodes. Each query node retrieves subquery results from the distributed storage module according to the query task and sends the subquery results to the coordinator. The coordinator processes the multiple subquery results, such as filtering, grouping, aggregating, and sorting, to obtain the initial query results.

[0041] In one example, the coordinator processes the results of multiple subqueries, which could be achieved by the coordinator using a vectorized acceleration strategy to process the results of multiple subqueries to obtain the initial query result.

[0042] Vectorization acceleration strategy refers to the strategy of accelerating numerical calculation by vectorizing execution instructions. That is, a single instruction can complete a batch of calculations, which can further improve the processing efficiency of multiple subquery results, thereby improving query speed.

[0043] Step 103: The analysis module processes the initial query results obtained by each target query engine based on the vectorization acceleration calculation strategy to obtain the target query results.

[0044] The initial query results obtained by each target query engine are processed, including filtering, grouping, aggregation, joining, and sorting. The specific processing method is related to the query conditions and is not limited here.

[0045] It is understandable that after obtaining the target query result, the analysis module will send the target query result to the interaction module to display the target query result.

[0046] In the aforementioned data query method, the interaction module in the data query system, after obtaining the user's input query conditions and at least one target query engine selected by the user from multiple query engines, converts the query conditions into target query statements and sends the target query statements to each target query engine. Each target query engine, based on the target query statement and using a dynamic pruning strategy, retrieves initial query results from the distributed storage module. The analysis module, based on a vectorized accelerated computation strategy, processes the initial query results retrieved by each target query engine to obtain the target query results. Thus, on the one hand, the dynamic pruning strategy can skip irrelevant data partitions, greatly reducing the amount of data read; on the other hand, the vectorized accelerated computation strategy enables parallel computing, improving computation speed. Therefore, this data query system can significantly improve data query speed. Furthermore, different query engines have different advantages; users can select a target query engine according to their needs, allowing different query tasks to be executed using the query engine they are best suited for, further improving query speed.

[0047] In one embodiment, the distributed storage module includes a first database and a second database; the first database stores hot data partitioned according to time and business type; the second database stores compressed cold data stored in a columnar format.

[0048] Hot data refers to data accessed at a frequency greater than or equal to a preset frequency within a target timeframe from the current time. For example, the first database stores frequently accessed data from the past three years. Furthermore, hot data can be written at high throughput using a tree structure.

[0049] The first database is partitioned by both time and business type to avoid hotspot issues, which also means avoiding data skew issues.

[0050] Cold data refers to data that is more than 3 years old and beyond the current target time. The compression ratio can reach 5:1, in other words, cold data is processed using an archiving method.

[0051] The distributed storage module also includes a metadata management unit, which is used to implement data lineage tracking, record data source, transformation logic, and access permissions.

[0052] In one embodiment, the distributed storage module stores transaction data for multiple accounts. The method further includes: an analysis module identifying whether each account is an abnormal account based on the transaction data; if an abnormal account exists among the multiple accounts, the account identifier of the abnormal account is sent to the interaction module; the interaction module displays the account identifier of the abnormal account to alert the user. The account identifier can be the account name.

[0053] In one optional implementation, the analysis module analyzes the account's transaction data to identify whether the account has frequent small-amount transfers; if so, the account is determined to be an abnormal account.

[0054] In one embodiment, the distributed storage module stores transaction data from multiple accounts, such as... Figure 2 As shown, another data query method is provided, which further includes:

[0055] Step 201: The analysis module retrieves transaction data from multiple accounts from the distributed storage module, and classifies each account into different levels based on the transaction data to obtain the corresponding account level.

[0056] In one example, the analysis module obtains the trading frequency and transaction amount of each account based on their transaction data. Based on these metrics, the accounts are then categorized into different account levels. It's understood that accounts with higher trading frequency and transaction amounts have higher account levels, thus identifying high-net-worth clients.

[0057] Step 202: Store the account level corresponding to each account in the distributed storage module for users to view.

[0058] After the analysis module stores the account levels corresponding to each account in the distributed storage module, when a user queries accounts with account levels higher than a preset level based on steps 101 to 103, the analysis module can filter out accounts with account levels higher than the preset level from the distributed storage module and send the account identifier and account level of the accounts with higher than the preset level to the interaction module. The interaction module generates and displays a chart based on the received account identifier and account registration. In addition, the interaction module can also implement responsive charts, automatically adjusting the layout on the terminal (e.g., mobile device).

[0059] In one embodiment, the analysis module obtains deposit data from multiple regions from the distributed storage module, processes the deposit data from multiple regions to obtain deposit distribution heatmap data, and sends the deposit distribution heatmap data to the interaction module; the interaction module generates and displays a deposit distribution heatmap by region based on the deposit distribution heatmap data.

[0060] In an optional embodiment, the analysis module can also retrieve deposit data from multiple regions from the distributed storage module, process the stored data from multiple regions to obtain deposit distribution heatmap data, and then store the deposit distribution heatmap data in the distributed storage module. After the user queries and obtains the deposit distribution heatmap data based on steps 101 to 103, the analysis module sends the deposit distribution heatmap data to the interaction module. The interaction module generates and displays a deposit distribution heatmap by region based on the deposit distribution heatmap data. In addition, the interaction module also supports drag-and-drop dashboard generation.

[0061] In one embodiment, the data query system further includes a security compliance module, and the method further includes: the security compliance module performing an anonymization operation on the data that needs to be stored in the distributed storage module, and storing the anonymized data in the distributed storage module. The anonymization operation involves encrypting sensitive fields (such as ID card numbers) in the data.

[0062] Correspondingly, each target query engine retrieves initial query results from the distributed storage module based on the target query statement and a dynamic pruning strategy. This includes: each target query engine sending a data retrieval request to the distributed storage module based on the target query statement and a dynamic pruning strategy; the distributed storage module responding to each data retrieval request sending the uncracked initial query results to the security and compliance module; and the security and compliance module decrypting the uncracked initial query results to obtain the initial query results, and then sending the initial query results to the corresponding target query engine.

[0063] In other words, the data returned by the distributed storage module to the target query engine is not sent directly to the target query engine. Instead, it is first sent to the security and compliance module, which decrypts it before sending it to the target query engine, thus achieving dynamic decryption during the query process.

[0064] In addition, if the data query system is in a development and testing environment, the security and compliance module will mask sensitive fields, such as replacing names with "AA".

[0065] In one embodiment, converting query conditions into target query statements includes: the interaction module sending the user's identity identifier to the security compliance module; the security compliance module determining the user's corresponding data query range identifier based on the identity identifier and sending the data query range identifier to the interaction module; and the interaction module converting the query conditions and data query range identifier into statements to obtain the target query statement.

[0066] The user's identity identifier is used to represent the user's role (teller, auditor, and account manager, etc.). Each role has a different data query scope, that is, different permissions. For example, a teller can only see data from their own branch. It can be understood that the security and compliance module can define roles and control field-level permissions.

[0067] The target query engine, based on the target query statement with a data query range identifier, can only retrieve data within the user's permissions from the distributed storage module.

[0068] In addition, auditors generally have more privileges and can query more data. Therefore, when users with the role of auditor query data, the security and compliance module will record all of the user's query behavior as audit logs and send them to the distributed storage module for storage, which is generally stored for more than 6 years.

[0069] In summary, this application has the following advantages: (1) High-efficiency query: sub-second response, supporting PB-level data; (2) Rich functionality: combining analysis and visualization to help data-driven decision-making; (3) Strong scalability: the distributed architecture can be horizontally scaled to adapt to data growth; (4) Security and compliance: meeting the security requirements of the financial industry; (5) Cost optimization: compared with traditional data warehouses, hardware and maintenance costs are reduced by more than 50%.

[0070] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0071] Based on the same inventive concept, this application also provides a data query system for implementing the data query method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data query system embodiments provided below can be found in the limitations of the data query method described above, and will not be repeated here.

[0072] In one exemplary embodiment, such as Figure 3 As shown, a data query system is provided, comprising: an interaction module, multiple query engines, a distributed storage module, and an analysis module, wherein:

[0073] The interaction module is used to execute the steps performed by the interaction module in any one of the above method embodiments; after being selected by the user, the query engine is used to execute the steps performed by the target query engine in any one of the above method embodiments; the distributed storage module is used to execute the steps performed by the distributed storage module in any one of the above method embodiments; and the analysis module is used to execute the steps performed by the analysis module in any one of the above method embodiments.

[0074] In one embodiment, the data query system further includes a security compliance module and a data acquisition module; the security compliance module is used to perform the steps performed by the security compliance module as described in any of the above method embodiments; the data acquisition module is used to perform the steps performed by the data acquisition module as described in any of the above method embodiments.

[0075] Each module in the aforementioned data query system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0076] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a data query method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0077] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0078] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any one of the above method embodiments.

[0079] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the above method embodiments.

[0080] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in any one of the above method embodiments.

[0081] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0082] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0083] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0084] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data query method, characterized in that, Applied to a data query system, the data query system including an interaction module, multiple query engines, a distributed storage module, and an analysis module, the method includes: After obtaining the query conditions input by the user and at least one target query engine selected by the user from the plurality of query engines, the interaction module converts the query conditions into target query statements and sends the target query statements to each of the target query engines. Each of the target query engines obtains initial query results from the distributed storage module based on the target query statement and a dynamic pruning strategy; The analysis module, based on a vectorized accelerated computing strategy, processes the initial query results obtained by each of the target query engines to obtain the target query results.

2. The method according to claim 1, characterized in that, The distributed storage module stores transaction data from multiple accounts, and the method further includes: The analysis module obtains transaction data of multiple accounts from the distributed storage module, and classifies each account into different levels based on the transaction data of each account to obtain the account level corresponding to each account. The account levels corresponding to each account are stored in the distributed storage module for users to access.

3. The method according to claim 2, characterized in that, The method further includes: The analysis module identifies whether each account is an abnormal account based on the transaction data of each account; if there is an abnormal account among multiple accounts, the account identifier of the abnormal account is sent to the interaction module. The interactive module displays the account identifier of the abnormal account to alert the user.

4. The method according to claim 1, characterized in that, The data query system also includes a security and compliance module, and the method further includes: The security and compliance module performs an anonymization operation on the data that needs to be stored in the distributed storage module, and then stores the anonymized data in the distributed storage module. Correspondingly, each of the target query engines obtains initial query results from the distributed storage module based on the target query statement and a dynamic pruning strategy, including: Each of the target query engines sends a data retrieval request to the distributed storage module based on the target query statement and a dynamic pruning strategy. In response to each of the data acquisition requests, the distributed storage module sends the undecrypted initial query results to the security and compliance module. The security and compliance module decrypts each of the undecrypted initial query results to obtain each of the initial query results, and sends each of the initial query results to the corresponding target query engine.

5. The method according to claim 4, characterized in that, The step of converting the query conditions into a target query statement includes: The interaction module sends the user's identity identifier to the security and compliance module; The security and compliance module determines the data query range identifier corresponding to the user based on the identity identifier, and sends the data query range identifier to the interaction module; The interaction module performs statement conversion on the query conditions and the data query range identifier to obtain the target query statement.

6. The method according to claim 1, characterized in that, The method further includes: The analysis module obtains deposit data from multiple regions from the distributed storage module, processes the deposit data from multiple regions to obtain deposit distribution heat map data, and sends the deposit distribution heat map data to the interaction module. The interactive module generates and displays a deposit distribution heatmap by region based on the deposit distribution heatmap data.

7. The method according to claim 1, characterized in that, The distributed storage module includes a first database and a second database; The first database stores hot data that is partitioned according to time and business type; The second database stores compressed cold data in a columnar format.

8. A data query system, characterized in that, The data query system includes an interaction module, multiple query engines, a distributed storage module, and an analysis module. The interaction module is used to perform the steps performed by the interaction module according to any one of claims 1-7; After being selected by the user, the query engine is used to perform the steps performed by the target query engine as described in any one of claims 1-7; The distributed storage module is used to perform the steps performed by the distributed storage module according to any one of claims 1-7; The analysis module is used to perform the steps performed by the analysis module according to any one of claims 1-7.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images