Data query method and device, computer equipment and storage medium

Abstract

The invention relates to a data query method and device, computer equipment and a storage medium. The method comprises the following steps: receiving a data query request sent by a client; analyzing the data query request to extract coding information corresponding to the target type from the data query request; carrying out validity verification on the coded information; if the coding information passes the validity verification, performing routing rule matching and data source selection processing on the coding information based on a preset rule engine to obtain a corresponding candidate data source; performing state evaluation on the candidate data sources to obtain a state evaluation result, and screening out a target data source from the candidate data sources according to the state evaluation result; obtaining a query statement corresponding to the data query request; sending the query statement into the target data source based on the connection pool so as to execute query processing on the query statement through the target data source to obtain a query result; and returning a query result to the client. The processing efficiency and the processing flexibility of the data query request are improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to data query methods, apparatus, computer equipment and storage media. Background Technology

[0002] In the development of modern enterprise application systems, multi-data source querying has become a common technical means to meet diverse business needs. As enterprise business scale continues to expand and data volume explodes, enterprises typically distribute data across multiple databases to achieve key objectives such as load balancing, data isolation, and business separation. Common multi-data source application scenarios cover various aspects including database sharding, read / write separation, multi-tenant systems, and data querying in multi-cloud environments.

[0003] However, in high-concurrency query scenarios of existing systems, the data source selection mechanism has significant shortcomings. Most current solutions only use a simple round-robin approach to achieve load balancing. This simplistic and unintelligent mechanism cannot dynamically adjust based on the actual load of each data source. In actual operation, this easily leads to some data sources becoming overloaded due to excessive query requests, while others remain idle. This uneven resource allocation not only severely impacts overall query performance, resulting in low query request processing efficiency, but also reduces system flexibility, making it difficult to adapt to complex and ever-changing business needs. Therefore, there is an urgent need to develop a data query technology based on an intelligent data source selection mechanism to improve the performance and flexibility of enterprise application systems in high-concurrency query scenarios. Summary of the Invention

[0004] The purpose of this application is to provide a data query method, apparatus, computer device, and storage medium to solve the technical problem that the existing data source selection mechanism based on simple polling has low processing efficiency and poor flexibility in query requests.

[0005] Firstly, a data query method is provided, including: Receive data query requests from clients; The data query request is parsed to extract the encoding information corresponding to the target type from the data query request; The validity of the encoded information is verified. If the encoded information passes the validity verification, then the encoded information is processed by routing rule matching and data source selection based on the preset rule engine to obtain the corresponding candidate data source; A status assessment is performed on the candidate data sources to obtain a status assessment result, and a target data source is selected from the candidate data sources based on the status assessment result; Obtain the query statement corresponding to the data query request; The query statement is sent to the target data source based on a preset connection pool, so that the query statement is processed by the target data source to obtain the corresponding query results; The query results are returned to the client.

[0006] Secondly, a data query device is provided, comprising: The receiving module is used to receive data query requests sent by the client; The extraction module is used to parse the data query request in order to extract the encoding information corresponding to the target type from the data query request; The verification module is used to verify the validity of the encoded information; The first processing module is used to perform routing rule matching and data source selection processing on the encoded information based on a preset rule engine if the encoded information passes the validity verification, so as to obtain the corresponding candidate data source. The second processing module is used to perform a status evaluation on the candidate data sources to obtain a status evaluation result, and to filter out the target data source from the candidate data sources based on the status evaluation result. The first acquisition module is used to acquire the query statement corresponding to the data query request; The third processing module is used to send the query statement to the target data source based on a preset connection pool, so as to perform query processing on the query statement through the target data source to obtain the corresponding query results; The return module is used to return the query results to the client.

[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described data query method.

[0008] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described data query method.

[0009] In the above-described data query method, apparatus, computer equipment, and storage medium, the following steps are taken: First, a data query request is received from a client; the data query request is parsed to extract encoded information corresponding to the target type; then, the encoded information is validated; if the encoded information passes the validation, routing rule matching and data source selection are performed on the encoded information based on a preset rule engine to obtain corresponding candidate data sources; subsequently, a status evaluation is performed on the candidate data sources to obtain a status evaluation result, and the target data source is selected from the candidate data sources based on the status evaluation result; next, a query statement corresponding to the data query request is obtained; further, the query statement is sent to the target data source based on a preset connection pool to perform query processing on the query statement through the target data source to obtain the corresponding query result; finally, the query result is returned to the client. Based on the above automated processing flow, this application parses the received data query request to extract encoded information. After the encoded information passes validity verification, it uses a rule engine to perform routing rule matching and data source selection to obtain candidate data sources. Then, it evaluates the status of the candidate data sources and selects the target data source based on the evaluation results, automatically and intelligently achieving flexibility and automation in data source switching. Subsequently, a connection pool is used to send the query statement corresponding to the data query request to the target data source for query processing. Finally, the query results are returned to the client, thus achieving efficient and flexible processing of data query requests and effectively improving processing efficiency and flexibility. Attached Figure Description

[0010] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart of an embodiment of the data query method according to this application; Figure 3 This is a schematic diagram of the structure of one embodiment of the data query device according to this application; Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0013] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0014] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0015] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0016] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0017] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, and a desktop computer, etc.

[0018] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0019] It should be noted that the data query method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the data query device is generally set in the server / terminal device.

[0020] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0021] Continue to refer to Figure 2 A flowchart illustrating an embodiment of the data query method according to this application is shown. The order of steps in the flowchart can be changed, and some steps can be omitted, depending on different needs. The data query method provided in this application embodiment can be applied to any scenario requiring data querying. The data query method includes the following steps: Step S201: Receive a data query request from the client.

[0022] In this embodiment, the data query method runs on an electronic device (e.g., Figure 1 The server / terminal device shown can receive data query requests from clients via wired or wireless connections. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future known wireless connection methods. The implementing entity of this application is specifically a data query system, which can be simply referred to as the system.

[0023] The system can receive data query requests from clients via web service interfaces, message queues, or direct API calls. Specifically, the system can deploy a web server (such as Tomcat or Nginx), and the client can send query requests via the HTTP protocol. Upon receiving the request, the server parses it into a request object, containing request headers, request bodies, and other information. For example, for RESTful APIs, the request body might contain query parameters in JSON format. Alternatively, the client can encapsulate the query request into a message and send it to a message queue (such as RabbitMQ or Kafka). The system, acting as a consumer, retrieves the message from the message queue, parses the message content, and extracts the query-related information. Message queues enable asynchronous processing, improving system throughput and responsiveness. Alternatively, the client can directly call the system's provided API interface. The system receives the request through an API framework (such as Spring MVC or Django), parsing request parameters and context information. For example, in a microservice architecture, different services communicate via APIs, with query requests being passed from one service to another.

[0024] Step S202: Parse the data query request to extract the encoding information corresponding to the target type from the data query request.

[0025] In this embodiment, the received data query request can be parsed to extract the encoded information corresponding to the following target types: Tenant ID (tenantId): Extracts the tenant identifier from the request context information or specific parameters. For example, in a multi-tenant system, the request header may contain a tenant ID field, which the system obtains by parsing the request header.

[0026] Sharding key (e.g., user ID): The sharding key is determined based on business logic and extracted from request parameters or data. For example, in a user data sharding scenario, the query request may contain a user ID parameter, which the system directly extracts as the sharding key.

[0027] Business Type Identifier (businessType): Identifies the business type from the request. This can be obtained through the request's URL path, specific fields in the request parameters, etc. For example, different business operations may correspond to different URL paths, and the system determines the business type based on the path.

[0028] Region Code: If the business logic involves regional information, the region code is extracted from the request. For example, the request may contain a code for the user's location; the system obtains this code by parsing the request parameters.

[0029] Time Range: For queries with time relevance, extract the time range information from the request parameters. For example, when querying data within a specific time period, the request parameters may include start and end times.

[0030] In addition, the extracted encoded information can be further standardized, including format verification, type conversion, and encoding normalization, to ensure the consistency and effectiveness of the encoding.

[0031] Step S203: Verify the validity of the encoded information.

[0032] In this embodiment, the specific implementation process of validating the encoded information described above will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0033] Step S204: If the encoded information passes the validity verification, then the encoded information is processed by routing rule matching and data source selection based on the preset rule engine to obtain the corresponding candidate data source.

[0034] In this embodiment, the specific implementation process of performing routing rule matching and data source selection on the encoded information based on the preset rule engine to obtain the corresponding candidate data source will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0035] Step S205: Perform a status assessment on the candidate data sources to obtain a status assessment result, and select the target data source from the candidate data sources based on the status assessment result.

[0036] In this embodiment, the specific implementation process of obtaining the state evaluation result by performing state evaluation on the candidate data sources will be described in further detail in subsequent specific embodiments of this application, and will not be elaborated on here. Alternatively, the state evaluation results of all candidate data sources can be numerically compared, and the data source corresponding to the highest value of the state evaluation result can be selected as the final target data source.

[0037] Step S206: Obtain the query statement corresponding to the data query request.

[0038] In this embodiment, the specific implementation process of obtaining the query statement corresponding to the data query request will be described in further detail in subsequent specific embodiments, and will not be elaborated on here. Additionally, if the data query request carries a query statement (SQL statement), the corresponding query statement can be directly extracted from the data query request.

[0039] Step S207: Based on a preset connection pool, the query statement is sent to the target data source so that the query statement is processed by the target data source to obtain the corresponding query result.

[0040] In this embodiment, connection pool parameters, including minimum number of connections, maximum number of connections, and idle timeout, can be configured based on the characteristics of the selected target data source (such as database type, performance requirements, etc.). For example, for databases with high performance requirements, the maximum number of connections can be appropriately increased. At system startup, a certain number of connections are pre-established according to the configured parameters to eliminate the connection establishment overhead during the first query, thereby improving query response speed. For instance, if the minimum number of connections is set to 10, the connection pool will immediately create 10 connections to the data source at startup. This way, subsequent query requests do not need to go through the connection establishment overhead and can directly use existing connections, thus quickly responding to queries and improving query response speed.

[0041] When the system receives a data query request from a client, it can use a high-performance network framework (such as Netty) to handle the request. Netty uses a non-blocking I / O model and processes requests in an event-driven manner. When a new query request arrives, Netty triggers the corresponding event, and the system can quickly respond to and process these requests, greatly improving the system's concurrency capabilities and enabling it to handle a large number of query requests arriving simultaneously. Furthermore, it implements an intelligent caching strategy for the results of frequently accessed queries, reducing database access pressure.

[0042] Additionally, when a data query request is detected that needs processing, the system obtains an available connection from the connection pool corresponding to the target data source. The connection pool, based on its internal management mechanism, selects a suitable connection and assigns it to the data query request. If there are no idle connections in the current connection pool, but the maximum number of connections has not been reached, a new connection will be created; if the maximum number of connections has been reached, the data query request will be processed according to the connection pool's configuration strategy (such as queuing or returning an error). Furthermore, during the processing of data query requests and when connections are idle, the system periodically (through a scheduled task) sends a simple query (such as SELECT 1) to the data source. This is to prevent connections from being reclaimed by the database server due to prolonged inactivity. For example, setting a keep-alive query to send every 5 minutes ensures that the connection is always available, avoiding subsequent query failures due to connection failures and impacting system performance.

[0043] Furthermore, before executing the SQL query for the data query request, the system checks whether the query results are already cached. If an in-memory cache (such as Redis) or a local cache (such as Caffeine) is used, the system searches the cache based on the unique identifier of the query request (such as the hash value of the SQL statement plus parameters). If a valid result exists in the cache (not expired), the cached result is returned directly without accessing the database, greatly reducing the access pressure on the database and improving query response speed. If no valid result exists in the cache, the system uses a connection obtained from the connection pool to send the SQL statement (i.e., the query statement) to the target data source for execution. After receiving the SQL statement, the database server retrieves data from the database according to the execution plan and returns the result to the system.

[0044] Subsequently, the system will store the query results returned by the database in a cache (if result caching is configured) and set a reasonable cache expiration time. For example, for data that does not change frequently, a longer expiration time (such as 1 hour) can be set; for data with high real-time requirements, a shorter expiration time (such as 5 minutes) can be set. In this way, subsequent identical query requests can directly retrieve results from the cache, improving query efficiency.

[0045] Step S208: Return the query results to the client.

[0046] In this embodiment, the system will perform necessary processing (such as format conversion) on the obtained query results before returning them to the client, thus completing the entire data query request processing flow. Furthermore, after the data query request is processed, the system will dynamically adjust connection resources based on connection usage and idle time. If connection usage is low and idle time exceeds a set threshold (e.g., 10 minutes), the connection pool will reclaim some connections, releasing system resources and preventing excessive idle connections from consuming memory and other resources, thereby improving the overall resource utilization of the system.

[0047] This application first receives a data query request from a client; then parses the data query request to extract encoded information corresponding to the target type; next, it verifies the validity of the encoded information; if the encoded information passes the validity verification, it performs routing rule matching and data source selection processing on the encoded information based on a preset rule engine to obtain corresponding candidate data sources; then, it performs state evaluation on the candidate data sources to obtain state evaluation results, and filters the target data source from the candidate data sources based on the state evaluation results; subsequently, it obtains a query statement corresponding to the data query request; further, it sends the query statement to the target data source based on a preset connection pool, so that the query statement is processed by the target data source to obtain the corresponding query results; finally, it returns the query results to the client. Based on the above automated processing flow, this application parses the received data query request to extract encoded information, and after the encoded information passes the validity verification, it performs routing rule matching and data source selection processing on the encoded information based on the use of a rule engine to obtain candidate data sources, and then performs state evaluation on the candidate data sources and filters the target data source from the candidate data sources based on the obtained state evaluation results, thus automatically and intelligently realizing the flexibility and automation of data source switching. Subsequently, by using a connection pool, the query statement corresponding to the data query request is sent to the target data source for query processing. Finally, the query result is returned to the client, thereby achieving efficient and flexible processing of data query requests and effectively improving the processing efficiency and flexibility of data query requests.

[0048] In some alternative implementations, step S204 includes the following steps: The encoded information is processed by a preset rule engine to obtain a successfully matched specified routing rule.

[0049] In this embodiment, the system stores predefined routing rules in a rule base, which can take the form of a database, configuration file, or other similar format. Each rule contains information such as encoding type, matching conditions, and the corresponding routing algorithm. Then, when the system receives a data query request and extracts the encoding information, the rule engine performs a multi-dimensional matching retrieval process in the rule base based on the type of encoding information (such as tenant ID, business type, region, etc.). The retrieval process can be improved in efficiency using data structures such as indexes and hash tables to quickly locate the routing rules related to the encoding information.

[0050] Multi-dimensional matching includes: Rule combination definition: Defining composite routing rules in the rule base, specifying the combination methods of different dimension codes and the corresponding routing strategies. For example, defining a combination rule for tenant ID, business type, and region; when a data query request contains codes for all three dimensions, matching is performed according to the predefined combination rule. Matching logic implementation: The system matches the extracted multi-dimensional codes one by one. First, it checks whether the tenant ID matches the tenant range in the rule; then it checks whether the business type matches; and finally, it checks whether the region information is within the specified range. Only when the codes for all dimensions meet the matching conditions is the rule considered to have matched successfully.

[0051] The corresponding specified routing algorithm is determined based on the specified routing rules.

[0052] In this embodiment, the routing algorithm to be used can be determined based on the matching specified routing rules. Specifically, the corresponding routing algorithm can be identified by parsing the content from the specified routing rules, and this identified algorithm can then be selected as the desired specified routing algorithm.

[0053] Obtain the preset initial data source.

[0054] In this embodiment, the aforementioned initial data source may refer to all the data sources originally set in the database.

[0055] The initial data source is filtered based on the specified routing algorithm to obtain the corresponding first data source.

[0056] In this embodiment, the specified routing algorithm may include one or more algorithms such as hash routing, range routing, and weighted routing. Specifically, the process of filtering the initial data source based on the specified routing algorithm includes: 1) Hash routing includes: Hash function selection: Choosing a suitable hash function to calculate the encoded value. Common hash functions include MD5 and SHA-1. The hash function should have good distribution, uniformly mapping different encoded values ​​to the hash space. Modulo operation: Performing a modulo operation on the hash value obtained from the hash calculation, with the modulus being the total number of database instances configured in the system. Based on the modulo result, selecting the corresponding data source. For example, if the modulo result is 0, then data source 1 is selected; if the result is 1, data source 2 is selected, and so on.

[0057] 2) Range routing includes: Range definition: The rule base defines the range boundaries of the encoded values ​​for each data source, namely minRange and maxRange. These range boundaries can be set according to business needs, such as dividing by time range, numerical range, etc. Range matching: The system compares the extracted encoded values ​​with the range boundaries defined for each data source. If the encoded value falls within the range of a certain data source, i.e., minRange ≤ encodingValue ≤ maxRange, then that data source is selected.

[0058] 3) Weighted routing includes: Weight configuration: Processing capacity weights are configured for each data source in the rule base. Weight values ​​can be set based on factors such as the data source's hardware configuration and historical performance metrics. Weighted random selection: The system selects a data source using a weighted random algorithm based on the weight values ​​of each data source. In practice, a random number can be generated, and the corresponding data source can be selected based on which weight range the random number falls into.

[0059] The first data source is selected as the candidate data source.

[0060] Based on the above processing flow, this application uses a rule engine to perform routing rule matching on the encoded information to obtain successfully matched specified routing rules. Then, based on a specified routing algorithm determined by the specified routing rules, the initial data source is filtered, and the resulting first data source is used as a corresponding candidate data source, ensuring the adaptability and accuracy of the candidate data sources. This facilitates subsequent state evaluation based on the candidate data sources, and allows for the selection of the optimal target data source from the candidate data sources based on the state evaluation results. Consequently, the subsequent use of the target data source ensures that data query requests can be processed efficiently and reliably.

[0061] In some optional implementations of this embodiment, step S205, which involves performing a state evaluation on the candidate data source to obtain a state evaluation result, includes the following steps: A health assessment is performed on the second data source to obtain the corresponding health status information; wherein, the second data source is any one of all the candidate data sources.

[0062] In this embodiment, the above-mentioned health assessment process includes: checking the connection availability of candidate data sources in real time, determining whether the connection is normal by attempting to establish a connection or sending a simple test request, while monitoring response time and error rate.

[0063] For example, connection check: The system periodically sends connection requests to data source 1. If the connection is successfully established, it indicates that the connection status of data source 1 is normal. Response time monitoring: The system records the response time of each interaction with data source 1. After a period of statistical analysis, the average response time is calculated to be 100 milliseconds. Assuming a preset threshold of 200 milliseconds, the average response time does not exceed the threshold, indicating that the response speed of data source 1 is normal. Error rate statistics: The system counts the number of errors that occur during interactions with data source 1, calculating an error rate of 0.5%. Assuming a preset threshold of 5%, the error rate does not exceed the threshold, indicating that data source 1 has good stability.

[0064] The second data source is subjected to load assessment to obtain the corresponding load information.

[0065] In this embodiment, the above-mentioned load assessment process includes: obtaining the current load status of candidate data sources, including indicators such as the number of active connections, CPU utilization, and memory utilization, which can be obtained by interacting with the data source management interface or by using monitoring tools.

[0066] For example, active connection statistics: The system displays 10 active connections for data source 1 in real time. Resource monitoring: The system interacts with the monitoring interface of the server where data source 1 is located and obtains that the CPU utilization is 30% and the memory utilization is 40%.

[0067] The health status information and the load information are calculated and processed based on a preset load balancing algorithm to obtain the corresponding calculation results.

[0068] In this embodiment, the load balancing algorithm described above can specifically employ a weighted round-robin algorithm for load balancing, including: score = (1 - loadFactor) × weight × healthFactor. Here, loadFactor represents the load factor of the data source (0-1), weight represents the weight of the data source (set according to its hardware configuration and historical performance metrics), and healthFactor represents the health factor of the data source (0-1).

[0069] Additionally, load factor calculation includes a weighted average of active connections, CPU utilization, and memory utilization. If the weights of active connections, CPU utilization, and memory utilization are set to 0.4, 0.3, and 0.3 respectively, based on actual needs, then according to the previous example, loadFactor = 0.4 × (10 / maximum active connections) + 0.3 × 30% + 0.3 × 40% (assuming a maximum active connection count of 50). Assume the calculated loadFactor = 0.22. Similarly, the health factor can also be calculated by weighting connection availability (1 for successful connections, 0 for failed connections), response time, and error rate.

[0070] Then, the calculated load factor and health factor are substituted into the above load balancing algorithm for calculation and processing, and the calculation result is used as the status evaluation result of the second data source.

[0071] The calculation result is used as the status assessment result of the second data source.

[0072] Based on the above processing flow, this application performs health assessment and load assessment on the second data source respectively, and then calculates and processes the obtained health status information and load information based on the use of a load balancing algorithm. The calculation results are used as the status assessment results of the second data source, thereby enabling automatic and accurate completion of the status assessment processing of the second data source and ensuring the data accuracy of the generated status assessment results.

[0073] In some alternative implementations, step S206 includes the following steps: Invokes the default query builder.

[0074] In this embodiment, the selection of the query builder is not specifically limited and can be determined according to actual business needs. Many programming languages ​​and frameworks provide query builders, which can progressively construct SQL statements based on various conditions of a query request. For example, the query builder mentioned above can be the query builder tool provided by the Java MyBatis framework, which can define query conditions through XML mapping files or annotations and dynamically generate SQL statements based on the input parameters.

[0075] The query builder processes each condition in the data query request step by step to obtain the corresponding initial query statement.

[0076] In this embodiment, the conditions in the above data query request can be processed step by step based on the selected query builder to generate the corresponding initial query statement.

[0077] Get the preset statement optimization strategy.

[0078] In this embodiment, the above-mentioned statement optimization strategy includes the following: 1) For high-performance scenarios. In high-concurrency, large-data-volume business systems, such as product queries on e-commerce platforms and transaction record queries in financial transaction systems, query performance directly affects user experience and system stability. Optimizing SQL statements can significantly reduce query time and improve system throughput. For example, by rewriting complex subqueries into join queries and making reasonable use of indexes, full table scans can be avoided, greatly improving query efficiency.

[0079] 2) For complex query scenarios. When a query involves joins between multiple tables, complex conditional statements, aggregate functions, etc., the original SQL statement may have performance issues. For example, an SQL statement containing multiple levels of subqueries may lead to an unreasonable database engine execution plan. Query performance can be improved by optimizing subqueries to rewrite them into more efficient join queries or by using temporary tables.

[0080] 3) For resource-constrained scenarios. In server environments with limited resources, such as when memory and CPU resources are scarce, optimizing SQL statements can reduce the resource consumption of the database server. This can be achieved by caching execution plans to avoid redundant calculations and optimizing pagination queries to reduce data scanning, thereby reducing the load on the database server and improving the overall stability of the system.

[0081] The implementation process of caching execution plans includes: for a rewritten SQL statement, the system checks whether the execution plan for that statement has already been cached. If it exists in the cache, the cached execution plan is used directly, avoiding the need to re-analyze the SQL statement and generate an execution plan for each query, thereby saving significant computing resources and improving query efficiency.

[0082] In addition, optimizing the implementation of paginated queries includes: if the query request is a paginated query, the system will adopt appropriate optimization strategies. For example, for paginated queries with large amounts of data, cursor pagination can be used, recording the position of the last query so that the next query can directly retrieve data from that position, avoiding scanning a large amount of data for each query; or a delayed join method can be used, first quickly locating the record IDs that meet the conditions through the index, and then using these IDs to join and retrieve the complete data, reducing the amount of data scanned and improving the performance of paginated queries.

[0083] The initial query statement is optimized based on the aforementioned statement optimization strategy to obtain the corresponding target query statement.

[0084] In this embodiment, the optimization process of the initial query statement can be performed based on the strategy content of the above statement optimization strategy, and the resulting target query statement can be used as the corresponding query statement.

[0085] The target query statement is used as the query statement.

[0086] Based on the above processing flow, this application uses a query builder to progressively construct and process each condition in the data query request to obtain an initial query statement. Then, it optimizes the initial query statement using a statement optimization strategy and uses the obtained target query statement as the corresponding query statement. This improves the intelligence of the generated query statement, enabling subsequent data queries based on the query statement to improve the query response speed and processing capacity, reduce the pressure on the database, and improve the overall performance of the system.

[0087] In some alternative implementations, step S203 includes the following steps: Obtain the preset format validation rules and business validation rules.

[0088] In this embodiment, the format verification rules include: checking the encoding format, such as whether the length and character set of the tenant ID conform to the predefined specifications. The business verification rules include: verifying the validity of the encoding in business operations, such as querying the database or cache to confirm whether the tenant exists in the system and whether the sharding key is within a valid range. Specifically, this can be verified by querying relevant information in the database or cache.

[0089] The encoded information is format-verified based on the format verification rules.

[0090] In this embodiment, the format verification of the encoded information can be performed based on the rules of the aforementioned format verification rules. If the encoding format of the encoded information is found to conform to the predefined specifications, the encoded information is determined to have passed the format verification. If the encoding format of the encoded information is found to not conform to the predefined specifications, the encoded information is determined to have failed the format verification, and thus the encoded information is directly determined to have failed the validity verification.

[0091] If the encoded information passes the format verification, then the encoded information is subjected to business verification based on the business verification rules.

[0092] In this embodiment, the above-mentioned encoded information can be verified based on the rule content of the above-mentioned business verification rules. If the encoded information is found to be valid in terms of business, the encoded information is determined to have passed the business verification; otherwise, the encoded information is determined to have failed the business verification.

[0093] If the encoded information passes the service verification, it is determined that the encoded information has passed the validity verification; otherwise, it is determined that the encoded information has failed the validity verification.

[0094] In this embodiment, the encoded information is determined to have passed the validity verification only if it is detected that both the format verification and the business verification have passed simultaneously; otherwise, the encoded information is determined to have failed the validity verification.

[0095] Based on the above processing flow, this application uses a combination of format verification rules and business verification rules to perform format verification and business verification on the encoded information. Only when the encoded information is detected to have passed both format verification and business verification will it be determined that the encoded information has passed the validity verification; otherwise, the encoded information is determined to have failed the validity verification. This can effectively ensure that the extracted encoded information meets the system specifications and business requirements, and avoid problems in subsequent processing due to encoding errors.

[0096] In some optional implementations of this embodiment, after step S207, the electronic device may further perform the following steps: The third data source is monitored for health based on a preset multi-dimensional indicator monitoring strategy to obtain corresponding health indicator data; wherein, the third data source is any one of all the data sources.

[0097] In this embodiment, the aforementioned health monitoring refers to the processing of multi-dimensional indicators (connection availability, response latency, error rate, and resource utilization), including: Connection availability: Lightweight queries (e.g., SELECT 1) are sent to each data source periodically (e.g., every 5 seconds). If three consecutive queries fail, the connection is marked as interrupted. The timestamp and result of each detection are recorded to form a historical connection status log. Response latency: The P50 (median), P90 (90th percentile), and P99 (99th percentile) values ​​of query response times are statistically analyzed and aggregated at the minute level. Latency trends are stored and visualized using a time-series database (e.g., Prometheus). Error rate: The ratio of the number of erroneous queries to the total number of queries is statistically analyzed using a sliding window (e.g., within the last 5 minutes). Different types of errors (e.g., timeout, permission denial, syntax errors) are distinguished, and the error rates for each type are calculated separately. Resource utilization: CPU, memory, and disk I / O usage are collected using built-in database monitoring tools (e.g., SHOW STATUS for MySQL, pg_stat_activity for PostgreSQL) or third-party tools (e.g., Prometheus Node Exporter). Set a basic threshold (e.g., CPU utilization > 80%), and trigger an alarm when the threshold is exceeded.

[0098] The health indicator data is analyzed based on preset fault detection rules to generate corresponding fault analysis results.

[0099] In this embodiment, the fault detection rules include: Heartbeat Failure: If the heartbeat packets from the data source (e.g., sent every 10 seconds) fail to respond three times consecutively, it is determined to be a fault. Continuous Error Exceedance: If the number of erroneous queries from the same data source exceeds 50% of the total number of queries within one minute, a fault determination is triggered. Based on the above fault detection rules, fault analysis processing of the aforementioned health indicator data can be performed, generating fault analysis results indicating either storage fault or no fault.

[0100] The system also includes a fault warning mechanism, the specific implementation process of which includes: 1) Threshold determination. A dynamic threshold is set for each health indicator (e.g., based on the 95th percentile of historical data ± 10% fluctuation range). When an indicator exceeds the threshold for two consecutive detection cycles (e.g., 2 minutes), a primary alarm is triggered; exceeding the threshold for five consecutive cycles escalates to a critical alarm. 2) Trend analysis. A sliding window algorithm (e.g., window size = 10 detection cycles) is used to calculate the month-on-month change rate of the indicator (e.g., the delay P99 value increases by 30% compared to the previous window). If the trend continues to deteriorate (e.g., increases for three consecutive windows), an early warning is triggered and marked as "potential fault risk". 3) Risk classification. Low risk: A single data source's response delay briefly exceeds the limit, without affecting business operations. Medium risk: The error rate of multiple data sources increases, or the core data source connection is interrupted but a backup exists. High risk: The main data source is completely unavailable and there is no effective backup, which may lead to business interruption. Different risk levels correspond to different alarm channels (e.g., low risk is only notified by email, high risk is notified urgently via SMS + telephone).

[0101] If the fault analysis result indicates that a fault exists, then the third data source is marked as faulty.

[0102] In this embodiment, the fault marking process includes: immediately updating the data source status table, marking the faulty data source as "unavailable," and recording the fault start time. The monitoring system can also be notified to generate a fault event log.

[0103] Obtain a specified data query request related to the third data source.

[0104] In this embodiment, the specified data query request may refer to a query request that has not yet been processed within the third data source, and may also include a query request that is about to be assigned to the third data source.

[0105] The specified data query request is transferred to a preset backup data source based on a preset transfer strategy.

[0106] In this embodiment, the above-mentioned transfer strategy includes: Traffic transfer: Update the load balancer configuration according to routing rules (such as proximity priority, selecting the geographically closest available backup data source). If multiple backups exist (such as primary → backup → offline cold backup), switch according to priority level to avoid concentrated impact on backup sources. Fault recovery: After the faulty data source is restored, perform a health check (such as connection test + simple query verification). Verify the consistency of primary and backup data through incremental log comparison (such as parsing Binlog or WAL logs), and automatically switch back traffic after confirming that there are no errors.

[0107] Specifically, the specified data query request can be transferred to the corresponding backup data source based on the policy content of the above transfer strategy.

[0108] In addition, the system provides data consistency assurance for traffic transfer. The specific implementation process includes: 1) Read / write separation. During failover, all write operations are directed to the primary data source (or the only available data source), while read operations are distributed to backup sources. Read and write traffic are forcibly separated through middleware (such as MySQL Router, ProxySQL) or application-layer routing rules to strictly distinguish between read and write operations and ensure data consistency. 2) Transaction integrity. Two-phase commit (2PC): The coordinator (such as the application server) sends a "ready" command to all participating data sources and commits after all confirmations. If any data source fails, the coordinator sends a "rollback" command to ensure that all data sources are in the same state. Compensation transaction mechanism: For scenarios where 2PC cannot be used (such as cross-heterogeneous databases), transaction operation logs are recorded, and after failure, retrying or reversing operations is performed through compensation tasks (such as scheduled tasks). 3) Data verification and synchronization. Incremental log parsing: The change logs of the primary data source (such as MySQL's Binlog, PostgreSQL's WAL) are captured in real time using parsing tools (such as Canal, Debezium). The parsed data is synchronized to the backup source via a message queue (such as Kafka) to ensure eventual consistency. Regular full verification is performed: a full data comparison is executed daily during off-peak hours (e.g., by verifying table data using MD5) to correct potential inconsistencies. Data consistency assurance guarantees the accuracy and integrity of data during fault handling.

[0109] Based on the above processing flow, this application performs fault analysis on health indicator data using fault detection rules. When a fault is detected, the third data source is automatically marked as faulty, and the specified data query request is intelligently transferred to a preset backup data source based on a transfer strategy. In this way, this application can promptly detect data source faults through real-time health monitoring, and fault warnings and automatic transfer can quickly transfer query requests to a backup data source when a fault occurs, minimizing the impact on business operations.

[0110] In some optional implementations of this embodiment, after step S207, the electronic device may further perform the following steps: During the processing of the data query request, the system's performance metrics data are collected in real time.

[0111] In this embodiment, multi-dimensional performance metrics can be collected and processed to obtain the following: query response time (P50, P90, P99 percentile response time); throughput (query requests per second (QPS) and data processing volume); connection pool status (number of active connections, waiting queue length, etc.); data source load (CPU, memory, and I / O utilization of each data source); routing success rate (proportion of successfully routed requests out of total requests); and failover frequency (the key system performance metric corresponding to the frequency at which the system automatically performs failover).

[0112] The performance index data is analyzed based on a preset performance analysis engine to generate corresponding performance analysis results.

[0113] In this embodiment, the aforementioned performance analysis engine is a pre-built automated engine with performance analysis capabilities. The process of analyzing the performance indicator data based on the performance analysis engine includes: 1) Anomaly detection. Sudden rise and fall detection: Using a sliding window algorithm (e.g., window size = 5 minutes), the month-on-month change rate of the indicator is calculated. If the P99 latency suddenly increases by more than 30%, it is marked as an anomaly. Seasonal decomposition (STL algorithm) is combined to remove interference from periodic fluctuations (e.g., daily peak periods). Periodic fluctuation identification: Fourier transform is used to analyze the periodic components of the time series to identify fluctuations at fixed frequencies (e.g., query peaks once per hour). Periodic fluctuations are separated from sudden anomalies. 2) Root cause analysis. Correlation analysis: The Pearson correlation coefficient between performance indicators (e.g., latency) and potential factors (e.g., specific SQL, data source CPU utilization) is calculated, and highly correlated combinations are screened. Example: If the correlation coefficient between the execution time of a certain SQL and the disk I / O latency of data source A is >0.8, it is determined to be a strong correlation. Time Series Analysis: Use Granger causality tests to verify causal relationships between indicators (e.g., whether a surge in connections to data source B leads to an increase in overall latency). 3) Load Forecasting. Time Series Model: Use ARIMA or Prophet models to fit historical QPS, latency, and other indicators to predict load trends for the next 24 hours. Adjust the forecast results by incorporating business events (e.g., manually marking peak periods).

[0114] Based on the performance analysis results, corresponding query optimization suggestions are generated.

[0115] In this embodiment, targeted query optimization suggestions can be generated by processing the generated performance analysis results. Specifically, query optimization suggestions include: Routing strategy optimization: Based on historical load data, a dynamic weight adjustment scheme is recommended (e.g., "Data source A handles 70% of read requests, data source B handles 30%"). If a significant difference in latency is found due to proximity routing strategies, it is recommended to switch to a load balancing priority strategy. Connection pool tuning: Analyze connection pool usage patterns (e.g., peak waiting queue length) and recommend adjusting the maxPoolSize or idleTimeout parameters. Example: If the number of active connections is consistently close to the maximum value, it is recommended to expand the connection pool. SQL performance optimization: Identify slow queries (e.g., SQL with execution time > P99 threshold) and analyze whether indexes are missing based on the execution plan. Recommended field combinations for index creation (e.g., high-frequency filtering fields in WHERE conditions). Resource expansion suggestions: When the predicted load exceeds 80% of the current resource capacity, a expansion warning is generated 7 days in advance, and the expansion scale is recommended (e.g., "add 2 database instances").

[0116] The query optimization suggestions are then output.

[0117] In this embodiment, the generated query optimization suggestions can be sent to the relevant system operation and maintenance personnel to provide corresponding directions and measures for improving system performance.

[0118] In addition, the system also has a mechanism for verifying the effectiveness of optimization measures after they are implemented. The specific implementation process includes: 1) A / B testing framework. Configuration isolation: Distribute traffic to the experimental group (new configuration) and the control group (old configuration) according to rules (such as user ID hash) to ensure a balanced distribution of traffic between the two groups. Support multi-dimensional traffic splitting (such as by business type, region). Metric comparison: Calculate the difference in P99 latency and error rate between the experimental group and the control group in real time. If the experimental group is significantly better than the control group (such as a 10% reduction in latency and a p-value < 0.05), the optimization is deemed effective. 2) Performance benchmark testing. Standardized test suite: Predefined test scenarios (such as "100 concurrent read requests + 50 concurrent write requests") are used to simulate real load using tools (such as JMeter). Tests are executed regularly (such as once a week) to generate benchmark reports containing throughput and latency. Historical comparison: Compare the current benchmark test results with historical data to identify performance degradation (such as a 15% decrease in QPS). 3) Continuous monitoring closed loop. Process Automation: When the performance analysis engine detects anomalies, it automatically triggers the generation of optimization suggestions, which are then pushed to the operations team through an approval process. After the optimization measures are implemented, the effects are verified using an A / B testing framework. If the targets are met, the configuration is solidified; otherwise, it is rolled back and re-analyzed. Knowledge Accumulation: The root causes, measures, and effects of each optimization are recorded in a knowledge base for rapid diagnosis of similar issues in the future.

[0119] Based on the above processing flow, this application provides comprehensive system performance data through multi-dimensional performance indicator collection. Performance analysis of the obtained performance indicator data using a performance analysis engine can promptly identify performance problems and pinpoint root causes. Furthermore, the performance analysis results generate corresponding query optimization suggestions, providing specific directions and measures for improving system performance. Thus, the subsequent use of query optimization suggestions is beneficial for continuously optimizing system performance, ensuring the system can operate efficiently and stably.

[0120] In some optional implementations of this embodiment, the system also has dynamic configuration management and hot update functions, and the specific implementation process includes: 1. Configuration Center Integration. Configuration Monitoring: The system monitors route configuration change events through a configuration center (such as Nacos or Apollo). Configuration Pulling: Periodically pulls the latest route configuration information from the configuration center, including data source definitions, routing rules, load balancing strategies, etc.

[0121] 2. Configuration Hot Update Handling. Change Detection: Detects the type and scope of configuration changes, including adding data sources, modifying routing rules, adjusting weights, etc. Configuration Validation: Validates the validity of new configurations to ensure correctness and consistency. Configuration Application: Applies the validated new configurations to the runtime environment: Routing Rule Update: Updates the routing rule table in memory; the new rules take effect immediately. Dynamic Data Source Registration: Implements dynamic registration and discovery of data sources through Redis's distributed management mechanism. Data Source Node Initialization Registration: Newly added data source nodes register with the configuration center using a custom encryption protocol. Connection Establishment: Data source nodes establish a connection with the configuration center through a pre-configured secure channel. Registration Request: Nodes send registration requests containing metadata such as data source identifiers and connection information. Route ID Assignment: The configuration center assigns a unique route ID to each data source to ensure uniqueness in the distributed environment. Encrypted Response Return: Transmits a response message containing the route ID to the data source node through a secure encrypted channel.

[0122] 3. Heartbeat Mechanism Maintenance. Heartbeat Sending: Data source nodes periodically send heartbeat packets to the configuration center to indicate that they are still active. Status Update: The configuration center updates the activity status of data source nodes based on the heartbeat information; the routing ID remains valid while the node is active. Failure Handling: If a data source node fails to send a heartbeat within a specified time, the node is marked as unavailable, and its query requests are rerouted to a backup data source.

[0123] 4. Communication Security Guarantee. Encrypted Transmission: AES-256 symmetric encryption is used to manage communication between the data source nodes and the configuration center, ensuring the security of transmitted data. Data Integrity: HMAC-SHA256 algorithm is used to verify the integrity of each configuration change request, preventing configuration tampering. Asynchronous Communication Mechanism: A non-blocking asynchronous communication mode is adopted to improve the efficiency of configuration updates in high-concurrency scenarios.

[0124] 5. Configuration Conflict Resolution. Distributed Lock Mechanism: Distributed locks (such as the Redlock algorithm) are used during configuration updates to ensure atomicity and consistency. Automatic Retry Strategy: When a configuration conflict is detected, an exponential backoff retry mechanism is triggered, gradually increasing the retry interval until configuration synchronization is successful. Version Control Mechanism: Configuration version control is implemented, ensuring that all nodes use the latest valid configuration by comparing version numbers.

[0125] In some optional implementations of this embodiment, the system also includes a multi-scenario adaptation and extension mechanism, the specific implementation process of which includes: 1. Scenario Adaptation Layer. Multi-tenancy adaptation provides tenant isolation level configuration, supporting both soft sharing and hard isolation modes, including: Soft Sharing Mode: Logically isolates data at the application layer using the tenant ID field, adds a tenant_id field to the database table structure, and automatically adds filtering conditions during queries (e.g., WHERE tenant_id = ?). Implements tenant-level access control, ensuring users can only access their own tenant's data (e.g., intercepting unauthorized cross-tenant queries through middleware). Hard Isolation Mode: Assigns an independent database instance or schema to each tenant, dynamically binding the mapping relationship between tenant ID and data source through configuration files. At the routing layer, routes directly to the corresponding data source based on the tenant ID, avoiding cross-instance queries.

[0126] Database sharding and table partitioning adaptation supports multiple sharding strategies, including: Consistent hash sharding: Using virtual node technology (e.g., 1000 virtual nodes) to evenly distribute data across multiple shards, only a small amount of data corresponding to the virtual nodes needs to be migrated when adding a new node. The hash value of the sharding key is calculated and mapped to the virtual node during writes, and the shard is located through the same logic during queries. Range sharding: Data is partitioned by time range (e.g., by month) or numerical range (e.g., by user ID interval), with predefined sharding boundary rules. During queries, the system directly routes to the corresponding shard based on conditions (e.g., create_time BETWEEN '2024-01-01' AND '2024-01-31'). Custom sharding: Provides a configuration interface, allowing users to define sharding logic (e.g., "sharding by region: Beijing -> Database 1, Shanghai -> Database 2"), and dynamically resolves the sharding key through scripts or a rule engine.

[0127] Read / Write Splitting Adaptation: Master-Slave Replication Latency Awareness: Periodically execute `SHOW SLAVE STATUS` (MySQL) or equivalent commands on the slave database to obtain replication latency. If the latency exceeds a threshold (e.g., 5 seconds), temporarily route read requests to the master database. Implement a latency compensation mechanism (e.g., read the master database's Binlog position to confirm data synchronization). Read / Write Routing Strategies: Tag Routing: Tag data sources (e.g., "Financial Business Dedicated Database"), specifying the tag range for queries through annotations or configuration. Proximity Routing: Based on client IP or region information, prioritize slave databases in the same region (e.g., route requests from the East China region to the East China slave database). Cloud Environment Adaptation: Hybrid Cloud Routing: Maintain a health status table for multi-region data sources, dynamically selecting the optimal data source based on latency detection results (e.g., through Ping tests). Implement cross-region traffic scheduling (e.g., if the master database is in AWS Tokyo and the slave database is in Alibaba Cloud Shanghai, prioritize reading from the Shanghai region). Elastic Scaling: Integrate cloud platform APIs (e.g., AWS Auto Scaling), automatically create new instances and add them to the data source pool when the load exceeds a threshold, confirming instance readiness through heartbeat detection.

[0128] 2. Extended Interface Design. Routing Strategy Extension: SPI Interface: Define a Router interface (e.g., RouteResultroute(Request request)). Users implement this interface and package it as a JAR file in the extension directory. The system dynamically loads it via ServiceLoader. Machine Learning Routing: Integrate lightweight predictive models (e.g., XGBoost) to predict the future load of each data source based on historical QPS and latency data, prioritizing the node with the lowest predicted load. Geographic Information Routing: Combine IP location libraries (e.g., MaxMind GeoIP) to obtain the client's geographic location, prioritizing the data source with the closest physical distance. Load Balancing Extension: Supports custom load balancing algorithm implementation. Dynamic Weight Adjustment: Calculates the real-time load score of each data source every minute (e.g., score = 0.5 * latency + 0.3 * CPU + 0.2 * number of connections), dynamically adjusting routing weights. Multi-Dimensional Scoring: Calculates a health score by combining latency (sliding window average), throughput (QPS), and error rate (percentage of failed requests), eliminating data sources with scores below the threshold. Fault Detection Extension: Supports custom fault detection mechanisms. Business semantic detection: Allows users to configure custom SQL detection (e.g., SELECT 1 FROM dual), and marks the data source as unavailable if it times out or fails. Connection pool status detection: Monitors connection pool leaks (e.g., active connections not released for a long time) and connection acquisition timeout events, triggering alarms or circuit breakers.

[0129] 3. Plugin-based architecture. Hot-plugging mechanism for dynamic loading: Plugins are isolated using an OSGi container or a custom class loader. Loading is automatically triggered when the plugin JAR file is placed in a specified directory (e.g., by listening for directory file creation events). Version compatibility: The plugin manifest file (MANIFEST.MF) declares the range of middleware versions required (e.g., Require-Middleware: 1.2.0-2.0.0), and compatibility is checked at startup. Sandbox isolation: Each plugin is allocated an independent thread pool and resource quota, and plugin permissions are restricted through SecurityManager (e.g., disabling file read / write).

[0130] 4. Configure Template Library. Pre-set Templates for Various Scenarios: Built-in Templates: Provides configuration templates for scenarios such as e-commerce (high-concurrency reads), finance (strong consistency), and IoT (massive writes), including sharding strategies, routing rules, connection pool parameters, etc. Dynamic Generation: Recommends optimized templates based on system diagnostic results (e.g., "High proportion of cross-database JOINs detected") (e.g., "Global table synchronization recommended"). Configuration Validation: Syntax Check: Parses the validity of fields in configuration files (e.g., YAML) (e.g., sharding keys cannot be null). Dependency Validation: Checks the matching between the sharding strategy and the number of data sources (e.g., range sharding requires at least 2 databases).

[0131] The multi-scenario adaptation and extension mechanism enables the system to adapt to different business scenarios and needs. The scenario adaptation layer provides various common business scenario adaptation solutions, such as multi-tenancy, database sharding, read-write separation, and cloud environment adaptation. The extension interface design provides a flexible way to further extend the system. Through the SPI interface and plug-in extension, third-party developers can easily add new functions to the system, improving its scalability and flexibility.

[0132] In some alternative implementations, the user information obtained is subject to user consent and complies with relevant laws and policies.

[0133] Furthermore, any software tools or components not belonging to our company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0134] Furthermore, the query method based on rule encoding and dynamically switching data sources at runtime proposed in this application has the following advantages compared to existing technologies: 1. Social Impact: This application significantly improves the flexibility and reliability of data querying in multi-data source environments through runtime dynamic routing mechanisms and unified data source management. This not only enables enterprises to quickly respond to complex business scenario changes (such as multi-tenant isolation, database sharding, read-write separation, etc.), but also enhances the overall stability and data service capabilities of the system, providing a more efficient and reliable data support foundation for digital transformation and business innovation across various industries.

[0135] 2. Economic Benefits: This application significantly reduces operating costs and development investment for enterprises in multi-data source environments by simplifying data source configuration processes, reducing system restart frequency, and lowering maintenance complexity. The system's efficient routing and resource scheduling capabilities improve database resource utilization, avoid resource idleness and waste, and its failover and load balancing mechanisms reduce system downtime and ensure business continuity, thereby bringing significant economic benefits and competitive advantages to enterprises.

[0136] 3. Technical effects: This application achieves efficient, secure and flexible scheduling of the multi-data source query system during runtime by introducing a coding-based dynamic routing mechanism and a unified routing management layer.

[0137] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0139] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0140] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a data query device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0141] like Figure 3 As shown, the data query device 300 described in this embodiment includes: a receiving module 301, an extraction module 302, a verification module 303, a first processing module 304, a second processing module 305, a first acquisition module 306, a third processing module 307, and a return module 308. Wherein: The receiving module 301 is used to receive data query requests sent by the client; Extraction module 302 is used to parse the data query request in order to extract the encoding information corresponding to the target type from the data query request; Verification module 303 is used to verify the validity of the encoded information; The first processing module 304 is used to perform routing rule matching and data source selection processing on the encoded information based on a preset rule engine if the encoded information passes the validity verification, so as to obtain the corresponding candidate data source. The second processing module 305 is used to perform a status evaluation on the candidate data sources to obtain a status evaluation result, and to filter out the target data source from the candidate data sources based on the status evaluation result. The first acquisition module 306 is used to acquire the query statement corresponding to the data query request; The third processing module 307 is used to send the query statement to the target data source based on a preset connection pool, so as to perform query processing on the query statement through the target data source to obtain the corresponding query results; The return module 308 is used to return the query results to the client.

[0142] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the data query method in the aforementioned embodiments, and will not be repeated here.

[0143] In some optional implementations of this embodiment, the first processing module 304 includes: The matching submodule is used to perform routing rule matching processing on the encoded information based on a preset rule engine to obtain the specified routing rule that is successfully matched. The first determining submodule is used to determine the corresponding specified routing algorithm based on the specified routing rule; The first acquisition submodule is used to acquire the preset initial data source; The filtering submodule is used to perform data source filtering processing on the initial data source based on the specified routing algorithm to obtain the corresponding first data source. The second determining submodule is used to select the first data source as the candidate data source.

[0144] In some optional implementations of this embodiment, the second processing module 305 includes: The first assessment submodule is used to perform health assessment processing on the second data source to obtain the corresponding health status information; wherein, the second data source is any one of all the candidate data sources; The second evaluation submodule is used to perform load evaluation processing on the second data source to obtain the corresponding load information. The calculation submodule is used to perform calculations on the health status information and the load status information based on a preset load balancing algorithm to obtain the corresponding calculation results. The third determining submodule is used to use the calculation result as the state evaluation result of the second data source.

[0145] In some optional implementations of this embodiment, the first acquisition module 306 includes: Call the submodule to invoke the preset query builder; The construction submodule is used to progressively construct and process each condition in the data query request based on the query builder to obtain the corresponding initial query statement; The second acquisition submodule is used to acquire preset statement optimization strategies; The optimization submodule is used to optimize the initial query statement based on the statement optimization strategy to obtain the corresponding target query statement; The fourth determination submodule is used to use the target query statement as the query statement.

[0146] In some optional implementations of this embodiment, the verification module 303 includes: The third acquisition submodule is used to acquire preset format validation rules and business validation rules; The first verification submodule is used to perform format verification on the encoded information based on the format verification rules; The second verification submodule is used to perform business verification on the encoded information based on the business verification rules if the encoded information passes the format verification. The determination submodule is used to determine that the encoded information has passed the validity verification if the encoded information passes the business verification, and otherwise determine that the encoded information has failed the validity verification.

[0147] In some optional implementations of this embodiment, the data query device further includes: The monitoring module is used to perform health monitoring on a third data source based on a preset multi-dimensional indicator monitoring strategy to obtain corresponding health indicator data; wherein, the third data source is any one of all the data sources; The first analysis module is used to perform fault analysis on the health indicator data based on preset fault detection rules and generate corresponding fault analysis results. A marking module is used to mark the third data source as faulty if the fault analysis result indicates that a fault exists. The second acquisition module is used to acquire a specified data query request related to the third data source; The transfer module is used to transfer the specified data query request to a preset backup data source based on a preset transfer strategy.

[0148] In some optional implementations of this embodiment, the data query device further includes: The data acquisition module is used to collect system performance index data in real time during the processing of the data query request. The second analysis module is used to analyze the performance index data based on a preset performance analysis engine and generate corresponding performance analysis results. The generation module is used to generate corresponding query optimization suggestions based on the performance analysis results. The output module is used to process the query optimization suggestions.

[0149] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0150] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0151] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0152] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for data querying methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0153] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions for the data query method.

[0154] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0155] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the data query method described above.

[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0157] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A data query method, characterized in that, Includes the following steps: Receive data query requests from clients; The data query request is parsed to extract the encoding information corresponding to the target type from the data query request; The validity of the encoded information is verified. If the encoded information passes the validity verification, then the encoded information is processed by routing rule matching and data source selection based on the preset rule engine to obtain the corresponding candidate data source; A status assessment is performed on the candidate data sources to obtain a status assessment result, and a target data source is selected from the candidate data sources based on the status assessment result; Obtain the query statement corresponding to the data query request; The query statement is sent to the target data source based on a preset connection pool, so that the query statement is processed by the target data source to obtain the corresponding query results; The query results are returned to the client.

2. The data query method according to claim 1, characterized in that, The step of performing routing rule matching and data source selection on the encoded information based on a preset rule engine to obtain corresponding candidate data sources specifically includes: The encoded information is processed by a preset rule engine to obtain a successfully matched specified routing rule. The corresponding specified routing algorithm is determined based on the specified routing rules; Obtain the preset initial data source; Based on the specified routing algorithm, the initial data source is filtered to obtain the corresponding first data source; The first data source is selected as the candidate data source.

3. The data query method according to claim 1, characterized in that, The step of performing a state assessment on the candidate data source to obtain a state assessment result specifically includes: A health assessment is performed on the second data source to obtain the corresponding health status information; wherein, the second data source is any one of all the candidate data sources. Perform load assessment on the second data source to obtain the corresponding load information; The health status information and the load status information are calculated and processed based on a preset load balancing algorithm to obtain the corresponding calculation results; The calculation result is used as the status assessment result of the second data source.

4. The data query method according to claim 1, characterized in that, The step of obtaining the query statement corresponding to the data query request specifically includes: Invoke the default query builder; Based on the query builder, the various conditions in the data query request are constructed step by step to obtain the corresponding initial query statement; Get the preset statement optimization strategy; The initial query statement is optimized based on the aforementioned statement optimization strategy to obtain the corresponding target query statement; The target query statement is used as the query statement.

5. The data query method according to claim 1, characterized in that, The step of validating the encoded information specifically includes: Obtain the preset format validation rules and business validation rules; The format of the encoded information is verified based on the format verification rules. If the encoded information passes the format verification, then the encoded information is subjected to business verification based on the business verification rules; If the encoded information passes the service verification, it is determined that the encoded information has passed the validity verification; otherwise, it is determined that the encoded information has failed the validity verification.

6. The data query method according to claim 1, characterized in that, After the step of sending the query statement to the target data source based on a preset connection pool, so as to perform query processing on the query statement through the target data source to obtain the corresponding query results, the method further includes: Based on a preset multi-dimensional indicator monitoring strategy, health monitoring is performed on a third data source to obtain corresponding health indicator data; wherein, the third data source is any one of all the data sources. Based on preset fault detection rules, the health indicator data is analyzed for faults, and corresponding fault analysis results are generated. If the fault analysis result indicates that a fault exists, then the third data source is marked as faulty. Obtain a specified data query request related to the third data source; The specified data query request is transferred to a preset backup data source based on a preset transfer strategy.

7. The data query method according to claim 1, characterized in that, After the step of sending the query statement to the target data source based on a preset connection pool, so as to perform query processing on the query statement through the target data source to obtain the corresponding query results, the method further includes: During the processing of the data query request, the system's performance metrics data are collected in real time. The performance index data is analyzed based on a preset performance analysis engine to generate corresponding performance analysis results. Based on the performance analysis results, corresponding query optimization suggestions are generated; The query optimization suggestions are then output.

8. A data query device, characterized in that, include: The receiving module is used to receive data query requests sent by the client; The extraction module is used to parse the data query request in order to extract the encoding information corresponding to the target type from the data query request; The verification module is used to verify the validity of the encoded information; The first processing module is used to perform routing rule matching and data source selection processing on the encoded information based on a preset rule engine if the encoded information passes the validity verification, so as to obtain the corresponding candidate data source. The second processing module is used to perform a status evaluation on the candidate data sources to obtain a status evaluation result, and to filter out the target data source from the candidate data sources based on the status evaluation result. The first acquisition module is used to acquire the query statement corresponding to the data query request; The third processing module is used to send the query statement to the target data source based on a preset connection pool, so as to perform query processing on the query statement through the target data source to obtain the corresponding query results; The return module is used to return the query results to the client.

9. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data query method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data query method as described in any one of claims 1 to 7.

Images