Data processing method and device based on MyBatis framework

Abstract

The invention provides a data processing method and device based on a MyBatis framework, and the method comprises the steps: obtaining input data corresponding to a data query request in response to the data query request; according to a specified dynamic SQL label in the MyBatis framework, judging whether the input data comprises specified type data or not; if yes, dynamic splicing processing is carried out according to the input data and a pre-designed configuration file, and an initial SQL query statement is generated; according to conditional statements in the initial SQL query statements and a query plan determined according to the initial SQL query statements, performing optimization processing on the initial SQL query statements according to a preset optimization strategy, and determining target SQL query statements; and determining a corresponding target caching strategy according to the target SQL query statement, and obtaining target data from a corresponding position according to the corresponding target caching strategy. In this way, through dynamic SQL optimization and an intelligent caching strategy, the query efficiency and the query result accuracy can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and device based on a MyBatis framework. Background Art

[0002] With the rapid development of information technology and communication technology, the Internet of Vehicles (IoV), as an emerging intelligent transportation system, is gradually changing people's travel methods and traffic management modes. The IoV system realizes intelligent management and services of vehicles through interconnection and information sharing between vehicles, infrastructure and cloud platforms. In order to support these functions, the IoV system usually involves a large amount of real-time data query and caching.

[0003] In the prior art, static SQL technology is often used for real-time data query and cache processing. This method requires manual concatenation of strings to construct SQL statements, which is not only cumbersome but also prone to errors. Especially when dealing with complex query logic, concatenation errors may cause SQL statements to fail to execute or return incorrect results. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a data processing method and device based on the MyBatis framework, which can significantly reduce the number of database queries and improve query efficiency through dynamic SQL optimization and intelligent caching strategies, and can also have strong adaptability to complex business scenarios, thereby improving the accuracy of processing results.

[0005] The present application embodiment provides a data processing method based on the MyBatis framework, the data processing method comprising:

[0006] In response to a data query request, obtaining input data corresponding to the data query request;

[0007] According to the specified dynamic SQL tag in the MyBatis framework, determine whether the input data includes data of a specified type;

[0008] If included, dynamically concatenate the input data with a pre-designed configuration file to generate an initial SQL query statement;

[0009] According to the conditional statement in the initial SQL query statement and the query plan determined according to the initial SQL query statement, the initial SQL query statement is optimized according to a preset optimization strategy to determine a target SQL query statement;

[0010] According to the target SQL query statement, a corresponding target cache strategy is determined, and target data is acquired from a corresponding position according to the corresponding target cache strategy.

[0011] Optionally, the step of dynamically concatenating the input data with a pre-designed configuration file to generate an initial SQL query statement includes:

[0012] Obtaining a dynamic SQL query template written according to a dynamic tag in the configuration file;

[0013] The input data is concatenated with the dynamic SQL query template to determine an initial SQL query statement.

[0014] Optionally, optimizing the initial SQL query statement according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement according to a preset optimization strategy to determine a target SQL query statement includes:

[0015] Parsing the initial SQL query statement using a query tool provided by the database to be accessed to determine a first execution plan for the initial query;

[0016] Evaluate the query conditions in the initial SQL query statement according to the first execution plan to determine the performance impact of each query condition;

[0017] According to the performance optimization principle, the query conditions in the initial SQL query statement are optimized according to the performance impact of each query condition to obtain a candidate SQL query statement;

[0018] Using the second execution plan determined according to the candidate SQL query statement to evaluate the set traversal type determined according to the candidate SQL query statement, and selecting a corresponding set traversal optimization strategy according to the evaluation result;

[0019] The candidate SQL query statements are re-optimized according to the corresponding set traversal optimization strategy to obtain the target SQL query statement.

[0020] Optionally, when optimizing the query conditions in the initial SQL query statement according to the performance impact of each query condition, at least one of the following processing strategies is adopted:

[0021] Determine key query conditions based on the performance impact of the query conditions, and create an index for each key query condition statement;

[0022] Use the dynamic SQL feature of MyBatis to adjust the fields in the initial SQL query statement;

[0023] Determine the target search column of the data to be queried based on the business scenario.

[0024] Optionally, the set traversal optimization strategy includes at least one of the following:

[0025] Use the dynamic tags for collection traversal in the MyBatis framework to process candidate SQL query statements;

[0026] Use paging query method;

[0027] Creating indexes for key fields in the candidate SQL query statements;

[0028] Adjust the data type according to the amount of data returned.

[0029] Optionally, determining a corresponding target cache strategy according to the target SQL query statement includes:

[0030] Parsing the target SQL query statement to determine the type and business scenario of the target SQL query statement;

[0031] Determine whether to start the cache strategy according to the type and business scenario of the target SQL query statement;

[0032] If started, the target level of the corresponding cache is determined according to the target SQL query statement, and the target data is searched from the cache of the determined target level.

[0033] Optionally, the data processing method further includes:

[0034] Use the pre-stored data in the database to perform consistency check on the cached data, and update the cached data based on the correction results.

[0035] Optionally, the data processing method further includes:

[0036] When the cache expiration condition is reached, the data in the cache is deleted according to the preset deletion rules.

[0037] Optionally, the data processing method further includes:

[0038] Determine the cache level corresponding to the cached data based on the frequency of the data;

[0039] And the cache data is stored and processed according to the corresponding cache level.

[0040] The embodiment of the present application also provides a data processing device based on the MyBatis framework, the data processing device comprising:

[0041] An acquisition module, used to respond to a data query request and acquire input data corresponding to the data query request;

[0042] A judgment module is used to judge whether the input data includes data of a specified type according to a specified dynamic SQL tag in the MyBatis framework;

[0043] A splicing module, used to dynamically splice the input data with a pre-designed configuration file to generate an initial SQL query statement if the data includes a specified type;

[0044] A determination module, configured to optimize the initial SQL query statement according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement according to a preset optimization strategy, and determine a target SQL query statement;

[0045] The query module is used to determine a corresponding target cache strategy according to the target SQL query statement, and obtain target data from a corresponding position according to the corresponding target cache strategy.

[0046] Optionally, when the splicing module is used to dynamically splice the input data with a pre-designed configuration file to generate an initial SQL query statement, the splicing module is used to:

[0047] Obtaining a dynamic SQL query template written according to a dynamic tag in the configuration file;

[0048] The input data is concatenated with the dynamic SQL query template to determine an initial SQL query statement.

[0049] Optionally, when the determination module is used to optimize the initial SQL query statement according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement according to a preset optimization strategy to determine a target SQL query statement, the determination module is used to:

[0050] Parsing the initial SQL query statement using a query tool provided by the database to be accessed to determine a first execution plan for the initial query;

[0051] Evaluate the query conditions in the initial SQL query statement according to the first execution plan to determine the performance impact of each query condition;

[0052] According to the performance optimization principle, the query conditions in the initial SQL query statement are optimized according to the performance impact of each query condition to obtain a candidate SQL query statement;

[0053] Using the second execution plan determined according to the candidate SQL query statement to evaluate the set traversal type determined according to the candidate SQL query statement, and selecting a corresponding set traversal optimization strategy according to the evaluation result;

[0054] The candidate SQL query statements are re-optimized according to the corresponding set traversal optimization strategy to obtain the target SQL query statement.

[0055] Optionally, when optimizing the query conditions in the initial SQL query statement according to the performance impact of each query condition, at least one of the following processing strategies is adopted:

[0056] Determine key query conditions based on the performance impact of the query conditions, and create an index for each key query condition statement;

[0057] Use the dynamic SQL feature of MyBatis to adjust the fields in the initial SQL query statement;

[0058] Determine the target search column of the data to be queried based on the business scenario.

[0059] Optionally, the set traversal optimization strategy includes at least one of the following:

[0060] Use the dynamic tags for collection traversal in the MyBatis framework to process candidate SQL query statements;

[0061] Use paging query method;

[0062] Creating indexes for key fields in the candidate SQL query statements;

[0063] Adjust the data type according to the amount of data returned.

[0064] Optionally, when the query module is used to determine a corresponding target cache strategy according to the target SQL query statement, the query module is used to:

[0065] Parsing the target SQL query statement to determine the type and business scenario of the target SQL query statement;

[0066] Determine whether to start the cache strategy according to the type and business scenario of the target SQL query statement;

[0067] If started, the target level of the corresponding cache is determined according to the target SQL query statement, and the target data is searched from the cache of the determined target level.

[0068] Optionally, the data processing device further includes an updating module, wherein the updating module is configured to:

[0069] Use the pre-stored data in the database to perform consistency check on the cached data, and update the cached data based on the correction results.

[0070] Optionally, the data processing device further includes a failure module, wherein the failure module is used to:

[0071] When the cache expiration condition is reached, the data in the cache is deleted according to the preset deletion rules.

[0072] Optionally, the data processing device further includes a cache module, and the cache module is used to:

[0073] Determine the cache level corresponding to the cached data based on the frequency of the data;

[0074] And the cache data is stored and processed according to the corresponding cache level.

[0075] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the data processing method as described above are performed.

[0076] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data processing method as described above are executed.

[0077] The embodiment of the present application provides a data processing method and device based on the MyBatis framework. The method includes: in response to a data query request, obtaining input data corresponding to the data query request; judging whether the input data includes data of a specified type according to a specified dynamic SQL tag in the MyBatis framework; if included, dynamically splicing the input data with a pre-designed configuration file to generate an initial SQL query statement; optimizing the initial SQL query statement according to a preset optimization strategy according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement, and determining a target SQL query statement; determining a corresponding target cache strategy according to the target SQL query statement, and obtaining target data from a corresponding position according to the corresponding target cache strategy.

[0078] In this way, this application significantly reduces the number of database queries, improves query efficiency, reduces unnecessary database access and calculations, and reduces system resource consumption through dynamic SQL optimization and intelligent caching strategies. In addition, through multi-layer caching mechanisms and intelligent caching decisions, the system's adaptability to complex business scenarios is enhanced, and data access speed can be accelerated to improve user experience.

[0079] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0081] Figure 1 A flowchart of a data processing method based on the MyBatis framework provided in an embodiment of the present application;

[0082] Figure 2 One of the structural schematic diagrams of a data processing device based on the MyBatis framework provided in an embodiment of the present application;

[0083] Figure 3 The second structural diagram of a data processing device based on the MyBatis framework provided in an embodiment of the present application;

[0084] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0086] With the rapid development of information technology and communication technology, the Internet of Vehicles (IoV), as an emerging intelligent transportation system, is gradually changing people's travel methods and traffic management modes. The IoV system realizes intelligent management and services of vehicles through interconnection and information sharing between vehicles, infrastructure and cloud platforms. In order to support these functions, the IoV system usually involves a large amount of real-time data query and caching.

[0087] In the prior art, static SQL technology is mostly used for real-time data query and cache processing. This method requires manual concatenation of strings to construct SQL statements, which is not only cumbersome but also prone to errors. Especially when dealing with complex query logic, concatenation errors may cause SQL statements to fail to execute or return incorrect results. And unoptimized SQL statements may cause performance bottlenecks. For example, full table scans, unnecessary table joins, missing indexes, or ineffective use of indexes will all reduce query performance. In high-concurrency scenarios, if SQL statements are not optimized, the database load may be too high, thereby affecting the performance of the entire application. In addition, when the data in the database changes, the data in the cache may not be updated in a timely manner, affecting data consistency.

[0088] Based on this, the embodiment of the present application provides a data processing method and device based on the MyBatis framework, which significantly reduces the number of database queries and improves query efficiency through dynamic SQL optimization and intelligent caching strategies, and also has strong adaptability to complex business scenarios, thereby improving the accuracy of processing results, and also ensuring data consistency.

[0089] See also Figure 1 , Figure 1 The following is a flow chart of a data processing method based on the MyBatis framework provided in the embodiment of the present application. Figure 1 As shown in , the data processing method provided in the embodiment of the present application includes:

[0090] S101. In response to a data query request, obtain input data corresponding to the data query request.

[0091] S102: According to the specified dynamic SQL tag in the MyBatis framework, it is determined whether the input data includes data of a specified type.

[0092] S103: If included, dynamically concatenate the input data with a pre-designed configuration file to generate an initial SQL query statement.

[0093] S104, optimizing the initial SQL query statement according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement according to a preset optimization strategy to determine a target SQL query statement;

[0094] S105 determines a corresponding target cache strategy according to the target SQL query statement, and obtains target data from a corresponding position according to the corresponding target cache strategy.

[0095] With respect to step S101, in this step, after receiving a data query request, the data query request is parsed to obtain corresponding input data.

[0096] The input data may be data of a parameter object containing query conditions that is passed when a user calls a method in the Mapper interface.

[0097] For step S102, for example, this step can be performed according to the MyBatis framework. <if>Tag, to determine whether the input data includes the status parameter.

[0098] If included, step S103 is executed; if not included, data processing (eg, query) operation is performed according to the input data according to conventional default logic.

[0099] With respect to step S103, the configuration file is a predefined Mapper XML file.

[0100] In an implementation manner provided in the present application, the dynamic splicing processing is performed based on the input data and the pre-designed configuration file to generate an initial SQL query statement, including:

[0101] S1031. Obtain a dynamic SQL query template written according to the dynamic tag in the configuration file.

[0102] S1032: Concatenate the input data with the dynamic SQL query template to determine an initial SQL query statement.

[0103] With respect to step S1031, after receiving the query request, MyBatis will parse the dynamic SQL in the Mapper XML file and obtain the dynamic SQL query template.

[0104] It should be noted that in MyBatis, dynamic SQL is usually defined in the Mapper XML file. You need to write SQL statements in this file and use the dynamic SQL tags provided by MyBatis to splice SQL according to conditions.

[0105] With respect to step S1032, the input data is filled into the corresponding position of the dynamic SQL template to complete the dynamic splicing process and obtain the initial SQL query statement.

[0106] With respect to step S104, the preset optimization strategies include a conditional statement optimization strategy and a set traversal optimization strategy.

[0107] In an implementation manner provided in the present application, the initial SQL query statement is optimized according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement according to a preset optimization strategy to determine a target SQL query statement, including:

[0108] S1041: parse the initial SQL query statement using a query tool provided by the database to be accessed to determine a first execution plan for the initial query.

[0109] S1042: Evaluate the query conditions in the initial SQL query statement according to the first execution plan to determine the performance impact of each query condition.

[0110] S1043. According to the performance optimization principle, the query conditions in the initial SQL query statement are optimized according to the performance impact of each query condition to obtain a candidate SQL query statement.

[0111] S1044: Use the second execution plan determined according to the candidate SQL query statement to evaluate the set traversal type determined according to the candidate SQL query statement, and select a corresponding set traversal optimization strategy according to the evaluation result.

[0112] S1045. Re-optimize the candidate SQL query statement according to the corresponding set traversal optimization strategy to obtain a target SQL query statement.

[0113] With respect to step S1041 , the query tool may be EXPLAIN or a similar tool.

[0114] By viewing the execution plan, you can understand how the query is executed by the database system and identify performance bottlenecks. In this way, according to the results of the execution plan, adjust the query logic to optimize performance. For example, if a certain connection operation causes a large amount of data exchange, you can consider reorganizing the query or using a temporary table to store intermediate results.

[0115] With respect to step S1042, the performance impact of the query condition in this step can be determined according to the amount of data returned. The amount of data returned can be predicted according to the first execution plan. The priority of the query condition can also be determined according to the performance impact.

[0116] In this way, we analyze the conditional statements in the query and identify which conditions have the greatest impact on the result set, that is, which conditions can significantly reduce the number of returned records. We can then consider the logical relationship between the conditions and evaluate the impact of their execution order on query performance.

[0117] For example, put the condition that can return the minimum number of records in the front, so that irrelevant data can be filtered out earlier.

[0118] For step S1043, in an embodiment provided in the present application, when optimizing the query conditions in the initial SQL query statement according to the performance impact of each query condition, at least one of the following processing strategies is adopted: determining the key query conditions according to the performance impact of the query conditions, and indexing each key query condition statement; utilizing the dynamic SQL feature of MyBatis to adjust the fields in the initial SQL query statement; and determining the target search column of the data to be queried according to the business scenario.

[0119] Determining the key query condition according to the performance impact of the query condition and creating an index for each key query condition statement includes: identifying columns frequently used in the query condition, determining them as key columns, and adding corresponding indexes for the key columns.

[0120] For example, vehicle status may narrow the search more than location. Create indexes for these key columns (vehicle status) to speed up database queries.

[0121] For example, the use of the dynamic SQL feature of MyBatis to adjust the fields in the initial SQL query statement includes: in MyBatis, using <if> 、 <choose> 、 <when> 、 <otherwise>etc. tags to dynamically include or exclude SQL fragments based on conditions.

[0122] For example, determining the target search column of the data to be queried according to the business scenario includes: identifying the initial SQL query statement, determining the business scenario, and determining the target search column of the data to be queried according to the identified specific business scenario.

[0123] In this way, the use of SELECT*5 can be reduced and the WHERE clause can be optimized.

[0124] With respect to step S1044, in this step, the candidate SQL query statement is parsed using tools such as EXPLAIN provided by the database to determine a second execution plan; and the set type corresponding to the candidate SQL query statement is identified at the same time; and then at least one set traversal optimization strategy is selected based on the determined second execution plan and set type.

[0125] For example, the collection type may include a list, a set, a map, or an array.

[0126] In one embodiment provided in the present application, the collection traversal optimization strategy includes at least one of the following: using dynamic tags for collection traversal in the MyBatis framework to process candidate SQL query statements; using a paging query method; indexing key fields in the candidate SQL query statements; and adjusting data types based on the amount of data returned.

[0127] The method of using the dynamic tag for set traversal in the MyBatis framework to process the candidate SQL query statement includes, for example, using the foreach tag in the MyBatis framework to process the candidate SQL query statement.

[0128] The paging query method is adopted, specifically: according to the second execution plan, the data volume is determined; if the data volume exceeds the threshold, the paging query method is executed.

[0129] The indexing of the key fields in the candidate SQL query statements may include, for example: if the set traversal involves a cross-table query or filtering is required based on a field in the set, indexing the corresponding field.

[0130] According to the principle of data balance, adjust the data type according to the amount of data returned.

[0131] With respect to step S105, in an implementation manner provided in the present application, determining a corresponding target cache strategy according to the target SQL query statement includes:

[0132] S1051: Parse the target SQL query statement to determine the type and business scenario of the target SQL query statement.

[0133] S1052: Determine whether to start a cache strategy based on the type and business scenario of the target SQL query statement.

[0134] S1053: If started, determine the corresponding target level of the cache according to the target SQL query statement, and search for target data from the cache of the determined target level.

[0135] For step S1052, identify whether the type and business scenario of the target SQL query statement meet the preset startup conditions. If so, execute step S1053.

[0136] If not satisfied, perform paging query and obtain target data from the database.

[0137] For step S1053, a multi-layer cache mechanism is preset in this solution, specifically including: introducing a second-level cache (Mapper interface level) and an optional distributed cache (shared across servers) based on the default first-level cache (SqlSession level) of MyBatis to form a multi-layer cache mechanism.

[0138] It should be noted that the first-level cache is automatically enabled and does not require additional configuration. It is based on the SqlSession level and is a private, local cache area. The scope of the first-level cache is the current SqlSession. When the SqlSession is closed, the first-level cache becomes invalid. When MyBatis executes a SQL query, it first checks whether there is already a result with the same SQL and parameters in the first-level cache. If it exists, the result is returned directly; if not, the SQL query is executed, the result is stored in the first-level cache, and the result is returned.

[0139] Second Level Cache: The second level cache needs to be enabled in the global configuration file of MyBatis (such as mybatis-config.xml). <settings>The cacheEnabled attribute of the tag is turned on and passed in the Mapper mapping file <cache>Tags configure specific parameters (such as cache strategy, capacity, etc.). The second-level cache is a cache based on the Mapper interface level and is a global cache area. Its scope is the entire Mapper interface and can share data across SqlSessions. When MyBatis executes a SQL query, it will first check whether the result of the same SQL and parameters already exists in the second-level cache. If it exists, the result is returned directly; if not, the SQL query is executed, the result is stored in the second-level cache, and the result is returned.

[0140] Three-level cache (external cache system): For large applications or distributed systems, it may be necessary to introduce an external cache system (such as Redis, Memcached, etc.) to form a three-level cache or more levels of cache structure. This usually requires integrating the client library of the external cache system in the MyBatis configuration file and specifying the use of the external cache system in the Mapper mapping file or by other means. As an independent cache layer, the external cache system can share data across multiple application instances. When MyBatis executes a query, it first checks whether the required data exists in the external cache system; if not, it executes a database query and stores the result in the external cache system for subsequent use.

[0141] With respect to step S1053, a level-by-level search is performed according to the target SQL query statement to determine whether there is a cache of the corresponding target level; if so, the target data is searched from the cache of the corresponding target level.

[0142] If it is determined that there is no cache at the corresponding target level, the target data is obtained from the database and cached.

[0143] In an implementation manner provided in the present application, the data processing method further includes: using pre-stored data in the database to perform consistency verification on the cached data, and updating the cached data according to the verification result.

[0144] Here, when the predetermined time is reached or the predetermined operation is satisfied, the cache data is checked for consistency using the pre-stored data in the database; and the cache data inconsistent with the pre-stored data is deleted.

[0145] In another implementation manner provided in the present application, the data processing method further includes: when a cache expiration condition is reached, deleting the data in the cache according to a preset deletion rule.

[0146] Here, it specifically includes: after the data update operation (such as insert, update, delete), use @CacheEvict annotation to automatically clear the relevant cache data. Or set an appropriate cache expiration time to ensure that the data in the cache will not be inconsistent with the data in the database for a long time. Among them, different deletion rules can be set according to business needs, such as LRU (least recently used), FIFO (first in first out), etc.

[0147] In another embodiment provided in the present application, the data processing method further includes: determining a cache level corresponding to the cache data according to the frequency of the data; and performing storage processing of the cache data according to the corresponding cache level.

[0148] For example, for data with low update frequency (such as vehicle model information), use the second-level cache or Redis cache (third-level cache) first. For data with high real-time requirements (such as the current vehicle location), use the cache with caution or set a shorter cache expiration time. Use the @Cacheable annotation on the service method to specify the cache name and cache key.

[0149] In addition, if the query result is empty, you can specify not to cache the empty result through the unless property. Customize the cache key generation logic to ensure that the same query conditions generate the same cache key to avoid unnecessary repeated queries.

[0150] In this way, this application significantly reduces the number of database queries, improves query efficiency, reduces unnecessary database access and calculations, and reduces system resource consumption through dynamic SQL optimization and intelligent caching strategies. In addition, through multi-layer caching mechanisms and intelligent caching decisions, the system's adaptability to complex business scenarios is enhanced, and data access speed can be accelerated to improve user experience.

[0151] Based on the same inventive concept, a data processing device corresponding to the data processing method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned data processing method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0152] See also Figure 2 , Figure 3 , Figure 2 One of the structural schematic diagrams of a data processing device based on the MyBatis framework provided in an embodiment of the present application, Figure 3 The second structural diagram of a data processing device based on the MyBatis framework provided in the embodiment of the present application. Figure 2 As shown in , the data processing device 200 includes:

[0153] The acquisition module 210 is used to respond to the data query request and acquire the input data corresponding to the data query request;

[0154] The judgment module 220 is used to judge whether the input data includes data of a specified type according to a specified dynamic SQL tag in the MyBatis framework;

[0155] A splicing module 230, for dynamically splicing the input data with a pre-designed configuration file to generate an initial SQL query statement if the data includes data of a specified type;

[0156] A determination module 240 is used to optimize the initial SQL query statement according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement according to a preset optimization strategy to determine a target SQL query statement;

[0157] The query module 250 is used to determine a corresponding target cache strategy according to the target SQL query statement, and obtain target data from a corresponding location according to the corresponding target cache strategy.

[0158] Optionally, when the splicing module 230 is used to dynamically splice the input data with a pre-designed configuration file to generate an initial SQL query statement, the splicing module 230 is used to:

[0159] Obtaining a dynamic SQL query template written according to a dynamic tag in the configuration file;

[0160] The input data is concatenated with the dynamic SQL query template to determine an initial SQL query statement.

[0161] Optionally, when the determination module 240 is used to optimize the initial SQL query statement according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement and to determine a target SQL query statement, the determination module 240 is used to:

[0162] Parsing the initial SQL query statement using a query tool provided by the database to be accessed to determine a first execution plan for the initial query;

[0163] Evaluate the query conditions in the initial SQL query statement according to the first execution plan to determine the performance impact of each query condition;

[0164] According to the performance optimization principle, the query conditions in the initial SQL query statement are optimized according to the performance impact of each query condition to obtain a candidate SQL query statement;

[0165] Using the second execution plan determined according to the candidate SQL query statement to evaluate the set traversal type determined according to the candidate SQL query statement, and selecting a corresponding set traversal optimization strategy according to the evaluation result;

[0166] The candidate SQL query statements are re-optimized according to the corresponding set traversal optimization strategy to obtain the target SQL query statement.

[0167] Optionally, when optimizing the query conditions in the initial SQL query statement according to the performance impact of each query condition, at least one of the following processing strategies is adopted:

[0168] Determine key query conditions based on the performance impact of the query conditions, and create an index for each key query condition statement;

[0169] Use the dynamic SQL feature of MyBatis to adjust the fields in the initial SQL query statement;

[0170] Determine the target search column of the data to be queried based on the business scenario.

[0171] Optionally, the set traversal optimization strategy includes at least one of the following:

[0172] Use dynamic tags for collection traversal in the MyBatis framework to process candidate SQL query statements;

[0173] Use paging query method;

[0174] Creating indexes for key fields in the candidate SQL query statements;

[0175] Adjust the data type according to the amount of data returned.

[0176] Optionally, when the query module 250 is used to determine a corresponding target cache strategy according to the target SQL query statement, the query module 250 is used to:

[0177] Parsing the target SQL query statement to determine the type and business scenario of the target SQL query statement;

[0178] Determine whether to start the cache strategy according to the type and business scenario of the target SQL query statement;

[0179] If started, the target level of the corresponding cache is determined according to the target SQL query statement, and the target data is searched from the cache of the determined target level.

[0180] Optional, such as Figure 3 As shown, the data processing device 200 further includes an updating module 260, and the updating module 260 is used to:

[0181] Use the pre-stored data in the database to check the consistency of the cached data, and update the cached data according to the correction results.

[0182] Optionally, the data processing device 200 further includes a failure module 270, and the failure module 270 is used to:

[0183] When the cache expiration condition is reached, the data in the cache is deleted according to the preset deletion rules.

[0184] Optionally, the data processing device 200 further includes a cache module 280, and the cache module 280 is used to:

[0185] Determine the cache level corresponding to the cached data based on the frequency of the data;

[0186] And the cache data is stored and processed according to the corresponding cache level.

[0187] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0188] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0189] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0190] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0191] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0192] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0194] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0195] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.< / cache> < / settings> < / otherwise> < / when> < / choose> < / if> < / if>

Claims

1. A data processing method based on MyBatis framework, characterized in that: The data processing method comprises: In response to a data query request, obtaining input data corresponding to the data query request; According to the specified dynamic SQL tag in the MyBatis framework, determine whether the input data includes data of a specified type; If included, dynamically concatenate the input data with a pre-designed configuration file to generate an initial SQL query statement; According to the conditional statement in the initial SQL query statement and the query plan determined according to the initial SQL query statement, the initial SQL query statement is optimized according to a preset optimization strategy to determine a target SQL query statement; According to the target SQL query statement, a corresponding target cache strategy is determined, and target data is acquired from a corresponding position according to the corresponding target cache strategy.

2. The data processing method according to claim 1, characterized in that: The step of dynamically combining the input data with the pre-designed configuration file to generate an initial SQL query statement includes: Obtaining a dynamic SQL query template written according to a dynamic tag in the configuration file; The input data is concatenated with the dynamic SQL query template to determine an initial SQL query statement.

3. The data processing method according to claim 1, characterized in that: The step of optimizing the initial SQL query statement according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement according to a preset optimization strategy to determine a target SQL query statement includes: Parsing the initial SQL query statement using a query tool provided by the database to be accessed to determine a first execution plan for the initial query; Evaluate the query conditions in the initial SQL query statement according to the first execution plan to determine the performance impact of each query condition; According to the performance optimization principle, the query conditions in the initial SQL query statement are optimized according to the performance impact of each query condition to obtain a candidate SQL query statement; Using the second execution plan determined according to the candidate SQL query statement to evaluate the set traversal type determined according to the candidate SQL query statement, and selecting a corresponding set traversal optimization strategy according to the evaluation result; The candidate SQL query statements are re-optimized according to the corresponding set traversal optimization strategy to obtain the target SQL query statement.

4. The data processing method according to claim 3, characterized in that: When optimizing the query conditions in the initial SQL query statement according to the performance impact of each query condition, at least one of the following processing strategies is adopted: Determine key query conditions based on the performance impact of the query conditions, and create an index for each key query condition statement; Use the dynamic SQL feature of MyBatis to adjust the fields in the initial SQL query statement; Determine the target search column of the data to be queried based on the business scenario.

5. The data processing method according to claim 3, characterized in that: The set traversal optimization strategy includes at least one of the following: Use dynamic tags for collection traversal in the MyBatis framework to process candidate SQL query statements; Use paging query method; Creating indexes for key fields in the candidate SQL query statements; Adjust the data type according to the amount of data returned.

6. The data processing method according to claim 1, characterized in that: Determining a corresponding target cache strategy according to the target SQL query statement includes: Parsing the target SQL query statement to determine the type and business scenario of the target SQL query statement; Determine whether to start the cache strategy according to the type and business scenario of the target SQL query statement; If started, the target level of the corresponding cache is determined according to the target SQL query statement, and the target data is searched from the cache of the determined target level.

7. The data processing method according to claim 1, characterized in that: The data processing method further includes: Use the pre-stored data in the database to check the consistency of the cached data, and update the cached data according to the correction results.

8. The data processing method according to claim 1, characterized in that: The data processing method further includes: When the cache expiration condition is reached, the data in the cache is deleted according to the preset deletion rules.

9. The data processing method according to claim 1, characterized in that: The data processing method further includes: Determine the cache level corresponding to the cached data based on the frequency of the data; And the cache data is stored and processed according to the corresponding cache level.

10. A data processing device based on MyBatis framework, characterized in that: The data processing device comprises: An acquisition module, used to respond to a data query request and acquire input data corresponding to the data query request; A judgment module is used to judge whether the input data includes data of a specified type according to a specified dynamic SQL tag in the MyBatis framework; A splicing module, used to dynamically splice the input data with a pre-designed configuration file to generate an initial SQL query statement if the data includes a specified type; A determination module, configured to optimize the initial SQL query statement according to a conditional statement in the initial SQL query statement and a query plan determined according to the initial SQL query statement, and determine a target SQL query statement according to a preset optimization strategy; The query module is used to determine a corresponding target cache strategy according to the target SQL query statement, and obtain target data from a corresponding position according to the corresponding target cache strategy.

Images