System and method for realizing database output based on LLM
The system addresses anomalies in query rewriting and caching by constructing a monitoring model to evaluate and provide early warnings, enhancing database query stability and user experience.
Patent Information
- Application Number
- CN202410670312.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-28
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-05-28
AI Technical Summary
Existing database output systems based on LLM face challenges in timely perception of anomalies in query rewriting and query caching, leading to reduced accuracy and timeliness of output, affecting user experience.
A system and method that constructs a query optimization hidden danger monitoring model using data access path conflict anomaly coefficients and cache invalidation mechanism delay coefficients to evaluate and provide early warnings for potential issues in query rewriting and caching.
Enhances the stability and reliability of database queries by timely detection and resolution of hidden dangers, improving user satisfaction and query performance.
Smart Images

Figure CN118708611B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of database output, and more specifically, to a system and method for implementing database output based on LLM. Background Art
[0002] The database output system based on LLM is a system that uses natural language processing technology to convert the natural language query input by the user into a query language that the database can understand, and accurately retrieve the required information from the database. This system combines artificial intelligence and database technology to provide users with a more convenient and intuitive way to retrieve data. The core technology of the system is the language model (LLM), which is a natural language processing model based on deep learning that can understand and generate natural language text. Through pre-training, LLM can learn large-scale text data, so that it has a deep understanding of language structure and semantics. In the system that implements database output, LLM first receives the natural language query input by the user, and then converts it into a structured database query language such as SQL through text understanding and semantic analysis. In this process, LLM can identify and parse the intention of the user's query and convert it into accurate and effective operations on the database. The system also involves database technology. As a tool for storing and managing large amounts of structured data, the database plays a vital role in the system. Through the database management system (DBMS), the system can efficiently store and organize data and adopt a series of query optimization technologies, including query rewriting and query caching. However, there are still challenges in timely perception of potential anomalies in query rewriting and query caching. When obvious errors occur in query rewriting and obvious delays occur in the cache invalidation mechanism, the accuracy and timeliness of the final output of the database may be reduced, affecting the user's query experience.
[0003] In order to solve the above defects, a technical solution is now provided. Summary of the invention
[0004] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present invention provide a system and method for implementing database output based on LLM to solve the problems raised in the above-mentioned background technology.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] The database output method based on LLM includes the following steps:
[0007] Step S1, for obtaining data access path information of a query rewriting process, and obtaining a data access path conflict anomaly coefficient according to the data access path information;
[0008] Step S2, for obtaining cache invalidation mechanism delay information in the cache query process, and obtaining a cache invalidation mechanism delay coefficient according to the cache invalidation mechanism delay information;
[0009] Step S3, for constructing a query optimization hidden danger monitoring model according to the data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient, comprehensively evaluating the abnormal hidden dangers existing in the current query subsystem, and generating a query optimization hidden danger index;
[0010] Step S4, for comparing the query optimization hidden danger index with a preset query optimization hidden danger index threshold, and classifying the abnormal hidden dangers existing in the current query subsystem;
[0011] Step S5 is used to issue an early warning on the accuracy and timeliness of the database output according to the classification results of abnormal hidden dangers existing in the query subsystem.
[0012] In a preferred embodiment, by acquiring data access path information of the query rewriting process, analyzing abnormal conflicts occurring in the generated data access path, and acquiring a data access path conflict abnormality coefficient, the degree of abnormal conflicts occurring in the data access path is measured;
[0013] The logic for obtaining the data access path conflict anomaly coefficient is as follows:
[0014] Obtain data access path data during the query rewriting process within the T time period;
[0015] Calculate the edit distance of two data access paths as follows:
[0016] Mark the two data access paths as data access path A and data access path B respectively;
[0017] Represent the data access path as a string, including nodes and edges;
[0018] Initialize and create a (m+1)×(n+1) two-dimensional array dp, where m and n are the lengths of data access path A and data access path B plus one respectively;
[0019] Initialize dp[i][0] = i, which indicates the number of edit operations required to delete the first i characters of data access path A into an empty string;
[0020] Initialize dp[0][j] = j, which indicates the number of edit operations required to convert an empty string into the first j characters of data access path B;
[0021] Fill the array, starting from the upper left corner and gradually filling the dp array: when A[i-1] and B[j-1] are the same, dp[i][j] = dp[i-1][j-1], indicating that no additional operation is required; when A[i-1] and B[j-1] are different, dp[i][j] = min(dp[i-1][j], dp[i][j-1], dp[i-1][j-1]) + 1, indicating the minimum value in the editing operation;
[0022] The result is returned, and the final edit distance is bj=dp[m][n], where m and n are the lengths of data access path A and data access path B respectively;
[0023] The path length weight qz is calculated based on the edit distance, and the expression is as follows
[0024] Calculate the similarity of two data access paths as follows:
[0025] Representing the data access path A and the data access path B in a vector form, marking the data access path A and the data access path B as vector A and vector B respectively, wherein each dimension in the vector represents a node of the data access path;
[0026] Calculate the dot product dj of vector A and vector B. The expression is as follows
[0027] Calculate the norm of vector A and vector B respectively, the expression is as follows and
[0028] Calculate the path similarity of two data access paths. The expression is as follows
[0029] The comprehensive similarity qinvnf of the data access path is calculated according to the path length weight qz and the path similarity similarity, which is expressed as follows: qinvnf = qz*similarity;
[0030] Calculate the data access path comprehensive similarities between different data access paths according to the data access path comprehensive similarity calculation method to obtain a data access path comprehensive similarity sequence;
[0031] The data access path comprehensive similarity in the data access path comprehensive similarity sequence is compared with the conflict threshold. When the data access path comprehensive similarity is greater than the conflict threshold, the data access path comprehensive similarities greater than the conflict threshold are accumulated and calculated to obtain the data access path conflict anomaly coefficient shx.
[0032] In a preferred embodiment, by obtaining cache invalidation mechanism delay information during the cache query process, possible delays of the cache invalidation mechanism are analyzed, and a cache invalidation mechanism delay coefficient is obtained to measure the time delay degree of the cache invalidation mechanism;
[0033] The logic for obtaining the cache invalidation mechanism delay coefficient is as follows:
[0034] Set a reference threshold sx for the delay time of cache invalidation events during query cache z ;
[0035] Obtain the delay time of cache invalidation events in the query cache process within the T time period and mark it as yc; compare the delay time of cache invalidation events with the reference threshold. When yc>sx z When , the delay time of the cache invalidation event is marked as the abnormal delay time t q , q indicates that yc>sx z The number of times is q = {1, 2, ..., e}, where e is a positive integer;
[0036] Calculate the cache invalidation mechanism delay coefficient, the expression is as follows Where afk represents the cache invalidation mechanism delay coefficient, and sxs represents the total number of cache invalidation events.
[0037] In a preferred embodiment, the data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient are normalized, and a query optimization hidden danger monitoring model is constructed based on the normalized data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient to generate a query optimization hidden danger index mak. The model is based on the following formula: mak=a1*shx+a2*afk, where a1 and a2 represent the preset proportional coefficients of the data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient, respectively, and a1 and a2 are both greater than 0.
[0038] In a preferred embodiment, the query optimization hidden danger index is compared with a preset query optimization hidden danger index threshold;
[0039] If the query optimization hidden danger index is greater than the preset query optimization hidden danger index threshold, a hidden danger abnormality signal is generated;
[0040] If the query optimization hidden danger index is less than or equal to a preset query optimization hidden danger index threshold, a hidden danger steady-state signal is generated.
[0041] In a preferred embodiment, when a hidden danger abnormality signal is generated, the query optimization hidden danger index generated in the subsequent multiple T time periods is obtained, and a hidden danger trend analysis set is established, and the hidden danger trend analysis set is marked as qus={makα}, where α={1,2,3…β}, β is a positive integer;
[0042] Calculate the standard deviation lqk of the query optimization hidden danger index in the hidden danger trend analysis set. The expression is as follows Where lav represents the average value of the query optimization hidden danger index in the hidden danger trend analysis set, and the expression is as follows
[0043] Compare the standard deviation of the query optimization risk index with a preset standard deviation threshold;
[0044] If the standard deviation of the query optimization risk index is greater than the preset standard deviation threshold, an early warning signal is generated;
[0045] If the standard deviation of the query optimization risk index is less than or equal to the preset standard deviation threshold, there is no need to generate an early warning signal.
[0046] Implementing a database output system based on LLM, including a query rewriting monitoring module, for obtaining data access path information during the query rewriting process, and obtaining a data access path conflict anomaly coefficient based on the data access path information;
[0047] A query cache monitoring module is used to obtain cache invalidation mechanism delay information during the query cache process, and obtain a cache invalidation mechanism delay coefficient based on the cache invalidation mechanism delay information;
[0048] The model building module is used to build a query optimization hidden danger monitoring model based on the data access path conflict anomaly coefficient and cache invalidation mechanism delay coefficient, conduct a comprehensive assessment of the abnormal hidden dangers existing in the current query subsystem, and generate a query optimization hidden danger index;
[0049] A classification module is used to compare the query optimization hidden danger index with a preset query optimization hidden danger index threshold, and classify the abnormal hidden dangers existing in the current query subsystem;
[0050] The early warning module is used to provide early warning prompts on the accuracy and timeliness of database output based on the classification results of abnormal hidden dangers in the query subsystem.
[0051] Technical effects and advantages of the present invention:
[0052] 1. The present invention obtains data access path information in the query rewriting process and cache invalidation delay information in the query caching process, and constructs a query optimization hidden danger monitoring model based on the data access path information and cache invalidation delay information, so as to comprehensively evaluate the abnormal hidden dangers existing in the current query optimization technology, thereby timely discovering possible problems with query rewriting and query caching, and improving the efficiency and performance of database queries. By monitoring and evaluating the abnormal hidden dangers existing in the query optimization technology and providing early warning prompts in time, the risks brought by the abnormal hidden dangers can be effectively reduced, and the stability and reliability of the system can be improved. By providing early warning prompts for the accuracy and timeliness of the database output, potential problems can be discovered and handled in time, thereby improving the user's query experience and enhancing user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with the accompanying drawings;
[0054] Figure 1 This is a schematic diagram of the structure of the method of Example 1 of the present invention;
[0055] Figure 2 Schematic diagram of the structure of the system of Example 2 of the present invention. DETAILED DESCRIPTION
[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0057] Example 1
[0058] Figure 1 The present invention provides a method for implementing database output based on LLM, which includes the following steps:
[0059] Step S1, for obtaining data access path information of a query rewriting process, and obtaining a data access path conflict anomaly coefficient according to the data access path information;
[0060] The data access path refers to the path of the data table, index or other storage structure that is determined during the query rewriting process to be accessed when actually querying the database. During the query rewriting process, the database system will generate one or more query plans based on the query statements entered by the user and the metadata information within the system. Each query plan represents a possible data access path. These paths may include different table connection orders, index usage methods, etc.
[0061] When an abnormal conflict occurs in the generated data access path, the following impacts may be caused on the stability of the query subsystem:
[0062] Performance degradation: Abnormal conflicts may result in the selection of inefficient data access paths, such as selecting a full table scan instead of an index search, which results in degraded query performance and prolonged query response time.
[0063] Resource competition: Abnormal conflicts may cause resource competition during query execution. For example, multiple queries request the same resource at the same time, resulting in resource contention and reducing the concurrent performance of the system.
[0064] Deadlock risk: Abnormal conflicts may cause deadlocks during query execution. For example, multiple queries hold resources at the same time and wait for other queries to release resources, causing deadlocks and making the system unable to continue to operate normally.
[0065] Data consistency issues: Abnormal conflicts may lead to improper selection of data access paths, resulting in data consistency being destroyed. For example, dirty or inconsistent data may be read, affecting the correctness and reliability of the system.
[0066] System crash: In extreme cases, abnormal conflicts may cause system crashes, such as an abnormal situation that cannot be handled, causing the system to terminate abnormally, resulting in service interruption and data loss;
[0067] Therefore, by obtaining the data access path information of the query rewriting process, analyzing the abnormal conflicts in the generated data access path, and obtaining the data access path conflict abnormality coefficient, the degree of abnormal conflicts in the data access path is measured;
[0068] The logic for obtaining the data access path conflict anomaly coefficient is as follows:
[0069] Obtain data access path data during the query rewriting process within the T time period;
[0070] Calculate the edit distance of two data access paths as follows:
[0071] Mark the two data access paths as data access path A and data access path B respectively;
[0072] Represent the data access path as a string, including nodes and edges;
[0073] It should be noted that nodes represent database table indexes, and edges represent connection operations;
[0074] Initialize and create a (m+1)×(n+1) two-dimensional array dp, where m and n are the lengths of data access path A and data access path B plus one respectively;
[0075] Initialize dp[i][0] = i, which indicates the number of edit operations required to delete the first i characters of data access path A into an empty string;
[0076] Initialize dp[0][j] = j, which indicates the number of edit operations required to convert an empty string into the first j characters of data access path B;
[0077] Fill the array, starting from the upper left corner and gradually filling the dp array: when A[i-1] and B[j-1] are the same, dp[i][j] = dp[i-1][j-1], indicating that no additional operation is required; when A[i-1] and B[j-1] are different, dp[i][j] = min(dp[i-1][j], dp[i][j-1], dp[i-1][j-1]) + 1, indicating the minimum value in the editing operation;
[0078] The result is returned, and the final edit distance is bj=dp[m][n], where m and n are the lengths of data access path A and data access path B respectively;
[0079] The path length weight qz is calculated based on the edit distance, and the expression is as follows
[0080] Calculate the similarity of two data access paths as follows:
[0081] Representing the data access path A and the data access path B in a vector form, marking the data access path A and the data access path B as vector A and vector B respectively, wherein each dimension in the vector represents a node of the data access path;
[0082] Calculate the dot product dj of vector A and vector B. The expression is as follows
[0083] Calculate the norm of vector A and vector B respectively, the expression is as follows and
[0084] Calculate the path similarity of two data access paths. The expression is as follows
[0085] The comprehensive similarity qinvnf of the data access path is calculated according to the path length weight qz and the path similarity similarity, which is expressed as follows: qinvnf = qz*similarity;
[0086] Calculate the data access path comprehensive similarities between different data access paths according to the data access path comprehensive similarity calculation method to obtain a data access path comprehensive similarity sequence;
[0087] Compare the data access path comprehensive similarity in the data access path comprehensive similarity sequence with the conflict threshold. When the data access path comprehensive similarity is greater than the conflict threshold, accumulate and calculate the data access path comprehensive similarities greater than the conflict threshold to obtain the data access path conflict anomaly coefficient shx;
[0088] From the above calculation, it can be seen that the larger the data access path conflict anomaly coefficient is, the higher the probability of abnormal hidden dangers that may exist in the query subsystem is. Conversely, the smaller the data access path conflict anomaly coefficient is, the lower the probability of abnormal hidden dangers that may exist in the query subsystem is.
[0089] Step S2, for obtaining cache invalidation mechanism delay information in the cache query process, and obtaining a cache invalidation mechanism delay coefficient according to the cache invalidation mechanism delay information;
[0090] Cache invalidation delay is a measure of the delay in the cache invalidation mechanism during cache query. It indicates the time delay required to re-acquire data due to cache invalidation. This delay is the extra time the query subsystem needs to spend when the cache is invalidated to detect data invalidation, re-acquire data, and update the cache.
[0091] When the cache invalidation mechanism is delayed, the following impacts may be caused on the stability of the query subsystem:
[0092] Increased response time: Cache invalidation delays cause the query subsystem to take extra time to re-acquire data, thereby increasing the query response time. If the delay is large, users may perceive that the system's response speed is slow, affecting the user experience;
[0093] Increase system load: When the cache fails, the query subsystem needs to re-acquire data from the data source and update the cache. If a large number of queries fail at the same time, the system load will increase suddenly, which may affect the stability and availability of the system;
[0094] Reduced system fault tolerance: If the cache invalidation mechanism is severely delayed, a large number of queries will directly access the data source and bypass the cache. Once the data source fails or the network is abnormal, it will directly affect the normal operation of the system. At this time, the system's fault tolerance will be reduced, which may lead to system unavailability;
[0095] Increased system complexity: In order to cope with the impact of cache invalidation mechanism delays, it may be necessary to introduce more complex cache strategies, optimize cache configuration, or increase system resources. This will increase system complexity and maintenance costs;
[0096] Therefore, by obtaining the cache invalidation mechanism delay information during the query cache process, the possible delay of the cache invalidation mechanism is analyzed, and the cache invalidation mechanism delay coefficient is obtained to measure the time delay degree of the cache invalidation mechanism;
[0097] The logic for obtaining the cache invalidation mechanism delay coefficient is as follows:
[0098] Set a reference threshold sx for the delay time of cache invalidation events during query cache z ;
[0099] Obtain the delay time of cache invalidation events in the query cache process within the T time period and mark it as yc; compare the delay time of cache invalidation events with the reference threshold. When yc>sx z When , the delay time of the cache invalidation event is marked as the abnormal delay time t q , q indicates that yc>sx z The number of times is q = {1, 2, ..., e}, where e is a positive integer;
[0100] Calculate the cache invalidation mechanism delay coefficient, the expression is as follows Where afk represents the cache invalidation mechanism delay coefficient, and sxs represents the total number of cache invalidation events;
[0101] From the above calculation, it can be seen that the larger the cache invalidation mechanism delay coefficient is, the higher the probability of abnormal hidden dangers that may exist in the query subsystem is. Conversely, the smaller the cache invalidation mechanism delay coefficient is, the lower the probability of abnormal hidden dangers that may exist in the query subsystem is.
[0102] Step S3, for constructing a query optimization hidden danger monitoring model according to the data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient, comprehensively evaluating the abnormal hidden dangers existing in the current query subsystem, and generating a query optimization hidden danger index;
[0103] The data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient are normalized, and a query optimization hidden danger monitoring model is constructed based on the normalized data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient to generate a query optimization hidden danger index mak. The model is based on the following formula mak=a1*shx+a2*afk, where a1 and a2 represent the preset proportional coefficients of the data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient, respectively, and a1 and a2 are both greater than 0;
[0104] It can be seen from the above calculation expression that the larger the data access path conflict anomaly coefficient and the larger the cache invalidation mechanism delay coefficient, the larger the query optimization hidden danger index, indicating that the probability of abnormal hidden dangers that may exist in the query subsystem is higher. Conversely, the smaller the data access path conflict anomaly coefficient and the smaller the cache invalidation mechanism delay coefficient, the smaller the query optimization hidden danger index, indicating that the probability of abnormal hidden dangers that may exist in the query subsystem is lower.
[0105] Step S4, for comparing the query optimization hidden danger index with a preset query optimization hidden danger index threshold, and classifying the abnormal hidden dangers existing in the current query subsystem;
[0106] If the query optimization hidden danger index is greater than the preset query optimization hidden danger index threshold, it indicates that the current query subsystem has abnormal hidden dangers, and a hidden danger abnormality signal is generated;
[0107] If the query optimization hidden danger index is less than or equal to the preset query optimization hidden danger index threshold, it indicates that the current query subsystem does not have abnormal hidden dangers, and a hidden danger steady-state signal is generated;
[0108] Step S5, for providing early warning prompts on the accuracy and timeliness of database output according to the abnormal hidden danger classification results existing in the query subsystem;
[0109] When a hidden danger abnormal signal is generated, the query optimization hidden danger index generated in the subsequent multiple T time periods is obtained, and a hidden danger trend analysis set is established, and the hidden danger trend analysis set is marked as qus={mak α}, where α={1,2,3…β}, β is a positive integer;
[0110] Calculate the standard deviation lqk of the query optimization hidden danger index in the hidden danger trend analysis set. The expression is as follows Where lav represents the average value of the query optimization hidden danger index in the hidden danger trend analysis set, and the expression is as follows
[0111] Compare the standard deviation of the query optimization risk index with a preset standard deviation threshold;
[0112] If the standard deviation of the query optimization risk index is greater than the preset standard deviation threshold, it indicates that the abnormal risks in the current query subsystem are continuously increasing, which may lead to a decrease in the execution efficiency of the query, thereby affecting the accuracy and timeliness of the database output, and generating an early warning signal;
[0113] If the standard deviation of the query optimization hidden danger index is less than or equal to the preset standard deviation threshold, it indicates that the abnormal hidden dangers existing in the current query subsystem are accidental and there is no need to generate an early warning signal.
[0114] The present invention obtains data access path information in the query rewriting process and cache invalidation delay information in the query caching process, and constructs a query optimization hidden danger monitoring model based on the data access path information and cache invalidation delay information, so as to comprehensively evaluate the abnormal hidden dangers existing in the current query optimization technology, thereby timely discovering possible problems with query rewriting and query caching, and improving the efficiency and performance of database queries. By monitoring and evaluating the abnormal hidden dangers existing in the query optimization technology and providing early warning prompts in time, the risks brought by the abnormal hidden dangers can be effectively reduced, and the stability and reliability of the system can be improved. By providing early warning prompts for the accuracy and timeliness of the database output, potential problems can be discovered and handled in time, thereby improving the user's query experience and enhancing user satisfaction.
[0115] Example 2
[0116] This embodiment is an introduction to the database output system based on LLM. Figure 2 As shown, it includes a query rewriting monitoring module, which is used to obtain data access path information of the query rewriting process, and obtain the data access path conflict anomaly coefficient according to the data access path information;
[0117] A query cache monitoring module is used to obtain cache invalidation mechanism delay information during the query cache process, and obtain a cache invalidation mechanism delay coefficient based on the cache invalidation mechanism delay information;
[0118] The model building module is used to build a query optimization hidden danger monitoring model based on the data access path conflict anomaly coefficient and cache invalidation mechanism delay coefficient, conduct a comprehensive assessment of the abnormal hidden dangers existing in the current query subsystem, and generate a query optimization hidden danger index;
[0119] A classification module is used to compare the query optimization hidden danger index with a preset query optimization hidden danger index threshold, and classify the abnormal hidden dangers existing in the current query subsystem;
[0120] The early warning module is used to provide early warning prompts on the accuracy and timeliness of database output based on the classification results of abnormal hidden dangers in the query subsystem;
[0121] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.
[0122] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0123] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and method described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0125] In several embodiments provided in this application, it should be understood that the disclosed systems and methods may be implemented in other ways.
[0126] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A database output method based on LLM, characterized in that: The steps include: Step S1, for obtaining data access path information of a query rewriting process, and obtaining a data access path conflict anomaly coefficient according to the data access path information; By obtaining the data access path information of the query rewriting process, analyzing the abnormal conflicts in the generated data access path, and obtaining the data access path conflict abnormality coefficient, the degree of abnormal conflicts in the data access path is measured; The logic for obtaining the data access path conflict anomaly coefficient is as follows: Obtain data access path data during the query rewriting process within the T time period; Calculate the edit distance of two data access paths as follows: Mark the two data access paths as data access path A and data access path B respectively; Represent the data access path as a string, including nodes and edges; Initialize and create a (m+1)×(n+1) two-dimensional array dp, where m and n are the lengths of data access path A and data access path B plus one respectively; Initialize dp[i][0] = i, which indicates the number of edit operations required to delete the first i characters of data access path A into an empty string; Initialize dp[0][j] = j, which indicates the number of edit operations required to convert an empty string into the first j characters of data access path B; Fill the array, starting from the upper left corner and gradually filling the dp array: when A[i-1] and B[j-1] are the same, dp[i][j] = dp[i-1][j-1], indicating that no additional operation is required; when A[i-1] and B[j-1] are different, dp[i][j] = min(dp[i-1][j], dp[i][j-1], dp[i-1][j-1]) + 1, indicating the minimum value in the editing operation; The result is returned, and the final edit distance is bj=dp[m][n], where m and n are the lengths of data access path A and data access path B respectively; The path length weight qz is calculated based on the edit distance, and the expression is as follows Calculate the similarity of two data access paths as follows: Representing the data access path A and the data access path B in a vector form, marking the data access path A and the data access path B as vector A and vector B respectively, wherein each dimension in the vector represents a node of the data access path; Calculate the dot product dj of vector A and vector B. The expression is as follows Calculate the norm of vector A and vector B respectively, the expression is as follows and Calculate the path similarity of two data access paths. The expression is as follows The comprehensive similarity qinvnf of the data access path is calculated according to the path length weight qz and the path similarity similarity, which is expressed as follows: qinvnf = qz*similarity; Calculate the data access path comprehensive similarities between different data access paths according to the data access path comprehensive similarity calculation method to obtain a data access path comprehensive similarity sequence; Compare the data access path comprehensive similarity in the data access path comprehensive similarity sequence with the conflict threshold. When the data access path comprehensive similarity is greater than the conflict threshold, accumulate and calculate the data access path comprehensive similarities greater than the conflict threshold to obtain the data access path conflict anomaly coefficient shx; Step S2, for obtaining cache invalidation mechanism delay information in the cache query process, and obtaining a cache invalidation mechanism delay coefficient according to the cache invalidation mechanism delay information; By obtaining the cache invalidation mechanism delay information during the query cache process, the possible delay of the cache invalidation mechanism is analyzed, and the cache invalidation mechanism delay coefficient is obtained to measure the time delay degree of the cache invalidation mechanism; The logic for obtaining the cache invalidation mechanism delay coefficient is as follows: Set a reference threshold sx for the delay time of cache invalidation events during query cache z ; Obtain the delay time of cache invalidation events in the query cache process within the T time period and mark it as yc; compare the delay time of cache invalidation events with the reference threshold. When yc>sx z When , the delay time of the cache invalidation event is marked as the abnormal delay time t q , q indicates that yc>sx z The number of times is q = {1, 2, ..., e}, where e is a positive integer; Calculate the cache invalidation mechanism delay coefficient, the expression is as follows Where afk represents the cache invalidation mechanism delay coefficient, and sxs represents the total number of cache invalidation events; Step S3, for building a query optimization hidden danger monitoring model according to the data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient, comprehensively evaluating the abnormal hidden dangers existing in the current query subsystem, and generating a query optimization hidden danger index; Step S4, for comparing the query optimization hidden danger index with a preset query optimization hidden danger index threshold, and classifying the abnormal hidden dangers existing in the current query subsystem; Step S5 is used to issue an early warning on the accuracy and timeliness of the database output according to the classification results of abnormal hidden dangers existing in the query subsystem.
2. The method for implementing database output based on LLM according to claim 1, characterized in that: The data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient are normalized, and a query optimization hidden danger monitoring model is constructed based on the normalized data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient to generate a query optimization hidden danger index mak. The model is based on the following formula: mak=a1*shx+a2*afk, where a1 and a2 represent the preset proportional coefficients of the data access path conflict anomaly coefficient and the cache invalidation mechanism delay coefficient, respectively, and a1 and a2 are both greater than 0.
3. The method for realizing database output based on LLM according to claim 2, characterized in that: Compare the query optimization potential risk index with a preset query optimization potential risk index threshold; If the query optimization hidden danger index is greater than the preset query optimization hidden danger index threshold, a hidden danger abnormality signal is generated; If the query optimization hidden danger index is less than or equal to a preset query optimization hidden danger index threshold, a hidden danger steady-state signal is generated.
4. The method for realizing database output based on LLM according to claim 3, characterized in that: When a hidden danger abnormal signal is generated, the query optimization hidden danger index generated in the subsequent multiple T time periods is obtained, and a hidden danger trend analysis set is established, and the hidden danger trend analysis set is marked as qus={mak α }, where α={1,2,3…β}, β is a positive integer; Calculate the standard deviation lqk of the query optimization hidden danger index in the hidden danger trend analysis set. The expression is as follows Where lav represents the average value of the query optimization hidden danger index in the hidden danger trend analysis set, and the expression is as follows Compare the standard deviation of the query optimization risk index with a preset standard deviation threshold; If the standard deviation of the query optimization risk index is greater than the preset standard deviation threshold, an early warning signal is generated; If the standard deviation of the query optimization risk index is less than or equal to the preset standard deviation threshold, there is no need to generate an early warning signal.
5. A database output system based on LLM, used to implement the database output method based on LLM as claimed in any one of claims 1 to 4, characterized in that: It includes a query rewriting monitoring module, which is used to obtain data access path information of the query rewriting process, and obtain the data access path conflict anomaly coefficient according to the data access path information; A query cache monitoring module is used to obtain cache invalidation mechanism delay information during the query cache process, and obtain a cache invalidation mechanism delay coefficient based on the cache invalidation mechanism delay information; The model building module is used to build a query optimization hidden danger monitoring model based on the data access path conflict anomaly coefficient and cache invalidation mechanism delay coefficient, conduct a comprehensive assessment of the abnormal hidden dangers existing in the current query subsystem, and generate a query optimization hidden danger index; A classification module is used to compare the query optimization hidden danger index with a preset query optimization hidden danger index threshold, and classify the abnormal hidden dangers existing in the current query subsystem; The early warning module is used to provide early warning prompts on the accuracy and timeliness of database output based on the classification results of abnormal hidden dangers in the query subsystem.
Citation Information
Patent Citations
Positioning data transmission management system and method based on big data
CN117573680A
Database System With Path Based Query Engine
US20080114803A1
Systems and methods for data mining and automated generation of search query rewrites
US20160179807A1
Machine Language Query Management for Low-Latency Database Analysis System
US20210089530A1
Access path for database optimizer
US20220188314A1
