Database object screening method, computer device, and storage medium

Abstract

The application relates to a database object screening method, a computer device and a storage medium, and relates to the technical field of natural language processing. The method comprises the following steps: performing semantic correlation calculation on a query request and each database object to obtain original correlation scores of the database objects; determining an object retention quantity based on probability distribution statistical features corresponding to the original correlation scores of the database objects; and determining a subset of screened database objects from the database objects based on the object retention quantity.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a database object filtering method, computer device, and storage medium. Background Technology

[0002] In the Natural Language to SQL (Structured Query Language) task, in order to convert the user's natural language into executable SQL statements, the semantic similarity between the query and the database objects is usually calculated, and the database objects are filtered based on a fixed threshold or a fixed number, which serves as the context for the subsequent generation of SQL statements by the large model.

[0003] However, using uniform filtering parameters cannot adapt to the long-tail distribution of query difficulty. For simple queries, filtering too many irrelevant objects can introduce noise, while for complex queries, insufficient filtering may miss key objects, resulting in poor database object filtering performance. Summary of the Invention

[0004] This application provides a database object filtering method, computer device, and storage medium, which can dynamically adjust the filtering granularity according to the query difficulty, effectively improve the accuracy and robustness of database object filtering, and enhance the generalization ability under different database domains and retrieval models. The technical solution is as follows.

[0005] Firstly, a database object filtering method is provided, the method comprising: Semantic relevance calculation is performed between the query request and each database object to obtain the original relevance score of each database object; Based on the probability distribution statistical characteristics corresponding to the original relevance scores of each database object, the number of objects to be retained is determined; Based on the number of objects to be retained, a subset of filtered database objects is determined from each database object.

[0006] In a second aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the database object filtering method described above.

[0007] Thirdly, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the above-described database object filtering method.

[0008] Fourthly, a computer program product is provided, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to execute to implement the database object filtering method provided in the various optional implementations described above.

[0009] The database object filtering method provided in this application calculates the original relevance score by performing semantic relevance calculation on the query request and each database object, determines the number of objects to be retained based on the probability distribution statistical characteristics corresponding to the original relevance scores of each database object, and finally determines the filtered subset of database objects from each database object based on the number of objects to be retained. This realizes the dynamic adjustment of the filtering granularity with the query difficulty, effectively improves the accuracy and robustness of database object filtering, and enhances the generalization ability under different database domains and retrieval models.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0012] Figure 1 A flowchart of a database object filtering method provided in an exemplary embodiment of this application is shown; Figure 2 A schematic diagram of an input interface module provided in an exemplary embodiment of this application is shown; Figure 3 A schematic diagram of a vector retrieval engine provided in an exemplary embodiment of this application is shown; Figure 4 A schematic diagram of a probability distribution calibrator provided in an exemplary embodiment of this application is shown; Figure 5 A schematic diagram of an exemplary dynamic pruning controller provided in this application is shown; Figure 6 This illustration shows a schematic diagram of the process for constructing a query statement provided by an exemplary embodiment of this application; Figure 7 A structural block diagram of a computer device illustrated in an exemplary embodiment of this application is shown; Figure 8 A structural block diagram of a computer device illustrated in another exemplary embodiment of this application is shown. Detailed Implementation

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0014] To address the issue that the retrieval granularity cannot be dynamically adjusted according to query complexity, this application provides a database object filtering method that can achieve uncertainty perception based on information entropy and dynamically generate filtering thresholds based on this. Figure 1 A flowchart illustrating a database object filtering method provided in an exemplary embodiment of this application is shown. This method can be executed by a computer device, which can be implemented as a server or a terminal device; Figure 1 As shown, the database object filtering method may include the following steps.

[0015] Step 110: Perform semantic relevance calculation on the query request and each database object to obtain the original relevance score of each database object.

[0016] The query request can be natural language query text input by the user through an interactive interface. In one possible implementation, after receiving the user's query request, the computer device can first perform text preprocessing on the query request to eliminate noise information in the original text and obtain a standardized query request. Text preprocessing can include, but is not limited to, word segmentation, stop word removal, abbreviation removal, character normalization, etc. Word segmentation refers to splitting compound words into independent word units according to preset splitting rules corresponding to the naming format. For example, underscore names can be split using underscore characters as boundaries, such as splitting user_id into user and id; camelCase names can be split based on capitalization to identify word boundaries and use uppercase letters as the starting identifier of new words, such as splitting userName into user and name. Different naming formats of compound words can have different splitting rules, and this application embodiment does not limit this. Stop word removal refers to filtering out preset stop words from the text to remove words with low semantic contribution. Stop words can be selected based on word frequency statistics, terminology coverage, etc. The rate or domain knowledge determines the process; deabbreviation processing refers to optionally using a preset abbreviation-full name mapping relationship to restore database abbreviations to complete words. This mapping table can be pre-configured based on domain standard terminology, database naming conventions, or historical naming habits; character normalization refers to converting characters in the text into standard forms according to preset conversion rules, such as converting characters to uppercase or lowercase, and uniformly formatting non-alphanumeric characters such as numbers and special symbols; in addition, in text preprocessing, entities and time expressions in the query request can also be identified and normalized according to predefined entity types and event format templates to ensure the standardization and consistency of subsequent semantic parsing.

[0017] In one possible implementation, the computer device may include an input interface module, which serves as an input interface and can be connected to a user terminal via a network interface to receive query requests input by the user through the user terminal. Figure 2 A schematic diagram of an input interface module provided in an exemplary embodiment of this application is shown, such as... Figure 2As shown, the input interface module 210 includes a network interface layer and a text preprocessing unit. The network interface layer is used to establish a communication connection with the user terminal, receive natural language query requests input by the user through the interactive interface, and transmit the received query requests to the text preprocessing unit. The network interface layer can support multiple communication protocols, including but not limited to HTTP (HyperText Transfer Protocol) and gRPC (gRPC Remote Procedure Call), to adapt to the data transmission needs of different deployment scenarios. The text preprocessing unit is used to perform text cleaning and normalization processing on the original query requests transmitted by the network interface layer to obtain standardized query requests.

[0018] When calculating the semantic relevance between a query request and various database objects, a computer device can convert the query request into a query vector, and then calculate the similarity between the query vector and each database object as the original relevance score for each database object. In one possible implementation, the computer device can call a pre-trained dual-tower coding model to vectorize the query request and each database object separately, obtaining query vectors and database object vector sets. Subsequently, by calculating the similarity between the query vector and each database object vector, the semantic relevance score between each database object and the query request is obtained, thus obtaining the original relevance score for each database object.

[0019] Before vectorizing database objects, to avoid differences in data formats, computer devices can first serialize each database object. During this process, the computer device can access the target database's metadata storage area through a database connection interface to extract full feature information representing the database's semantics, including table names, column names, data types, table and column comments, primary key identifiers, and foreign key constraints. Then, using a predetermined serialization template, the structured metadata is converted into a linear text string. For example, this serialization template can include at least one of the following: compact serialization, semantically enhanced serialization, and topology-aware serialization. Compact serialization refers to concatenating table names and column names to adapt to scenarios with limited context length, with the format [Table Name]. [Column Name], for example, users.phone_number. Semantically enhanced serialization refers to concatenating table names, column names, data types, and comment information into a sentence with complete semantics, with the format Table: [Table Name] | Column: [Column Name] | Type: [Data Type] | Content: [Comment], for example, Table: employees | Column: dob | Type: date | Content: date of birth of the employee; Topology-aware serialization refers to explicitly describing foreign key relationships as text, with the format Column [Column Name] in Table [Table Name] is a foreign key referring to [Target Table].[Target Column].

[0020] The dual-tower encoding model can contain two logically independent encoders: a query encoder and a pattern encoder. The parameters of the two encoders can be shared or not. The encoding region can be a pre-trained language model based on the Transformer architecture, such as BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly Optimized BERT Approach), or DeBERTa (Decoding-enhanced BERT with Disentangled Attention), or a retrieval-specific model fine-tuned through contrastive learning, such as BGE (BAAI General Embedding) or Contriever (Contrastive Retriever). During the encoding process, the serialized database object and the query request are input into the corresponding encoders, and the model converts the text into a token ID sequence and adds special markers (such as [CLS]). After multi-layer self-attention computation, a fixed-dimensional output vector is extracted through a pooling strategy. The pooling strategy can adopt either of the following methods: CLSPooling, which takes the last hidden state vector corresponding to the special tag [CLS]; or Mean Pooling, which takes the arithmetic mean of all output token vectors. For example, if the encoder function is f... θ If the vector dimension is d, the query vector can be represented as: Database object vectors can be represented as .

[0021] The vectorization process of query requests and database objects can be performed synchronously, i.e., vectorizing the query request and database objects after receiving the query request; or, in another possible implementation, to reduce online computation latency and improve response efficiency, the vectorization process of query requests and database objects can be performed asynchronously. In this case, the vectorization process of the query request is performed online, while the vectorization process of the database objects is performed offline. The pre-processed database object vectors are stored in an offline vector index library for relevance calculation. That is, during the offline indexing phase, when the system initializes or the database structure changes, all database objects (c1, ..., c...) in the database are vectorized. N Perform the above extraction, serialization, and vectorization operations to generate a vector set {h}. c1 , ..., hcN The data is stored in an offline vector index library; during the online inference phase, upon receiving a query request, the query vector of the query request is calculated in real time, and the database object in the offline index library is called to perform semantic relevance calculation.

[0022] In another possible implementation, the computer device may include a vector retrieval engine, which may include an online encoder and a similarity calculation unit. Figure 3 A schematic diagram of a vector retrieval engine provided in an exemplary embodiment of this application is shown, such as... Figure 3 As shown, the input end of the vector retrieval engine 310 is connected to the input interface model to receive query requests output by the input interface model; the online encoder is used to convert the received query request into a query vector; the similarity calculation unit is used to calculate the similarity between the query vector and the database object vectors of each database object stored in the offline vector index library; in the online processing stage, after receiving the query request, the computer device encodes the query request into a query vector in real time through the online encoder, and the similarity calculation unit uses the query vector to perform similarity retrieval in the offline vector index library, calculates the similarity between the query vector and the vectors of each database object, and obtains the original relevance score of each database object.

[0023] When performing semantic relevance calculations, computer devices can use a variety of optional metrics to calculate similarity, including cosine similarity, dot product similarity, and Euclidean distance.

[0024] Cosine similarity measures the similarity between two vectors by calculating the cosine of the angle between them. The formula for calculating cosine similarity can be expressed as:

[0025] Where · represents the dot product operation, Let L2 norm (i.e., Euclidean norm) represent the vector, and the output score ranges from [-1, 1]. Represents the query vector. Represents a vector of database objects.

[0026] Dot product similarity refers to the sum of the products of corresponding dimensions of two vectors after L2 normalization of the query vector and the database object vector, to measure their alignment in the vector space. For example, the formula for calculating dot product similarity can be expressed as:

[0027] After L2 normalization, the vector magnitude is 1. The dot product result directly reflects the directional consistency between the two vectors. The larger the value, the closer the two vectors are in space and the higher the semantic relevance. It is suitable for fast calculation in large-scale data scenarios.

[0028] Euclidean distance measures how close the query vector and the database object vector are in the vector space by calculating the Euclidean distance between them and converting the distance value into a similarity score. For example, the formula for calculating Euclidean distance can be expressed as:

[0029] The smaller the distance value, the closer the two vectors are in space and the higher their semantic similarity. After calculating the Euclidean distance, it can be mapped to similarity for easier comparison. For example, it can be converted by taking the reciprocal or using the Gaussian kernel function.

[0030] In one possible implementation, to improve the computational efficiency of semantic relevance, this embodiment employs matrix operations for parallel computation. During matrix operations, firstly, the vectors of each database object are stacked into a database object matrix, represented as follows: Here, the i-th row corresponds to the i-th database object vector; then, a broadcast operation is performed using the query vector, that is, matrix multiplication or broadcast dot product operation is performed between the query vector and the database object matrix. Taking matrix multiplication as an example, the calculation formula can be expressed as:

[0031] The result of the calculation is a score component with an N×1 dimension, represented as follows: Each element corresponds to the original relevance score of a database object. This parallel computation method effectively improves data processing efficiency in large database scenarios.

[0032] After calculating the raw relevance scores for each database object, the computer device can bind each score to the corresponding database object to construct a set of raw relevance scores, represented as: The raw relevance score reflects the semantic similarity between a database object and a query request; a higher score indicates a stronger relevance. For example, if the query request is "query user's start date," the raw relevance scores for the corresponding database objects are as follows: (table: managers, col: name) -> score: 0.75, indicating that the field `managers.Name` has a certain semantic relationship with "user," but not directly with "start date," with a score of 0.75; (table: managers, col: hire_date) -> score: 0.92, indicating that the field `managers.hire_date` is semantically related to both "user" and "start date," with a score of 0.92; (table: products, col: price) -> score: 0.12, indicating that the field `products.Price` has no obvious semantic relationship with the query intent, with a score of 0.12. As can be seen from the above examples, database objects semantically related to the query request have higher scores, while irrelevant objects have lower scores.

[0033] Step 120: Determine the number of objects to be retained based on the probability distribution statistical characteristics corresponding to the original relevance scores of each database object.

[0034] After obtaining the original relevance scores of each database object, the computer device can normalize these scores, mapping them to the [0, 1] interval. Based on the confidence probability values ​​obtained from this mapping, the number of objects to be retained can be dynamically determined. Different probability distribution statistical characteristics correspond to different object retention numbers, and the object retention number changes with the probability distribution. Therefore, based on the probability distribution statistical characteristics corresponding to the original relevance scores of each database object, the number of objects to be retained is determined, including: The original relevance scores are subjected to probabilistic mapping to obtain the confidence probability value of each database object. The number of objects to retain is determined based on the probability distribution of the confidence probability values ​​of each database object.

[0035] In other words, after obtaining the corresponding confidence probability values ​​of each database object and constructing the probability distribution of the scores, the computer device can calculate the information entropy of this probability distribution. The entropy value measures the dispersion of the score distribution. A larger entropy value indicates a more uniform score distribution, with smaller differences in scores among databases, suggesting that the semantics of the current query request are relatively ambiguous, and multiple database objects may be relevant to the query request. A smaller entropy value indicates a more concentrated score distribution, with a few database objects scoring significantly higher than others, suggesting that the semantics of the current query are relatively clear. For example, if the probability distribution exhibits a peaked distribution, i.e., p... max Close to 1, the rest p i A probability distribution close to 0 indicates that the semantics of the current query request are very clear, and only a very small number of database objects are related to the query request. At this time, it is in a low-entropy state, and fewer objects should be retained to reduce noise interference. If the probability distribution is flat, that is, the top-ranked confidence values ​​are very close, for example, p1 is 0.15, p2 is 0.15, and p3 is 0.15, it indicates that the current semantics are relatively ambiguous, and multiple database objects may be related to the query request. At this time, it is in a high-entropy state, and more objects should be retained to avoid missed recall.

[0036] Based on the magnitude of information entropy, computer devices can dynamically determine the number of objects to retain. When the entropy value is large, more database objects are retained to avoid missed recalls; when the entropy value is small, fewer objects are retained to reduce noise interference.

[0037] In one possible implementation, the computer device can be configured with a preset probability value to control the coverage of the screening. This preset probability value can be set according to the actual application scenario; for example, if the preset probability value is set to 0.9, it indicates that a total confidence level of 90% is expected to be covered. In this case, based on the probability distribution of the confidence probability values ​​of each database object, the number of objects to be retained is determined, including: The confidence probability values ​​are accumulated sequentially in descending order until the accumulation ends, which includes the first time the preset probability value is reached or exceeded. The number of accumulated terms when the accumulation termination condition is met is determined as the number of items to be retained in the object.

[0038] The computer device can sort the database objects based on their confidence probability values, such as in descending or ascending order. Taking descending order as an example, this yields an ordered index sequence π. Within this ordered index sequence... , where π(1) represents the index of the database object with the highest probability, and π(N) represents the index of the database object with the lowest probability.

[0039] Then, the computer device sequentially accumulates the probability values ​​in the ordered index sequence until the accumulation termination condition first reaches or exceeds the preset probability threshold. The accumulation position at this point is the minimum truncation position k. * k * This indicates the optimal context window size required for the current query, i.e., the number of objects to retain; schematically, this process can be represented as:

[0040] Furthermore, in another possible implementation, the computer device can determine the number of objects to retain by calculating a sparsity index of the probability distribution. In this case, the number of objects to retain is determined based on the probability distribution of the confidence probability values ​​of each database object, including: The sparsity index of the probability distribution is calculated based on the confidence probability values ​​of each database object; Based on the sparsity index, the number of objects to be retained is determined by a preset mapping function.

[0041] The sparsity index measures the concentration of the probability distribution. In one possible implementation, the sparsity index can be the information entropy mentioned above; in another, it can be the Gini coefficient or the kurtosis index. The preset mapping function maps the sparsity index to the corresponding number of objects retained. This mapping relationship can be set according to actual needs or obtained through statistical learning. This application does not limit the mapping relationship. Illustratively, when the information entropy is below a first threshold, the number of objects retained is a first preset value; when the information entropy is above a second threshold, the number of objects retained is a second preset value; when the information entropy is between the first and second thresholds, the number of objects retained increases linearly with the increase of the entropy value, and so on.

[0042] After calculating the original relevance scores of each database object, due to the potential isotropic phenomenon in deep learning vector models, the original relevance scores may be concentrated in a narrow high-score range. Therefore, to effectively distinguish the degree of relevance to the query request, this embodiment of the application can enhance the contrast of the original relevance during the determination of probability distribution statistical features to sharpen high-confidence terms and suppress noise terms, converting the original relevance scores into a statistically significant posterior probability distribution, laying the foundation for subsequent dynamic screening. Based on this, probabilistic mapping processing is performed on each original relevance score to obtain the confidence probability value of each database object, including: The original relevance scores of each database object are contrast-enhanced to obtain the enhanced relevance scores of each database object. Probabilistic mapping is performed based on the enhanced relevance scores of each database object to obtain the confidence probability value of each database object; the probabilistic mapping process includes one of the following: probability distribution standardization mapping, independent confidence calculation mapping, or linear interval mapping.

[0043] Contrast enhancement can be used to amplify the differences between the original relevance scores. In one possible implementation, contrast enhancement can employ exponential scaling, i.e., performing an exponential operation on the original relevance scores. Where α is the scaling factor, the larger α is, the stronger the contrast enhancement effect; or, power function scaling: , where β is the power exponent and is greater than 1; or temperature scaling.

[0044] Taking temperature scaling as an example, the original relevance scores of each database object are contrast-enhanced to obtain the enhanced relevance scores of each database object, including: Based on a preset temperature coefficient, the original correlation scores of each database object are temperature scaled to obtain the enhanced correlation scores of each database object. The temperature coefficient is a positive number less than 1.

[0045] For example, the temperature coefficient can be set to a range of (0, 1], or more specifically, to [0.01, 0.1]. The formula for calculating temperature scaling can be expressed as:

[0046] τ is a temperature coefficient. When τ < 1, the absolute difference between the original correlation scores is scaled to 16 and 14, and the difference is expanded to 2. For example, the difference between the original correlation scores of 0.8 and 0.7 is 0.1. When τ = 0.05, the difference becomes 16 and 14, and the difference is expanded to 2.

[0047] After contrast enhancement, the enhanced relevance score for each database object is obtained. Then, through probabilistic mapping, the confidence probability value of each database object is obtained. The probability distribution standardization mapping can be achieved by mapping the enhanced relevance score to a probability distribution using the Softmax function. The Softmax function ensures that the sum of the probabilities of all database objects is 1. Normalization unifies the units of measurement, making the confidence scores of each database object relatively comparable; that is, the probability value reflects the degree of matching between each database object relative to the others. For example, the probability distribution standardization mapping can be expressed as:

[0048] Where N is the total number of database objects, exp(•) is an exponential function with base e, and p i This represents the confidence probability that the i-th database object is the target object, given a query request Q.

[0049] Independent confidence calculation mapping refers to performing probabilistic mapping independently for each database object, without requiring the sum of the confidence probabilities of all objects to be 1, and the confidence probabilities of different database objects do not affect each other; this independent confidence calculation mapping can be calculated using the Sigmoid function, and can be expressed as:

[0050] Linear interval mapping can refer to using a linear normalization method to map the enhanced relevance scores of each database object to the interval [0, 1], thereby obtaining the confidence probability value of each database object. Linear interval mapping can be expressed as:

[0051] Then, the process of determining the number of objects to be retained is carried out based on the confidence probability value of each database object, which will not be repeated here.

[0052] In one possible implementation, the computer device may include a probability distribution calibrator that performs probabilistic mapping on the received raw correlation scores via circuitry or logic units; for example, probabilistic mapping is performed after temperature scaling. Figure 4 A schematic diagram of a probability distribution calibrator provided in an exemplary embodiment of this application is shown, such as... Figure 4 As shown, the probability distribution calibrator 410 includes a contrast enhancement unit and a probabilistic mapping unit. The contrast enhancement unit is used to perform temperature scaling processing on the received original correlation score using a preset temperature parameter to obtain an enhanced correlation score. The probabilistic mapping unit can process the enhanced correlation score using any of the aforementioned probabilistic mapping processing methods to obtain the confidence probability value of each database object and generate a probability distribution.

[0053] Step 130: Based on the number of objects to be retained, determine the filtered subset of database objects from each database object.

[0054] Computer devices can extract objects from database objects based on the object retention count. Illustratively, based on the object retention count, a filtered subset of database objects is determined from each database object, including: Sort each database object in descending order of its confidence probability value to obtain a sequence of database objects; Extract the number of database objects to be retained sequentially from the sequence of database objects, and construct a subset of database objects.

[0055] In addition, in one possible implementation, if multiple database objects have the same confidence probability value and are at the truncation boundary, the multiple database objects can be retained or selected according to preset rules, such as according to the order of storage time, etc. This application embodiment does not limit this.

[0056] In one possible implementation, the computer device may include a dynamic pruning controller for performing the aforementioned process of determining the number of objects to retain and the database subset filtering process; for example, determining the number of objects to retain using a preset probability value. Figure 5 A schematic diagram of an exemplary dynamic pruning controller provided in this application is shown, such as... Figure 5 As shown, the dynamic pruning controller 510 may include a sorting unit, an accumulator, an accumulation end judgment unit, and an object extraction unit. The sorting unit sorts each database object in descending order based on its confidence probability value. The accumulator sequentially accumulates the input probabilities. For each accumulated item, the accumulation end judgment unit performs an accumulation end condition judgment. If the current accumulated value has not reached the preset probability value, it returns to the accumulator to continue accumulating the next item, repeating the process until the accumulated value reaches or exceeds the preset probability value, thus determining the number of objects to retain. The object extraction unit extracts a subset of the corresponding database objects based on this number of objects to retain.

[0057] To prevent extreme probability distributions from causing under-selection due to excessively low preset probability values ​​or object overflow due to excessively high preset probability values, this embodiment configures a constraint boundary value for the object retention quantity. This constraint boundary value includes a minimum retention quantity and a maximum retention quantity. The object retention quantity is adjusted based on the constraint boundary value, and a subset of database objects is determined based on the adjusted object retention quantity. In this case, determining the filtered subset of database objects from each database object based on the object retention quantity includes: The number of objects to be retained is adjusted based on the constraint boundary values ​​to obtain the adjusted number of objects to be retained. Based on the revised number of objects to retain, a filtered subset of database objects is determined from each database object.

[0058] For example, the process of adjusting the number of objects retained can be represented as follows:

[0059] in, Indicates the number of objects to retain after the correction. Indicates the minimum number to retain. Indicates the maximum number to retain.

[0060] By setting a minimum retention quantity, the number of objects retained can be adjusted to the minimum retention quantity if the number of objects is less than the minimum retention quantity, thus acting as a lower bound protection. This mechanism ensures that even with a very high confidence level in the current query request, at least a minimum number of database objects are retained to provide a certain degree of fault tolerance and prevent the omission of critical information due to a single object misjudgment. Conversely, by setting a maximum retention quantity, the number of objects retained can be adjusted to the maximum retention quantity if the number of objects is greater than the maximum retention quantity, acting as an upper bound circuit breaker. This mechanism limits the number of objects retained to the maximum retention quantity even with significant uncertainty in the current query request, preventing inference timeouts or exceeding the context window limit caused by subsequent input of large models. It should be noted that the values ​​of the minimum and maximum retention quantities can be configured based on actual needs or obtained statistically; this embodiment does not impose any restrictions on this.

[0061] After obtaining the subset of database objects determined through dynamic pruning based on the query request, the computer device can generate a query statement corresponding to the query request based on this subset. For example, the computer device can use the subset of database objects as context input to guide the large model in generating the SQL statement corresponding to the query request. Due to the adaptive filtering in the preceding steps, the constructed keywords have the dual advantages of high signal-to-noise ratio and low tokenization, meaning the context length of the input large model is controlled within a reasonable range, thereby improving the accuracy of SQL statement generation and inference efficiency.

[0062] For example, the process of constructing an SQL statement can be implemented as follows: A subset of database objects is restructured and serialized into a format easily understood by the large model; the serialized subset of database objects is combined with the query request to obtain a prompt text, which drives the large model to generate an executable structured query statement. This prompt text may include a role instruction area, a dynamic data area, and a task trigger area; the role instruction area is used to set the role of the large model and constrain the output format; the dynamic data area is filled with the subset of database objects; and the task trigger area contains query instructions and guiding words. Finally, the constructed prompt text is input into the large model to obtain the SQL statement output by the large model.

[0063] In addition, after generating the SQL statement, the computer device can execute the SQL statement in a sandbox environment. If an error occurs, the error message is fed back to the large model for self-correction, and the above process is repeated until an executable SQL statement is generated.

[0064] In summary, the database object filtering method provided in this application calculates the original relevance score by performing semantic relevance calculation on the query request and each database object, determines the number of objects to be retained based on the probability distribution statistical characteristics corresponding to the original relevance scores of each database object, and finally determines the filtered subset of database objects from each database object based on the number of objects to be retained. This achieves dynamic adjustment of the filtering granularity according to the query difficulty, effectively improves the accuracy and robustness of database object filtering, and enhances the generalization ability under different database domains and retrieval models.

[0065] Based on such Figure 1 The database object filtering method shown is as follows: Figure 6 The illustration shows a schematic diagram of the process for constructing a query statement provided in an exemplary embodiment of this application, such as... Figure 6 As shown, in the process of constructing the query statement, the computer device first preprocesses the received query request through the interface input module 610 to obtain a standardized query request; the vector retrieval engine 620 calculates the semantic relevance between the query request and each database object to obtain the original relevance score of each database object; the probability distribution calibrator 630 performs probability mapping processing after contrast enhancement of the original relevance score to obtain a probability distribution; the dynamic pruning controller 640 is used to dynamically determine the number of objects to retain according to the probability distribution, and to filter database objects to obtain a subset of database objects according to the number of objects to retain; the generative reasoning module 650 constructs an executable SQL statement based on the subset of database objects and the query request instruction large model. The generative reasoning module may include a prompt text assembler, a large language model, and a post-processing parser. The prompt text assembler is used to generate prompt text based on the subset of database objects and the query request, the large language model is used to generate an SQL statement based on the prompt text, and the post-processing parser is used to perform syntax checking and verification on the SQL statement to obtain an executable SQL statement. Furthermore, after receiving the SQL statement, the computer device can send the SQL statement back to the user terminal. The user can view the SQL statement through the user terminal and confirm, modify, or execute it directly according to actual needs. In this way, not only is the transparency of the query process improved, but the user's controllability and credibility over the output results are also enhanced.

[0066] Through the above process, a complete conversion process from natural language input to structured query statements was achieved. High-quality prompt text was constructed through adaptive filtering and dynamic pruning. Then, executable SQL statements were generated through a large model, which effectively reduced semantic deviations caused by redundant information interference or unclear context, and improved the accuracy and generation efficiency of query statements.

[0067] Figure 7A structural block diagram of a computer device 700 illustrated in an exemplary embodiment of this application is shown. This computer device can be implemented as a server as described above in this application. The computer device 700 includes a central processing unit 701, a system memory 704 including random access memory (RAM) 702 and read-only memory (ROM) 703, and a system bus 705 connecting the system memory 704 and the central processing unit 701. The computer device 700 also includes a mass storage device 706 for storing an operating system 709, application programs 710, and other program modules 711.

[0068] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 704 and mass storage device 706 described above can be collectively referred to as memory.

[0069] According to various embodiments of this application, the computer device 700 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 700 can be connected to a network 708 via a network interface unit 707 connected to the system bus 705, or the network interface unit 707 can be used to connect to other types of networks or remote computer systems (not shown).

[0070] The memory also includes at least one instruction, at least one program, code set, or instruction set, which are stored in the memory. The central processing unit 701 executes the at least one instruction, at least one program, code set, or instruction set to implement all or part of the steps in the database object filtering method shown in the above embodiments.

[0071] Figure 8 A structural block diagram of a computer device 800 illustrating another exemplary embodiment of this application is shown. This computer device 800 can be implemented as the aforementioned terminal device, such as a smartphone, tablet computer, laptop computer, desktop computer, etc. The computer device 800 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0072] Typically, computer device 800 includes a processor 801 and a memory 802.

[0073] In some embodiments, the computer device 800 may also optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 803 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, and a power supply 808.

[0074] In some embodiments, the computer device 800 further includes one or more sensors 809. The one or more sensors 809 include, but are not limited to, an accelerometer 810, a gyroscope 811, a pressure sensor 812, an optical sensor 813, and a proximity sensor 814.

[0075] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on the computer device 800, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0076] In one exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program, the at least one computer program being loaded and executed by a processor to perform the following: Figure 1 All or part of the steps in the illustrated embodiments.

[0077] In one exemplary embodiment, a computer program product is also provided, comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the above-described actions. Figure 1 All or part of the steps in the illustrated embodiments.

[0078] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0079] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for filtering database objects, characterized in that, The method includes: Semantic relevance calculation is performed between the query request and each database object to obtain the original relevance score of each database object; Based on the probability distribution statistical characteristics corresponding to the original relevance scores of each database object, the number of objects to be retained is determined; Based on the number of objects to be retained, a subset of filtered database objects is determined from each database object.

2. The method according to claim 1, characterized in that, The determination of the number of objects to be retained based on the probability distribution statistical characteristics corresponding to the original relevance scores of each database object includes: The original relevance scores are subjected to probabilistic mapping to obtain the confidence probability value of each database object. The number of objects to be retained is determined based on the probability distribution of the confidence probability values ​​of each database object.

3. The method according to claim 2, characterized in that, Determining the number of objects to retain based on the probability distribution of the confidence probability values ​​of each database object includes: The confidence probability values ​​are accumulated sequentially in descending order until the accumulation ends, including the first time the preset probability value is reached or exceeded. The number of accumulated terms when the accumulation termination condition is met is determined as the number of items to be retained for the object.

4. The method according to claim 2, characterized in that, The determination of the number of objects to be retained based on the probability distribution of the confidence probability values ​​of each database object includes: The sparsity index of the probability distribution is calculated based on the confidence probability values ​​of each database object; Based on the sparsity index, the number of objects to be retained is determined by a preset mapping function.

5. The method according to any one of claims 2 to 4, characterized in that, The step of determining a filtered subset of database objects from each database object based on the number of objects to be retained includes: The database objects are sorted in descending order according to their confidence probability values ​​to obtain a sequence of database objects; Extract the number of database objects to be retained from the sequence of database objects in order, and construct the subset of database objects.

6. The method according to claim 1, characterized in that, The object retention quantity is configured with constraint boundary values, including a minimum retention quantity and a maximum retention quantity; the step of determining a filtered subset of database objects from each database object based on the object retention quantity includes: The number of objects to be retained is corrected based on the constraint boundary values ​​to obtain the corrected number of objects to be retained. Based on the revised number of objects to be retained, a filtered subset of database objects is determined from each database object.

7. The method according to claim 2, characterized in that, The step of performing probabilistic mapping on each of the original relevance scores to obtain the confidence probability value of each database object includes: The original relevance scores of each database object are contrast-enhanced to obtain the enhanced relevance scores of each database object. Based on the enhanced relevance scores of each database object, a probabilistic mapping process is performed to obtain the confidence probability value of each database object; wherein, the probabilistic mapping process includes one of the following: probability distribution standardization mapping process, independent confidence calculation mapping process, or linear interval mapping process.

8. The method according to claim 7, characterized in that, The process of contrast-enhancing the original relevance scores of each database object to obtain enhanced relevance scores for each database object includes: The original relevance scores of each database object are temperature scaled based on a preset temperature coefficient to obtain the enhanced relevance scores of each database object. The temperature coefficient is a positive number less than 1.

9. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to implement the database object filtering method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the database object filtering method as described in any one of claims 1 to 8.

Images