Method for judging service type of unknown service database based on large model

Abstract

The invention relates to the field of business type judgment, in particular to a method for judging the business type of an unknown business database based on a large model, which comprises the following steps of: connecting the unknown business database through a standardized interface to obtain metadata and sample data; extracting multi-dimensional meta-information according to database types; performing semantic extension on the meta-information by utilizing a pre-training language model BERT; fusing multiple features through an attention mechanism to generate a fused feature, and outputting a service type reasoning prompt in combination with a historical reasoning result; after LLM multi-round reasoning, the matching degree is judged through comprehensive confidence, a confidence matrix is constructed, and a final service type is output; the method realizes automatic and high-precision judgment of the service type of the unknown service database, and is suitable for scenes of database management, data analysis and the like.

Description

Technical Field

[0001] This invention relates to the field of business type determination, specifically a method for determining the business type of an unknown business database based on a large model. Background Technology

[0002] With the widespread application of information technology, the scale and complexity of various business databases have increased significantly, making the demand for automated analysis of unknown business databases increasingly urgent. Traditional methods for determining business types mainly rely on manual parsing of database metadata, which has the following significant drawbacks:

[0003] Inefficient: Manual analysis is time-consuming, especially for complex databases where quick judgment is difficult; Inaccurate: It relies on the experience of analysts, is easily affected by subjective factors, and is difficult to handle issues such as missing metadata, abbreviations, or semantic ambiguity; Lack of intelligent means: Traditional methods do not fully utilize artificial intelligence technology and cannot achieve deep semantic understanding and reasoning of database features.

[0004] Large Language Models (LLMs) have demonstrated powerful capabilities in semantic understanding and reasoning, providing a new direction for automated database analysis. However, existing technologies still face key challenges when applying large models to business type judgment: the feature fusion mechanism is simple and struggles to capture the comprehensive business characteristics of the database; the reasoning process is uncontrollable: the lack of dynamic prompting engineering and multi-round reasoning mechanisms makes it impossible to iteratively optimize the judgment logic based on intermediate results, leading to insufficient reasoning accuracy in complex scenarios. Therefore, a method based on large models for business type judgment of unknown business databases is needed to solve these problems. Summary of the Invention

[0005] To address the technical problems raised in the background, this invention provides a method for determining the business type of an unknown business database based on a large model.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] This invention provides the following technical solution: a method for determining the business type of an unknown business database based on a large model, comprising:

[0008] Step F101: Establish a connection with an unknown business database through a standardized interface. Relational and non-relational databases are supported. Connection parameters include database address, authentication information, and database type identifier. Authentication information consists of username and password.

[0009] Step F102: Extract multidimensional metadata from relational databases and non-relational databases respectively;

[0010] Specifically, the relational database extraction process is as follows: Table names, table comments, and record counts are obtained through system table queries; field names, lengths, nullability, and field comments are queried; then, inter-table relationships are extracted through foreign key constraint tables. This yields the number of tables and the depth of relationships, which are then integrated into the first feature set F. sem ;

[0011] The extraction process for a non-relational database is as follows: obtain the collection name and the number of documents, iterate through the document samples in the collection, and count the frequency (TR) of a field. The frequency formula is expressed as: The data types of each field are extracted, including text, numeric, and date. Business keywords are extracted from text fields and associated with business scenarios. The word segmentation tool used is jieba, and entities in the text are identified and extracted using a pre-defined entity recognition model. The numerical meanings in numeric fields are obtained through statistical features, including mean, variance, and extreme value range. These statistical features are matched with business rules to obtain the business numerical meanings. The business rules are as follows: If the value is a positive integer and the variance is less than β², it is classified as a quantity. Time patterns are identified from date fields, specifically by determining the time granularity through field format. Table count, field frequency, statistical features, and time granularity are integrated into a second feature set F. date .

[0012] Step F103: Semantically expand the extracted meta-information using the pre-trained language model BERT to generate a complete business semantic description;

[0013] Specifically, the extracted meta-information is divided and meaningless stop words are removed to obtain a lexical set Q = {q1, q2, ..., q}. n}, where n represents the total number of lexical units. The lexical set is input into a pre-trained language model, and context encoding is performed on the lexical set to generate word vector representations. The encoding formula is: h = BERT([CLS]|q1、q2.....,q n |), where h is the hidden state matrix containing all lexical context information, ([CLS]|q1、q2.....,q n |) represents a sequence of lexical units; then, for abbreviations or semantic gaps in the lexical units, the semantics are generated by matching word vector similarity with a standard business terminology database. Specifically, the standard business terminology database F = {f1, f2, ..., f...} is set up. k}, k is the total number of terms, and each term f j Corresponding to a standard word vector emb(f) j The standard word vectors are generated by BERT pre-training; the cosine similarity Sim(q,f) between the hidden state of a word and all terms in the terminology database is obtained. jThe closest term is selected as the completion result, and the similarity calculation logic is as follows: The completion rule is: if Sim(q,f) j If q ≥ 0.7, then q is completed as f. j Conversely, the original word elements are retained and marked as requiring manual confirmation;

[0014] The latent state h of the lexical, the table relationship α, and the data type μ of the field are concatenated to obtain the candidate fusion features. Its calculation logic is as follows: Where emb(α) is the vectorized representation of the association relationship, and emb(μ) is the one-hot encoding of the data type; the generative language model BART is used to decode the second feature set to generate a business scenario, and its input formula for the generative model is expressed as:

[0015] Input = Field name: q; Data type: μ; Table join: α; Inferring the business scenario, a fixed-dimensional semantic vector is generated using a large language model (LLM), which is then used as the third feature set F. struct .

[0016] Step F104: Generate fused features from the first feature set and the second feature set of the business database through the attention mechanism, and then output business type reasoning hints by combining the historical reasoning results.

[0017] Specifically, the first, second, and third feature sets are projected onto the same dimension to ensure that the attention score can be calculated. Then, the attention score A is calculated using the first feature set as the query, the second feature set as the key, and the third feature set as the value. ttention (F sem The calculation logic for L, M) is as follows: in Let L be the normalization factor, Softmax be a function, and L = [F] sem ,F struct ], [F struct ,F sem Attention scores are calculated in parallel using multiple attention heads, and then concatenated and projected to obtain the fused feature F. fusion ;

[0018] Extract historical reasoning records from the server, including previous reasoning results and context, and represent them in a structured form as H = {(J1,A1),(J2,A2),........(J g A g )}, where J m For the prompt question in the m-th round, A mThe output of the LLM is denoted by g, which represents the number of rounds in the historical dialogue, i.e., the number of reasoning interactions performed. The fusion features and historical reasoning records are input into the LLM. The LLM includes a prompt generation layer that generates natural language descriptions based on the fusion features. The level of detail in the description is dynamically adjusted by the highest confidence level of the previous round of reasoning, expressed by the formula: The detailed description includes specific indicators of the first feature, second feature, and third feature set, and the concise description is set to reserved keywords to generate business type reasoning prompts.

[0019] Step F105: Determine the accuracy of the business type judgment through LLM multi-round inference. The input of each round of inference is the business type inference hint, and the output is the candidate business type, confidence level and inference basis.

[0020] Specifically, the LLM is configured with multiple inference layers. Business type inference hints are input into the initial inference round. The output of the initial inference round is represented as Output = {candidate type list, original confidence list, inference basis}. The candidate type list is a list of candidates for business database type judgment, arranged in descending order of confidence. The original confidence list is the original confidence of the corresponding type. The inference basis is the inference hints of business database features. If the original confidence of the initial inference round is less than a preset threshold, an iterative correction round is triggered, and new features with acquired meta-information are added. The iterative correction round adjusts the confidence of the candidate types through the new and old features. The correlation between the corrected confidence and the new features is positive. If the confidence of the output of the iterative round is greater than or equal to the preset threshold or the maximum number of iterations is reached, the inference ends, and the corrected candidate type list is obtained.

[0021] Step F106: Determine the degree of matching between the business database and the candidate class selection list by comprehensive confidence score, construct a confidence score matrix, and output the final business type;

[0022] Specifically, the original confidence scores l of each type are transformed into probability-based distributed confidence scores C using the Softmax function. LLM The formula is expressed as: Among them l ε The original confidence level for the ε-th business type output by LLM. δ represents the total number of candidate business types, and δ represents the number of the candidate business type; the fusion feature F of the business database is used to... fusion The feature similarity confidence level C is obtained by calculating the cosine similarity with the standard feature vectors in the standard business terminology library. sim The overall confidence level C is obtained by merging the two confidence levels through a weighted average. final Semantic reasoning that prioritizes LLM is expressed as follows: in The weight is fixed at 0.68; the rows are database features, the columns are business types, and the cell value is the overall confidence score C.final Construct a confidence matrix; obtain the highest confidence level within the confidence matrix, and determine the business database type corresponding to the highest confidence level as the final type.

[0023] Compared with the prior art, the beneficial effects of the present invention are: Confidence verification: LLM optimizes the judgment step by step through multiple rounds of inference, initial inference, iterative correction, and result convergence. It dynamically adjusts the confidence level by combining historical dialogues and newly added features to reduce the impact of hallucination problems. It calculates the confidence level by combining the probability distribution of LLM and feature similarity, constructs a confidence matrix to quantify the degree of matching, and filters low-quality results through threshold rules to ensure the reliability of the judgment results.

[0024] Multi-dimensional feature fusion enhances semantic understanding: hierarchical extraction techniques extract meta-information, semantic meta-information, and data layer features, which are dynamically weighted and fused through an attention mechanism to form a comprehensive feature vector containing structure, semantics, and data. This avoids the one-sidedness of single-dimensional judgment. A pre-trained language model performs semantic completion and scene generation on the meta-information to solve abbreviation and ambiguity problems and improve the accuracy of semantic understanding. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The following drawings are not drawn to scale according to the actual size, but are intended to illustrate the main idea of ​​the present invention.

[0026] Figure 1 This is a diagram illustrating the method steps of the present invention. Detailed Implementation

[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are also within the scope of protection of the present invention. This application provides an efficient farmland irrigation method and system based on machine learning.

[0028] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the method for determining the business type of an unknown business database based on a large model in this application includes:

[0030] Step F101: Establish a connection with an unknown business database through a standardized interface. Relational and non-relational databases are supported. Connection parameters include database address, authentication information, and database type identifier. Authentication information consists of username and password.

[0031] Step F102: Extract multidimensional metadata from relational databases and non-relational databases respectively;

[0032] Specifically, the relational database extraction process is as follows: First, obtain table names, table comments, and record counts by querying the system tables. Then, query field names, lengths, nullability, and field comments. Next, extract inter-table relationships through foreign key constraints. It should be noted that the system tables are special tables in the database used to store metadata. The table name is the naming identifier for the table in the database, the field name is the naming identifier for the field in the table, and the relationships are formed by foreign key constraints. This process is used to obtain the number of tables and the depth of relationships, which are then integrated into the first feature set F. sem ;

[0033] The extraction process for a non-relational database is as follows: obtain the collection name and the number of documents, iterate through the document samples in the collection, and count the frequency (TR) of a field. The frequency formula is expressed as: The data types of each field are extracted, including text, numeric, and date. Business keywords are extracted from text fields and associated with business scenarios, such as extracting fields like "complaints" and "returns" and associating them with after-sales service. The word segmentation tool used is jieba, and a preset entity recognition model is used to identify and extract entities from the text. The numerical meanings of numeric fields, such as amount and quantity, are obtained through statistical features, including mean, variance, and extreme value range. These statistical features are matched with business rules to obtain the business numerical meanings. The business rules are as follows: If the value is a positive integer and the variance is less than β², it is classified as a quantity. Time patterns are identified from date fields, specifically by determining the time granularity through the field format, such as year, day, hour, or minute. The table count, field frequency, statistical features, and time granularity are then integrated into a second feature set F. date .

[0034] Step F103: Use the pre-trained language model BERT to semantically expand the extracted meta-information, solve the problems of missing, ambiguous and abbreviated meta-information, and generate a complete business semantic description.

[0035] Specifically, the extracted meta-information is divided and meaningless stop words are removed to obtain a lexical set Q = {q1, q2, ..., q}. n}, where n represents the total number of lexical units. The lexical set is input into a pre-trained language model, and context encoding is performed on the lexical set to generate word vector representations. The encoding formula is: h = BERT([CLS]|q1、q2.....,q n |), where h is the hidden state matrix containing all lexical context information, ([CLS]|q1、q2.....,q n |) represents a sequence of lexical units; then, for abbreviations or semantic gaps in the lexical units, the semantics are generated by matching word vector similarity with a standard business terminology database. Specifically, the standard business terminology database F = {f1, f2, ..., f...} is set up. k}, k is the total number of terms, and each term f j Corresponding to a standard word vector emb(f) j The standard word vectors are generated by BERT pre-training; the cosine similarity Sim(q,f) between the hidden state of a word and all terms in the terminology database is obtained. j The closest term is selected as the completion result, and the similarity calculation logic is as follows: The completion rule is: if Sim(q,f) j If q ≥ 0.7, then q is completed as f. j Conversely, the original word elements are retained and marked as requiring manual confirmation;

[0036] The latent state h of the lexical, the table relationship α, and the data type μ of the field are concatenated to obtain the candidate fusion features. Its calculation logic is as follows: Here, emb(α) is the vectorized representation of the association relationship, and emb(μ) is the one-hot encoding of the data type. The BART generative language model is used to decode the second feature set to generate a business scenario. The input formula for this generative model is: Input = Field Name: q; Data Type: μ; Table Association: α. The business scenario is inferred by generating a fixed-dimensional semantic vector using the Large Language Model (LLM), which serves as the third feature set F. struct .

[0037] Step F104: Generate fused features from the first feature set and the second feature set of the business database through the attention mechanism, and then output business type reasoning hints by combining the historical reasoning results.

[0038] Specifically, the first, second, and third feature sets are projected onto the same dimension to ensure that the attention score can be calculated. Then, the attention score A is calculated using the first feature set as the query, the second feature set as the key, and the third feature set as the value. ttention (F sem The calculation logic for L, M) is as follows: in Let L be the normalization factor, Softmax be a function, and L = [F] sem ,F struct ], [F struct ,F sem Attention scores are calculated in parallel using multiple attention heads, and then concatenated and projected to obtain the fused feature F. fusion ;

[0039] Extract historical reasoning records from the server, including previous reasoning results and context, and represent them in a structured form as H = {(J1,A1),(J2,A2),........(J g A g )}, where J m For the prompt question in the m-th round, A m The output of the LLM is denoted by g, which represents the number of rounds in the historical dialogue, i.e., the number of reasoning interactions performed. The fusion features and historical reasoning records are input into the LLM. The LLM includes a prompt generation layer that generates natural language descriptions based on the fusion features. The level of detail in the description is dynamically adjusted by the highest confidence level of the previous round of reasoning, expressed by the formula: Where Cprev represents the confidence matrix or confidence value of a certain business type obtained in the previous round of calculation. The detailed description includes specific indicators of the first feature, second feature and third feature set. The concise description is set as reserved keywords, thereby generating business type reasoning prompts.

[0040] Step F105: Determine the accuracy of the business type judgment through LLM multi-round inference. The input of each round of inference is the business type inference hint, and the output is the candidate business type, confidence level and inference basis.

[0041] Specifically, the LLM is configured with multiple inference layers. Business type inference hints are input into the initial inference round. The output of the initial inference round is represented as Output = {candidate type list, original confidence list, inference basis}. The candidate type list is a list of candidates for business database type judgment. Business database types include e-commerce, finance, medical and education, etc., and are arranged in descending order of confidence. The original confidence list is the original confidence of the corresponding type. The inference basis is the inference hints of business database features. If the original confidence of the initial inference round is less than a preset threshold, an iterative correction round is triggered, and new features with acquired meta-information are added. The iterative correction round adjusts the confidence of the candidate types through the new and old features. The correlation between the corrected confidence and the new features is positive. If the confidence of the output of the iterative round is greater than or equal to the preset threshold or the maximum number of iterations is reached, the inference ends, and the corrected candidate type list is obtained.

[0042] Step F106: Determine the degree of matching between the business database and the candidate class selection list by comprehensive confidence score, construct a confidence score matrix, and output the final business type;

[0043] Specifically, the original confidence scores l of each type are transformed into probability-based distributed confidence scores C using the Softmax function. LLM The formula is expressed as: Among them l ε The original confidence level for the ε-th business type output by LLM. δ represents the total number of candidate business types, and δ represents the number of the candidate business type; the fusion feature F of the business database is used to... fusion The feature similarity confidence level C is obtained by calculating the cosine similarity with the standard feature vectors in the standard business terminology library. sim The overall confidence level C is obtained by merging the two confidence levels through a weighted average. final Semantic reasoning that prioritizes LLM is expressed as follows: in The weight is fixed at 0.68; the rows are database features, the columns are business types, and the cell value is the overall confidence score C. final Construct a confidence matrix to visualize the degree of matching across multiple dimensions; obtain the highest confidence level within the confidence matrix and determine the business database type corresponding to the highest confidence level as the final type.

[0044] The foregoing description is illustrative of the invention and should not be construed as limiting it. Although several exemplary embodiments of the invention have been described, those skilled in the art will readily understand that many modifications can be made to the exemplary embodiments without departing from the novel teachings and advantages of the invention. Therefore, all such modifications are intended to be included within the scope of the invention as defined in the claims. It should be understood that the foregoing description is illustrative of the invention and should not be construed as limiting it to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The invention is defined by the claims and their equivalents.

Claims

1. A method for determining the business type of an unknown business database based on a large model, characterized in that, include: Establish connections to unknown business databases through standardized interfaces, supporting both relational and non-relational databases. Connection parameters include database address, authentication information, and database type identifier. Based on the different categories of relational and non-relational databases, multidimensional metadata is extracted from them respectively; The extracted meta-information is semantically expanded by the pre-trained language model BERT to generate a complete business semantic description. The first feature set, the second feature set, and the third feature set of the business database are used to generate fused features through an attention mechanism, and then combined with historical inference results to output business type inference hints. The degree of matching between the business database and the candidate class selection list is determined by the comprehensive confidence score, a confidence score matrix is ​​constructed, and the final business type is output.

2. The method for determining the business type of an unknown business database based on a large model according to claim 1, characterized in that, The degree of matching between the business database and the candidate class selection list is determined by comprehensive confidence score, and a confidence score matrix is ​​constructed to output the final business type, specifically as follows: The original confidence scores of each type are transformed into probability-based distributed confidence scores using the Softmax function; the cosine similarity of the fused features of the business database and the standard feature vectors in the standard business terminology library is used to calculate the feature similarity confidence score, and the two confidence scores are fused by weighted average to obtain the comprehensive confidence score, which prioritizes semantic reasoning based on LLM; a confidence matrix is ​​constructed with database features as rows, business types as columns, and cell values ​​as the comprehensive confidence score; the highest confidence score is obtained within the confidence matrix, and the business database type corresponding to the highest confidence score is determined as the final type.

3. The method for determining the business type of an unknown business database based on a large model according to claim 1, characterized in that, The accuracy of business type judgment is determined through multi-round inference using LLM. The input for each round of inference is a business type inference hint, and the output is a candidate business type, confidence level, and inference basis. Specifically: LLM sets up multiple inference layers. Business type inference hints are input into the initial inference wheel. The output of the initial inference wheel is represented as Output = {candidate type list, original confidence list, inference basis}. The candidate type list is the candidate list for business database type judgment, which is arranged in descending order of confidence. The original confidence list is the original confidence of the corresponding type. The inference basis is the inference hint of business database features. If the initial confidence level of the initial inference round is less than the preset threshold, an iterative correction round is triggered, and new features with acquired meta-information are added. The iterative correction round adjusts the confidence level of the candidate types using the new and old features. The corrected confidence level is positively correlated with the new features. If the confidence level output by the iterative round is greater than or equal to the preset threshold or the maximum number of iterations is reached, the inference ends, and a corrected candidate type list is obtained.

4. The method for determining the business type of an unknown business database based on a large model according to claim 1, characterized in that, The extracted meta-information is semantically expanded by the pre-trained language model BERT to generate a complete business semantic description. The extracted meta-information is divided and meaningless stop words are removed to obtain a word set. This word set is then input into a pre-trained language model, where context encoding is performed to generate word vector representations. Each word vector contains a latent state matrix containing the context information of all word elements. The semantics are then completed by matching the word vectors against a standard business terminology database. Specifically, a standard business terminology database is set up, with each term corresponding to a standard word vector, which is generated by BERT pre-training. The cosine similarity between the latent state of a word element and all terms in the terminology database is obtained, and the closest term is selected as the completion result. By concatenating features such as latent state of lexical elements, table relationships, and field data types, candidate fusion features are obtained. The second feature set is decoded using the generative language model BART to generate a business scenario. A fixed-dimensional semantic vector is then generated using the large language model LLM, which serves as the third feature set.

5. The method for determining the business type of an unknown business database based on a large model according to claim 1, characterized in that, The attention mechanism is used to generate fused features from multiple sets of features in the business database, and combined with historical inference results to output business type inference hints, specifically: The first, second, and third feature sets are projected onto the same dimension to ensure that attention scores can be calculated. Then, the first feature set is used as the query, the second feature set as the key, and the third feature set as the value to calculate the attention score. The attention scores are calculated in parallel by multiple attention heads, and finally the fused features are concatenated and projected to obtain the fused features. Extract historical reasoning records from the server, including previous reasoning results and context. The structured representation is as follows: input the fusion features and historical reasoning records into the LLM. The LLM has a prompt generation layer that generates natural language descriptions based on the fusion features. The level of detail in the description is dynamically adjusted by the highest confidence level of the previous round of reasoning. The detailed description includes specific indicators of the first feature, second feature, and third feature set. The concise description is set to retain keywords. In this way, business type reasoning prompts are generated.

6. The method for determining the business type of an unknown business database based on a large model according to claim 1, characterized in that, Based on the different categories of relational and non-relational databases, multidimensional metadata is extracted separately, specifically as follows: The relational database extraction process is as follows: obtain the table name, table comments and record count through system table query, query field name, length, whether it can be null and field comments, and then extract the relationship between tables through foreign key constraint table, thereby obtaining the number of tables and the relationship depth and integrating them into the first feature set; The extraction process for a non-relational database is as follows: Obtain the collection name and document count; iterate through the document samples in the collection and count the frequency of each field; extract the data type of each field (text, numeric, and date types); extract business keywords from text fields and associate them with business scenarios; use jieba as the word segmentation tool; then identify and extract entities from the text using a pre-defined entity recognition model; obtain the numerical meaning of numeric fields through statistical features, including mean, variance, and extreme value range; match the statistical features with business rules to obtain the business numerical meaning. The business rules are... If the value is a positive integer and the variance is less than β2, it is classified as a quantity. Time patterns are identified from date fields, specifically by determining the time granularity through field format. The number of tables, field frequency, statistical features, and time granularity are integrated into a second feature set.

Images