Database semantic model automatic construction and dynamic updating method and system and medium

Abstract

This application belongs to the field of artificial intelligence and discloses a method, system, and medium for automatically constructing and dynamically updating a database semantic model. The method includes: establishing a mapping relationship between database tables and business entities as the first-stage semantic model; establishing an association relationship between database tables, database fields, and business semantics as the second-stage semantic model; optimizing the second-stage semantic model using a semantic knowledge base as the third-stage semantic model; collecting historical question-and-answer data, extracting semantic mapping relationships between natural language expressions and database fields from valid question-and-answer samples, and training a large language model; dynamically adjusting the confidence level of the semantic mapping relationships based on the frequency and accuracy of their use in historical question-and-answer data, and synchronizing the updated semantic mapping relationships to the semantic knowledge base. This method reduces modeling costs and improves the model's semantic understanding ability and accuracy.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence, and specifically relates to a method, system and medium for automatic construction and dynamic updating of database semantic models. Background Technology

[0002] To lower the barrier to data use, natural language query technology has gradually become an important interaction method for enterprise data analysis systems. Users can directly express their analysis needs through natural language, and the system will automatically generate corresponding data query statements to complete the access and analysis of database data.

[0003] In existing technologies, natural language query systems typically employ a conversion method from natural language to structured query language (SQL). This approach first performs semantic parsing on the user-input natural language question, and then generates the corresponding SQL query statement by combining it with database structure information. To achieve an accurate mapping between natural language expression and the underlying data structure, most systems need to build a semantic layer between the database structure and the natural language. This layer involves manually maintaining information such as field names, business semantics, synonyms, and indicator definitions to assist the natural language parsing process.

[0004] Chinese patent application CN115408506A discloses a method and system for converting natural language into SQL statements. By performing semantic parsing on the natural language input by the user and combining it with database table structure information for field matching, the method can automatically convert natural language questions into SQL query statements.

[0005] Chinese patent application CN113743142A discloses a database query method and apparatus based on natural language. By performing word segmentation, semantic understanding and database pattern matching on natural language questions, it realizes the conversion of user natural language query requests into database query statements.

[0006] Chinese patent application CN114156842A discloses a database query method and system based on semantic parsing. By performing semantic analysis on natural language questions and combining it with database schema information for matching, it realizes the conversion of natural language query statements into structured SQL statements.

[0007] While the aforementioned existing technologies can achieve automatic conversion from natural language to SQL statements to varying degrees, they still have the following technical limitations: (1) Semantic model construction is highly dependent on manual configuration. It usually requires data engineers or business experts to configure the relationship between database tables and business entities, as well as the mapping relationship between fields and business semantics. The construction cost is high. When the number of database tables and the scale of fields are large, the workload of semantic modeling increases significantly, resulting in a long system deployment cycle.

[0008] (2) When the database structure changes, such as adding a new database table, changing the field name, or adjusting the field semantics, the existing semantic model needs to be manually maintained. Otherwise, the natural language parsing results may be inconsistent with the actual database structure, thus affecting the accuracy of the query. Especially in scenarios where the database structure is frequently adjusted, the maintenance cost of the semantic model is high.

[0009] (3) There are differences in data structure and business semantics between different business systems. Existing semantic models are usually built for specific business systems or specific data domains, making it difficult to reuse them directly in different business scenarios, which increases the difficulty of deploying intelligent data query systems in multi-business system environments.

[0010] (4) The existing semantic model is a static configuration file. With business iteration and the diversification of user questioning methods, the accuracy of semantic parsing is difficult to improve independently, and the long-term user experience continues to be limited.

[0011] Therefore, there is an urgent need for a semantic modeling solution that can automatically construct semantic models, support dynamic and continuous updates, be reusable across business operations, and have self-learning capabilities. Summary of the Invention

[0012] To address the aforementioned issues, this application provides a method, system, and medium for automatically constructing and dynamically updating a database semantic model. This reduces modeling costs and continuously optimizes semantic mapping relationships based on database structure information and historical question-and-answer data, thereby improving the model's semantic understanding capabilities and accuracy.

[0013] The technical solution is as follows: A method for automatically constructing and dynamically updating a database semantic model includes: Obtain table metadata from the database, including table names, table comments, and inter-table relationships. Input the table metadata into a large language model for semantic parsing, identify business entities, and establish a mapping relationship between database tables and business entities, which serves as the first-stage semantic model. Read the field metadata of the target database table, including field name, field remarks and field data type. Input the metadata of each field and the business entity information of the database table to which the field belongs into the large language model for field semantic parsing, identify the business semantics of each field, and establish the association between database table, database field and business semantics as the second stage semantic model. The second-stage semantic model is matched with historical semantic mapping data stored in a pre-built semantic knowledge base, and the second-stage semantic model is optimized based on the matching results to obtain the third-stage semantic model. Collect historical question-and-answer data, extract the semantic mapping relationship between natural language expressions and database fields from valid question-and-answer samples, generate semantic learning samples, and use the semantic learning samples to train the large language model; Based on the frequency and accuracy of semantic mapping relationships in historical question-and-answer data, the confidence level of semantic mapping relationships is dynamically adjusted in the third-stage semantic model, and the updated semantic mapping relationships are synchronized to the semantic knowledge base.

[0014] A system for automatically constructing and dynamically updating a database semantic model, comprising: The database semantic parsing module is used to obtain table metadata of the database, including table names, table comments and inter-table relationships. The table metadata is input into the large language model for semantic parsing to identify business entities and establish a mapping relationship between database tables and business entities, which serves as the first stage of the semantic model. The field semantic extension module is used to read the field metadata of the target database table, including field name, field remarks and field data type. It inputs the metadata of each field and the business entity information of the database table to which the field belongs into the large language model for field semantic parsing, identifies the business semantics of each field, and establishes the association between the database table, the database field and the business semantics, as the second stage semantic model. The semantic model optimization module is used to match the second-stage semantic model with the historical semantic mapping data stored in the pre-built semantic knowledge base, and optimize the second-stage semantic model according to the matching results to obtain the third-stage semantic model. The semantic backflow learning module is used to extract the semantic mapping relationship between natural language expressions and database fields from effective question-answering samples, generate semantic learning samples, and use the semantic learning samples to train the large language model; The dynamic semantic model update module is used to dynamically adjust the confidence of semantic mapping relationships in the third-stage semantic model based on the frequency and accuracy of semantic mapping relationships in historical question-and-answer data, and synchronize the updated semantic mapping relationships to the semantic knowledge base.

[0015] Based on the same inventive concept, this application also provides a computer storage medium storing computer-executable instructions, which, when executed, implement the aforementioned method for automatic construction and dynamic updating of a database semantic model.

[0016] Compared with the prior art, this application has the following advantages: This method automatically constructs mapping relationships between database tables, database fields, and business semantics. Compared to the traditional method that relies on manual maintenance of field dictionaries or semantic tags, this method can automatically complete semantic expansion and updates when the database structure changes or new fields are added. This significantly reduces the configuration and maintenance costs of the semantic model and improves the natural language question-and-answer system's ability to understand business expressions. At the same time, it reuses and corrects semantic mapping relationships based on a semantic knowledge base, improving the accuracy of the semantic model and its cross-scenario deployment efficiency. It generates effective learning samples from historical question-and-answer data to train the large language model, enabling the semantic model to have self-learning capabilities based on real business feedback. Based on the frequency and accuracy of the use of each semantic mapping relationship in historical question-and-answer data, it dynamically adjusts the weights of semantic mapping relationships, improving the continuous optimization capability of the semantic model.

[0017] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the method for automatic construction and dynamic updating of the database semantic model in the embodiments of this application; Figure 2 This is a schematic diagram of the automatic construction and dynamic updating system for the database semantic model in this application embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] like Figure 1 As shown in the figure, this application provides a method for automatically constructing and dynamically updating a database semantic model, including: S1. Obtain table metadata from the database, including table names, table comments, and inter-table relationships. Input the table metadata into a large language model for semantic parsing, identify business entities, and establish a mapping relationship between database tables and business entities, which serves as the first-stage semantic model. Table name: The name of a database table. Each database table typically represents a separate business entity or object. For example, the "Orders table" generally contains data related to user orders, while the "Users table" contains information about system users.

[0022] Table comments: Descriptive information accompanying each database table helps in understanding the table's function and purpose. During database table parsing, table descriptions / comments are automatically read. For example, the "Users table" might include the description "This table records the basic information of all registered users, such as name, address, and contact information," clearly identifying the business entity corresponding to the table and its application in actual business operations.

[0023] Inter-table relationships: These are established through foreign key constraints and table joins in the database, identifying the data interaction and dependency patterns between different database tables. For example, the order table and the user table are typically linked by a "user ID" field, indicating that a user can have multiple orders.

[0024] This step extracts the structured descriptive information (table metadata) of the target database. The table name provides the basic identity of the database table, the table comments supplement the business function description of the database table, and the relationships between tables reveal the business dependency logic between the tables.

[0025] The table metadata and pre-defined business semantic prompts are input into a large language model (such as commercial models like GPT-4 or Wenxin Yiyan, or a finely tuned open-source model) for semantic understanding. These business semantic prompts are a set of rules that guide the model to parse database tables according to a unified semantic understanding framework, understanding the relationship between database structure and business requirements. The prompts can be customized according to business scenarios, ensuring the model can flexibly handle query needs from different domains. For example, in an e-commerce scenario, for a "user table," the business semantic prompt might be: "This table records all users' basic information, including name, contact information, registration date, etc."; for an "order table," the prompt might be: "This table records all users' order information, including order number, product details, purchase date, etc."

[0026] The large language model performs semantic understanding and comprehensive analysis on the input table metadata based on business semantic prompt word templates, identifies the actual business function of each database table, and automatically assigns corresponding business entity type labels to each database table. Mapping each database table to its corresponding business entity yields the first-stage semantic model. The core task of the first-stage semantic model is to clarify the mapping relationship between business entities and database tables, laying the foundation for subsequent query parsing.

[0027] For example, for a database table named "order_info" with the comment "stores user order information", the semantic model infers that its corresponding business entity is "order". At the same time, for the relationship between tables, the model can understand the business meaning reflected by the relationship that "the order table is associated with the user table through the user ID", that is, the one-to-many business relationship that "a user can have multiple orders", thereby verifying and supplementing the relationship between different business entities.

[0028] S2. Read the field metadata of the target database table, including field name, field remarks and field data type. Input the metadata of each field and the business entity information of the database table to which the field belongs into the big language model for field semantic parsing, identify the business semantics of each field, and establish the association between the database table, the database field and the business semantics as the second stage semantic model. Field Name: The name that identifies each field in a database table, used to identify the specific data attribute of that field in the database. Field Comment: Used to describe the meaning of the field in the database, such as supplementing information about the specific meaning, value range, or business purpose of the data recorded by the field. Field Data Type: The data type defined for a field in the database, used to constrain the structural characteristics and value range of the data stored in that field. Common data types include integer, floating-point, string, and date / time types.

[0029] The field metadata and the business entity information corresponding to the database table to which the field belongs (obtained based on semantic model reasoning in the first stage) are input into the large language model. A pre-set semantic parsing prompt template guides the model to perform semantic parsing and reasoning about the field's meaning. The large language model comprehensively analyzes the field name, field notes, and field data type of each field, and automatically identifies the actual meaning of the field in the business scenario by combining this with the business entity information of the database table to which the field belongs. For example, for a field named "customer_id", the model identifies it as related to "customer identifier" and infers its corresponding business semantics as "customer ID" or "customer unique identifier" based on the database table and context information.

[0030] Through the semantic parsing process described above, the mapping relationship between database fields and business semantics is automatically generated, and a triplet association structure of database tables, database fields, and business semantics is established. That is, each field not only records its own business semantic label, but is also associated with the business entity through its database table, so that the underlying data structure in the database can establish a unified semantic expression with the business terminology system.

[0031] S3. Match the second-stage semantic model with the historical semantic mapping data stored in the pre-built semantic knowledge base, and optimize the second-stage semantic model according to the matching results to obtain the third-stage semantic model. The semantic knowledge base is a database that centrally stores, manages, and reuses semantic mapping experience. By calling historical semantic mapping data from the semantic knowledge base, the semantic mapping relationships in the second-stage semantic model are supplemented, corrected, and improved to form a semantic mapping set, resulting in the third-stage semantic model. Compared with the second-stage semantic model, the third-stage semantic model has higher completeness, higher accuracy, and stronger expressive coverage.

[0032] S4. Collect historical question-and-answer data, extract the semantic mapping relationship between natural language expressions and database fields from valid question-and-answer samples, generate semantic learning samples, and use the semantic learning samples to train the large language model; After the third-phase semantic model went live, the Natural Language Question Data Service provided mapping support between database tables and business entities, and between database fields and business semantics, for the semantic parsing process of the large language model. To further improve the accuracy of natural language parsing and the model's self-learning capability, a historical question-and-answer feedback learning mechanism was used to save user-submitted natural language query requests, model-generated SQL query statements, and query results, obtaining historical question-and-answer data. Combined with user feedback and a multi-agent logic review mechanism, when the query results were determined to conform to business logic, the question-and-answer process was considered a valid question-and-answer sample. The semantic mapping relationship between the natural language expression and database fields in the valid question-and-answer samples was accumulated as semantic learning samples for offline training of the large language model, continuously optimizing the semantic parsing capability based on real business data.

[0033] S5. Based on the frequency and accuracy of semantic mapping relationships in historical question-and-answer data, dynamically adjust the confidence of semantic mapping relationships in the third-stage semantic model, and synchronize the updated semantic mapping relationships to the semantic knowledge base.

[0034] By statistically analyzing historical question-and-answer data, the confidence level of semantic mapping relationships is dynamically adjusted. This automatically increases the confidence level of high-frequency and accurate semantic mapping relationships while decreasing the confidence level of low-frequency or erroneous semantic mapping relationships, thereby improving the accuracy of natural language parsing. The updated semantic rules are then synchronized to the semantic knowledge base, continuously optimizing semantic understanding capabilities during long-term system operation, achieving the intelligent effect of the natural language question-and-answer system becoming "more accurate with use."

[0035] In this embodiment, by automatically constructing the mapping relationship between database tables, database fields, and business semantics, compared to the traditional method that relies on manual maintenance of field dictionaries or semantic tags, this method can automatically complete semantic expansion and updates when the database structure changes or new fields are added. This significantly reduces the configuration and maintenance costs of the semantic model and improves the natural language question-and-answer system's ability to understand business expressions. Reusing and correcting semantic mapping relationships based on a semantic knowledge base improves the accuracy of the semantic model and its cross-scenario deployment efficiency. Valid learning samples generated from historical question-and-answer data are used to train the large language model, enabling the model to have self-learning capabilities based on real business feedback. The weights of semantic mapping relationships are dynamically adjusted according to the frequency and accuracy of their use in historical question-and-answer data, improving the continuous optimization capability of the semantic model.

[0036] In this embodiment, constructing a semantic knowledge base includes: (1) Collect historical semantic mapping rules created in historical projects and other business systems. The historical semantic mapping rules shall at least include business indicator definitions, industry common terms, and the correspondence between field names and business semantics. During the process of enterprise digital transformation, different business systems have accumulated certain semantic mapping experience over long-term operation. This includes the correspondence between fields and business terms (e.g., "user_id" corresponds to "user ID"), the definition of business metrics (e.g., the calculation method for "monthly active users"), and industry-standard terms and their synonyms (e.g., "customer," "user," and "account" are synonyms). These semantic mapping rules have been validated by actual business operations and possess high accuracy and reliability. By collecting structured semantic mapping data from these existing business systems, standardizing the format, and writing it into a semantic knowledge base, cross-system semantic experience can be reused.

[0037] (2) Collect effective semantic mapping relationships in the historical question-and-answer process, wherein the effective semantic mapping relationships include the correspondence between the verified natural language expressions and the database fields; During the natural language question-and-answer process, the user's input natural language questions, the SQL query statements generated by the model, and the query results are recorded as historical question-and-answer data. When the user provides feedback or the logic verifies that the query results are correct, the corresponding semantic mapping relationship is extracted and written into the semantic knowledge base, gradually accumulating semantic expression patterns in real business environments.

[0038] (3) The collected historical semantic mapping rules and the effective semantic mapping relationships are written into the semantic knowledge base.

[0039] Preferably, the semantic knowledge base can be implemented at the physical level using relational databases, key-value stores, or vector databases, with the appropriate storage solution selected based on the actual volume of business data. The semantic knowledge base supports retrieval by one or more of the following: business scenario, database table name, field name, and business semantic tags, to quickly retrieve relevant historical semantic rules during subsequent matching.

[0040] According to this embodiment, the second-stage semantic model is matched with the historical semantic mapping data stored in the semantic knowledge base, and the second-stage semantic model is optimized based on the matching results, including: (1) When the business semantics of a database field are missing in the second stage semantic model, and there is historical semantic mapping data in the semantic knowledge base that matches the business semantics, the matching business semantics are added to the second stage semantic model; Specifically, for a data field in the second-stage semantic model, if its corresponding business semantics have not yet been successfully recognized by the large language model (i.e., the field lacks a business semantic label in the second-stage semantic model), but a semantic mapping relationship matching the field name or field annotation exists in the semantic knowledge base, then the corresponding business semantics will be automatically retrieved from the semantic knowledge base and added to the second-stage semantic model. For example, if the second-stage semantic model has a field named "employee_status" with the annotation "employee status", but its business semantic label is empty, and the semantic knowledge base has already stored the business semantics corresponding to "employee_status" as "employed status", then this business semantics will be added to the second-stage semantic model.

[0041] (2) When the semantic mapping relationship in the second stage semantic model conflicts with the historical semantic mapping data in the semantic knowledge base, the semantic mapping data in the semantic knowledge base shall prevail; Since the semantic knowledge base is derived from historical project verification or real business Q&A accumulation, it has a high degree of credibility. Therefore, in the event of a conflict, the knowledge base rules shall prevail, which can effectively avoid mapping errors caused by reasoning bias in the large language model. For example, if the second-stage semantic model maps the field "status" to "state", while the standard semantics of this field under the corresponding business entity in the semantic knowledge base is "account status", the mapping relationship will be automatically corrected to "account status".

[0042] (3) When there are synonyms in the semantic knowledge base that are related to the business semantics of the second stage semantic model, add the synonyms to the second stage semantic model.

[0043] When multiple natural language expressions are mapped to the same field, these expressions are synonyms. When the semantics of a field are found to be highly similar to existing semantic rules in the semantic knowledge base, the business semantic label of that field can be automatically adjusted or synonym relationships can be supplemented, thereby forming more complete semantic mapping information. For example, the second-stage semantic model identifies the business semantics of the field "total_sales" as "total sales amount," while the semantic knowledge base stores synonyms for this business semantics, including "total sales revenue," "total sales amount," and "total revenue." These synonyms are then automatically added to the synonym mapping table of the "total_sales" field, enabling natural language parsing to recognize more expressions.

[0044] This embodiment revises, supplements, and improves the second-stage semantic model by calling historical semantic mapping data from the semantic knowledge base. This not only allows for the correction of newly generated semantic models using historical experience but also enables the sharing of semantic understanding capabilities across different business systems. This gives the semantic model the ability to continuously evolve and migrate across scenarios, effectively improving the semantic understanding accuracy and stability of the overall natural language question-and-answer system. Furthermore, as the system's runtime increases and question-and-answer data continues to accumulate, the data in the semantic knowledge base becomes increasingly rich, and its optimization effect on the semantic model will gradually strengthen, forming a virtuous cycle.

[0045] The natural language question and answer system is composed of a large language model, a third-stage semantic model, and a database. Based on this natural language question and answer service, a full set of question and answer records is made for each user query, resulting in a historical question and answer data collection.

[0046] Each question and answer data item includes: Natural language questions entered by the user; SQL query statement generated by the big language model: The big language model parses the natural language query request and calls the semantic mapping information stored in the third-stage semantic model during the parsing process to automatically generate the SQL query statement; Query results: The query results returned after the generated SQL query statement is executed on the target database; User feedback information: Feedback information provided by users based on the query results.

[0047] The above four types of information together constitute a complete question and answer record, which is stored in the question and answer log database to form historical question and answer data.

[0048] To prevent model degradation, the collected historical question-and-answer data needs to be filtered, and only valid question-and-answer samples need to be selected. Semantic mapping relationships between natural language expressions and fields are extracted from the valid question-and-answer samples to generate semantic learning samples. The semantic learning samples are then used to train the large language model.

[0049] In this embodiment, valid question-and-answer samples are filtered from the collected question-and-answer data based on the user feedback information and preset logical verification rules. The specific filtering method is as follows: (1) User feedback verification; When a user confirms that the query result is correct, the question and answer data is marked as a valid sample.

[0050] (2) Logical verification; For question-and-answer data without explicit user feedback, the SQL query statements and their execution results are automatically validated using preset logical validation rules, including but not limited to: Rule validation: This verifies whether the SQL query statement conforms to preset syntax and business rules. For example, it checks whether the SQL query statement contains the necessary WHERE conditions, whether it uses the correct table and field names, and whether it complies with the database's access control rules. Data consistency verification: This is used to verify whether the query results are consistent with the original data in the database. For example, for aggregate queries (such as SUM and COUNT), cross-validation can be performed to check whether there are logical contradictions between query results of different granularities.

[0051] Business logic rationality verification: This is used to verify whether the query results conform to preset business logic constraints. For example, the query for "sales amount this month" should return a positive number, and the query for "number of users" should not return a result exceeding the maximum number of users. When the execution result clearly violates the basic business logic, the query result can be determined to be unreasonable.

[0052] Question and answer data that meets the above verification rules can be marked as valid samples even without explicit user feedback. The two verification methods can be used independently or in combination; when the two methods yield inconsistent results, the user feedback verification shall prevail.

[0053] After selecting valid question-and-answer samples, natural language expressions are extracted from the natural language questions in the valid question-and-answer samples, and relevant database fields are extracted from the SQL statements in the valid question-and-answer samples; the correspondence between the natural language expressions and the database fields is used as a semantic mapping relationship.

[0054] The specific process is as follows: First, the natural language questions in the valid question-answer samples are processed (such as word segmentation, part-of-speech tagging, named entity recognition, dependency parsing, etc.) to extract keywords and phrases that represent business indicators (such as "sales amount" and "number of users"), business objects (such as "products" and "orders"), or business conditions (such as "this month" and "Guangdong region"), and these are used as natural language expressions. Secondly, the SQL statements in the valid question-and-answer samples are parsed. By identifying the structures such as SELECT clause, FROM clause, and WHERE clause in the SQL statement, the database table names and field names referenced by the SQL statement are extracted. Then, establish a correspondence between natural language expressions and database fields. For example, establish a semantic mapping relationship between "sales amount" extracted from the natural language question "What is the sales amount this month?" and the "total_amount" field in the SQL statement "SELECT SUM(total_amount)".

[0055] Finally, the extracted semantic mapping relationships are encapsulated in a structured manner to generate semantic learning samples. Each semantic learning sample contains at least: a natural language expression (such as "sales amount"), a corresponding database field (such as "total_amount"), and an initial confidence value for the mapping relationship between the two.

[0056] Once the semantic learning samples stored in the backflow data pool have accumulated to a certain scale, the model training process is initiated. The accumulated semantic learning samples are used to train the large language model in order to optimize the large language model's ability to recognize the mapping relationship between natural language expressions and database fields.

[0057] Preferably, the model can be optimized using either of the following two training methods: (1) Reinforcement learning method: The natural language expression in the semantic learning sample is used as input. The large language model generates the corresponding field selection result according to the current policy. The field selection result is compared with the correct field labeled in the learning sample, and the reward value is calculated. The parameters of the large language model are adjusted based on the reward value through the policy gradient algorithm. Positive reward is given for the correct mapping result, and negative reward is given for the incorrect mapping result, so that the large language model gradually learns a more accurate mapping policy.

[0058] (2) Offline fine-tuning method: construct semantic learning samples as supervised training data, use natural language expressions as input sequences and corresponding database field labels as output sequences to perform supervised fine-tuning on the large language model; by continuously optimizing the model parameters in multiple rounds of training, the large language model gradually learns the correspondence between natural language expressions and database fields.

[0059] In this embodiment, statistical analysis is performed on historical question-and-answer data to calculate two types of statistical indicators: Semantic mapping frequency: The total number of times each natural language expression and database field semantic mapping relationship is used in historical question and answer is counted, and the frequency of use is calculated, which reflects the activity level of the mapping relationship in real business queries.

[0060] The accuracy rate of semantic mapping relationships: The percentage of each semantic mapping relationship that has been confirmed as correct by users or passed logical verification in the history of question answering is used as the accuracy rate of that mapping relationship.

[0061] Furthermore, based on the frequency and accuracy of each semantic mapping relationship in historical question-answering data, the confidence / weight of the semantic mapping relationship is dynamically adjusted. This confidence / weight is used to control the matching priority of the semantic mapping relationship in the subsequent natural language parsing process, and the semantic mapping relationship with higher confidence / weight is adopted first.

[0062] According to a preferred embodiment, the confidence level of the semantic mapping relationship is dynamically adjusted in the third-stage semantic model, including but not limited to: (1) For semantic mapping relationships whose frequency of use is higher than the first preset threshold and whose accuracy is higher than the second preset threshold, increase their confidence; Mapping relationships that are frequently used and consistently verified in real-world business scenarios represent the most common user expressions and the most reliable mapping rules. Prioritizing their use in subsequent parsing can significantly improve response speed and accuracy in high-frequency query scenarios. For example, with a first preset threshold set to 100 times and a second preset threshold set to 90%, when "sales amount" → "total_amount" is used frequently and with high accuracy, its confidence level increases from 0.85 to 0.95.

[0063] (2) For semantic mapping relationships that are not used within the statistical period, reduce their confidence level; Semantic mappings that have not been used for a long time may have had changes in the corresponding business terms, the corresponding business metrics may no longer be used, or the way the semantic mapping is expressed may not conform to the user's actual expression habits. Lowering its confidence can avoid prioritizing the matching of invalid or outdated mappings during parsing.

[0064] (3) For semantic mapping relationships with an accuracy rate lower than the third preset threshold, reduce their confidence or remove them.

[0065] For semantic mappings with an accuracy rate below a third preset threshold, their confidence level is automatically reduced. When the confidence level falls below a set minimum threshold, the mapping is removed from the current semantic model. This adjustment strategy is based on the following principle: semantic mappings with consistently low accuracy indicate a discrepancy between the natural language expression and the database field, and continued use may interfere with subsequent query parsing; when the discrepancy becomes severe enough, it should be removed from the model to maintain the simplicity and effectiveness of the semantic model structure.

[0066] The three adjustment strategies described above can be executed independently or in combination. When the same semantic mapping relationship simultaneously meets multiple adjustment conditions, the adjustments are cumulatively applied.

[0067] During the dynamic adjustment process, the confidence level of synonym mapping follows independent statistical logic. In this embodiment, when multiple natural language expressions are mapped to the same database field, these multiple natural language expressions are synonyms. For multiple natural language expressions that are synonyms, the frequency of use of the semantic mapping relationship between different natural language expressions and the same database field is statistically analyzed; based on the frequency of use of each semantic mapping relationship, the confidence level of each semantic mapping relationship is adjusted.

[0068] Frequently used synonyms receive higher confidence scores, while less frequently used synonyms receive lower confidence scores. For example, the field "total_amount" has three synonym expressions: "sales amount" is used 1000 times, increasing the confidence score to 0.95; "sales revenue" is used 200 times, maintaining a confidence score of 0.80; and "total revenue" is used 20 times, decreasing the confidence score to 0.55. This gradually adapts to the mainstream language expression habits of the user group, making the semantic parsing results more consistent with the expression preferences of most users.

[0069] After adjusting the confidence level as described above, the adjusted confidence level parameters are applied to the corresponding semantic mapping relationships in the current semantic model, thereby achieving iterative optimization of the current semantic model parameters. Preferably, an incremental learning approach is used to update the parameters of the current semantic model, that is, only the adjusted confidence level parameters are used as incremental patches to update the corresponding semantic mapping relationships in the current semantic model. This eliminates the need to retrain the entire model structure, resulting in low computational cost, fast update speed, and real-time or near-real-time effectiveness of the adjusted confidence level. Furthermore, the update process does not interrupt the natural language query service, ensuring the availability and stability of the model under high-concurrency query scenarios.

[0070] This embodiment can also configure multiple update strategies according to actual business needs: real-time update strategy, which takes effect immediately after each confidence adjustment; periodic update strategy, which applies all adjustment results in batches according to a preset time period (such as 2:00 AM every day); and on-demand update strategy, which triggers an update when the cumulative amount of adjustments reaches a preset threshold.

[0071] After updating the parameters of the current semantic model, the newly added or corrected semantic mapping relationships are standardized and synchronized to the semantic knowledge base as new or corrected semantic rules, achieving unified management and sharing of semantic knowledge. In this embodiment, the current semantic model is the set of semantic mapping relationships currently used by the system. The confidence parameter in the model changes dynamically with user query behavior, reflecting the system's real-time adaptation to current user expression habits. The semantic knowledge base is a long-term stored semantic asset, storing semantic rules that have been verified by real business and can be reused across projects, without storing temporary confidence fluctuation values. Therefore, what is synchronized to the knowledge base is the confirmed correct semantic mapping relationship itself (i.e., the correspondence rules between natural language expressions and database fields), rather than the real-time confidence parameter.

[0072] Through the aforementioned knowledge base synchronization operation, the newly added or revised semantic rules confirmed during this optimization process have been added to the semantic knowledge base. When a new business system is integrated or an existing system needs to rebuild its semantic model, the accumulated semantic rules in the knowledge base can be directly invoked, thereby enabling the sharing and reuse of semantic experience across different business systems. Simultaneously, the continuous enrichment of the semantic knowledge base provides a more comprehensive historical experience foundation for the automatic construction and optimization of the next round of semantic models. In this way, the semantic model can not only be continuously optimized within a single system but also achieve experience sharing and capability migration across multiple business systems, thereby improving the overall intelligence level of the data analysis platform.

[0073] The semantic model update mechanism described in this embodiment realizes the transformation of the semantic model from static configuration to continuous evolution, enabling the natural language query system to continuously self-optimize during long-term operation. As user query data accumulates, a more complete semantic mapping system gradually forms, continuously improving the accuracy and stability of natural language parsing. This mechanism not only significantly reduces the problem of traditional semantic models relying on manual maintenance but also enables the system to have continuous learning and self-optimization capabilities, thereby achieving an intelligent effect of "becoming more accurate with use," and providing an efficient and scalable semantic model update solution for enterprise-level natural language data analysis systems.

[0074] like Figure 2As shown in the embodiments of this application, an automatic construction and dynamic update system for a database semantic model is also provided, including: a database semantic parsing module, a field semantic extension module, a semantic model optimization module, a semantic backflow learning module, and a dynamic semantic model update module.

[0075] The field semantic extension module is used to read the field metadata of the target database table, including field name, field remarks and field data type. It inputs the metadata of each field and the business entity information of the database table to which the field belongs into the large language model for field semantic parsing, identifies the business semantics of each field, and establishes the association between the database table, the database field and the business semantics, as the second stage semantic model.

[0076] The semantic model optimization module is used to match the second-stage semantic model with the historical semantic mapping data stored in the pre-built semantic knowledge base, and optimize the second-stage semantic model according to the matching results to obtain the third-stage semantic model.

[0077] The semantic backflow learning module is used to extract the semantic mapping relationship between natural language expressions and database fields from effective question-answering samples, generate semantic learning samples, and use the semantic learning samples to train the large language model.

[0078] The dynamic semantic model update module is used to dynamically adjust the confidence of semantic mapping relationships in the third-stage semantic model based on the frequency and accuracy of semantic mapping relationships in historical question-and-answer data, and synchronize the updated semantic mapping relationships to the semantic knowledge base.

[0079] Regarding the system in the above embodiments, the specific manner in which each unit module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0080] Based on the same inventive concept, this application also provides an electronic device, including: a memory and a processor, wherein the processor is used to read and execute a computer program stored in the memory to realize the aforementioned method for automatic construction and dynamic updating of a database semantic model.

[0081] Based on the same inventive concept, this application also provides a computer storage medium storing computer-executable instructions, which, when executed, implement the aforementioned method for automatic construction and dynamic updating of a database semantic model.

[0082] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0083] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a particular embodiment can be found in the relevant descriptions of other embodiments. Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatically constructing and dynamically updating a database semantic model, characterized in that, The method includes: Obtain table metadata from the database, including table names, table comments, and inter-table relationships. Input the table metadata into a large language model for semantic parsing, identify business entities, and establish a mapping relationship between database tables and business entities, which serves as the first-stage semantic model. Read the field metadata of the target database table, including field name, field remarks and field data type. Input the metadata of each field and the business entity information of the database table to which the field belongs into the large language model for field semantic parsing, identify the business semantics of each field, and establish the association between database table, database field and business semantics as the second stage semantic model. The second-stage semantic model is matched with historical semantic mapping data stored in a pre-built semantic knowledge base, and the second-stage semantic model is optimized based on the matching results to obtain the third-stage semantic model. Based on the natural question count service provided by the third-stage semantic model, historical question and answer data is collected, and the semantic mapping relationship between natural language expressions and database fields is extracted from the effective question and answer samples to generate semantic learning samples. The semantic learning samples are then used to train the large language model. Based on the frequency and accuracy of semantic mapping relationships in historical question-and-answer data, the confidence level of semantic mapping relationships is dynamically adjusted in the third-stage semantic model, and the updated semantic mapping relationships are synchronized to the semantic knowledge base.

2. The method according to claim 1, characterized in that, Building a semantic knowledge base includes: Collect historical semantic mapping rules created in historical projects and other business systems. These historical semantic mapping rules include at least business indicator definitions, industry-standard terms, and the correspondence between field names and business semantics. Collect valid semantic mapping data from historical question-and-answer processes. The valid semantic mapping relationship includes the correspondence between verified and accurate natural language expressions and database fields. The collected historical semantic mapping rules and the effective semantic mapping relationships are written into a semantic knowledge base, and the semantic knowledge base supports retrieval by one or more of the following: business scenario, database table name, field name, and business semantic tag.

3. The method according to claim 1, characterized in that, The second-stage semantic model is matched with historical semantic mapping data stored in the semantic knowledge base, and the second-stage semantic model is optimized based on the matching results, including: When the business semantics of a database field are missing in the second-stage semantic model, and historical semantic mapping data in the semantic knowledge base matches the business semantics, the matching business semantics are added to the second-stage semantic model. When the semantic mapping relationship in the second-stage semantic model conflicts with the historical semantic mapping data in the semantic knowledge base, the semantic mapping data in the semantic knowledge base shall prevail. When a synonym for the business semantics of the second-stage semantic model exists in the semantic knowledge base, the synonym is added to the second-stage semantic model.

4. The method according to claim 1, characterized in that, The question-and-answer data includes natural language questions entered by users, SQL query statements generated by large language models, query results, and user feedback information.

5. The method according to claim 4, characterized in that, The step of extracting the semantic mapping relationship between natural language expressions and database fields from valid question-answer samples includes: Based on the user feedback information and the preset logical verification rules, valid question and answer samples are selected from the collected question and answer data; Natural language expressions are extracted from the natural language questions in the valid question-answer samples, and relevant database fields are extracted from the SQL query statements in the valid question-answer samples. The correspondence between the natural language expression and the database field is used as a semantic mapping relationship.

6. The method according to claim 5, characterized in that, The step of filtering valid question-and-answer samples from the collected question-and-answer data based on the user feedback information and preset logical verification rules includes: When the user confirms that the query results are correct, the question and answer data is marked as a valid sample; For question-and-answer data without explicit user feedback, the SQL query statements and their execution results are automatically validated using preset logical validation rules, including but not limited to: Rule validation is used to verify whether the SQL query statement conforms to preset syntax rules and business rules; Data consistency verification is used to verify whether the query results are consistent with the data in the database; Business logic rationality validation is used to verify whether the query results conform to the preset business logic constraints.

7. The method according to claim 1, characterized in that, The method of dynamically adjusting the confidence level of semantic mapping relationships in the third-stage semantic model based on the frequency and accuracy of semantic mapping relationships in historical question-answering data includes, but is not limited to: For semantic mapping relationships that are used more frequently than the first preset threshold and have an accuracy rate higher than the second preset threshold, increase their confidence. For semantic mapping relationships that are not used within the statistical period, their confidence level should be reduced. For semantic mapping relationships with an accuracy rate lower than the third preset threshold, their confidence level is reduced or they are removed.

8. The method according to claim 1, characterized in that, The method further includes: For multiple natural language expressions that are synonyms, the frequency of use of the semantic mapping relationship between different natural language expressions and the same database field is counted. When multiple natural language expressions are mapped to the same database field, the multiple natural language expressions are synonyms. Adjust the confidence level of each semantic mapping relationship based on its usage frequency.

9. A system for automatically constructing and dynamically updating a database semantic model, characterized in that, The system includes: The database semantic parsing module is used to obtain table metadata of the database, including table names, table comments and inter-table relationships. The table metadata is input into the large language model for semantic parsing to identify business entities and establish a mapping relationship between database tables and business entities, which serves as the first stage of the semantic model. The field semantic extension module is used to read the field metadata of the target database table, including field name, field remarks and field data type. It inputs the metadata of each field and the business entity information of the database table to which the field belongs into the large language model for field semantic parsing, identifies the business semantics of each field, and establishes the association between the database table, the database field and the business semantics, as the second stage semantic model. The semantic model optimization module is used to match the second-stage semantic model with the historical semantic mapping data stored in the pre-built semantic knowledge base, and optimize the second-stage semantic model according to the matching results to obtain the third-stage semantic model. The semantic backflow learning module is used to extract the semantic mapping relationship between natural language expressions and database fields from effective question-answering samples, generate semantic learning samples, and use the semantic learning samples to train the large language model; The dynamic semantic model update module is used to dynamically adjust the confidence of semantic mapping relationships in the third-stage semantic model based on the frequency and accuracy of semantic mapping relationships in historical question-and-answer data, and synchronize the updated semantic mapping relationships to the semantic knowledge base.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed, implement the method described in any one of claims 1-8.

Images