An intelligent query method and device based on pre-normalization of a business term dictionary

Abstract

The application provides an intelligent query method and device based on business term dictionary pre-normalization, and relates to the technical field of natural language processing and data query. The application introduces an independent business term dictionary as a pre-normalization layer, and uses the business term dictionary including at least a standard term identifier, a term synonymous expression set and term field classification information to accurately match and judge the ambiguity of entity terms in user queries. Deterministic mapping processing strategies are adopted for three cases of unique matching, polysemy matching and non-matching, which effectively eliminates term ambiguity and improves query accuracy. Specifically, through direct mapping of the unique matching case, the fast processing of known unambiguous terms is ensured; through the ambiguity clarification sub-process of the polysemy matching case, the human-computer interaction interruption is minimized while ensuring accuracy; and through marking and reserving the non-matching case, progressive learning ability is provided for unknown terms.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and data query technology, and in particular to an intelligent query method, apparatus, computing device, and computer-readable storage medium based on the pre-normalization of a business term dictionary. Background Technology

[0002] In recent years, intelligent query technology based on Large Language Model (LLM) has been widely applied to natural language access in enterprise internal databases and document systems. Existing technologies mainly include three types of solutions: First, the Natural Language to SQL (NL2SQL) method, where the LLM directly generates SQL (Structured Query Language) statements based on the user's natural language question; second, the Retrieval-Augmented Generation (RAG) method, which retrieves semantically similar document fragments from the knowledge base through vector retrieval and then feeds them into the LLM to generate the answer; and third, the method of hard-coding business terminology mapping rules into downstream data query tools, allowing the downstream data query tools to perform normalization automatically.

[0003] However, these solutions have significant shortcomings when dealing with queries in specialized enterprise domains, such as the testing, inspection, and certification (TIC) industry: In the NL2SQL solution, LLM is prone to generating incorrect SQL or phantom fields due to a lack of domain knowledge; the RAG solution relies on text vector semantic similarity, making it difficult to distinguish the different meanings of the same term in different business domains; and hard-coded solutions suffer from difficulties in updating, high maintenance costs, and inconsistent mapping logic among multiple downstream data query tools. Furthermore, existing solutions generally lack ambiguity clarification mechanisms, traceability capabilities, and good adaptability to terminology-intensive industries.

[0004] Prior art document CN116662582B discloses a domain-specific business knowledge retrieval method based on natural language, which uses vector retrieval for fuzzy recall, but does not provide explicit ambiguity determination or human-computer clarification interaction mechanisms, nor does it provide a business terminology dictionary as an independent, hot-updable semantic service. Prior art document US10592505B2 discloses a database natural language interface based on autonomous agents and a thesaurus, but its thesaurus index nodes do not carry domain metadata, lack an explicit ambiguity determination mechanism for polysemy, and do not provide an independent semantic service interface.

[0005] The aforementioned existing technologies have the following drawbacks when facing inquiries from specialized fields within enterprises (especially in the Testing, Inspection, and Certification (TIC) industry): 1) Lack of ambiguity clarification mechanism: When the term entered by the user has corresponding meanings in multiple fields, the existing solution either randomly selects one of the interpretations or directly returns "not found", lacking the ability to interact with the user for clarification.

[0006] 2) Lack of traceability: When there are discrepancies in the query results, it is difficult to pinpoint whether the error is due to incorrect terminology recognition, incorrect mapping of business terminology dictionary, or error in the query by downstream data query tools.

[0007] 3) Insufficient adaptability to industries with dense terminology: The TIC industry contains a large number of national standards, industry standards, enterprise internal standards and customer common expressions. The same testing item may have dozens of different names (including standardized terms, abbreviations, foreign names, customer colloquial names, internal codes, etc.). Relying solely on the language understanding capabilities of general LLM cannot meet the actual business needs. Summary of the Invention

[0008] In view of this, the technical problem to be solved by the present invention is: how to provide an intelligent query method that can effectively eliminate terminological ambiguity in users' natural language queries, so as to solve the problem that existing natural language query technology is not accurate, reliable and user experience when facing queries in professional fields (especially terminology-intensive fields).

[0009] Firstly, this invention provides an intelligent query method based on pre-normalization of a business terminology dictionary, comprising the following steps: S1, receiving a natural language query input by a user; S2, extracting entity terms from the natural language query, the extracted entity terms collectively constituting a term set to be normalized; S3, performing query matching on each entity term in the term set to be normalized in a pre-constructed business terminology dictionary, wherein each entry in the business terminology dictionary includes at least a standardized terminology identifier, a set of synonyms, and terminology domain classification information; S4, performing ambiguity judgment based on the query matching results, the ambiguity judgment including the following three scenarios: a) Unique match: the entity term exists in the business terminology dictionary and corresponds to only one standardized terminology identifier; b) Polysemous match: the entity term corresponds to multiple standardized terminology identifiers in different domains in the business terminology dictionary; c) Unmatched: The entity term does not exist in the business terminology dictionary; S5. Perform mapping processing according to the ambiguity judgment results to obtain normalized entity terms; The normalized entity terms together constitute the normalized term set. The mapping processing specifically includes: For uniquely matched entity terms, directly map them to the corresponding canonical term identifier; For ambiguous matched entity terms, trigger the ambiguity clarification sub-process to determine their corresponding canonical term identifier; For unmatched entity terms, mark them as unnormalized items and retain the original text; S6. Use the normalized term set to call the downstream data query tool to obtain the query results; S7. Return the query results to the user.

[0010] Optionally, in step S2, extracting entity terms from the natural language query specifically includes: performing word segmentation and preprocessing on the natural language query; identifying candidate entities through a named entity recognition model; using the longest matching algorithm, with matching priority from longest to shortest, verifying and merging candidate entities in the term synonym set of the business term dictionary; and adding successfully matched entities to the term set to be normalized.

[0011] Optionally, the named entity recognition model adopts an entity recognition model based on a pre-trained language model, and the candidate entities output by the entity recognition model are constrained, verified, corrected, or trained and optimized through a business term dictionary.

[0012] Optionally, in step S3, after performing query matching in the pre-built business terminology dictionary, a matching confidence score is calculated for each query matching result. The matching confidence score is obtained by weighting and combining at least one of the following factors: string similarity, source authority of the hit synonym in the business terminology dictionary, correlation between the terminology domain classification information of the normalized entity term in the current session and the terminology domain classification information corresponding to the candidate canonical term identifier, and the proportion of users who accepted it in historical sessions.

[0013] Optionally, in step S4, for ambiguous entity terms, if among multiple candidate standard term identifiers there is a unique candidate standard term identifier whose term domain classification information can be uniquely determined by the current session context and whose matching confidence is higher than a preset threshold, then the unique candidate standard term identifier is directly adopted as the unique matching result; otherwise, the ambiguity clarification sub-process is triggered to determine its corresponding standard term identifier.

[0014] Optionally, in step S5, the ambiguity clarification sub-process includes: checking whether there are resolution results of the same entity term in the session cache; if so, directly adopting them; if not, checking whether there are other uniquely normalized entity terms in the current query; if there are other uniquely normalized terms, performing context-based implicit resolution, calculating the matching confidence of candidate canonical term identifiers; if the matching confidence is higher than a preset threshold, automatically adopting them; if the confidence is lower than a preset threshold, generating a list of candidate options for the user and constructing a clarification question; receiving the user's selection, writing the user's selection result into the session-level cache, and outputting the normalization result.

[0015] Optionally, the session-level cache is session-scoped, and entries in the session-level cache are valid for the lifetime of the session and can be set with an idle lifespan. Entries in the session-level cache become invalid when one of the following occurs: the session ends or the idle lifespan is exceeded; the user explicitly switches business domains within the session; the business terminology dictionary undergoes a hot update that changes the resolution results of the entity terminology; or the user explicitly corrects the previous resolution results.

[0016] Optionally, the business terminology dictionary provides query capabilities through an independent semantic service interface; the semantic service interface is deployed and versioned independently; both the business terminology dictionary and the semantic service interface are decoupled from downstream data query tools; and the business terminology dictionary supports online hot updates.

[0017] Optionally, hot updates are triggered by receiving changes to the business terminology dictionary file, receiving hot update instructions, or polling the business terminology dictionary version number at time intervals. The hot update process employs at least one concurrency control mechanism, either a read-write lock mechanism or a versioned snapshot mechanism, to ensure data consistency and atomicity.

[0018] Secondly, this invention provides an intelligent query device based on pre-normalization of a business terminology dictionary, used to implement the intelligent query method described above, comprising: an entity extraction module, used to receive a natural language query input by a user and extract entity terms from the query to obtain a set of terms to be normalized; a business terminology dictionary query module, communicatively connected to the entity extraction module, used to perform query matching on each term in the set of terms to be normalized in a pre-constructed business terminology dictionary, wherein each entry in the business terminology dictionary includes at least a standardized term identifier, a set of synonyms for terms, and terminology domain classification information; and an ambiguity judgment module, communicatively connected to the business terminology dictionary query module, used to determine whether each term has domain ambiguity based on the query matching results, the ambiguity judgment including the following three cases: a) unique matching; b) polysemous matching; c) The system is divided into three modules: a clarification and interaction module (connected to the ambiguity judgment module), a session caching module (connected to the clarification interaction module), and a downstream data query tool invocation module. The clarification interaction module triggers an ambiguity clarification sub-process to determine the corresponding canonical term identifier for terms whose ambiguity judgment result is a polysemous match. The session caching module caches and reuses the confirmed term resolution results within the same session. The downstream data query tool invocation module connects to the session caching module, the clarification interaction module, and the ambiguity judgment module. It uses the normalized term set to invoke the downstream data query tool and returns the query results. For uniquely matched entity terms, the ambiguity judgment module directly maps them to the corresponding canonical term identifier and sends them to the downstream data query tool invocation module. For unmatched entity terms, the ambiguity judgment module marks them as unnormalized items, preserves the original text, and sends them to the downstream data query tool invocation module.

[0019] Implementing this invention offers the following advantages: By introducing an independent business terminology dictionary as a pre-normalization layer, and using this dictionary—which includes at least standardized terminology identifiers, a set of synonyms, and terminology domain classification information—to accurately match and resolve ambiguities in user queries, deterministic mapping strategies are employed for unique matches, polysemous matches, and no matches. This effectively eliminates terminology ambiguity, improves query accuracy, and ensures a balance between processing efficiency and accuracy. Specifically, direct mapping for unique matches ensures rapid processing of known unambiguous terms; the ambiguity clarification sub-process for polysemous matches minimizes human-computer interaction interruptions while maintaining accuracy; and marking and retaining unmatched cases provides a progressive learning capability for unknown terms. The synergistic effect of these technical features enables efficient, accurate, and fluent responses to terminology-intensive industry query needs, significantly improving the accuracy, reliability, and user experience of natural language queries in terminology-intensive domains. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the intelligent query method in one embodiment; Figure 2 This is a schematic diagram of the structure of a business terminology dictionary in one embodiment; Figure 3 This is a schematic diagram illustrating the relationship between the semantic service interface and the downstream data query tool in one embodiment; Figure 4 This is a schematic diagram of a sub-process for ambiguity clarification in one embodiment; Figure 5 This is a schematic diagram of the structure of an intelligent query device in one embodiment. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0022] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0023] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0024] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0025] Example 1 In this embodiment, as Figure 1 The intelligent query method shown includes the following steps: (This is based on the pre-normalization of a business terminology dictionary) S1. Receive natural language queries input by the user; S2. Extract entity terms from natural language queries. The extracted entity terms together constitute a set of terms to be normalized. S3. Query and match each entity term in the set of terms to be normalized in the pre-built business term dictionary. Each entry in the business term dictionary includes at least a canonical term identifier, a set of term synonyms, and term domain classification information. S4. Perform ambiguity judgment based on the query matching results. Ambiguity judgment includes the following three scenarios: a) Unique match: The entity term exists in the business term dictionary and corresponds to only one canonical term identifier; b) Polysemous matching: Entity terms correspond to multiple standardized terminology identifiers from different domains in the business terminology dictionary; c) No match: The entity term does not exist in the business terminology dictionary; S5. Based on the results of the ambiguity judgment, perform mapping processing to obtain normalized entity terms; the normalized entity terms together constitute the normalized term set. The mapping processing specifically includes: For a uniquely matching entity term, it is directly mapped to the corresponding canonical term identifier; For entity terms with multiple meanings, trigger the ambiguity clarification subprocess to determine the corresponding canonical term identifier; For unmatched entity terms, mark them as unnormalized terms and retain the original text; S6. Use the normalized term set to call the downstream data query tool to obtain the query results; S7. Return the query results to the user.

[0026] In this embodiment, as Figure 2 As shown, each business terminology dictionary entry includes at least the following fields: (1) Standardized terminology identifier, used to represent the standardized terminology form that can be recognized by downstream data query tools; the standardized terminology identifier can be at least one of the following: standard word, test item number, method number, standard number, unique ID, URI or database primary key.

[0027] (2) A set of synonyms for terms, used to record at least one of the following: synonyms, abbreviations, foreign names, industry idioms, customer idioms, historical names, internal codes, vectorized representations or hash representations of entity terms.

[0028] (3) Terminology domain classification information, used to identify the business domain, testing category, product category, standard system, laboratory department, data source type or hierarchical classification label to which the term belongs.

[0029] (4) Synonym group identifier, used to mark synonyms or near-synonyms across entries.

[0030] (5) Remarks information, used to record metadata such as ambiguity handling rules, usage constraints, data source, audit status, and update time.

[0031] S21. Perform word segmentation and preprocessing on natural language queries. Preprocessing includes at least one of the following: converting full-width characters to half-width characters; unifying different forms of connectors and dashes into ordinary hyphens; compressing consecutive whitespace into a single space; unifying the capitalization of English, legal abbreviations, standard numbers, or method numbers; cleaning up punctuation marks that do not affect semantics; and retaining key separators in standard numbers, method numbers, legal abbreviations, and product models.

[0032] For example, user-inputted "gbt8410", "GB / T 8410", and "gb t 8410" can be uniformly standardized to "GB / T8410"; "ISO105-B06" can be uniformly standardized to "ISO 105-B06"; and "salt spray / NSS / neutral saltspray" can retain different candidate expressions in the set of synonyms for the test item.

[0033] S22. Identify candidate entities using a Named Entity Recognition (NER) model; specifically, the NER model can be implemented using at least one of the following methods: Rule matching based on a business terminology dictionary, such as constructing an AC (Aho-Corasick) automaton, prefix tree, or inverted index using a set of synonyms from the business terminology dictionary, can be used to scan natural language queries and recall candidate entities.

[0034] The prefix tree organizes synonyms of terms layer by layer according to the common prefixes of characters or words. It is used to match layer by layer from the current position of the query text and prioritize the acquisition of longer complete terms. For example, when both "GB / T 8410" and "GB / T 8410-2006" exist in the term synonym set, for the query text "detect by GB / T 8410-2006", the prefix tree continues to match along the common prefix "GB / T 8410" to "-2006" and recalls the longer complete synonym "GB / T 8410-2006".

[0035] The inverted index uses character fragments, tokens, or standard numbers as index keys, recording the correspondence between each index key and the business terminology dictionary entries containing that index key. It is used to first recall candidate entries, and then filter them based on complete descriptions, terminology domain classification information, and matching confidence. For example, for the query text "detection of heavy metal migration in plastic toys," corresponding candidate entries can be recalled based on index keys such as "plastic toys," "heavy metals," and "migration," and the corresponding standardized terminology identifiers can be determined by combining this with toy domain classification information. Both the prefix tree and the inverted index are generated from the business terminology dictionary to improve the efficiency of candidate entity recall and terminology matching.

[0036] Based on statistical machine learning models, such as Conditional Random Fields (CRF) and Hidden Markov Models (HMM). Pre-trained language models (PLMs) based on deep learning, such as fine-tuned BERT (Bidirectional Encoder Representations from Transformers), BiLSTM-CRF (Bidirectional Long Short-Term Memory-Conditional Random Field), or dedicated entity recognition models fine-tuned for TIC domain corpora.

[0037] In this embodiment, the NER model uses a pre-trained language model based on deep learning, and fine-tunes the pre-trained language model using a business term dictionary as a constraint.

[0038] Specifically, the NER model can label text fragments in a user query as different entity types, including product entities, testing item entities, standard number entities, regulatory entities, method number entities, customer entities, and sample attribute entities. For example, for a user's natural language query "heavy metal testing of plastic toys, quoting according to GB 24613", the NER model can output candidate entity boundaries and types as: "plastic toys / product entity", "heavy metal testing / testing item entity", and "GB24613 / standard number entity".

[0039] Subsequently, the business terminology dictionary query module verifies and corrects the candidate entities based on the term synonym set, standardized term identifiers, and terminology domain classification information. Verification and correction include: when a candidate entity matches a complete term in the business terminology dictionary, the complete term boundary is prioritized; when a candidate entity only matches a substring in the business terminology dictionary, the entity boundary is expanded or contracted according to the longest matching strategy; when the candidate entity type is inconsistent with the terminology domain classification information in the business terminology dictionary, the matching confidence of the candidate entity is reduced, or the candidate entity is sent to the ambiguity clarification process.

[0040] For example, for the candidate entity "heavy metal detection", if the business terminology dictionary contains multiple standardized terminology identifiers such as "heavy metal migration detection in the toy field", "heavy metal migration detection in food contact materials", and "heavy metal detection of hazardous substances in electronic and electrical systems", the business terminology dictionary query module combines the terminology field classification information of "plastic toys" and "GB 24613" already identified in the current query to prioritize correcting "heavy metal detection" to heavy metal migration detection under the toy field; if the matching confidence of this correction result is higher than the preset threshold, it will be automatically adopted; if it is lower than the preset threshold, the ambiguity clarification process will be triggered.

[0041] S23. Using the Maximum Matching Algorithm, candidate entities are verified and merged in the set of synonyms in the business term dictionary with a matching priority from longest to shortest. For example, if the business term dictionary contains two synonyms, "heavy metal" and "heavy metal content", the longer one, "heavy metal content", will be used first in the query text "heavy metal content".

[0042] S24. Add the successfully matched entities to the term set to be normalized. For complex term boundary issues (such as nesting, omission, referencing, and whether it's a single term or a combination of terms), a hybrid strategy of "identifying candidate spans (text fragments / intervals) + dictionary longest match verification + domain constraint optimization" is used, as shown in the example below: Nested / combined ambiguity: For "non-methane total hydrocarbons", if there is a corresponding standardized term identifier in the business term dictionary, the longest match will be prioritized to identify the whole as a single term, instead of splitting it into "methane" and "total hydrocarbons"; only when there is no "non-methane total hydrocarbons" in the dictionary but its substring exists will it degenerate to identification by substring, and the whole will be recorded in the term pool to be reviewed for subsequent supplementation; Omission / Reference: For queries containing reference or omission (such as "what about its heavy metals"), complete the query by combining the normalized terms in the conversation context, and then let the subsequent matching process take place. Recall and precision: Compared to the longest match which relies solely on exact string matching, introducing domain constraints can improve recall and precision for complex expressions. The NER model identification and the business term dictionary verification complement each other. The NER model is responsible for discovering candidates whose literal meanings are not strictly matched, while the business term dictionary is responsible for constraining to standardized terms and eliminating false alarms.

[0043] In step S3 of this embodiment, after performing a query match in the pre-built business terminology dictionary, a matching confidence score is calculated for each matching result. The matching confidence score is obtained by weighting and combining at least one of the following factors: string similarity, source authority of the hit synonym expression in the business terminology dictionary, correlation between the terminology domain classification information of the normalized entity terminology in the current session and the terminology domain classification information of the candidate canonical terminology identifier, and the proportion of users who accepted it in historical sessions.

[0044] Specifically, a matching confidence score is calculated for each matching result, which is used for ambiguity judgment in step S4 and such. Figure 4 The implicit resolution shown is used for automatic acceptance. The matching confidence is obtained by weighting the following factors: String similarity (sim): The string similarity between the original text and the matched synonym, such as based on edit distance or Jaccard (Jaccard Similarity) normalized to [0,1]. Authority level (auth): The source authority of the synonym in the business terminology dictionary (value [0,1], terms extracted from standard documents are higher than terms automatically extracted from historical documents, for example, standard source is 1.0, manual review is 0.8, and automatic extraction is 0.5). Domain context relevance (context, ctx): The relevance between the term domain classification information of the normalized entity term in the current session and the term domain classification information of the candidate canonical term identifier corresponding to the entity term. It can be defined as the score of the normalized co-occurrence frequency of the normalized entity term and the corresponding candidate canonical term identifier in the business term dictionary / historical corpus (value [0,1]). Historical success rate (history, hist): The percentage of times this mapping (synonym → canonical terminology identifier) ​​has been accepted (not corrected) by the user in the most recent N sessions, using a sliding window statistic (example N=1000; can be configured for in-session or global statistics, global statistics are the default).

[0045] Match confidence calculation formula (example): Formula (1); Where w1, w2, w3, and w4 are the weight coefficients for string similarity, authority level, domain context relevance, and historical success rate, respectively, and all take values ​​of [0,1], and w1+w2+w3+w4= 1.

[0046] All weighting coefficients are adjustable empirical parameters; example values ​​are w1=0.4, w2=0.2, w3=0.3, and w4=0.1. A confidence threshold of θ is set: when the confidence of a candidate specification term identifier is ≥ θ, it is considered automatically adoptable; otherwise, the clarification process begins. θ is a configurable parameter that can be determined through offline evaluation or online testing; an example value of θ=0.95 is used to strike a balance between automatic resolution efficiency and the risk of false resolution.

[0047] In this embodiment, in step S4, for ambiguous matching entity terms, if there is a unique candidate standard term identifier among multiple candidate standard term identifiers whose term domain classification information can be uniquely determined by the current session context and whose matching confidence is higher than a preset threshold, then the unique candidate standard term identifier is directly adopted as the unique matching result; otherwise, the ambiguity clarification sub-process is triggered to determine its corresponding standard term identifier.

[0048] In an optional embodiment, the pseudocode (example) corresponding to step S4 is as follows: function match_and_score(term, session_ctx): candidates =dict.lookup_aliases(term) # Set of candidate canonical term identifiers for c in candidates:c.score = w1*sim(term, c.alias) + w2*auth(c.alias) + w3*ctx(session_ctx.domains, c.domain) + w4*hist(term, c.std, window=N) sort candidates byscore desc if len(candidates) == 0: return UNMATCHED # Case c if len(candidates) == 1: return UNIQUE(candidates[0]) # Case a if candidates[0].domain is unique and candidates[0].score>= θ: return UNIQUE(candidates[0]) # Implicit resolution and automatic acceptance return AMBIGUOUS(candidates) # Case b, trigger ambiguity clarification subprocess In this embodiment, step S5, the ambiguity clarification sub-process includes: Check if there are resolution results for the same entity terms in the session cache; if so, use them directly. If not, check if there are other uniquely normalized entity terms in the current query; If there are other uniquely normalized terms, then context-based implicit resolution is performed, the confidence of the candidate canonical term identifier is calculated, and if the confidence is higher than a preset threshold, it is automatically adopted; if the confidence is lower than the preset threshold, a list of candidate options is generated for the user and a clarification question is constructed. Receive user selections, write the user selection results to the session-level cache, and output the normalized results.

[0049] Specifically, such as Figure 4As shown, after entering the sub-process, it first determines whether the resolution result of the same term already exists in the session cache. If a match is found (the resolution result of the same term exists in the session cache), it is directly adopted; if no match is found, it determines whether there are other uniquely normalized terms in the current query. If so, it performs context-based implicit resolution, calculates its matching confidence, and automatically adopts it if it is higher than the preset threshold θ. If it is lower than the preset threshold θ, it generates a list of candidate options for the user and constructs a clarification question. After receiving the user's selection, it writes it to the session-level cache, outputs the normalization result, and proceeds to step S6.

[0050] In this embodiment, the session-level cache is session-scoped, and entries in the session-level cache are valid for the lifetime of the session, with an idle lifespan that can be set. Furthermore, entries in the session-level cache become invalid under any of the following circumstances: The session has ended or the Time To Live (TTL) has been exceeded. Users can explicitly switch business domains within a session (to avoid context pollution). If the business terminology dictionary is hot-updated and the changes affect the resolution results of the entity terminology (ensuring that the session-level cache is consistent with the business terminology dictionary); The user explicitly corrected the previous resolution result.

[0051] Specifically, the session-level cache is scoped to the session. The key of the scope is the original term (or the original term + candidate domain), and the value is the confirmed canonical term identifier and its term domain classification information. The session-level cache is valid for the lifetime of the session and a TTL can be set (30 minutes in the example).

[0052] In step S5 of this embodiment, terms marked as unnormalized items are recorded in the term pool to be reviewed, and then added to the business term dictionary after being reviewed by the administrator; the supplemented business term dictionary takes effect immediately through a hot update mechanism.

[0053] Specifically, unmatched terms are marked as "unnormalized terms" and included in the normalized term set while retaining their original text form. They are also recorded in the term pool to be reviewed by the administrator, who will periodically review them and add reasonable new terms to the business term dictionary. The supplemented business term dictionary takes effect immediately through a hot update mechanism, and subsequent identical queries can be normalized normally.

[0054] It should be noted that the vast majority of key terms in the natural language query are normalized to the canonical terminology identifiers required by downstream data query tools. Only a very small number of missed terms are retained and explicitly labeled as unnormalized terms. Therefore, downstream data query tools operate within a constrained context, primarily using canonical terminology identifiers and containing only a small amount of labeled original text, rather than facing a completely unprocessed raw query. Downstream data query tools can adopt deterministic degradation strategies based on this explicit labeling, including but not limited to: For downstream data query tools of LLM type, prompt them that "this item is unnormalized raw text. If the meaning cannot be determined, you should request clarification or ignore the constraint. Do not arbitrarily create fields" to suppress illusions. For strict SQL generator-type downstream data query tools, unnormalized items are treated as soft constraints or optional filtering conditions (such as ignoring the unknown constraint, or generating an executable query only for the normalized part and prompting "there is an unidentified constraint" in the results) to avoid direct errors due to unknown fields; When necessary, the upstream application layer that calls the semantic service interface, or the semantic service interface itself, can trigger a secondary clarification of the unnormalized term based on the unnormalized term marker. Through a closed loop of "explicit annotation + downstream data query tool downgrade + backflow of pending term pool + hot update", unknown terms are gradually eliminated, rather than causing overall failure in a single natural language query.

[0055] In this embodiment, as Figure 3 As shown, the business terminology dictionary provides query capabilities through an independent semantic service interface; the semantic service interface is deployed and versioned independently, decoupled from downstream data query tools; and the business terminology dictionary supports online hot updates.

[0056] Specifically, updating the business terminology dictionary does not require modifying the code of downstream data query tools; it supports multiple downstream data query tools concurrently calling the same business terminology dictionary; and it supports online hot updates of the business terminology dictionary—changes to the business terminology dictionary content take effect without restarting the service and do not interrupt ongoing natural language query sessions. Hot updates can be triggered by receiving business terminology dictionary file change events, receiving dedicated hot update commands, or polling the business terminology dictionary version number at time intervals.

[0057] In this embodiment, hot updates are triggered by receiving a business terminology dictionary file change event, receiving a hot update instruction, or polling the business terminology dictionary version number at time intervals.

[0058] Specifically, to ensure data consistency and atomicity during hot updates under high-concurrency natural language queries, this embodiment employs at least one of the following concurrency control mechanisms: Read-Write Lock mechanism: Natural language queries (read operations) do not block each other and can be performed concurrently; business term dictionary updates (write operations) must wait for all current read operations to complete before execution, and prevent new read operations from entering during hot updates. After the update is completed, the write lock is released, thereby avoiding reading intermediate states; Versioning / Snapshot Mechanism: The business terminology dictionary is published in the form of an immutable version (snapshot). Each natural language query is bound to a dictionary version number upon entry. An update generates a new version, and ongoing old queries continue to use their bound old versions until completion, while new queries use the new version. Version switching is completed through atomic pointer switching (such as double buffering, copy-on-write), ensuring instantaneous consistency during switching. Old versions are reclaimed after they are no longer referenced by natural language queries.

[0059] The above control mechanisms can be used individually or in combination (e.g., primarily using version snapshots, supplemented by read-write locks for incremental writes) to maintain low query latency while ensuring consistency.

[0060] In this embodiment, as Figure 5 As shown, the call process of steps S1 to S7 is recorded in a traceable manner. The traceable record is a globally unique trace identifier (trace_id) generated for each query. The traceability record should include at least the following: the original query, the extracted entity terms, the business terminology dictionary matching results and confidence levels, the ambiguity resolution process, the final normalized terms, and the results of downstream data query tool calls.

[0061] Specifically, traceability is implemented in the following ways: Unique Identifier: A globally unique trace_id, such as a UUID (Universally Unique Identifier) ​​or Snowflake Algorithm ID, is generated for each natural language query at the request entry point. This trace_id is passed through layers of the call context, such as request headers, thread context MDC (Mapped Diagnostic Context), or distributed tracing span context, including entity extraction, business terminology dictionary lookup, ambiguity resolution, clarification interaction, session caching, and downstream data query tool calls. This trace_id is also carried in every log / record generated, thus associating records distributed across different steps and even different service nodes as the entire chain of the same natural language query.

[0062] In this embodiment, traceability records can be stored in at least one way, such as writing structured logs (e.g., JSON Lines) to disk, writing to a relational database table (with trace_id as the index column), or writing to a retrieval / time-series engine (e.g., Elasticsearch); they can be retrieved and summarized by dimensions such as trace_id, time range, user, and normalized terms. In this embodiment, traceability records can also be integrated with distributed tracing. Specifically, in scenarios involving distributed or high-concurrency natural language queries, a distributed tracing framework (such as OpenTelemetry / Jaeger) can be further integrated to express the call chain and execution time using span parent-child relationships. This facilitates the reconstruction of the complete execution trajectory of a query by trace_id, supporting business auditing, problem localization, and performance analysis.

[0063] Example 2 This embodiment provides an application example of an intelligent query method based on Embodiment 1.

[0064] In this embodiment, the natural language query input in step S1 is, for example, "Heavy metal testing of plastic toys, price quoted according to GB24613". Step S2 extracts entity terms from the natural language query to obtain a set of terms to be normalized: {plastic toys, heavy metal detection, GB 24613}; Step S3 involves querying and matching each entity term in the set of terms to be normalized against a pre-built business term dictionary: In the matching results, "plastic toys" and "GB 24613" are unique matches. "Heavy metal testing" is a polysemous match (migration amount of 8 heavy metals in the toy industry / total content of 4 heavy metals in the food industry). Step S4 performs ambiguity determination based on the query matching results. The result of the ambiguity determination is: "Plastic toys" and "GB 24613" are unique matches; "heavy metal detection" is a polysemous match. Since the toy-related term "plastic toys" and the standard number "GB 24613" already exist as unique matches in the query, the ambiguity judgment module enters the contextual implicit resolution process in the polysemous matching branch, automatically resolving "heavy metal detection" into 8 heavy metal migration detection items under the toy-related field; if its implicit resolution confidence is higher than the threshold θ, it is automatically adopted without interrupting the user; if its implicit resolution confidence is not higher than the threshold θ, the ambiguity clarification subprocess is triggered.

[0065] Step S5: Perform mapping processing based on the ambiguity judgment result, directly mapping "plastic toys" and "GB 24613" to the corresponding standard term identifiers, and determine the standard term identifier corresponding to "heavy metal detection" through contextual implicit resolution. The normalized term set is formed by all normalized entity terms.

[0066] Step S6 uses the normalized term set to call the downstream data query tool (in this embodiment, the final goal is to query the quotation, so the downstream data query tool corresponds to the downstream quotation system interface). Step S7: Return the quotation results to the user and display the normalized key terms for the user to check.

[0067] Example 3 This embodiment, based on Embodiment 1, provides an intelligent query device based on the pre-normalization of a business terminology dictionary.

[0068] In this embodiment, the intelligent query device is used to implement the intelligent query method in the embodiment.

[0069] In this embodiment, as Figure 5 As shown, the intelligent query device includes: The entity extraction module is used to receive natural language queries input by users and extract entity terms from the queries to obtain a set of terms to be normalized. The business terminology dictionary query module is connected to the entity extraction module. The business terminology dictionary query module is used to query and match each term in the set of terms to be normalized in the pre-built business terminology dictionary. Each entry in the business terminology dictionary includes at least a canonical term identifier, a set of term synonyms, and term domain classification information. The ambiguity determination module is connected to the business terminology dictionary query module. This module determines whether each term has domain ambiguity based on the query matching results. Ambiguity determination includes the following three scenarios: a) Unique match; b) Ambiguous match; c) No match; The clarification interaction module is connected to the ambiguity judgment module. The clarification interaction module is used to trigger the ambiguity clarification sub-process to determine the corresponding standard term identifier for terms whose judgment result from the ambiguity judgment module is a polysemous match. The session caching module is connected to the clarification interaction module. The session caching module is used to cache and reuse the confirmed term resolution results within the same session. The downstream data query tool calling module communicates with the session cache module, the clarification interaction module, and the ambiguity judgment module. The downstream data query tool calling module is used to call the downstream data query tool using the normalized term set and return the query results. Specifically, for uniquely matched entity terms, the ambiguity judgment module directly maps them to the corresponding canonical term identifier and sends them to the downstream data query tool calling module; for unmatched entity terms, the ambiguity judgment module marks them as unnormalized terms and sends them to the downstream data query tool calling module after retaining the original text.

[0070] In this embodiment, as Figure 5As shown, the intelligent query device also includes a traceability record module, which is communicatively connected to the entity extraction module, the business terminology dictionary query module, the ambiguity judgment module, the clarification interaction module, the session cache module, and the downstream call module. The traceability record module is used to aggregate and store process information from the entity extraction module, the business terminology dictionary query module, the ambiguity judgment module, the clarification interaction module, the session cache module, and the downstream call module based on a globally unique trace_id.

[0071] In this embodiment, the process information includes at least: the user's original natural language query, extracted entity terms, business terminology dictionary matching results and confidence levels, ambiguity resolution process, final normalized terms, and downstream call results.

[0072] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. An intelligent query method based on the pre-normalization of a business terminology dictionary, characterized in that, Includes the following steps: S1. Receive natural language queries input by the user; S2. Extract entity terms from the natural language query, and the extracted entity terms together constitute a set of terms to be normalized; S3. Query and match each entity term in the set of terms to be normalized in a pre-built business term dictionary, wherein each entry in the business term dictionary includes at least a canonical term identifier, a set of term synonyms, and term domain classification information. S4. Perform ambiguity judgment based on the query matching results. Ambiguity judgment includes the following three scenarios: a) Unique match: The entity term exists in the business term dictionary and corresponds to only one canonical term identifier; b) Polysemous matching: Entity terms correspond to multiple standardized terminology identifiers from different domains in the business terminology dictionary; c) No match: The entity term does not exist in the business term dictionary; S5. Perform mapping processing based on the results of the ambiguity judgment to obtain normalized entity terms; the normalized entity terms together constitute a normalized term set, and the mapping processing specifically includes: For a uniquely matching entity term, it is directly mapped to the corresponding canonical term identifier; For entity terms with multiple meanings, trigger the ambiguity clarification subprocess to determine the corresponding canonical term identifier; For unmatched entity terms, mark them as unnormalized terms and retain the original text; S6. Use the normalized term set to call the downstream data query tool to obtain the query results; S7. Return the query results to the user.

2. The intelligent query method as described in claim 1, characterized in that, In step S2, extracting entity terms from the natural language query specifically includes: The natural language query is segmented and preprocessed. Candidate entities are identified using a named entity recognition model; The candidate entities are verified and merged in the set of synonyms in the business term dictionary using the longest matching algorithm, with matching priority from longest to shortest. The entities that are successfully matched are added to the set of terms to be normalized.

3. The intelligent query method as described in claim 2, characterized in that, The named entity recognition model adopts an entity recognition model based on a pre-trained language model, and the candidate entities output by the entity recognition model are constrained, verified, corrected, or trained and optimized through the business term dictionary.

4. The intelligent query method as described in claim 1, characterized in that, In step S3, after performing query matching in the pre-built business terminology dictionary, a matching confidence score is calculated for each query matching result. The matching confidence score is obtained by weighting and combining at least one of the following factors: string similarity, source authority of the hit synonym expression in the business terminology dictionary, correlation between the terminology domain classification information of the normalized entity terminology in the current session and the terminology domain classification information corresponding to the candidate canonical terminology identifier, and the proportion of users who accepted it in historical sessions.

5. The intelligent query method as described in claim 4, characterized in that, In step S4, for polysemous entity terms, if there is a unique candidate standard term identifier among multiple candidate standard term identifiers whose term domain classification information can be uniquely determined by the current session context and whose matching confidence is higher than a preset threshold, then the unique candidate standard term identifier is directly adopted as the unique matching result; otherwise, the ambiguity clarification sub-process is triggered to determine its corresponding standard term identifier.

6. The intelligent query method as described in claim 1, characterized in that, In step S5, the ambiguity clarification sub-process includes: Check if there are resolution results for the same entity terms in the session cache; if so, use them directly. If not, check if there are other uniquely normalized entity terms in the current query; If there are other uniquely normalized terms, then context-based implicit resolution is performed, the matching confidence of the candidate canonical term identifiers is calculated, and if the matching confidence is higher than a preset threshold, it is automatically adopted; if the confidence is lower than the preset threshold, a list of candidate options is generated for the user and a clarification question is constructed. Receive user selections, write the user selection results to the session-level cache, and output the normalized results.

7. The intelligent query method as described in claim 6, characterized in that, The session-level cache is session-scoped, and entries in the session-level cache are valid for the lifetime of the session, with an idle lifespan that can be set. An entry in the session-level cache becomes invalid under any of the following conditions: The session has ended or the idle survival time has expired; Users can explicitly switch business areas within a session; The business terminology dictionary has undergone a hot update and the changes affect the resolution results of the entity terminology; The user explicitly corrected the previous resolution result.

8. The intelligent query method as described in claim 1, characterized in that, The business terminology dictionary provides query capabilities through an independent semantic service interface; the semantic service interface is deployed and versioned independently; both the business terminology dictionary and the semantic service interface are decoupled from downstream data query tools; and the business terminology dictionary supports online hot updates.

9. The intelligent query method as described in claim 8, characterized in that, The hot update is triggered by receiving a change event in the business terminology dictionary file, receiving the hot update instruction, or polling the business terminology dictionary version number at time intervals. The hot update process uses at least one concurrency control mechanism, either a read-write lock mechanism or a versioned snapshot mechanism, to ensure data consistency and atomicity.

10. An intelligent query device based on the pre-normalization of a business terminology dictionary, used to implement the intelligent query method as described in any one of claims 1 to 9, characterized in that, include: The entity extraction module is used to receive natural language queries input by users and extract entity terms from the queries to obtain a set of terms to be normalized. The business terminology dictionary query module is communicatively connected to the entity extraction module. The business terminology dictionary query module is used to query and match each term in the set of terms to be normalized in a pre-built business terminology dictionary. Each entry in the business terminology dictionary includes at least a canonical term identifier, a set of synonyms for terms, and terminology domain classification information. The ambiguity determination module is communicatively connected to the business terminology dictionary query module. This module determines whether each term has domain ambiguity based on the query matching results. Ambiguity determination includes the following three scenarios: a) Unique match; b) Ambiguous match; c) No match; The clarification interaction module is communicatively connected to the ambiguity judgment module. The clarification interaction module is used to trigger an ambiguity clarification sub-process to determine the corresponding standard term identifier for terms whose judgment result from the ambiguity judgment module is a polysemous match. A session caching module is communicatively connected to the clarification interaction module. The session caching module is used to cache and reuse the confirmed term resolution results within the same session. The downstream data query tool invocation module is communicatively connected to the session cache module, the clarification interaction module, and the ambiguity judgment module. The downstream data query tool invocation module is used to invoke the downstream data query tool using the normalized term set and return the query results. Specifically, for uniquely matching entity terms, the ambiguity judgment module directly maps them to the corresponding canonical term identifier and sends them to the downstream data query tool calling module; for unmatched entity terms, the ambiguity judgment module marks them as unnormalized terms and sends them to the downstream data query tool calling module after retaining the original text.

Images