An intelligent inquiry method based on deep semantic matching

Abstract

The application discloses an intelligent inquiry method based on deep semantic matching, which comprises the following steps: step one, obtaining original inquiry text and preprocessing to generate standard inquiry text and domain term set; step two, generating inquiry intention type and inquiry semantic vector through an improved BERT model; step three, performing knowledge filtering and path retrieval based on the domain term set and an ant colony algorithm to obtain a candidate knowledge text set; step four, performing semantic matching on the candidate knowledge text set and the inquiry semantic vector to obtain a matched knowledge text set; step five, generating inquiry response text through a large language model based on the inquiry intention type and the matched knowledge text set; step six, screening credible inquiry text through similarity matching based on the inquiry response text; and step seven, outputting the credible inquiry text and collecting feedback data. The improved BERT model and the ant colony algorithm improve the credibility of intelligent inquiry.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to an intelligent query method based on deep semantic matching. Background Technology

[0002] With the continuous development of service digitalization, industry knowledge service platforms, and intelligent customer service systems, intelligent inquiry technology for business consulting and professional domain Q&A has received widespread attention. Existing intelligent inquiry methods typically rely on keyword matching or rule templates to achieve information retrieval and response generation, but in practical applications, they generally suffer from the following problems:

[0003] User queries are characterized by diverse expressions, complex sentence structures, and implicit domain terminology. Traditional query parsing methods based on keywords or shallow features struggle to accurately identify users' true intentions, easily leading to intent confusion or question-answer deviations. Existing methods often employ static rules or simple similarity ranking during knowledge retrieval, making it difficult to achieve efficient, globally optimized retrieval path selection across large-scale domain knowledge bases and external documents. This results in insufficient relevance of candidate knowledge texts, impacting the quality of subsequent question answers. While some query methods incorporate deep learning models for semantic representation, they often treat intent recognition, semantic vector generation, and knowledge matching processes in isolation, lacking a unified semantic modeling and collaborative optimization mechanism. This makes it difficult to simultaneously consider intent constraints, semantic matching accuracy, and response credibility in complex query scenarios. Generative question-answering models, lacking effective semantic filtering and feedback mechanisms, are prone to producing responses inconsistent with standard answers or lacking credibility, failing to meet the application requirements for high professionalism and accuracy.

[0004] Therefore, how to provide an intelligent query method based on deep semantic matching is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an intelligent query method based on deep semantic matching. This invention comprehensively utilizes natural language processing, deep learning, and intelligent optimization algorithms. Through an improved BERT model, it achieves question parsing, intent recognition, and semantic vector representation of user query text. It also incorporates a domain-term-driven ant colony algorithm for path retrieval and knowledge filtering from the domain knowledge base and external documents. Under the constraint of query intent type, a large language model is introduced to generate query response text. Simultaneously, semantic similarity matching and feedback mechanisms are used to verify the credibility of the query results and adaptively optimize them. This invention possesses advantages such as strong semantic understanding capabilities, good retrieval path optimization, high credibility of question-answering results, and strong self-learning capabilities.

[0006] An intelligent query method based on deep semantic matching according to an embodiment of the present invention includes the following steps:

[0007] Step 1: Obtain the original query text input by the user, perform text preprocessing on the original query text to obtain the standard query text and domain terminology set;

[0008] Step 2: Input the standard query text into the improved BERT model for query parsing, intent recognition, and semantic convergence to obtain the query intent type and query semantic vector; the improved BERT model includes a word segmenter, an embedding layer, a BERT encoder, a classification layer, and a semantic convergence layer;

[0009] Step 3: Based on the domain terminology set, the ant colony algorithm is used to perform knowledge filtering and path retrieval on the pre-built domain knowledge base and the external domain document set to obtain a candidate knowledge text set;

[0010] Step 4: Semantically encode the candidate knowledge text set to generate a knowledge semantic vector set, and perform semantic matching between the knowledge semantic vector set and the query semantic vector to obtain the matching knowledge text set;

[0011] Step 5: Under the constraint of query intent type, the matching knowledge text set and query semantic vector are used to generate query response text through a large language model;

[0012] Step Six: Perform similarity matching between the query response text and the preset set of standard response texts to filter and generate credible query texts;

[0013] Step 7: Output credible query text and collect feedback data to optimize and update path retrieval and semantic matching.

[0014] Optionally, step one specifically includes:

[0015] The text preprocessing includes word segmentation, text normalization, and structuring.

[0016] The word segmentation process uses a Chinese word segmentation tool to segment the original query text into multiple word segments, which are then arranged into a text word segmentation sequence according to the segmentation order.

[0017] The text normalization process filters out noise symbols and normalizes whitespace characters in the text segmentation sequence to obtain a normalized text segmentation sequence.

[0018] The structured processing performs string concatenation operations on the standardized text segmentation sequence according to the segmentation order to obtain standard query text;

[0019] A domain terminology list is set up, and the standardized text segmentation sequence is identified with domain terms according to the domain terminology list to obtain a set of domain terms.

[0020] Optionally, step two specifically includes:

[0021] The standard query text is processed by a word segmenter to perform sub-word segmentation and sequence encoding to generate a text tag sequence. The text tag sequence includes several text tags, which are divided into valid tags and padding tags.

[0022] Each text token in the text token sequence is numbered and mapped using the built-in vocabulary of the BERT intent recognition model to obtain a token number sequence;

[0023] Based on the text tag sequence, an attention mask sequence is constructed. Specifically, when the text tag is a valid tag, the corresponding attention mask value is 1, and when the text tag is a padding tag, the corresponding attention mask value is 0.

[0024] The tag number sequence is mapped to a tag embedding matrix through the embedding layer;

[0025] The label embedding matrix and attention mask sequence are input into the BERT encoder to obtain the hidden state matrix;

[0026] Based on the hidden state matrix, the hidden state vector corresponding to the classification label position is selected, and the hidden state vector is linearly mapped through the classification layer to obtain the query intent classification vector.

[0027] Softmax normalization is applied to the query intent classification vector to generate a query intent probability vector.

[0028] Perform the maximum probability selection operation on the query intent probability vector to obtain the query intent type index;

[0029] Set up a query intent type mapping relationship, and map the query intent type index to the query intent type according to the query intent type mapping relationship;

[0030] The hidden state matrix is ​​input into the semantic convergence layer, and a mask-weighted average operation is performed on the hidden state vectors corresponding to the effective labels to generate the semantic convergence vector. The semantic convergence vector is then converted into a query semantic vector through a linear mapping matrix.

[0031] Optionally, the step of inputting the label embedding matrix and the attention mask sequence into the BERT encoder to obtain the hidden state matrix specifically includes:

[0032] The BERT encoder includes several Transformer encoding layers;

[0033] In each Transformer encoding layer, the input embedding matrix is ​​transformed into a query matrix, a key matrix, and a value matrix through three sets of trainable linear mapping matrices; where the input embedding matrix of the first Transformer encoding layer is the label embedding matrix, and the input embedding matrix of the second and subsequent Transformer encoding layers is the output hidden state matrix of the previous Transformer encoding layer.

[0034] Based on the attention mask sequence, an attention mask matrix is ​​constructed. Specifically, if the j-th element of the attention mask sequence is 1, then the value of the element in the i-th row and j-th column of the attention mask matrix is ​​set to 0; if the j-th element of the attention mask sequence is 0, then the value of the element in the i-th row and j-th column of the attention mask matrix is ​​set to negative infinity.

[0035] The attention score matrix is ​​obtained by scaling the dot product of the query matrix and the key matrix. The attention score matrix is ​​then added element-wise to the attention mask matrix and normalized using Softmax to obtain the attention weight matrix.

[0036] The attention weight matrix and the value matrix are multiplied to obtain the attention feature matrix; the attention feature matrix is ​​residually concatenated with the input embedding matrix, and the intermediate hidden state matrix is ​​obtained through layer normalization.

[0037] The intermediate hidden state matrix is ​​input into the feedforward network, which includes two layers of linear mapping and a nonlinear activation function, to obtain the feedforward hidden state matrix.

[0038] The feedforward hidden state matrix and the intermediate hidden state matrix are residually connected and layer normalized to obtain the output hidden state matrix of the Transformer coding layer.

[0039] The output hidden state matrix of the last Transformer coding layer is used as the hidden state matrix of the BERT encoder.

[0040] Optionally, step three specifically includes:

[0041] A domain terminology set is obtained, and based on the domain terminology set, terminology matching and filtering are performed in a pre-built domain knowledge base and an external domain document set to obtain a limited knowledge text set; the limited knowledge text set includes several knowledge texts that meet the limiting conditions of the domain terminology set.

[0042] Each knowledge text in the limited knowledge text set is mapped to a retrieval node to obtain a retrieval node set. Based on the retrieval node set, a retrieval path graph is constructed, and the semantic edges between retrieval nodes are the connection relationships of the retrieval path.

[0043] Based on the retrieval path graph, the pheromone matrix and heuristic function matrix of the ant colony algorithm are initialized, and the number of iteration rounds, number of ants, maximum path length and path evaluation rules are set.

[0044] The path evaluation rule is obtained by weighted fusion of domain terminology coverage and path length;

[0045] The domain terminology coverage is specifically defined as follows: taking the union of the domain terminology sets corresponding to all retrieval nodes contained in the retrieval path, and calculating the ratio of the number of elements in the intersection of the union and the domain terminology set to the total number of elements in the domain terminology set;

[0046] The path length is specifically defined as the reciprocal of the number of search nodes contained in the search path.

[0047] In the current iteration, set the starting search node for each ant and construct a set of search paths based on the number of ants;

[0048] Update the pheromone matrix based on the set of retrieval paths in the current iteration round;

[0049] The search path set from the last iteration round is filtered through a path selection operation to obtain a candidate search path set;

[0050] Based on the candidate retrieval path set, extract the knowledge text contained in all candidate retrieval paths, and remove duplicates to obtain the candidate knowledge text set.

[0051] Optionally, in the current iteration, setting the starting retrieval node for each ant and constructing a retrieval path set based on the number of ants specifically includes:

[0052] At the current retrieval node, based on the pheromone matrix and the heuristic function matrix, each ant calculates the transition probability from retrieval node i to retrieval node j;

[0053] Based on the transition probability, each ant performs a path construction operation to obtain a retrieval path. Specifically, it samples the set of reachable retrieval nodes of the current retrieval node i according to the transition probability to obtain the next retrieval node j, and adds the next retrieval node j to the retrieval path. The sampling and addition operation is repeated until the path termination condition is met. The path termination condition includes reaching the maximum path length or the current retrieval node having no reachable retrieval nodes.

[0054] The search paths constructed by all ants are combined into a search path set.

[0055] Optionally, the path selection operation specifically includes: calculating the evaluation value of each retrieval path according to the path evaluation rules, sorting the paths in descending order according to the evaluation values, and selecting the top K retrieval paths to form a candidate retrieval path set.

[0056] Optionally, step four specifically includes:

[0057] Each candidate knowledge text in the candidate knowledge text set is semantically encoded using the BERT encoder to obtain a candidate knowledge semantic vector.

[0058] Cosine similarity is used to calculate the cosine similarity between the query semantic vector and each candidate knowledge semantic vector to obtain a semantic matching score.

[0059] The candidate knowledge texts are sorted in descending order according to their semantic matching scores.

[0060] Set a semantic matching threshold and a text filtering quantity. Select candidate knowledge texts whose semantic matching scores are greater than or equal to the semantic matching threshold and whose ranking is within the text filtering quantity as matching knowledge texts. Aggregate all matching knowledge texts to obtain a set of matching knowledge texts.

[0061] Optionally, step five specifically includes:

[0062] Each matching knowledge text in the matching knowledge text set is semantically encoded using the BERT encoder to generate a set of matching knowledge semantic vectors.

[0063] The matching knowledge semantic vector set is weighted and averaged to obtain the matching knowledge convergence vector; the matching knowledge convergence vector is concatenated with the query semantic vector to generate a joint semantic feature vector.

[0064] The query intent type is used as the generation control condition, and the joint semantic feature vector is input into the large language model to generate query response text.

[0065] Optionally, step six specifically includes:

[0066] A standard response text set is defined, which includes several standard response texts, and each standard response text corresponds to a standard response index;

[0067] The query response text and each standard response text in the standard response text set are semantically encoded using the BERT encoder to generate a set of response semantic vectors and a set of standard response semantic vectors.

[0068] The similarity between the response semantic vector and each standard response semantic vector is calculated using the cosine similarity formula to obtain a similarity score vector.

[0069] Perform a maximum value selection operation on the similarity score vector to obtain the maximum similarity score, and obtain the index of the standard response in the standard response text set with the maximum similarity score;

[0070] Set a credibility threshold. If the maximum similarity score is greater than or equal to the credibility threshold, then the standard answer text corresponding to the standard answer index will be regarded as the credible query text.

[0071] If the maximum similarity score is less than the credibility threshold, the process of regenerating the query response text is triggered.

[0072] Optionally, step seven specifically includes:

[0073] The feedback data includes query semantic vectors, matching knowledge text sets, and similarity score vectors.

[0074] Based on the feedback data, the retrieval paths in the path retrieval process are evaluated, the retrieval paths that generate credible query text are marked as valid paths, and the retrieval nodes in the valid paths are used as positive feedback path samples.

[0075] The similarity score vector is used as the path feedback weight. According to the pheromone update rule of the ant colony algorithm, the pheromone corresponding to the effective path is positively updated, and the pheromone corresponding to the retrieval path that does not generate credible query text is decayed and updated. The pheromone is the matrix element of the corresponding retrieval node pair in the pheromone matrix.

[0076] Based on feedback data, the semantic matching threshold and the number of texts filtered during the semantic matching process are adaptively updated.

[0077] Optionally, the inference process of the improved BERT model is executed by a GPU parallel computing unit; the path retrieval process of the ant colony algorithm is executed by a multi-core processor parallel thread; and the generation process of the large language model is executed by an FPGA inference acceleration unit.

[0078] The beneficial effects of this invention are:

[0079] First, this invention performs text preprocessing on the original query text input by the user, and jointly completes question parsing, intent recognition and semantic convergence in the improved BERT model to obtain the query intent type and query semantic vector. This avoids the problem of the separation between intent recognition and semantic features in traditional query systems, and improves the consistency and stability of query semantic understanding.

[0080] Secondly, based on the domain terminology set, this invention introduces the ant colony algorithm to perform knowledge filtering and path retrieval on the domain knowledge base and external domain document set. Through pheromone updating and path evaluation rules, the retrieval path is globally optimized, enabling the candidate knowledge text set to cover more relevant knowledge content while meeting the domain terminology constraints, effectively reducing irrelevant knowledge and improving the targeting and efficiency of knowledge retrieval.

[0081] Furthermore, under the constraint of query intent type, this invention inputs the matching knowledge text set and query semantic vector into a large language model to generate query response text, and generates credible query text by filtering through similarity matching with the standard response text set. Combined with feedback data, the path retrieval and semantic matching processes are optimized and updated, so that the query response process forms a closed-loop optimization mechanism, thereby continuously improving the accuracy, reliability and consistency of query response in multi-round queries and complex scenarios.

[0082] In summary, this invention organically combines the improved BERT model, ant colony algorithm, and large language model to achieve query understanding, knowledge retrieval, semantic matching, and generate credible question answers within a deep semantic matching framework. This effectively improves the semantic understanding capability, knowledge matching accuracy, and query response credibility of intelligent query in complex domain applications. Attached Figure Description

[0083] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0084] Figure 1 This is a schematic diagram of an intelligent query method based on deep semantic matching proposed in this invention;

[0085] Figure 2 This is a flowchart of the improved BERT model structure in an intelligent query method based on deep semantic matching proposed in this invention.

[0086] Figure 3 This is a knowledge path retrieval structure diagram based on the ant colony algorithm in an intelligent query method based on deep semantic matching proposed in this invention. Detailed Implementation

[0087] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0088] refer to Figures 1-3 A method for intelligent querying based on deep semantic matching includes the following steps:

[0089] Step 1: Obtain the original query text input by the user, perform text preprocessing on the original query text to obtain the standard query text and domain terminology set;

[0090] Step 2: Input the standard query text into the improved BERT model for query parsing, intent recognition, and semantic convergence to obtain the query intent type and query semantic vector; the improved BERT model includes a word segmenter, an embedding layer, a BERT encoder, a classification layer, and a semantic convergence layer;

[0091] Step 3: Based on the domain terminology set, the ant colony algorithm is used to perform knowledge filtering and path retrieval on the pre-built domain knowledge base and the external domain document set to obtain a candidate knowledge text set;

[0092] Step 4: Semantically encode the candidate knowledge text set to generate a knowledge semantic vector set, and perform semantic matching between the knowledge semantic vector set and the query semantic vector to obtain the matching knowledge text set;

[0093] Step 5: Under the constraint of query intent type, the matching knowledge text set and query semantic vector are used to generate query response text through a large language model;

[0094] Step Six: Perform similarity matching between the query response text and the preset set of standard response texts to filter and generate credible query texts;

[0095] Step 7: Output credible query text and collect feedback data to optimize and update path retrieval and semantic matching.

[0096] In this embodiment, step one specifically includes:

[0097] The text preprocessing includes word segmentation, text normalization, and structuring.

[0098] The word segmentation process uses a Chinese word segmentation tool to segment the original query text into multiple word segments, which are then arranged into a text word segmentation sequence according to the segmentation order.

[0099] The text normalization process filters out noise symbols and normalizes whitespace characters in the text segmentation sequence to obtain a normalized text segmentation sequence.

[0100] The structured processing performs string concatenation operations on the standardized text segmentation sequence according to the segmentation order to obtain standard query text;

[0101] A domain terminology list is set up, and the standardized text segmentation sequence is identified with domain terms according to the domain terminology list to obtain a set of domain terms.

[0102] For example, the user's original query text is "How should I troubleshoot an interface timeout exception that occurs during system operation?". After word segmentation, the resulting word segmentation sequence includes "system operation", "process", "occur", "interface", "timeout", "exception", "should", "how", "troubleshoot", and "?". After text normalization to remove irrelevant symbols and unify their expressions, a standardized text segmentation sequence is obtained. In the structured processing stage, the standardized text segmentation sequence is concatenated in sequence to generate a standard query text. Subsequently, the standardized text segmentation sequence is matched and identified based on a pre-defined domain terminology. Among them, "interface", "timeout", "exception", and "troubleshoot" match the terms in the domain terminology and are identified as domain terms, thus generating a domain terminology set {interface, timeout, exception, troubleshoot}.

[0103] In this embodiment, step two specifically includes:

[0104] The standard query text is processed by a word segmenter to perform sub-word segmentation and sequence encoding to generate a text tag sequence. The text tag sequence includes several text tags, which are divided into valid tags and padding tags.

[0105] Each text token in the text token sequence is numbered and mapped using the built-in vocabulary of the BERT intent recognition model to obtain a token number sequence;

[0106] Based on the text tag sequence, an attention mask sequence is constructed. Specifically, when the text tag is a valid tag, the corresponding attention mask value is 1, and when the text tag is a padding tag, the corresponding attention mask value is 0.

[0107] The tag number sequence is mapped to a tag embedding matrix through the embedding layer;

[0108] The label embedding matrix and attention mask sequence are input into the BERT encoder to obtain the hidden state matrix;

[0109] Based on the hidden state matrix, the hidden state vector corresponding to the classification label position is selected, and the hidden state vector is linearly mapped through the classification layer to obtain the query intent classification vector.

[0110] Softmax normalization is applied to the query intent classification vector to generate a query intent probability vector.

[0111] Perform the maximum probability selection operation on the query intent probability vector to obtain the query intent type index;

[0112] Set up a query intent type mapping relationship, and map the query intent type index to the query intent type according to the query intent type mapping relationship;

[0113] For example, the user's original query text is "What are the latest policy requirements for grid connection of distributed photovoltaic projects?". After text preprocessing, a standard query text is obtained and input into the BERT intent recognition model. The BERT intent recognition model pre-defines multiple query intent types, which represent the semantic purpose of the user's query. Different query intent types correspond to different query processing paths. For example, query intent types include policy inquiry intent, business process consultation intent, application condition consultation intent, and information retrieval intent. After the standard query text is processed by the BERT encoder and classification layer, a query intent probability vector is output. In the query intent probability vector, the probability value representing business process consultation intent is the highest. After the maximum probability selection operation, the query intent type corresponding to this query text is determined to be business process consultation intent.

[0114] The hidden state matrix is ​​input into the semantic convergence layer, and a mask-weighted average operation is performed on the hidden state vectors corresponding to the effective labels to generate the semantic convergence vector. The semantic convergence vector is then converted into a query semantic vector through a linear mapping matrix.

[0115] In this embodiment, the step of inputting the label embedding matrix and the attention mask sequence into the BERT encoder to obtain the hidden state matrix specifically includes:

[0116] The BERT encoder includes several Transformer encoding layers;

[0117] In each Transformer encoding layer, the input embedding matrix is ​​transformed into a query matrix, a key matrix, and a value matrix through three sets of trainable linear mapping matrices; where the input embedding matrix of the first Transformer encoding layer is the label embedding matrix, and the input embedding matrix of the second and subsequent Transformer encoding layers is the output hidden state matrix of the previous Transformer encoding layer.

[0118] Based on the attention mask sequence, an attention mask matrix is ​​constructed. Specifically, if the j-th element of the attention mask sequence is 1, then the value of the element in the i-th row and j-th column of the attention mask matrix is ​​set to 0; if the j-th element of the attention mask sequence is 0, then the value of the element in the i-th row and j-th column of the attention mask matrix is ​​set to negative infinity.

[0119] The attention score matrix is ​​obtained by scaling the dot product of the query matrix and the key matrix. The attention score matrix is ​​then added element-wise to the attention mask matrix and normalized using Softmax to obtain the attention weight matrix.

[0120] The attention weight matrix and the value matrix are multiplied to obtain the attention feature matrix; the attention feature matrix is ​​residually concatenated with the input embedding matrix, and the intermediate hidden state matrix is ​​obtained through layer normalization.

[0121] The intermediate hidden state matrix is ​​input into the feedforward network, which includes two layers of linear mapping and a nonlinear activation function, to obtain the feedforward hidden state matrix.

[0122] The feedforward hidden state matrix and the intermediate hidden state matrix are residually connected and layer normalized to obtain the output hidden state matrix of the Transformer coding layer.

[0123] The output hidden state matrix of the last Transformer coding layer is used as the hidden state matrix of the BERT encoder.

[0124] In this embodiment, step three specifically includes:

[0125] A domain terminology set is obtained, and based on the domain terminology set, terminology matching and filtering are performed in a pre-built domain knowledge base and an external domain document set to obtain a limited knowledge text set; the limited knowledge text set includes several knowledge texts that meet the limiting conditions of the domain terminology set.

[0126] The domain knowledge base is a pre-built and locally stored data set used to store structured or textual knowledge content related to the target application domain; the external domain document set is a document set obtained through external data sources or external interfaces, used to provide supplementary textual information related to the target application domain.

[0127] Each knowledge text in the limited knowledge text set is mapped to a retrieval node to obtain a retrieval node set. Based on the retrieval node set, a retrieval path graph is constructed, and the semantic edges between retrieval nodes are the connection relationships of the retrieval path.

[0128] Based on the retrieval path graph, the pheromone matrix and heuristic function matrix of the ant colony algorithm are initialized, and the number of iteration rounds, number of ants, maximum path length and path evaluation rules are set.

[0129] The path evaluation rule is obtained by weighted fusion of domain terminology coverage and path length;

[0130] The domain terminology coverage is specifically defined as follows: taking the union of the domain terminology sets corresponding to all retrieval nodes contained in the retrieval path, and calculating the ratio of the number of elements in the intersection of the union and the domain terminology set to the total number of elements in the domain terminology set;

[0131] The path length is specifically defined as the reciprocal of the number of search nodes contained in the search path.

[0132] In this invention, domain terminology coverage directly reflects the semantic coverage of the knowledge text contained in the retrieval path to the query domain terminology set. Higher coverage indicates that the retrieval path can hit more domain terms directly related to the core semantics of the query, which helps to filter out paths that are only relevant at local nodes but whose overall semantics deviate from the query intent, thereby improving the domain consistency and relevance of the candidate knowledge text set. Path length, measured by taking the reciprocal of the number of retrieval nodes, is used to characterize the simplicity of the retrieval path. The shorter the path, the fewer retrieval node jumps are required to obtain relevant knowledge text, helping to avoid semantic drift and irrelevant node accumulation problems caused by excessively long paths, and reducing the computational complexity of the path search process. By weighted fusion of domain terminology coverage and path length, the ant colony algorithm can simultaneously guide ants to prioritize retrieval paths with sufficient semantic coverage and compact structure during path search, thereby improving the stability of path retrieval results and overall retrieval efficiency while ensuring knowledge relevance.

[0133] In the current iteration, a starting search node is set for each ant, and a set of search paths is constructed based on the number of ants, specifically as follows:

[0134] At the current retrieval node, based on the pheromone matrix and the heuristic function matrix, each ant calculates the transition probability from retrieval node i to retrieval node j;

[0135] Based on the transition probability, each ant performs a path construction operation to obtain a retrieval path. Specifically, it samples the set of reachable retrieval nodes of the current retrieval node i according to the transition probability to obtain the next retrieval node j, and adds the next retrieval node j to the retrieval path. The sampling and addition operation is repeated until the path termination condition is met. The path termination condition includes reaching the maximum path length or the current retrieval node having no reachable retrieval nodes.

[0136] For example, in a certain iteration, the starting search node of an ant is set as the knowledge text node that best matches "equipment maintenance" in the domain terminology set. At this starting search node, the ant calculates the transition probability from the current search node i to each search node j in its reachable search node set based on the pheromone matrix and heuristic function matrix. Assuming that the reachable search nodes include "fault diagnosis instructions", "maintenance process documents", and "operation specification text", the ant randomly samples according to the corresponding transition probabilities, selects the search node corresponding to "maintenance process documents" with the highest transition probability as the next search node, and adds this search node to the current search path. Subsequently, with this search node as the new current search node, the transition probability calculation and sampling operation are repeated, and search nodes such as "operation specification text" are added to the search path in turn, until the search path length reaches the preset maximum path length or there are no reachable search nodes for the current search node, thus forming a complete search path.

[0137] Combine all the search paths constructed by the ants into a search path set;

[0138] Update the pheromone matrix based on the set of retrieval paths in the current iteration round;

[0139] The search path set from the last iteration round is filtered through a path selection operation to obtain a candidate search path set;

[0140] Based on the candidate retrieval path set, extract the knowledge text contained in all candidate retrieval paths, and remove duplicates to obtain the candidate knowledge text set.

[0141] In this embodiment, the path selection operation specifically includes: calculating the evaluation value of each retrieval path according to the path evaluation rules, sorting the paths in descending order according to the evaluation values, and selecting the top K retrieval paths to form a candidate retrieval path set.

[0142] In this embodiment, the step of setting the starting search node for each ant in the current iteration round and constructing a search path set based on the number of ants specifically includes:

[0143] At the current retrieval node, based on the pheromone matrix and the heuristic function matrix, each ant calculates the transition probability from retrieval node i to retrieval node j;

[0144] Based on the transition probability, each ant performs a path construction operation to obtain a retrieval path. Specifically, it samples the set of reachable retrieval nodes of the current retrieval node i according to the transition probability to obtain the next retrieval node j, and adds the next retrieval node j to the retrieval path. The sampling and addition operation is repeated until the path termination condition is met. The path termination condition includes reaching the maximum path length or the current retrieval node having no reachable retrieval nodes.

[0145] For example, in a certain iteration, the starting search node of an ant is set as the knowledge text node that best matches "equipment maintenance" in the domain terminology set. At this starting search node, the ant calculates the transition probability from the current search node i to each search node j in its reachable search node set based on the pheromone matrix and heuristic function matrix. Assuming that the reachable search nodes include "fault diagnosis instructions", "maintenance process documents", and "operation specification text", the ant randomly samples according to the corresponding transition probabilities, selects the search node corresponding to "maintenance process documents" with the highest transition probability as the next search node, and adds this search node to the current search path. Subsequently, with this search node as the new current search node, the transition probability calculation and sampling operation are repeated, and search nodes such as "operation specification text" are added to the search path in turn, until the search path length reaches the preset maximum path length or there are no reachable search nodes for the current search node, thus forming a complete search path.

[0146] The search paths constructed by all ants are combined into a search path set.

[0147] In this embodiment, step four specifically includes:

[0148] Each candidate knowledge text in the candidate knowledge text set is semantically encoded using the BERT encoder to obtain a candidate knowledge semantic vector.

[0149] Cosine similarity is used to calculate the cosine similarity between the query semantic vector and each candidate knowledge semantic vector to obtain a semantic matching score.

[0150] The candidate knowledge texts are sorted in descending order according to their semantic matching scores.

[0151] Set a semantic matching threshold and a text filtering quantity. Select candidate knowledge texts whose semantic matching scores are greater than or equal to the semantic matching threshold and whose ranking is within the text filtering quantity as matching knowledge texts. Aggregate all matching knowledge texts to obtain a set of matching knowledge texts.

[0152] In this embodiment, step five specifically includes:

[0153] Each matching knowledge text in the matching knowledge text set is semantically encoded using the BERT encoder to generate a set of matching knowledge semantic vectors.

[0154] The matching knowledge semantic vector set is weighted and averaged to obtain the matching knowledge convergence vector.

[0155] The matching knowledge convergence vector and the query semantic vector are concatenated to generate a joint semantic feature vector.

[0156] The query intent type is used as the generation control condition, and the joint semantic feature vector is input into the large language model to generate query response text.

[0157] In this embodiment, step six specifically includes:

[0158] A standard response text set is defined, which includes several standard response texts, and each standard response text corresponds to a standard response index;

[0159] The query response text and each standard response text in the standard response text set are semantically encoded using the BERT encoder to generate a set of response semantic vectors and a set of standard response semantic vectors.

[0160] The similarity between the response semantic vector and each standard response semantic vector is calculated using the cosine similarity formula to obtain a similarity score vector.

[0161] Perform a maximum value selection operation on the similarity score vector to obtain the maximum similarity score, and obtain the index of the standard response in the standard response text set with the maximum similarity score;

[0162] Set a credibility threshold. If the maximum similarity score is greater than or equal to the credibility threshold, then the standard answer text corresponding to the standard answer index will be regarded as the credible query text.

[0163] If the maximum similarity score is less than the credibility threshold, the process of regenerating the query response text is triggered.

[0164] In this embodiment, step seven specifically includes:

[0165] The feedback data includes query semantic vectors, matching knowledge text sets, and similarity score vectors.

[0166] Based on the feedback data, the retrieval paths in the path retrieval process are evaluated, the retrieval paths that generate credible query text are marked as valid paths, and the retrieval nodes in the valid paths are used as positive feedback path samples.

[0167] The similarity score vector is used as the path feedback weight. According to the pheromone update rule of the ant colony algorithm, the pheromone corresponding to the effective path is positively updated, and the pheromone corresponding to the retrieval path that does not generate credible query text is decayed and updated. The pheromone is the matrix element of the corresponding retrieval node pair in the pheromone matrix.

[0168] For example, in a complete query processing, the system generates 10 retrieval paths. The similarity scores of the query response texts output by paths 3 and 7 with the standard response text set are 0.92 and 0.88 respectively, both exceeding the preset credibility threshold, and are therefore marked as valid paths. The remaining paths fail to generate credible query text and are considered invalid paths. The system uses the similarity score vector as path feedback weights to update the elements in the pheromone matrix: for retrieval node pairs contained in path 3, such as "equipment specification node" to "maintenance standard node" and "maintenance standard node" to "operation process node," the pheromone value between each pair of nodes will be positively updated based on the similarity score of 0.92; while for node pairs appearing in unselected paths, such as "old version document node" to "irrelevant terminology node" in path 5, since their query responses do not reach the credibility threshold, the system reduces their corresponding pheromone values ​​by a decay coefficient to prevent pheromone from continuously accumulating on low-value paths. Through positive reinforcement and attenuation, the pheromone matrix gradually strengthens high-quality paths and suppresses low-quality paths, providing better path guidance for ant colony iteration.

[0169] Based on feedback data, the semantic matching threshold and the number of texts filtered during the semantic matching process are adaptively updated.

[0170] In this embodiment, the inference process of the improved BERT model is executed by the GPU parallel computing unit; the path retrieval process of the ant colony algorithm is executed by the parallel thread of the multi-core processor; and the generation process of the large language model is executed by the FPGA inference acceleration unit.

[0171] Example 1

[0172] To verify the feasibility of this invention in practice, the method was applied to the intelligent inquiry system of an industrial knowledge service platform. This platform provides question-and-answer services for engineers and compliance personnel in manufacturing enterprises, offering textual information such as standard clause analysis, industry specification matching, equipment usage instructions, and quality judgment rules. Users submit complex inquiries with business context, such as: "What are the specific safety requirements for lithium battery packaging lines?", "What are the standard maintenance cycle times for equipment under high temperature and humidity conditions?", and "Which clauses mandate that operators wear personal protective equipment?".

[0173] Traditional intelligent query systems are mostly based on keyword matching or manually constructed rule templates. When dealing with complex problems involving domain terminology, implicit intent, or semantic jumps, they have the following drawbacks: it is difficult to accurately identify the business intent category corresponding to the query; it is difficult to efficiently extract text with high relevance to the query from tens of thousands of standard documents; and the answers are often generalized and lack specificity, affecting user confidence.

[0174] During implementation, text preprocessing operations are performed on the user-input query text, including word segmentation, symbol filtering, and structured concatenation, and domain terms are extracted based on a pre-defined standard glossary. An improved BERT model is used to simultaneously perform query intent recognition and query semantic vector generation on the standard query text. In the knowledge retrieval stage, candidate text sets are filtered based on domain terms and using an ant colony algorithm. In the semantic matching stage, cosine similarity calculation is performed between candidate knowledge texts and query semantic vectors. By setting a threshold and the number of selections, texts with high semantic fit are output as the final response material. Subsequently, query intent tags, matching knowledge vectors, and query vectors are fused and input into a large language model to generate natural language answers. Finally, credible question-and-answer texts are selected through semantic comparison with a standard answer database.

[0175] To verify the effectiveness of the method of this invention, a test set containing 2087 real enterprise user queries was constructed, covering 14 query intents and 412 terminology types. The invention was then compared with the following three schemes: Scheme A is a traditional keyword matching system; Scheme B is a standard BERT semantic matching question-answering system; and Scheme C is an intent recognition and standard template generation system. The statistical results are shown in Table 1.

[0176] Table 1 Comparison of performance of different solutions in enterprise knowledge query tasks

[0177] Comparison indicators Comparison Option A Comparison with Option B Comparison Option C Method of the present invention Intent recognition accuracy (%) 76.4 86.2 90.5 95.1 Query response accuracy (Top-1) 68.1 82.7 84.4 92.6 Inquiry response fluency rating (out of 5) 3.02 4.12 4.26 4.71 Top-3 Relevance Hit Rate (%) 72.9 83.3 86.0 94.5 Average coverage of matching paths (%) 59.3 71.5 77.8 89.4 Average response time (seconds) 0.91 1.53 1.67 1.78

[0178] As shown in Table 1, the method of this invention significantly outperforms the comparative schemes in several core metrics. Regarding intent recognition accuracy, the method of this invention reaches 95.1%, an improvement of 18.7 percentage points compared to comparative scheme A and nearly 9 percentage points compared to comparative scheme B. This indicates that the joint modeling of the BERT encoder and semantic convergence in the method of this invention can more accurately identify the semantic purpose of complex queries. In terms of Top-1 query response accuracy, the method of this invention achieves 92.6%, an improvement of 8.2 percentage points compared to comparative scheme C, demonstrating that the method of this invention can output answers that better match the user's query. Regarding query response fluency, this invention achieves a score of 4.71, indicating natural and fluent language expression and a clear content structure.

[0179] Furthermore, in terms of the Top-3 hit rate for relevance, the method of this invention achieves a hit rate of 94.5%, significantly higher than all comparative schemes, indicating that this invention possesses a higher relevance preservation capability during the multi-candidate generation process. Regarding the average coverage of matching paths, the method of this invention, relying on the ant colony algorithm's path retrieval strategy, achieves a coverage rate of 89.4%, significantly better than the comparative schemes, effectively solving the problem of insufficient relevance of candidate content in traditional systems. The average response time of this invention is 1.78 seconds, slightly higher than the comparative schemes, but still within an acceptable range for practical applications, and significantly improves intelligence and reliability.

[0180] This invention, while ensuring response efficiency, comprehensively improves the semantic recognition, content generation, and knowledge matching capabilities of intelligent query systems, demonstrating strong practicality and engineering feasibility.

[0181] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An intelligent query method based on deep semantic matching, characterized in that, Includes the following steps: Step 1: Obtain the original query text input by the user, perform text preprocessing on the original query text to obtain the standard query text and domain terminology set; Step 2: Input the standard query text into the improved BERT model for query parsing, intent recognition, and semantic convergence to obtain the query intent type and query semantic vector; the improved BERT model includes a word segmenter, an embedding layer, a BERT encoder, a classification layer, and a semantic convergence layer; Step 3: Based on the domain terminology set, the ant colony algorithm is used to perform knowledge filtering and path retrieval on the pre-built domain knowledge base and the external domain document set to obtain a candidate knowledge text set; Step 4: Semantically encode the candidate knowledge text set to generate a knowledge semantic vector set, and perform semantic matching between the knowledge semantic vector set and the query semantic vector to obtain the matching knowledge text set; Step 5: Under the constraint of query intent type, the matching knowledge text set and query semantic vector are used to generate query response text through a large language model; Step Six: Perform similarity matching between the query response text and the preset set of standard response texts to filter and generate credible query texts; Step 7: Output credible query text and collect feedback data to optimize and update path retrieval and semantic matching.

2. The intelligent query method based on deep semantic matching according to claim 1, characterized in that, Step one specifically includes: The text preprocessing includes word segmentation, text normalization, and structuring. The word segmentation process uses a Chinese word segmentation tool to segment the original query text into multiple word segments, which are then arranged into a text word segmentation sequence according to the segmentation order. The text normalization process filters out noise symbols and normalizes whitespace characters in the text segmentation sequence to obtain a normalized text segmentation sequence. The structured processing performs string concatenation operations on the standardized text segmentation sequence according to the segmentation order to obtain standard query text; A domain terminology list is set up, and the standardized text segmentation sequence is identified with domain terms according to the domain terminology list to obtain a set of domain terms.

3. The intelligent query method based on deep semantic matching according to claim 1, characterized in that, Step two specifically includes: The standard query text is processed by a word segmenter to perform sub-word segmentation and sequence encoding to generate a text tag sequence. The text tag sequence includes several text tags, which are divided into valid tags and padding tags. Each text token in the text token sequence is numbered and mapped using the built-in vocabulary of the BERT intent recognition model to obtain a token number sequence; Based on the text tag sequence, an attention mask sequence is constructed. Specifically, when the text tag is a valid tag, the corresponding attention mask value is 1, and when the text tag is a padding tag, the corresponding attention mask value is 0. The tag number sequence is mapped to a tag embedding matrix through the embedding layer; The label embedding matrix and attention mask sequence are input into the BERT encoder to obtain the hidden state matrix; Based on the hidden state matrix, the hidden state vector corresponding to the classification label position is selected, and the hidden state vector is linearly mapped through the classification layer to obtain the query intent classification vector. Softmax normalization is applied to the query intent classification vector to generate a query intent probability vector. Perform the maximum probability selection operation on the query intent probability vector to obtain the query intent type index; Set up a query intent type mapping relationship, and map the query intent type index to the query intent type according to the query intent type mapping relationship; The hidden state matrix is ​​input into the semantic convergence layer, and a mask-weighted average operation is performed on the hidden state vectors corresponding to the effective labels to generate the semantic convergence vector. The semantic convergence vector is then converted into a query semantic vector through a linear mapping matrix.

4. The intelligent query method based on deep semantic matching according to claim 3, characterized in that, The step of inputting the label embedding matrix and attention mask sequence into the BERT encoder to obtain the hidden state matrix specifically includes: The BERT encoder includes several Transformer encoding layers; In each Transformer encoding layer, the input embedding matrix is ​​transformed into a query matrix, a key matrix, and a value matrix through three sets of trainable linear mapping matrices; where the input embedding matrix of the first Transformer encoding layer is the label embedding matrix, and the input embedding matrix of the second and subsequent Transformer encoding layers is the output hidden state matrix of the previous Transformer encoding layer. Based on the attention mask sequence, an attention mask matrix is ​​constructed. Specifically, if the j-th element of the attention mask sequence is 1, then the value of the element in the i-th row and j-th column of the attention mask matrix is ​​set to 0; if the j-th element of the attention mask sequence is 0, then the value of the element in the i-th row and j-th column of the attention mask matrix is ​​set to negative infinity. The attention score matrix is ​​obtained by scaling the dot product of the query matrix and the key matrix. The attention score matrix is ​​then added element-wise to the attention mask matrix and normalized using Softmax to obtain the attention weight matrix. The attention weight matrix and the value matrix are multiplied to obtain the attention feature matrix; the attention feature matrix is ​​residually concatenated with the input embedding matrix, and the intermediate hidden state matrix is ​​obtained through layer normalization. The intermediate hidden state matrix is ​​input into the feedforward network, which includes two layers of linear mapping and a nonlinear activation function, to obtain the feedforward hidden state matrix. The feedforward hidden state matrix and the intermediate hidden state matrix are residually connected and layer normalized to obtain the output hidden state matrix of the Transformer coding layer. The output hidden state matrix of the last Transformer coding layer is used as the hidden state matrix of the BERT encoder.

5. The intelligent query method based on deep semantic matching according to claim 1, characterized in that, Step three specifically includes: A domain terminology set is obtained, and based on the domain terminology set, terminology matching and filtering are performed in a pre-built domain knowledge base and an external domain document set to obtain a limited knowledge text set; the limited knowledge text set includes several knowledge texts that meet the limiting conditions of the domain terminology set. Each knowledge text in the limited knowledge text set is mapped to a retrieval node to obtain a retrieval node set. Based on the retrieval node set, a retrieval path graph is constructed, and the semantic edges between retrieval nodes are the connection relationships of the retrieval path. Based on the retrieval path graph, the pheromone matrix and heuristic function matrix of the ant colony algorithm are initialized, and the number of iteration rounds, number of ants, maximum path length and path evaluation rules are set. The path evaluation rule is obtained by weighted fusion of domain terminology coverage and path length; The domain terminology coverage is specifically defined as follows: taking the union of the domain terminology sets corresponding to all retrieval nodes contained in the retrieval path, and calculating the ratio of the number of elements in the intersection of the union and the domain terminology set to the total number of elements in the domain terminology set; The path length is specifically defined as the reciprocal of the number of search nodes contained in the search path. In the current iteration, set the starting search node for each ant and construct a set of search paths based on the number of ants; Update the pheromone matrix based on the set of retrieval paths in the current iteration round; The search path set from the last iteration round is filtered through a path selection operation to obtain a candidate search path set; Based on the candidate retrieval path set, extract the knowledge text contained in all candidate retrieval paths, and remove duplicates to obtain the candidate knowledge text set.

6. The intelligent query method based on deep semantic matching according to claim 5, characterized in that, In the current iteration, the starting search node for each ant is set, and a search path set is constructed based on the number of ants. Specifically, this includes: At the current retrieval node, based on the pheromone matrix and the heuristic function matrix, each ant calculates the transition probability from retrieval node i to retrieval node j; Based on the transition probability, each ant performs a path construction operation to obtain a retrieval path. Specifically, it samples the set of reachable retrieval nodes of the current retrieval node i according to the transition probability to obtain the next retrieval node j, and adds the next retrieval node j to the retrieval path. The sampling and addition operation is repeated until the path termination condition is met. The path termination condition includes reaching the maximum path length or the current retrieval node having no reachable retrieval nodes. The search paths constructed by all ants are combined into a search path set.

7. The intelligent query method based on deep semantic matching according to claim 5, characterized in that, The path selection operation specifically includes: calculating the evaluation value of each retrieval path according to the path evaluation rules, sorting the paths in descending order according to the evaluation values, and selecting the top K retrieval paths to form a candidate retrieval path set.

8. The intelligent query method based on deep semantic matching according to claim 1, characterized in that, Step four specifically includes: Each candidate knowledge text in the candidate knowledge text set is semantically encoded using the BERT encoder to obtain a candidate knowledge semantic vector. Cosine similarity is used to calculate the cosine similarity between the query semantic vector and each candidate knowledge semantic vector to obtain a semantic matching score. The candidate knowledge texts are sorted in descending order according to their semantic matching scores. Set a semantic matching threshold and a text filtering quantity. Select candidate knowledge texts whose semantic matching scores are greater than or equal to the semantic matching threshold and whose ranking is within the text filtering quantity as matching knowledge texts. Aggregate all matching knowledge texts to obtain a set of matching knowledge texts.

9. The intelligent query method based on deep semantic matching according to claim 1, characterized in that, Step five specifically includes: Each matching knowledge text in the matching knowledge text set is semantically encoded using the BERT encoder to generate a set of matching knowledge semantic vectors. The matching knowledge semantic vector set is weighted and averaged to obtain the matching knowledge convergence vector; the matching knowledge convergence vector is concatenated with the query semantic vector to generate a joint semantic feature vector. The query intent type is used as the generation control condition, and the joint semantic feature vector is input into the large language model to generate query response text.

10. The intelligent query method based on deep semantic matching according to claim 1, characterized in that, Step six specifically includes: A standard response text set is defined, which includes several standard response texts, and each standard response text corresponds to a standard response index; The query response text and each standard response text in the standard response text set are semantically encoded using the BERT encoder to generate a set of response semantic vectors and a set of standard response semantic vectors. The similarity between the response semantic vector and each standard response semantic vector is calculated using the cosine similarity formula to obtain a similarity score vector. Perform a maximum value selection operation on the similarity score vector to obtain the maximum similarity score, and obtain the index of the standard response in the standard response text set with the maximum similarity score; Set a credibility threshold. If the maximum similarity score is greater than or equal to the credibility threshold, then the standard answer text corresponding to the standard answer index will be regarded as the credible query text. If the maximum similarity score is less than the credibility threshold, the process of regenerating the query response text is triggered.

11. The intelligent query method based on deep semantic matching according to claim 1, characterized in that, Step seven specifically includes: The feedback data includes query semantic vectors, matching knowledge text sets, and similarity score vectors. Based on the feedback data, the retrieval paths in the path retrieval process are evaluated, the retrieval paths that generate credible query text are marked as valid paths, and the retrieval nodes in the valid paths are used as positive feedback path samples. The similarity score vector is used as the path feedback weight. According to the pheromone update rule of the ant colony algorithm, the pheromone corresponding to the effective path is positively updated, and the pheromone corresponding to the retrieval path that does not generate credible query text is decayed and updated. The pheromone is the matrix element of the corresponding retrieval node pair in the pheromone matrix. Based on feedback data, the semantic matching threshold and the number of texts filtered during the semantic matching process are adaptively updated.

12. The intelligent query method based on deep semantic matching according to claim 1, characterized in that, The inference process of the improved BERT model is executed by a GPU parallel computing unit; the path retrieval process of the ant colony algorithm is executed by a multi-core processor parallel thread; and the generation process of the large language model is executed by an FPGA inference acceleration unit.

Images