Remote bank fuzzy question and answer accurate matching method based on deep learning

Abstract

The invention discloses a remote bank fuzzy question-answer accurate matching method based on deep learning. The method comprises the following steps: firstly, constructing a business semantic system (clear business class-scene-specific intention), a term and variant mapping library and a sentence pattern template containing a replaceable entity slot position; extending the original similar question pairs to generate derivative sentences and candidate similar sentences, filtering, clustering to generate cross-sentence pattern similar pairs, and selecting negative samples to control the literal overlapping rate; then constructing a twin BERT network, mixing original and enhanced samples into a training set, and training in two stages by using improved triple loss; finally, sentence vectors are generated through the trained twin network, semantic similarity is judged according to cosine similarity, question pairs which are judged to be similar through a model are subjected to secondary verification through a service rule base, and score reduction is carried out if the question pairs do not pass verification.

Description

Technical Field

[0001] This invention relates to the field of banking business data processing, and in particular to a method for accurate matching of fuzzy questions and answers in remote banking based on deep learning. Background Technology

[0002] Remote banking, as a core interactive scenario for the digitalization of financial services, often encounters user inquiries characterized by "vague expressions," such as "My credit card cannot be used for purchases" and "What should I do if my card cannot be swiped?" While the literal meanings of these two statements differ significantly, their core semantic meaning both point to a failed credit card transaction. This necessitates that the system overcome the limitations of "literal matching" and achieve precise semantic similarity matching. However, current technology faces multiple bottlenecks:

[0003] First, the scarcity of labeled samples leads to insufficient semantic learning in the model. In the banking sector, due to constraints such as data privacy protection and high labeling costs, the number of labeled "similar question pairs" under a single business intent is generally scarce (e.g., only 100 pairs of "credit card spending questions" are labeled). Because deep learning models lack sufficient semantically related samples, they cannot grasp the mapping rules of "different expressions - the same intent", and are prone to misjudging the above semantically similar questions as different intents, resulting in low matching accuracy.

[0004] Second, the complexity of user expressions exacerbates the difficulty of semantic recognition. Banking terminology contains numerous non-standard expressions: there are colloquialisms such as "cards that can be overdrawn" and "cards issued by the bank," as well as pinyin / industry abbreviations like "XYK" and "online banking," and misspellings and homophones such as "overdraft" and "billing date." Traditional text processing methods cannot establish a connection between standard terms and their variations, leading to biases in semantic feature extraction and further reducing matching accuracy.

[0005] Third, traditional matching methods have poor adaptability. Early methods such as TF-IDF and Word2Vec relied on word overlap and could not distinguish the differences in banking business logic (for example, although "consumption failure" and "repayment failure" both contain "failure", they belong to different scenarios); the general BERT model has not been fine-tuned for the banking field, has insufficient understanding of professional concepts such as "limit" and "billing date", and has difficulty capturing domain-specific semantic associations, thus failing to meet the needs of accurate matching.

[0006] Fourth, misjudgments negatively impact service efficiency and user experience. Model misjudgments can lead to two types of consequences: either matching user queries to incorrect intents (e.g., matching "failed transaction" with "credit limit inquiry"), resulting in invalid responses; or, due to insufficient matching confidence, transferring automatically answerable questions to manual processing, increasing customer service workload and extending user waiting time, severely restricting the automation rate and service quality of remote banking Q&A systems. There is an urgent need for a semantic similarity matching scheme that can address sample scarcity, adapt to terminology variations, and align with banking business logic. Summary of the Invention

[0007] To address the problems mentioned in the background section, this invention provides a deep learning-based method for accurate matching of fuzzy questions and answers in remote banking.

[0008] The first aspect discloses a deep learning-based method for accurate matching of fuzzy question-answering in remote banking, including the following steps:

[0009] Construct a business semantic system, determine the major business categories, the scenario classification of each major category, and the specific intent of each scenario, and form a structured intent framework;

[0010] Establish a terminology-variant mapping library to match variations for core business terms and form mapping relationships;

[0011] Design several sentence templates with replaceable entity slots for each specific intent;

[0012] Based on the constructed sentence template, the original labeled similar question pairs are expanded, generating several derived sentences for each original sample pair; and several candidate similar sentences are generated for each original question.

[0013] After filtering, similar sentences that meet the rules are retained;

[0014] For different original questions with the same specific intent, clustering is used to generate cross-sentence similarity pairs;

[0015] Negative samples are selected from different specific intentions to ensure that their literal overlap with positive samples meets a threshold.

[0016] Constructing a twin BERT network structure;

[0017] The original similarity pairs and the augmented samples are mixed proportionally to construct the training dataset;

[0018] An improved triplet loss function is used, and training is performed in two stages.

[0019] For the input question and candidate answer sentences, sentence vectors are generated through a trained Siamese network, and the semantic similarity is determined by cosine similarity.

[0020] For question pairs that the model determines to be similar, a second verification is performed using the business rule base. If the verification fails, the similarity score is reduced.

[0021] Furthermore, the Siamese BERT network structure includes: the input layer concatenates text word vectors with specific intent labels, the encoding layer adopts the Chinese BERT-base model, and the pooling layer generates a multi-dimensional sentence vector by concatenating the [CLS] token vector with the mean of the sentence vector.

[0022] Furthermore, the improved triplet loss function is used for two-stage training, including: the first stage freezes the parameters of the first few layers of BERT; the second stage unfreezes all parameters and uses cosine annealing learning rate scheduling and early stopping mechanism.

[0023] Furthermore, the construction of the business semantic system involves determining the major business categories, the scenario classifications within each category, and the specific intents of each scenario, forming a structured intent framework. Specifically, this involves constructing a business semantic system with a three-level structure of "major business category - scenario classification - specific intent," covering the core business areas of the bank; forming a tree-like structured intent framework through hierarchical association, clarifying the scenario classifications included in each major business category and the specific intents included in each scenario classification; and assigning unique identifier codes to all classifications and intents to facilitate accurate positioning and association during subsequent data processing and model invocation.

[0024] Furthermore, the establishment of a terminology and variant mapping library, which matches variants to form mapping relationships for core business terms, specifically involves: selecting core business terms containing key concepts such as business names, operational behaviors, and status descriptions from the business categories, scenario classifications, and specific intentions covered by the three-level business semantic system; collecting variant expressions for each core term by analyzing remote bank user interaction logs, customer service records, and business expert interviews; variants including colloquial expressions covering users' daily communication habits, abbreviations containing pinyin initials and industry-standard abbreviations, and misspellings containing common input errors and homophonic errors; and establishing a mapping relationship between terms and variants to form a mapping library according to the structure of "standard term ID, variant type, variant content, frequency of occurrence, and associated business scenario".

[0025] Furthermore, the step of designing several sentence templates with replaceable entity slots for each specific intent is as follows: For each specific intent under the three-level business semantic system, a sentence template with replaceable entity slots is designed. First, the core semantic components of each intent are determined; then, templates covering declarative, interrogative, and request sentences are designed based on these components. The replaceable entity slot types in the templates match the business characteristics of the banking domain. The business entity slots are selected from the terminology and variant mapping library to find the standard terms and variants corresponding to the intent. Templates are designed based on real user interaction scenarios. Sentences are generated through the permutation and combination of slot candidate content, and a sufficient number of semantically consistent sentences are generated for each intent. The template library and the terminology and variant mapping library are updated in conjunction. When a new variant is added to the mapping library, it is added to the candidate content of the corresponding slot.

[0026] Furthermore, based on the constructed sentence template, the original labeled similar question pairs are expanded, generating several derived sentences for each original sample; several candidate similar sentences are generated for each original question; after filtering, similar sentences that meet the rules are retained; specifically: based on the constructed sentence template, the original labeled similar question pairs are expanded, their respective third-level business intents are identified, and sentence templates containing replaceable entity slots under the corresponding intents are matched; corresponding variants are extracted from the terminology and variant mapping library and replaced according to slot type to generate derived sentences, ensuring that the derived sentences are semantically consistent with the original sentences; for each original question, a T5 model fine-tuned with banking domain data is used to generate candidate similar sentences; the candidate similar sentences are filtered in multiple layers, first by text similarity index, then by checking whether they contain the same business entities as the original question, and finally by checking whether they conform to the business logic of the corresponding third-level intent; high-quality similar sentences are selected from the filtered candidate sentences according to semantic diversity; the derived sentences generated by the template expansion and the selected high-quality similar sentences together constitute the enhanced samples.

[0027] Furthermore, for different original questions under the same specific intent, clustering is used to generate cross-sentence similarity pairs; negative samples are selected from different specific intents to ensure that their literal overlap rate with positive samples meets a threshold. Specifically, for all original questions under the same third-level specific intent, standardization preprocessing is first performed through a terminology and variant mapping library to convert colloquial expressions, abbreviations, and misspellings into corresponding standard terms. Then, the data is input into a Chinese BERT-base model fine-tuned for the banking domain, and the unique label embedding vector of the third-level intent is concatenated. After processing by the encoding layer, semantic vectors are generated through the pooling layer, and all vectors are L2 standardized. Then, K-means semantic vector clustering is performed. After clustering, sentences are randomly selected from different clusters and paired to form cross-sentence similarity pairs. Each pair of samples must come from a different cluster corresponding to the original question. All generated cross-sentence similarity pairs are submitted to experts in the corresponding business domain for semantic similarity annotation. Then, negative samples are selected. The source of negative samples is limited to other third-level intents that are different from the third-level intents of positive samples. Priority is given to third-level intents under different second-level business scenarios. When selecting, a bank-domain customized word segmenter is used to segment positive samples and extract the core word set. Then, sentences are screened from the candidate negative sample library, and the literal overlap rate between negative samples and positive samples is calculated. Only sentences with an overlap rate that meets the threshold are retained, while controlling the ratio of positive to negative samples.

[0028] Furthermore, the first stage freezes the parameters of the first few layers of BERT; the second stage unfreezes all parameters and adopts cosine annealing learning rate scheduling and early stopping mechanism. Specifically, the improved triplet loss function takes anchor sentences, positive sample sentences with the same third-level business intent, and negative sample sentences with different third-level business intents as inputs; it incorporates the loss value calculated based on the business category similarity determined by the third-level business semantic system; it guides the model to learn semantic features that "narrow the distance between positive samples and widen the distance between negative samples"; training is carried out on a training dataset from the banking domain; the AdamW optimizer is used and overfitting is suppressed through weight decay; the first stage freezes the parameters of the first few layers of BERT and trains only some layer parameters; the second stage unfreezes all BERT parameters and adjusts the initial learning rate; the training process adopts cosine annealing learning rate scheduling strategy and early stopping mechanism.

[0029] Furthermore, for the input question and candidate answer sentences, sentence vectors are generated through a trained Siamese network. Cosine similarity is used to determine semantic similarity. Specifically, a terminology and variant mapping library is used to convert colloquial expressions, abbreviations, and misspellings into standard terms. Simultaneously, a banking domain intent recognition module determines the third-level business intent of the input question and generates corresponding intent label embedding vectors. The preprocessed text is then input into the trained Siamese BERT network. The input layer segments the text to generate word vectors and concatenates them with the intent label embedding vectors in the sequence dimension. The encoding layer performs semantic encoding using a domain-fine-tuned BERT model, pooling... The `ing` layer extracts the [CLS] token vector and the mean vector of the text word vectors, concatenates them through linear mapping to generate a unified sentence vector, and then outputs it through the dropout layer. Subsequently, the cosine similarity between the input question sentence vector and the candidate answer sentence vectors is calculated based on the input question sentence vector. The candidate answer sentences are derived from the standard question-and-answer library of the third-level intent to which the input question belongs. Finally, the similarity judgment threshold is determined based on the validation set effect. If the input question matches multiple potential third-level intents, the cosine similarity of the candidate answer sentences under each intent is calculated, and the highest score is compared with the threshold. If the score meets the threshold, the two are judged to be semantically similar, and the highest score and the corresponding third-level intent are recorded to provide a basis for subsequent business rule verification.

[0030] Furthermore, the similarity score of question pairs determined by the model is reduced through a secondary verification using a business rule base. If the verification fails, the similarity score is lowered. Specifically: a business rule base is built based on a three-level business semantic system, and a dedicated secondary verification rule is configured for each third-level intent, clearly defining the corresponding core business entity, key actions, and state elements. After the model determines that the question pairs are semantically similar, the third-level intent to which they belong is first determined through the banking domain intent recognition module, and the dedicated verification rule for that intent is called from the business rule base. Colloquial expressions, abbreviations, and misspellings in the question pairs are uniformly converted into standard terms based on the terminology and variant mapping library. The core entities in the question pairs are extracted using the trained banking domain entity recognition model, and key actions and state descriptions are extracted using the key information extraction module. The extracted core entities, key actions, and state descriptions are compared item by item with the verification rules for that intent. If all match, the verification passes and the initial similarity score of the model is retained. If any item does not match, the verification fails and the original similarity score is reduced. If the reduced score is lower than the judgment threshold, it is re-determined as semantically dissimilar.

[0031] On the other hand, an electronic device is disclosed, including:

[0032] At least one processor; and

[0033] A memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0035] The beneficial effects of this invention are as follows: This application effectively solves the problem of insufficient semantic association learning in remote banking fuzzy question answering due to insufficient labeled similar question pairs, significantly improving the accuracy and robustness of semantic similarity matching. By constructing a business semantic system and terminology variant mapping library, combined with sentence template expansion and cross-sentence similarity pair clustering, the semantic consistency under different expressions can be fully explored, making up for the insufficient amount of original labeled samples. This allows the model to systematically learn the semantic association rules unique to the banking field, and is no longer unable to identify core semantic associations due to sample scarcity.

[0036] Leveraging a design that integrates a twin BERT network with embedded business intent tags, the model can deeply capture the semantic features behind the text, overcoming the limitations of literal matching. It accurately identifies question pairs with significant literal differences but similar semantics, such as "My credit card cannot be used" and "What should I do if my card cannot be swiped?", avoiding misjudging questions with similar meanings as having different intents. An improved triplet loss function and a phased training strategy further enhance the model's ability to distinguish between positive and negative samples, improving its generalization performance and adapting to question-answering needs in different business scenarios. Simultaneously, a secondary verification step using the business rule base effectively corrects potential model misjudgments, verifying core business elements for similar question pairs to ensure matching results align with the bank's actual business logic and reduce the risk of incorrect matching. The overall solution flexibly adapts to diverse user expression habits, including colloquialisms, abbreviations, and misspellings, achieving accurate and reliable semantic matching in remote banking question-answering scenarios, improving user interaction experience and customer service response efficiency. Attached Figure Description

[0037] Figure 1 This is a flowchart of the deep learning-based fuzzy question-answering accurate matching method for remote banking, as described in this application. Detailed Implementation

[0038] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0039] This application discloses a deep learning-based method for accurate matching of fuzzy question-answering in remote banking, such as... Figure 1 The steps include:

[0040] S100. Construct a business semantic system, determine the major business categories, the scenario classification of each major category, and the specific intent of each scenario, and form a structured intent framework;

[0041] S101. Establish a terminology and variant mapping library to match variants for core business terms and form mapping relationships; design several sentence templates with replaceable entity slots for each specific intent;

[0042] S102. Based on the constructed sentence template, the original labeled similar question pairs are expanded, generating several derived sentences for each original sample pair; several candidate similar sentences are generated for each original question; after filtering, similar sentences that meet the rules are retained.

[0043] S103. For different original questions under the same specific intent, cluster them to generate cross-sentence similarity pairs; select negative samples from different specific intents to ensure that their literal overlap rate with positive samples meets the threshold.

[0044] S104. Construct the Siamese BERT network structure; mix the original similar pairs and augmented samples proportionally to construct the training dataset; use the improved triplet loss function and train in two stages;

[0045] S105. In the scenario of fuzzy question answering in remote banking, sentence vectors are generated for the input question and candidate answer sentences through a trained Siamese network, and the semantic similarity is determined by cosine similarity.

[0046] S106. For question pairs that the model determines to be similar, a second verification is performed using the business rule base. If the verification fails, the similarity score is reduced.

[0047] The construction of the business semantic system involves determining major business categories, scenario classifications within each category, and specific intents for each scenario, forming a structured intent framework. Specifically, this involves defining 15 major business categories as primary categories, covering core banking business areas such as credit cards, savings, loans, transfers, wealth management, foreign exchange, precious metals, mobile banking, online banking, account management, payment and settlement, credit inquiry, branch services, customer service consultation, and complaints and suggestions. Each major business category is further divided into 6-8 core scenarios as secondary categories. For example, the credit card category includes 8 scenarios such as consumption, repayment, reporting lost cards, card replacement, credit limit adjustment, bill inquiry, activation, and cancellation; the savings category includes 7 scenarios such as account opening, deposits and withdrawals, reporting lost cards, account closure, interest inquiry, and transfers. Each secondary scenario is further subdivided into 8-12 specific question intents as tertiary categories. The categories, such as credit card spending scenarios, cover 12 intents including "failed transaction," "limit inquiry," "overseas spending restrictions," "points inquiry," "transaction bill details," "duplicate charges," "SMS notifications," "merchant inquiry," "installment payment settings," "refund processing," "password questions," and "temporary credit limit spending." Savings scenarios, such as deposit and withdrawal scenarios, cover 10 intents including "deposit failure," "withdrawal limit," "ATM withdrawal malfunction," "large deposit regulations," and "intercity deposit and withdrawal fees." A hierarchical tree-structured intent framework is formed, clearly defining the second-level scenarios within each first-level category and the third-level intents within each second-level scenario. All categories and intents are uniquely coded to facilitate accurate location and association during subsequent data processing and model invocation.

[0048] The establishment of a terminology and variant mapping library matches variants for core business terms, forming mapping relationships. Specifically, this involves selecting 800 core business terms from the 15 major business categories, secondary scenarios, and tertiary intents covered by the three-level business semantic system. These terms include key concepts such as business names (e.g., "credit card," "online banking"), operational behaviors (e.g., "consumption," "repayment," "transfer"), and status descriptions (e.g., "failure," "limit," "loss report"). For each core term, variant expressions are collected by analyzing remote banking user interaction logs (including text inquiries and voice-to-text records), customer service records, and interviews with business experts over the past three years. Conversational expressions must cover the user's daily life. Common communication habits are included, such as "credit card" corresponding to "my card", "the card issued by the bank", "the card that can be overdrawn", etc., and "failed transaction" corresponding to "cannot swipe card", "payment not successful", "cannot deduct money from purchase", etc.; abbreviations include initials of pinyin (such as "XYK" corresponding to "credit card" and "WYYH" corresponding to "online banking"), industry-standard abbreviations (such as "online banking" corresponding to "online banking" and "credit card" corresponding to "credit card"); misspellings cover common input errors (such as "consumption" and "payment" corresponding to "consumption" and "payment"), homophonic errors (such as "overdraft" corresponding to "overdraft" and "billing date" corresponding to "billing date"), etc., to ensure that the number of variations for each term is no less than 20.

[0049] All terms and variants are mapped according to the structure of "standard term ID - variant type - variant content - frequency of occurrence - associated business scenario", forming a mapping library with a total number of no less than 5,000 entries. In addition, 100-200 new variants are added each month based on new user interaction data. These are reviewed by business experts and then included in the library to ensure the comprehensiveness and accuracy of the mapping relationship.

[0050] The method involves designing several sentence templates with replaceable entity slots for each specific intent. In practice, this is done by first identifying the core semantic components of each specific intent in the three-level business semantic system, including user references (such as "my", "I", "myself", "myself"), business entities (such as credit cards, debit cards, online banking, etc. associated with the intent), operational behaviors (such as consumption, repayment, withdrawal, inquiry, etc.), and problem status / requests (such as failure, cannot be swiped, not successful, what to do, etc.). Then, based on these elements, 10-15 sentence templates are designed to ensure coverage of different user expression sentences such as declarative sentences, interrogative sentences, and request sentences. Each template includes replaceable entity slots. The slot types must match the business characteristics of the banking industry. For example, user referent slots can include expressions that conform to daily communication habits, such as "my," "I," "myself," and "family members." Business entity slots need to select standard terms and variations corresponding to the intent from the terminology-variant mapping library. For example, business entity slots for the intent "credit card transaction failed" include "credit card," "XYK," "cards that can be overdrawn," and "bank-issued consumer cards." Operation behavior slots include "consumption," "swipe card," "payment," and "payment." Problem status slots include "failed," "cannot swipe," "not successful," and "cannot deduct money." When designing templates, it is necessary to combine real user interaction scenarios. For example, declarative templates such as "[user referent] encountered a [problem status slot] when using [business entity slot][operation behavior slot]," interrogative templates such as "Why did [user referent] encounter a [problem status slot] when using [business entity slot][operation behavior slot]?" and request templates such as "[user referent] wants to solve the problem of [business entity slot][operation behavior slot][problem status slot], how should I do it?" are used to ensure sentence diversity to match different user expression habits.

[0051] Each template generates sentences by arranging and combining candidate content for each slot. For example, a template for the intent "credit card transaction failed" can be combined to generate statements such as "My credit card transaction failed," "I can't swipe my XYK card," and "I want to solve the problem of unsuccessful payment with my overdraft card; what should I do?" Each intent must generate at least 300 semantically consistent sentences through combinations of all templates. Templates and generated sentences must be reviewed by three or more core banking business experts (covering areas such as credit cards, savings, and loans). The review focuses on the reasonableness of slot replacements, whether sentences conform to business logic, and whether they reflect the user's actual expression. If the review fails, the candidate content for the slot or the template structure must be adjusted until the approval rate reaches 100%. Simultaneously, the template library must be updated in conjunction with the terminology-variant mapping library. When a new variant is added to the mapping library (e.g., a new user expression of "overdraft card" corresponding to "credit card"), it is promptly added to the candidate content of the corresponding slot. This ensures that the sentences generated by the templates continuously cover new user expressions, providing basic materials that align with business scenarios for data augmentation of similar question pairs.

[0052] The constructed sentence template expands the original labeled similar question pairs, generating several derived sentences for each original sample pair; it also generates several candidate similar sentences for each original question; after filtering, it retains the similar sentences that meet the rules. Specifically, in implementation, for the original labeled similar question pairs, the third-level business intent to which they belong (e.g., "credit card consumption scenario - consumption failure" intent) is first clarified, and a pre-designed sentence template containing replaceable entity slots under this intent is matched (e.g., "[my / me / myself]'s [credit card / XYK / overdraftable card] cannot [consume / swipe / pay]", "[credit card / card] can't [consume / swipe] anymore", etc.); during expansion, pairs are extracted from the term-variant mapping library. Based on the intended user referential variants, business entity variants, and operational behavior variants, precise replacements are performed according to the slot types in the template. For example, taking the original similar pair "My credit card cannot be used" and "What should I do if my credit card transaction fails" as the basis, derivative sentences such as "My XYK card cannot be used", "Why can't I use my card?" and "What should I do if my overdraft card fails to pay" can be generated based on the template. Each original sample needs to generate 20-30 derivative sentences, and the derivative sentences must be semantically consistent with the original sentences. 5% of the derivative samples are manually sampled, and three banking business experts determine the semantic consistency. Only when all three experts determine "consistent" is the derivative sentence included in the valid sample, ensuring 100% consistency in the sampling inspection.

[0053] Subsequently, a T5 model fine-tuned with banking data was used to generate candidate similar sentences. This T5 model first needed to be pre-trained and fine-tuned based on 500,000 core banking business Q&A texts (covering user questions and customer service replies from 15 major business categories) to ensure that the model mastered banking terminology and expression habits. During generation, each original question was used as input, and the generation temperature was set to 0.7 (balancing generation diversity and accuracy). Five candidate similar sentences were generated for each original question. Then, the candidate sentences were filtered in three layers: the first layer calculated the BLEU value between the candidate sentence and the original question, and only candidate sentences with a BLEU value ≥ 0.6 were retained (ensuring basic text similarity).

[0054] The second layer uses a banking entity recognition model (trained with banking business entity data, capable of recognizing core entities such as "credit card," "savings," and "loan") to verify candidate sentences, retaining only those containing the same business entity as the original question (e.g., if the original sentence contains "credit card," then candidate sentences without "credit card" and related variations are filtered out). The third layer calls the business rule base (containing the core logic of each business intent, such as the "consumption failure" intent needing to be associated with the "consumption / card swipe" action and the "failure / cannot be" status) to verify whether the candidate sentences conform to the business logic of the corresponding third-level intent. For example, if the original sentence is "credit card consumption failure," and the candidate sentence is "credit card repayment failure," although it contains the "credit card" entity, it does not conform to the "consumption" action logic, so it is filtered out.

[0055] Finally, from the candidate sentences that have passed through three layers of filtering, 2-3 high-quality similar sentences are selected based on semantic diversity (to avoid repetition of expressions), and together with the derivative sentences generated by template expansion, they constitute the enhanced sample.

[0056] The method involves clustering different original questions under the same specific intent to generate cross-sentence similarity pairs; selecting negative samples from different specific intents to ensure that their literal overlap rate with positive samples meets a threshold; specifically, in implementation, for all original questions under the same third-level specific intent, each original question is first standardized and preprocessed using a term-variant mapping library to convert colloquial expressions, abbreviations, typos, etc., into corresponding standard terms (e.g., converting "My XYK cannot be swiped" into "My credit card consumption failed"). Then, it is input into a Chinese BERT-base model finely tuned for the banking domain (with the same encoding layer as the semantic similarity model, trained with 500,000 bank question-and-answer texts), and the unique label embedding vector of the third-level intent is concatenated. After processing by the encoding layer, a 768-dimensional semantic vector is generated through a pooling layer (concatenating the mean of the [CLS] token vector and the sentence vector). All vectors are then L2 standardized to eliminate differences in magnitude to ensure clustering effect.

[0057] Next, K-means semantic vector clustering is performed. The number of clusters is set to the total number of original questions under the three-level intent (e.g., if an intent contains 10 original questions, then 10 clusters are set). The number of iterations is set to 500 to ensure clustering convergence. After clustering, sentences are randomly selected from different clusters and paired to form cross-sentence similarity pairs (e.g., "My credit card cannot be used" from cluster 1 is paired with "What should I do if my card cannot be swiped" from cluster 2). Each pair of samples must come from different clusters corresponding to the original questions to avoid repeated pairing with sentences derived from the original questions.

[0058] All generated cross-sentence similarity pairs need to be submitted to three experts in the corresponding business domain (e.g., credit card domain experts are responsible for credit card-related intent samples) for semantic similarity labeling. The labeling standard is "whether it expresses the same question intent". Only when all three experts' labeling results are "yes" will the pair be included in the valid positive sample library. If the labeling is inconsistent, it needs to be confirmed by the expert group to ensure the semantic accuracy of cross-sentence similarity pairs.

[0059] Negative samples were then selected, with the source of negative samples limited to other third-level intents that are different from the third-level intents of the positive samples. Priority was given to selecting third-level intents under different second-level business scenarios (e.g., if the positive sample is "credit card-consumption-consumption failure", then it is selected from different second-level scenario intents such as "savings-account opening-account opening process" and "loan-application-loan conditions", avoiding the selection of the same second-level scenario intent such as "credit card-repayment-repayment failure"), in order to ensure significant semantic differences.

[0060] During the selection process, a customized word segmenter for the banking sector was first used to segment the positive samples and extract the core word set (e.g., the core words for the positive sample "credit card consumption failure" are "credit card", "consumption", and "failure"). Then, sentences were screened from the candidate negative sample library, and the word overlap rate between the negative and positive samples was calculated. The overlap rate = (number of identical words between the negative and positive samples) / (total number of words in the positive samples) × 100%. Only sentences with an overlap rate ≤ 30% were retained (e.g., the positive sample "credit card consumption failure" has 3 identical words, the negative sample "debit card loss reporting process" has 0 identical words, and the overlap rate of 0% meets the requirements; the negative sample "credit card repayment failure" has 2 identical words, and the overlap rate of 66.7% is excluded).

[0061] Meanwhile, the ratio of positive to negative samples is strictly controlled at 1:3, meaning that each positive sample (including original similarity pairs, template extended samples, and cross-sentence similarity pairs) corresponds to three different qualified negative samples, and the same negative sample is not repeatedly associated with multiple positive samples to avoid the model learning repetitive negative features. In addition, the negative sample library is updated monthly based on new user question data, supplementing new expressions under different intentions. Business experts review the semantic differences and literal overlap between the new negative samples and the intentions of each positive sample to ensure the timeliness of the negative sample library and provide accurate negative supervision signals for model training.

[0062] The construction of the Siamese BERT network structure specifically includes: the input layer concatenating text word vectors with specific intent labels; the encoding layer adopting the Chinese BERT-base model; and the pooling layer generating a multi-dimensional sentence vector by concatenating the [CLS] token vector with the mean of the sentence vector. In practice, the dual-branch structure of the Siamese BERT network is first determined, and the two branches completely share parameters to ensure the consistency of feature extraction.

[0063] During input layer processing, the user's query text is first segmented using the Tokenizer of the Chinese BERT-base model, converting it into a corresponding token ID sequence. Then, a 768-dimensional text word vector sequence is generated through the model's built-in word embedding layer. Simultaneously, based on the third-level specific intent to which the query belongs (e.g., "credit card-consumption-consumption failure"), a unique integer identifier is assigned to each third-level intent. This identifier is input into an independent embedding layer to generate a 768-dimensional specific intent label embedding vector. Subsequently, the text word vector sequence and the intent label embedding vector are concatenated along the sequence dimension. For example, if the length of the text word vector sequence is L (according to the BERT-base maximum sequence length limit, L does not exceed 512), the concatenation forms a mixed input vector sequence of length L+1, ensuring that the model can combine prior information about business intent when extracting text semantic features.

[0064] The encoding layer uses a network initialized based on a Chinese BERT-base pre-trained model. After initialization, it needs to be fine-tuned using 500,000 question-and-answer texts covering 15 major banking business categories (including real user questions and standard customer service replies). During the fine-tuning process, the parameters of the first 4 layers of the model are frozen, and only the last 8 layers and the classification layer are trained. The optimizer is AdamW, the initial learning rate is set to 5e-5, and the training is carried out for 10 epochs. The fine-tuning task is set as binary classification of sentence pairs based on semantic similarity (labeling "similar" or "dissimilar"). After the model's accuracy on the validation set stabilizes, all layers are unfrozen and training continues for 5 epochs. This allows the model to fully learn the semantic expressions and business logic associations specific to the banking domain, avoiding semantic biases of the general BERT model in banking scenarios.

[0065] During pooling, the corresponding text word vectors (excluding tokens corresponding to concatenated intent tag embedding vectors) are first separated from the mixed input vector sequence output by the encoding layer. The mean of all token vectors in this part is calculated to obtain the 768-dimensional sentence vector mean. At the same time, the encoding vector (also 768-dimensional) corresponding to the first [CLS] token in the sequence is extracted. This vector contains the global semantic information of the entire text. Then, the [CLS] token vector and the sentence vector mean are input into a linear mapping layer. This layer maps the two 768-dimensional vectors into 384-dimensional vectors and then concatenates them to finally generate a unified 768-dimensional sentence vector, which retains global semantic features and incorporates the statistical information of local word vectors. To reduce the risk of overfitting, a dropout layer with a dropout probability of 0.1 is added after the pooling layer to randomly deactivate the generated sentence vectors before they are fed into the subsequent similarity calculation module.

[0066] The process involves mixing the original similar pairs with enhanced samples in a proportional manner to construct a training dataset. Specifically, the original similar pairs undergo quality verification and screening. These pairs originate from real user interaction scenarios in remote banking and must be annotated and confirmed for semantic similarity by three or more banking business experts (the annotation standard is "expressing the same question intent with no difference in core business elements"). Annotation consistency must reach 100% before inclusion. Simultaneously, it is ensured that the original similar pairs cover all 15 major business categories and their corresponding secondary scenarios and tertiary intents. The number of original similar pairs for a single tertiary intent is no less than 20 pairs, and the total size of the original similar pairs is controlled between 10,000 and 15,000 pairs based on business coverage.

[0067] Subsequently, the number and attribution of various types of augmented samples were counted. Augmented positive samples include derivative sentences generated by template expansion (20-30 sentences per pair of original samples, totaling approximately 200,000-450,000 sentences), high-quality similar sentences generated by the T5 model (2-3 sentences per original question, totaling approximately 20,000-45,000 sentences), and cross-sentence similar pairs (no less than 50 pairs per level 3 intent, totaling approximately 80,000-100,000 pairs). They need to be classified separately according to the level 3 intent dimension to ensure that the number of augmented positive samples for each level 3 intent is 5 times the number of original similar pairs for that intent, and strictly follow the mixing ratio of 1:5 between original similar pairs and augmented positive samples.

[0068] During the blending process, firstly, the original similar pairs under the same Level 3 intent are merged with the enhanced positive samples to form the positive sample set for that intent. Then, at a ratio of 1:3 (positive to negative samples), a corresponding number of negative samples are selected from the negative sample library (negative samples must satisfy a literal overlap rate of ≤30% with positive samples and belong to different Level 3 intents) and added to the sample set for that intent, forming a complete sample set for a single intent. Subsequently, the sample sets of all Level 3 intents are blended. During blending, sample weights need to be allocated according to the proportion of user inquiries in each business category. For example, if the inquiries in the credit card field account for 30%, then the proportion of samples in this field in the total dataset should also be controlled at around 30% to avoid excessive concentration of samples in a certain business field, which could lead to model bias.

[0069] After mixing, the dataset needs to be divided into training, validation, and test sets in an 8:1:1 ratio. Stratified sampling is used during the division process to ensure that the sample proportions of each business category and each level-3 intent in the training, validation, and test sets are consistent with the total sample.

[0070] Simultaneously, data cleaning was carried out, using text deduplication tools to remove duplicate samples (including complete repetition and semantically highly repetitive samples, judged by cosine similarity ≥ 0.95), and 5% of the samples (covering all business categories) were manually sampled to verify the accuracy of sample labeling (whether the similarity labels are correct and whether the business entities match) and semantic consistency (whether the enhanced samples and the original samples express the same intent). Samples that failed the sampling inspection needed to be re-labeled or removed to ensure the purity of the training dataset.

[0071] Finally, the processed training dataset is organized according to the model input format. Each sample must include the anchor question text, the comparison question text, the corresponding level 3 intent identifier (the anchor sentence and the comparison sentence must be consistent), and a similarity label (1 indicates similarity, 0 indicates dissimilarity). The text part needs to be preprocessed using a term-variant mapping library (e.g., converting "XYK" to "credit card") to ensure the standardization of the model input text. At the same time, the total size of the training dataset is calculated to meet the requirement that the number of training samples for a single business scenario (secondary classification) is no less than 5,000 pairs, and the total size of the training set is no less than 100,000 pairs, so as to provide sufficient and balanced sample support for the subsequent training of the semantic similarity model.

[0072] The improved triplet loss function is used, and training is conducted in two stages: the first stage freezes the parameters of the first few layers of BERT; the second stage unfreezes all parameters and uses cosine annealing learning rate scheduling and an early stopping mechanism. Specifically, the core components and parameter settings of the improved triplet loss function are first defined. This function takes the anchor sentence (a), positive sample sentences (p, semantically similar to the anchor sentence and belonging to the same level 3 business intent), and negative sample sentences (n, semantically dissimilar to the anchor sentence and belonging to different level 3 business intents) as inputs, and calculates the loss value among the three.

[0073] Wherein, d(a,p) represents the Euclidean distance between the anchor sentence and the positive sample sentence, which are encoded by the Siamese BERT network into 768-dimensional sentence vectors, and d(a,n) represents the Euclidean distance between the anchor sentence and the negative sample sentence; α is the marginal value, set to 0.8 in combination with the semantic similarity judgment requirements of the banking domain, to ensure that the distance difference between the positive and negative samples and the anchor sentence reaches the effective distinction threshold; β is the business category similarity weight coefficient, set to 0.3, to balance the influence of semantic features and business prior information; sim_b(a,p) is the business category similarity between the anchor sentence and the positive sample sentence, which is determined according to the three-level business semantic system. If the two belong to the same third-level intent, the value is 1; if they belong to the same second-level scenario but different third-level intents, the value is 0.5; and if they belong to different second-level scenarios, the value is 0. This parameter integrates the business hierarchy association into the loss calculation, avoiding the model from relying solely on the surface features of the text and ignoring the business logic association.

[0074] When calculating the loss function, first calculate d(a,p), d(a,n) and sim_b(a,p) respectively, and then substitute them into the formula L=max(0,d(a,p)-d(a,n)+0.8-0.3×sim_b(a,p)). If the calculation result is negative, take 0 to ensure that the loss value is non-negative, effectively guiding the model to learn the semantic features of "closing the distance between positive samples and widening the distance between negative samples".

[0075] The training phase adopts a phased strategy, and the training process is based on the constructed banking domain training dataset (original similar pairs and enhanced samples are mixed at a ratio of 1:5, with a total scale of no less than 100,000 pairs and no less than 5,000 pairs for a single business scenario). The optimizer is AdamW, and the weight decay coefficient is set to 0.01 to suppress overfitting.

[0076] During the first phase of training, the parameters of the first 6 layers of the Chinese BERT-base model in the Siamese BERT network encoding layer are frozen, and only the parameters of the last 6 layers, the pooling layer, and the related mapping layer are trained. The initial learning rate is set to 5e-5, and the training rounds are 8 epochs. In each round of training, samples are input in batches (batch size is set to 32 to adapt to model training efficiency and hardware resources). The loss value of the samples in each batch is calculated according to the improved triplet loss function, and the trainable parameters are updated through backpropagation.

[0077] In the second phase of training, all 12 layers of the BERT model's parameters are unfrozen, and the initial learning rate is adjusted to 2e-5 (to reduce the learning rate and avoid parameter oscillations). The training epochs are set to 15 epochs, and samples are input in batches, with loss calculated and parameters updated. The entire training process employs a cosine annealing learning rate scheduling strategy. Starting from the initial learning rate, the learning rate is gradually reduced according to the cosine function curve as the training epochs increase, with a learning rate cycle adjustment every 3 epochs to ensure stable convergence of model parameters in later training stages. An early stopping mechanism is also introduced: every 3 epochs, the model's matching accuracy is evaluated using a validation set (accounting for 10% of the total dataset, stratified sampling to ensure the business category distribution is consistent with the training set). If the validation set accuracy decreases for two consecutive times (with a decrease threshold of 0.5%), training is immediately stopped, preserving the current optimal model parameters to prevent overfitting in later training stages.

[0078] During training, it is also necessary to record the loss value, training set accuracy, and validation set accuracy of each training round in real time, and generate training logs to facilitate subsequent model performance analysis and parameter tuning.

[0079] The process involves generating sentence vectors for the input question and candidate answer sentences using a trained Siamese network, and determining semantic similarity based on cosine similarity. Specifically, the input question and candidate answer sentences undergo standardization preprocessing: a pre-built terminology-variant mapping library is used to convert colloquial expressions (e.g., "card cannot be swiped"), abbreviations (e.g., "XYK"), and misspellings (e.g., "consumption") in the input question into corresponding standard terms (e.g., "credit card consumption failed"). The candidate answer sentences also undergo this preprocessing to ensure that both textual expressions are based on a unified business terminology system. Simultaneously, a banking domain intent recognition module is used to determine the third-level business intent of the input question (e.g., "credit card consumption scenario - consumption failed"), obtain the unique identifier corresponding to this third-level intent, and generate a 768-dimensional intent label embedding vector (consistent with the intent embedding dimension during model training).

[0080] Next, the preprocessed input question and candidate answer sentences are input into the trained Siamese BERT network: The input layer first performs word segmentation on the text, converting it into a token ID sequence that meets the requirements of the Chinese BERT-base model, generating corresponding word vectors, and then concatenating the word vectors with the aforementioned three-level intent tag embedding vectors in the sequence dimension (e.g., if the length of the text word vector sequence is L, then the length of the concatenated sequence is L+1); The encoding layer performs semantic encoding on the concatenated vector sequence using a BERT model fine-tuned with 500,000 bank question-and-answer texts, capturing semantic associations unique to the banking domain; The pooling layer extracts the [CLS] token vector (containing global semantic information) and the mean vector of the text word vector part from the encoded sequence, and after passing through the linear mapping layer, both are mapped to 384-dimensional vectors and concatenated to generate a unified 768-dimensional sentence vector. Finally, the dropout layer (dropout probability 0.1) outputs the final sentence vector, ensuring that it is completely consistent with the sentence vector generation logic during model training.

[0081] Then, the cosine similarity between the two is calculated: taking the sentence vector generated from the input question as the benchmark, the cosine value is calculated with the sentence vector of each candidate answer sentence. This value is used to measure the angle between the two vectors. The closer the value is to 1, the more similar the semantics are, and the closer it is to 0, the greater the semantic difference is. The candidate answer sentences must come from the standard question-answer library corresponding to the third-level intent of the input question (e.g., the candidate answer library for the intent "credit card consumption failed" contains standard replies such as "credit card consumption failed may be due to insufficient credit limit, you can check the credit limit through mobile banking") to ensure that the candidate answers match the business scenario of the input question.

[0082] Finally, similarity judgment is performed: the judgment threshold is determined based on the validation set performance during the model training phase. On a validation set containing 10,000 pairs of real user questions and standard answers, when the cosine similarity threshold is set to 0.75, the model achieves a matching accuracy of 92%, a recall of 90%, and the best F1 score. This effectively covers question pairs with significant literal differences but similar semantics, such as "My credit card cannot be used" and "What should I do if my card cannot be swiped" (the cosine similarity calculated between these two is usually between 0.8 and 0.85), and also effectively filters candidate answers that are semantically irrelevant to the input question (such as the input question being...). For questions related to credit card spending, the similarity to candidate answers such as "debit card loss reporting process" is mostly below 0.4. If the input question is matched with multiple potential level 3 intents by the intent recognition module (e.g., an ambiguous question may match two intents: "credit card spending failed" and "credit card limit insufficient"), then the cosine similarity of the candidate answer sentences under each intent is calculated, and the highest score is compared with the 0.75 threshold. If the score is ≥0.75, the input question is determined to be semantically similar to the candidate answer sentence. At the same time, the highest score and the corresponding level 3 intent are recorded to provide a basis for subsequent business rule verification.

[0083] Cosine similarity measures semantic similarity solely through the angle between vectors, which has two limitations: first, it ignores the differences in business importance across different dimensions; for example, "credit card" and "debit card" may occupy similar dimensions in sentence vectors but have completely different business attributes; second, it does not consider the semantic intensity information contained in the vector amplitude, such as failing to reflect the semantic certainty difference between "failed transaction" and "possible transaction failure." This approach addresses the characteristics of remote banking scenarios, which involve numerous terminology variations, complex terminology, and strict business logic.

[0084] Furthermore, this application provides a novel method for integrating business weights and multiple distance metrics, including: configuring input data, a 768-dimensional sentence vector generated by a Siamese BERT network, and an input question vector. Candidate answer sentence vector Dimension weight vector Extracted from the term variant mapping library, core business terms (such as "consumption" and "repayment") have higher corresponding dimension weight values; the business rule matching degree γ is taken from the verification results of the business rule library, γ∈0,1, representing the degree of matching of core entities and action elements;

[0085] We perform dimensional weighting, assigning weights based on the importance of business terms to strengthen the influence of core dimensions:

[0086] Where ⊙ represents element-wise multiplication. To determine the frequency of occurrence of the business term corresponding to the i-th dimension in the mapping library, ensure that the dimension with higher frequency core terms (such as "credit card") has a higher weight.

[0087] Perform hybrid distance calculations, fusing cosine similarity and Manhattan distance, while simultaneously capturing differences in direction and magnitude:

[0088]

[0089] Where α = 0.7 is the adjustment factor, which prioritizes the retention of semantic direction information of cosine similarity; ||·||1 is the Manhattan distance, and the denominator is standardized to ensure that the value range is 0,1;

[0090] Dynamically adjust business rules, introduce a rule matching degree γ-correction score, and penalize cases where core elements do not match:

[0091] Sim final =Sim hybrid • (γ+∈) When core entities (such as “credit card” vs “debit card”) or key actions (such as “spending” vs “repayment”) do not match, γ takes a value of 0.3 to 0.5 (set according to the business risk level), and ∈ = 0.2 is the basic offset to ensure the validity of the minimum score.

[0092] W is the dimension weight vector, which is obtained by joint training of the frequency of terms in the terminology mapping library and the annotations of business experts. The dimension weights corresponding to core business terms are ≥0.8. α is the hybrid distance adjustment factor, which takes a value of 0.7 to make semantic direction more important than amplitude difference. γ is the business rule matching degree, which is jointly calculated by the entity recognition model and the key information extraction module. When there is a perfect match, γ = 1.

[0093] Then output the result, Sim final For values ​​∈0,1, a score ≥0.75 is considered semantically similar. This method is compatible with the original threshold but offers better discrimination. In easily confused scenarios such as "consumption failure" and "repayment failure", this method, through the weight difference between the "consumption" and "repayment" dimensions (weights of 0.92 and 0.88 respectively), combined with business rule verification, can correct the original cosine similarity score of 0.82 to 0.85 (correct match) and 0.63 (incorrect match), significantly improving the compliance of business logic.

[0094] The similarity score of question pairs determined by the model is further validated through a business rule base. If the validation fails, the similarity score is reduced. In practice, this involves first building a business rule base based on a three-level business semantic system, configuring a dedicated secondary validation rule for each level three intent. The rule must clearly define the core business entity corresponding to the intent (covering all variant expressions of the entity in the terminology-variant mapping library) and key actions and state elements. For example, the validation rule for the intent "credit card consumption failed" must include "core entity is 'credit card' and variants such as 'XYK' and 'card that can be overdrawn', key actions must be related expressions such as 'consumption', 'swipe card', and 'payment', and state must include descriptions such as 'failed', 'cannot swipe', and 'not successful'". The validation rule for the intent "savings withdrawal limit query" must include "core entity is 'savings card' and variants such as 'deposit card' and 'debit card', key actions must be expressions such as 'query' and 'check', and key information must include content such as 'limit' and 'quote restriction'". All rules must be reviewed by experts in the corresponding business domain to ensure they conform to the actual business logic of the bank and the common expression habits of users.

[0095] Once the model determines that a pair of questions is semantically similar (cosine similarity ≥ 0.75), it first uses the banking domain intent recognition module to determine the level 3 intent to which the question pair belongs, and then calls the specific verification rules for that intent in the business rule base. Subsequently, with the help of the terminology-variant mapping library, colloquial expressions, abbreviations, and typos in the question pair are uniformly converted into standard terms. For example, "My card cannot be swiped" is converted into "My credit card transaction failed", and "Check savings card limit" is converted into "Check savings card withdrawal limit". Then, the core entities in the question pair are extracted using the pre-trained banking domain entity recognition model (fine-tuned with 500,000 banking business texts, capable of accurately identifying core entities of various business categories). The key information extraction module (trained based on banking business action and state corpus) extracts key actions and state descriptions. Next, the extracted core entities, key actions, and state descriptions are compared item by item with the verification rules for that intent. If all the rule requirements are met (such as the existence of core entities, matching of key actions, and correspondence of state descriptions), the verification passes, and the initial similarity score calculated by the model is retained.

[0096] If any non-compliance exists, such as the model misjudging "credit card repayment failure" as similar to "what to do if the card cannot be swiped" (belonging to the intent of "credit card consumption failure"), and upon verification it is found that the key action of the former is "repayment" rather than "consumption", which does not meet the rule of the intent of "credit card consumption failure", then the verification is deemed to have failed, and the original similarity score is reduced by 0.2. If the score after reduction is lower than 0.75, then it is re-determined to be semantically dissimilar.

[0097] Meanwhile, the business rule base needs to be linked with the daily collection of new user questions and manual annotation results. Every week, the rules should be optimized for abnormal cases that fail the verification (such as users adding expressions that are not covered by the rules, such as "overdraft card cannot be swiped"). The verification logic corresponding to new entity variants or action expressions should be added. The optimized rules need to be reviewed and confirmed by business experts to ensure that the secondary verification process can not only correct model misjudgments, but also adapt to the dynamic changes of user expressions and business scenarios.

[0098] This application also provides an embodiment of an electronic device. The electronic device is manifested in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: one or more processors or processing units, memory, and buses connecting different components (including memory and processing units).

[0099] A bus refers to one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. Examples of these architectures include, but are not limited to, Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses.

[0100] Electronic devices typically include a variety of computer-readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, and removable and non-removable media.

[0101] The memory may include computer-readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. Electronic devices may further include other removable / non-removable, volatile / non-volatile computer device storage media. By way of example only, the storage system may be used to read and write non-removable, non-volatile magnetic media.

[0102] The electronic device can also communicate with one or more external devices (e.g., keyboard, pointing device, camera, etc.), may include a display, and may communicate with one or more devices that enable a user to interact with the electronic device, and / or with any device that enables the electronic device to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via an input / output (I / O) interface. Furthermore, the electronic device can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. The processor executes various functional applications and data processing by running programs stored in memory, such as implementing the deep learning-based remote banking fuzzy question-answering accurate matching method provided in the above embodiments of the present invention.

[0103] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0104] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A remote bank fuzzy question and answer accurate matching method based on deep learning, characterized in that, The method comprises the steps of: building a business semantic system, determining business categories, scene classifications of each category, and specific intents of each scene, and forming a structured intent framework; establishing a term and variant mapping library to match variants for core business terms and form a mapping relationship; designing several sentence templates containing replaceable entity slots for each specific intent; based on the constructed sentence templates, expanding the original annotated similar question pairs, and generating several derived sentences for each original sample; after filtering, retaining similar sentences that meet the rules; clustering different original questions under the same specific intent to generate cross-sentence similar pairs; selecting negative samples from different specific intents to ensure that the literal overlap rate with positive samples meets a threshold; building a twin BERT network structure; mixing the original similar pairs and the enhanced samples in proportion to build a training data set; using an improved triple loss function for two-stage training; generating sentence vectors for input questions and candidate answer sentences through the trained twin network, and determining whether they are semantically similar through cosine similarity; for question pairs determined by the model to be similar, secondary verification is performed through the business rule library, and if the verification fails, the similarity score is reduced.

2. The deep learning-based remote bank fuzzy question and answer accurate matching method according to claim 1, characterized in that, The twin BERT network structure comprises: an input layer that embeds and concatenates text word vectors and specific intent labels, a coding layer that uses a Chinese BERT-base model, and a pooling layer that generates several-dimensional sentence vectors by concatenating [CLS] token vectors and sentence vector averages. 3.The deep learning-based remote banking fuzzy question and answer accurate matching method according to claim 1, characterized in that, The improved triple loss function is used for two-stage training, which comprises: freezing the parameters of the first several layers of BERT in the first stage; and unfreezing all parameters in the second stage, using cosine annealing learning rate scheduling and early stopping mechanism.

4. The deep learning-based remote bank fuzzy question and answer accurate matching method according to claim 1, characterized in that, The business semantic system is built, business categories, scene classifications of each category, and specific intents of each scene are determined, and a structured intent framework is formed, specifically: a three-level structure is formed according to "business category-scene classification-specific intent", covering the core business field of the bank; a tree structured intent framework is formed through hierarchical association, clearly defining the scene classifications contained in each business category and the specific intents contained in each scene classification; all classifications and intents are labeled with unique identification codes for accurate positioning and association during subsequent data processing and model calling. 5.The deep learning-based remote banking fuzzy question and answer accurate matching method according to claim 1, characterized in that, The term and variant mapping library is established to match variants for core business terms and form a mapping relationship, specifically: core business terms containing business names, operation behaviors, and state description key concepts are selected from the business categories, scene classifications, and specific intents covered by the three-level business semantic system; through analysis of remote bank user interaction logs, artificial customer service records, and business expert interviews, variant expressions are collected for each core term, including colloquial expressions covering user daily communication habits, abbreviations containing pinyin initials and industry common abbreviations, and misspelled words containing common input errors and homophonic errors; a mapping relationship between terms and variants is established according to the structure of "standard term ID, variant type, variant content, occurrence frequency, and associated business scene" to form a mapping library. 6.The deep learning-based remote banking fuzzy question and answer accurate matching method according to claim 1, characterized in that, The sentence pattern templates containing replaceable entity slots are designed for each specific intent, specifically: the core semantic elements of each intent are determined; templates covering declarative sentences, interrogative sentences, and request sentences are designed based on these elements; the replaceable entity slot types in the templates match the business characteristics in the banking field; the business entity slots are filtered from the term and variant mapping library to select the standard terms and variants corresponding to the intent; the templates are designed according to the real interactive scenarios of users; sentences are generated by arranging and combining the slot candidate contents, and a sufficient number of semantically consistent sentences are generated for a single intent; the template library is updated in conjunction with the term and variant mapping library, and when new variants are added to the mapping library, they are supplemented to the candidate contents of the corresponding slots.

7. The deep learning-based remote bank fuzzy question and answer accurate matching method according to claim 1, characterized in that, The constructed sentence pattern templates are used to expand the original annotated similar question pairs, and several derived sentences are generated for each original sample; several candidate similar sentences are generated for each original question; after filtering, similar sentences that meet the rules are retained; specifically: the constructed sentence pattern templates are used to expand the original annotated similar question pairs, the corresponding intent is determined, and the sentence pattern templates containing replaceable entity slots under the corresponding intent are matched; the corresponding variants are extracted from the term and variant mapping library, and the derived sentences are generated by replacing the slots according to the slot types, and it is ensured that the derived sentences are semantically consistent with the original sentences; A T5 model fine-tuned on banking domain data is used to generate candidate similar sentences for each original question; the candidate similar sentences are filtered in multiple layers, first by a text similarity index, then by checking whether they contain the same business entity as the original question, and finally by checking whether they meet the business logic of the corresponding third-level intent; high-quality similar sentences are selected from the filtered candidate sentences according to semantic diversity; The derived sentences generated by template expansion and the selected high-quality similar sentences together constitute the augmented samples. 8.The deep learning-based remote banking fuzzy question and answer precision matching method according to claim 1, characterized in that, For different original questions under the same specific intent, cross-sentence similar pairs are generated by clustering; negative samples are selected from different specific intents to ensure that their literal overlap rate meets a threshold, specifically: for all original questions under the same third-level specific intent, first, the standardization preprocessing is performed through the term and variant mapping library to convert colloquial expressions, abbreviations, and misspelled words into corresponding standard terms; then, the Chinese BERT-base model fine-tuned on the banking domain is input, and the unique label embedding vector of the third-level intent is simultaneously spliced; after the encoding layer processing, the semantic vector is generated through the pooling layer, and all vectors are L2 standardized; then, K-means semantic vector clustering is performed; after clustering is completed, cross-sentence similar pairs are formed by randomly selecting sentences from different clusters and combining them two by two, and each pair of samples needs to come from different original question corresponding clusters; all generated cross-sentence similar pairs are submitted to the corresponding business domain experts for semantic similarity annotation. Then the negative sample selection is carried out, the source of the negative sample is limited to other three-level intents different from the three-level intent to which the positive sample belongs, and the three-level intent under different secondary business scenes is preferentially selected, the bank domain customized tokenizer is used to tokenize the positive sample and extract the core word set, then the sentences in the candidate negative sample library are screened, the literal overlap rate of the negative sample and the positive sample is calculated, only the sentences with an overlap rate meeting the threshold are retained, and the positive and negative sample ratio is controlled.

9. The deep learning-based remote bank fuzzy question and answer accurate matching method according to claim 1, characterized in that, In the first stage, the parameters of the first several layers of BERT are frozen, and in the second stage, all parameters are unfrozen, and a cosine annealing learning rate scheduling and early stopping mechanism is adopted, specifically: the improved triplet loss function takes the anchor sentence, the positive sample sentence of the same three-level business intent and the negative sample sentence of different three-level business intent as input; the business category similarity calculation loss value is integrated according to the judgment of the three-level business semantic system; the model is guided to learn the semantic features of "shortening the distance of positive samples and lengthening the distance of negative samples"; The training is carried out based on the bank domain training data set; the AdamW optimizer is used to suppress overfitting through weight decay; in the first stage, the parameters of the first several layers of BERT are frozen, and only part of the layer parameters are trained; in the second stage, all parameters of BERT are unfrozen and the initial learning rate is adjusted; The cosine annealing learning rate scheduling strategy and early stopping mechanism are adopted in the training process. 10.The deep learning-based remote banking fuzzy question and answer accurate matching method according to claim 1, characterized in that, The input question and the candidate answer sentence are converted into sentence vectors by the trained twin network, and the cosine similarity is used to determine whether they are semantically similar, specifically: the term and variant mapping library is called to convert colloquial expressions, abbreviations and misspelled words into standard terms, and the three-level business intent of the input question is determined by the bank domain intent recognition module and the corresponding intent label embedding vector is generated; then the preprocessed text is input into the trained twin BERT network, the input layer tokenizes the text to generate word vectors, which are then concatenated with the intent label embedding vector in the sequence dimension, the encoding layer encodes the semantics through the domain fine-tuned BERT model, the pooling layer extracts the [CLS] token vector and the average vector of the text word vector, and then the unified sentence vector is generated after linear mapping and concatenation, and then output through the dropout layer; then the cosine similarity between the input question vector and each candidate answer vector is calculated, the candidate answer vector is derived from the standard question and answer library of the three-level intent to which the input question belongs; finally, the similarity threshold is determined according to the validation set effect, if the input question matches multiple potential three-level intents, the cosine similarity of the candidate answer sentence under each intent is calculated and the highest score is compared with the threshold, if the score meets the standard, it is determined that the two are semantically similar, and the highest score and the corresponding three-level intent are recorded to provide a basis for subsequent business rule verification.

Images