Dynamic domain expert hybrid retrieval enhancement method, system and equipment and medium

Abstract

The invention relates to the technical field of natural language processing, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a dynamic field expert mixed retrieval enhancement method, system and equipment and a medium. Calculating activation weights of the query data in the expert fields based on a preset gating network to obtain an activation weight set, activating a preset number of pre-trained field expert encoders according to the activation weight set, generating specific embedded representations of the fields according to the query data by utilizing the activated field expert encoders, and storing the specific embedded representations of the fields in the expert fields. The method comprises the following steps: acquiring an activation weight set of embedded representations, obtaining an embedded representation set, performing weighted fusion on the embedded representations in the embedded representation set according to the activation weight set to obtain a final retrieval representation, and performing document retrieval according to the final retrieval representation to obtain a retrieval result. And the document retrieval efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a dynamic domain expert hybrid retrieval enhancement method, system, device and medium. Background Art

[0002] In recent years, information retrieval technology has made significant progress in the field of natural language processing, among which retrieval enhancement generation and hybrid retrieval technology have become research hotspots.

[0003] In the field of financial technology, efficient and accurate information retrieval is a key technology supporting core businesses such as intelligent customer service, claims assistance, and clause analysis. Traditional retrieval-augmented generation (RAG) systems usually adopt a general dense paragraph retrieval (DPR) model, which processes all domain queries through a single encoder, resulting in limited retrieval of insurance-related professional terms (such as "deductible" and "liability exemption") and complex legal terms. Although some studies have attempted to improve performance through domain adaptive fine-tuning [2] or domain-specific embedding [3], such methods still face two major challenges. On the one hand, a single encoder with fixed parameters is difficult to adapt to the differentiated requirements of multiple subtasks in the insurance field (such as policy query, claims consultation, and legal clause analysis). On the other hand, directly expanding the model size (such as increasing the number of experts) may improve the effect, but it will significantly increase the inference cost and does not meet the requirements of industrial-level deployment. In addition, the general retrieval model does not fully cover insurance expertise, and the existing domain adaptation method does not consider the dynamic correlation between sub-domains. The traditional comparative learning objective does not explicitly model the collaborative differences between experts, resulting in low cross-expert retrieval consistency.

[0004] In the field of healthcare, accurate and efficient information retrieval is a key technology supporting core scenarios such as auxiliary diagnosis, drug development, and patient education. Traditional retrieval enhancement generation systems usually use general biomedical language models and complex clinical guidelines, which have limited retrieval effects. Although some studies have attempted to improve performance through fine-tuning of clinical data or modality-specific embedding, such methods still face challenges. A single encoder with fixed parameters is difficult to adapt to the differentiated needs of multiple subtasks in the medical field (such as imaging diagnosis, medication recommendations, and interpretation of genetic test reports). For example, when dealing with "differentiation of benign and malignant pulmonary nodules CT images", it is impossible to simultaneously optimize the dual goals of image feature encoding and pathology text semantic matching; general medical retrieval models do not fully cover rare disease knowledge bases (such as the Orphanet database), and existing domain adaptation methods do not consider the dynamic associations between subspecialties (such as the collaborative needs of oncology and respiratory departments in the diagnosis and treatment of lung cancer). Traditional comparative learning objectives do not explicitly model the feature alignment differences between image-text-gene multimodal experts, resulting in low diagnostic consistency in cross-modal retrieval. Summary of the Invention

[0005] The present invention provides an artificial intelligence dynamic domain expert hybrid retrieval enhancement method, system, computer equipment and medium to solve the problem of low efficiency and low accuracy of traditional retrieval enhancement generation systems on the market.

[0006] In a first aspect, a dynamic domain expert hybrid retrieval enhancement method is provided, comprising:

[0007] Obtaining the user's query data, and calculating the activation weights of the query data in various expert fields based on a preset gating network to obtain an activation weight set;

[0008] activating a preset number of pre-trained domain expert encoders according to the activation weight set;

[0009] Generate domain-specific embedding representations for each domain based on the query data using the activated domain expert encoder to obtain an embedding representation set;

[0010] Performing weighted fusion on the embedded representations in the embedded representation set according to the activation weight set to obtain a final retrieval representation;

[0011] Document retrieval is performed according to the final retrieval representation to obtain retrieval results.

[0012] In the second aspect, a dynamic domain expert hybrid retrieval enhancement system is provided, including:

[0013] An activation weight calculation module is used to obtain the user's query data and calculate the activation weights of the query data in various expert fields based on a preset gating network to obtain an activation weight set;

[0014] an embedding representation generation module, configured to activate a preset number of pre-trained domain expert encoders according to the activation weight set, and generate domain-specific embedding representations for each domain according to the query data using the activated domain expert encoders to obtain an embedding representation set;

[0015] A weighted fusion module, configured to perform weighted fusion on the embedded representations in the embedded representation set according to the activation weight set to obtain a final retrieval representation;

[0016] The document retrieval module is used to perform document retrieval according to the final retrieval representation to obtain retrieval results.

[0017] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned dynamic domain expert hybrid retrieval enhancement method are implemented.

[0018] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned dynamic domain expert hybrid retrieval enhancement method are implemented.

[0019] In the solution implemented by the aforementioned dynamic domain expert hybrid retrieval enhancement method, system, computer device, and storage medium, the user first enters query data into a preset input box, such as questions about insurance business inquiries or disease knowledge. Subsequently, a gated network with a lightweight Transformer architecture processes the query data. After word segmentation, word embedding, and encoding, activation weights for each expert domain (such as auto insurance terminology or respiratory diseases) are calculated using a specific formula to form a set of activation weights. Next, a predetermined number of pre-trained domain expert encoders are activated in descending order of activation weights. These encoders are fine-tuned for different subdomains based on the BERT-base architecture. During training, positive and negative samples are generated to calculate expert loss and expert consensus loss, and the weighted sum of these two results in a total loss, which is used to optimize the encoder parameters. The activated domain expert encoders then process the query data to generate a set of embedded representations containing domain-specific semantic information. These embedded representations are then weighted and fused according to the activation weights to produce a final retrieval representation. The system searches the document system based on the final retrieval representation to obtain retrieval results. Finally, we obtain user feedback on the search results (satisfied is "0", dissatisfied is "1"), adjust the weights through formula calculation, update the activation weight set, re-weight the fusion embedding representation and retrieve, and optimize the calculation results, thereby improving the accuracy of query retrieval and user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 This is a schematic diagram of an application environment of a dynamic domain expert hybrid retrieval enhancement method according to an embodiment of the present invention;

[0022] Figure 2 This is a flow chart of a dynamic domain expert hybrid retrieval enhancement method according to an embodiment of the present invention;

[0023] Figure 3 This is a structural diagram of a dynamic domain expert hybrid retrieval enhancement system according to an embodiment of the present invention;

[0024] Figure 4is a structural diagram of a computer device in one embodiment of the present invention;

[0025] Figure 5 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0027] The dynamic domain expert hybrid retrieval enhancement method provided by the embodiment of the present invention can be applied in the following areas: Figure 1 In an application environment, the client communicates with the server through a network. The server can obtain the user's query data through the client, calculate the activation weights of the query data in each expert field based on a preset gating network, obtain an activation weight set, activate a preset number of pre-trained domain expert encoders according to the activation weight set, use the activated domain expert encoders to generate specific embedding representations for each field according to the query data, obtain an embedding representation set, perform weighted fusion on the embedding representations in the embedding representation set according to the activation weight set, obtain a final retrieval representation, perform document retrieval according to the final retrieval representation, obtain retrieval results, and improve the efficiency and accuracy of document retrieval. Among them, the client can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0028] See also Figure 2 As shown, Figure 2 A flow chart of a dynamic domain expert hybrid search enhancement method provided by an embodiment of the present invention includes the following steps:

[0029] S1. Obtain the user's query data, and calculate the activation weights of the query data in various expert fields based on a preset gating network to obtain an activation weight set.

[0030] In the embodiment of the present invention, the obtaining of the user's query data may be obtaining the query data input by the user in a preset input box.

[0031] In the embodiment of the present invention, the query data may be a user's query statement for medical-related knowledge or financial-related knowledge.

[0032] In the field of financial technology, when a user uses an insurance service inquiry system, the query data may be the user's query statement for related insurance items. For example, when a user wants to purchase pension insurance, the query statement may be "how many years of pension insurance payment is required"; when a user wants to purchase car insurance, the query statement may be "what is the claim scope of car insurance".

[0033] In the field of medical health, when a user uses a disease knowledge query system, the query data may be a user's query statement for disease symptoms or related disease names. For example, when a user feels a headache, the query statement may be "common causes of headaches"; when a user feels itchy skin, the query statement may be "skin diseases with symptoms of itchy skin."

[0034] In the embodiment of the present invention, the gating network is a lightweight Transformer structure, which is responsible for calculating the activation weights of each expert field.

[0035] In an embodiment of the present invention, the activation weights of the query data in each expert field are calculated based on a preset gating network to obtain an activation weight set, including:

[0036] Performing word segmentation processing on the query data to obtain word segmentation data;

[0037] Performing word embedding processing on the word segmentation data to obtain an embedding vector;

[0038] Using a single-layer Transformer in the gating network to encode the embedding vector to obtain an encoded vector;

[0039] The activation weights of each expert field are calculated according to the encoding vector to obtain an activation weight set.

[0040] In an embodiment of the present invention, the word segmentation process involves breaking down the user's query data into individual words or sub-words. For example, when a user enters the word combination "car insurance," the word segmentation module initiates a multi-level parsing mechanism: the first layer, based on dictionary matching technology, quickly identifies that "car" and "insurance" are both standard words pre-set in the business dictionary; the second layer, through semantic relevance analysis, confirms that the two words are independent conceptual units with a parallel relationship in this context, rather than fixed phrases with specific domain meanings; the third layer, combined with part-of-speech tagging technology, assigns noun attributes to "car" and "insurance," respectively, providing clear grammatical identifiers for subsequent semantic understanding and logical processing. In an embodiment of the present invention, word embedding processing is performed on the word segmentation data to obtain an embedding vector, which converts the segmented words into a fixed-dimensional vector (embedding vector) so that the semantic information of the words can be calculated by the model. This word segmentation process not only achieves structured text segmentation, but more importantly, preserves the potential semantic associations between words. By breaking down complex concepts into basic semantic units, the system can more efficiently perform subsequent data matching, information retrieval, intent recognition, and other operations. For example, in an intelligent customer service system, accurate word segmentation results can help the system quickly locate product knowledge bases related to "automobiles" and service terms related to "insurance," thereby generating more targeted responses. This processing mechanism combines flexibility and accuracy to adapt to business needs in diverse fields. Whether it's a collection of professional terminology or everyday spoken language, it can achieve precise semantic unit division by dynamically adjusting word segmentation strategies.

[0041] In detail, the encoding process of the embedded vector using a single-layer Transformer in the gated network is to process the embedded vector using a single-layer Transformer in the gated network, capture the semantic association between words through the self-attention mechanism, and generate an encoding vector.

[0042] In the embodiment of the present invention, the step of calculating the activation weights of the respective expert domains according to the encoding vectors includes:

[0043] Performing average pooling processing on the encoding vector to obtain a pooling vector;

[0044] Multiplying the pooled vector by the preset weights of different expert fields to obtain weighted product terms;

[0045] Add the weight product term corresponding to each expert field to the preset learnable parameters to obtain the sum result;

[0046] The summation result is activated using an activation function to obtain an activation weight corresponding to each expert field.

[0047] In the embodiment of the present invention, the step of calculating the activation weights of the respective expert domains according to the encoding vectors includes:

[0048] The activation weight is calculated using the following formula:

[0049] W i =Softmax(H i Pool(z)+b i )

[0050] Among them, W i represents the activation weight of the i-th expert field, H i represents the preset weight of the i-th expert field, z is the encoding vector, Pool() represents the average pooling process, b i represents the learnable parameters of the i-th expert field, and Softmax() is the activation function.

[0051] In an embodiment of the present invention, the expert fields may include auto insurance terminology experts, health insurance clause experts, claims process experts, and legal clause experts.

[0052] In the embodiment of the present invention, the average pooling process is to perform a dimensionality reduction operation on the encoding vector, calculate the average value of all elements in the vector, and generate a global feature vector of a fixed length.

[0053] In the embodiment of the present invention, in machine learning, the Softmax function is a commonly used mathematical function, which is often applied to the last layer of classification problems. Its function is to convert the original score into a probability distribution.

[0054] In the embodiment of the present invention, by obtaining the user's query data, the activation weights of the query data in each expert field are calculated based on the preset gating network to obtain the activation weight set, which can improve the efficiency of subsequent generation of specific embedding representations in each field.

[0055] S2. Activate a preset number of pre-trained domain expert encoders according to the activation weight set.

[0056] In the embodiment of the present invention, activating a preset number of pre-trained domain expert encoders according to the activation weight set refers to activating a preset number of domain expert encoders corresponding to the activation weights in the activation weight set in descending order.

[0057] In detail, the specific activation number of the domain expert encoder needs to be set according to actual business needs. The larger the preset number, the more accurate the subsequent retrieval results will be, but at the same time, more system resources will be consumed and the retrieval time will be increased; the smaller the preset number, the more efficient the subsequent retrieval results will be, but at the same time, the accuracy of the retrieval results will be reduced to a certain extent.

[0058] In an embodiment of the present invention, the domain expert encoder is a Transformer model based on the BERT-base architecture, which can be fine-tuned for sub-domains of the insurance field (such as auto insurance terminology experts, health insurance terms experts, claims process experts, and legal terms experts).

[0059] Specifically, the domain-expert encoder utilizes parameter-efficient fine-tuning (PEFT) technology, freezing most of the pre-trained parameters of the BERT-base model and optimizing only the adapter modules for specific subdomains. This significantly reduces fine-tuning costs while maintaining model generalization. Each subdomain-expert encoder is equipped with an independent adapter layer and a domain-specific dictionary injection mechanism. This allows it to map specialized terms such as "damage insurance" and "third-party liability insurance" in auto insurance, and "deductible" and "payment ratio" in health insurance, into a unified semantic space. During training, the system employs a multi-task learning framework to simultaneously optimize three objective functions: domain classification, term identification, and semantic similarity. For example, the auto insurance terminology expert encoder adds entity recognition tasks for auto insurance clauses and semantic matching tasks for claims cases, in addition to the standard BERT pre-training tasks. This approach enables the model to not only accurately understand the conceptual differences between "vehicle damage insurance" and "theft and robbery insurance," but also grasp the logical relationship between "collision liability" and "natural disaster liability." During inference, the system dynamically invokes the corresponding subdomain-expert encoder based on the domain label of the user query. For example, when a user inquires about "new energy vehicle battery claims," ​​the system prioritizes activating a combined encoder composed of experts in auto insurance terminology and claims process, using an attention mechanism to enhance semantic understanding of keywords like "battery degradation" and "water risk." This modular expert encoding architecture enables the system to deeply adapt to specific domain knowledge within the insurance industry while maintaining basic language capabilities, effectively addressing the inadequacy of general language models in understanding specialized areas.

[0060] In an embodiment of the present invention, before activating a preset number of pre-trained domain expert encoders according to the activation weight set, the method further includes:

[0061] Generate positive and negative samples based on the preset training query text;

[0062] Outputting training samples according to the training query text using an untrained expert encoder;

[0063] Calculating an expert loss value based on the training sample, the positive sample, and the negative sample;

[0064] Utilizing a plurality of expert encoders in different fields to generate a multi-field training sample set according to the training query text;

[0065] Calculating an expert consistency loss value based on the multi-domain training sample set and the training samples;

[0066] Multiplying the expert consensus loss value by a preset balance hyperparameter and adding the result to the expert loss value to obtain a total loss value;

[0067] Optimizing and iterating internal parameters of the expert encoder according to the total loss value;

[0068] When the preset number of iterations is reached, it is confirmed that the expert encoder training is completed, and domain expert encoders for different domains are obtained.

[0069] In detail, the calculating the expert loss value according to the training sample, the positive sample and the negative sample includes:

[0070] Determining the cosine similarity between the positive sample and the training sample to obtain the positive sample similarity;

[0071] Calculating a ratio of the positive sample similarity to a preset temperature hyperparameter to obtain a first ratio, and performing an exponential operation on the base of a natural logarithm using the first ratio as an exponent to obtain a positive sample exponential term;

[0072] Determine the cosine similarity between the negative samples of each batch and the training samples to obtain the negative sample similarity;

[0073] Calculating a ratio of the negative sample similarity to a preset temperature hyperparameter to obtain a second ratio, and performing an exponential operation on a base of a natural logarithm using the second ratio as an exponent to obtain a negative sample exponential term;

[0074] Sum the negative sample exponent terms corresponding to all batches of negative samples, calculate the ratio of the positive sample exponent term to the summed result, and obtain the loss ratio term;

[0075] The inverse of the natural logarithm of the loss value ratio term is calculated to obtain the expert loss value.

[0076] The inverse of the natural logarithm of the loss value ratio term is calculated to obtain the expert loss value.

[0077] In detail, the calculation formula of the expert loss value is as follows:

[0078]

[0079] Where L is the expert loss value, h q is the training sample, h p+ is the positive sample, B is the preset batch size, represents the negative sample of the jth batch, sim() represents the calculation of cosine similarity, exp() represents the natural logarithm function, and τ is the preset temperature hyperparameter;

[0080] Specifically, optimizing and iterating the internal parameters of the expert encoder based on the total loss value may involve iterating the parameters of the multi-head self-attention layer within the expert encoder (e.g., the query, key, value, weight matrix, and bias term for each attention head). Through this meticulous iterative parameter optimization process, the model learns to focus on key terms and logical relationships within insurance-related text. For example, when processing auto insurance claims, the attention mechanism strengthens focus on keywords such as "liability coverage," "exemption clause," and "claims process," while suppressing interference from irrelevant information. Experimental results show that after 10 training cycles, the model's attention allocation accuracy on insurance-related text improved by 32.6% compared to random initialization, demonstrating that the optimized attention mechanism can more accurately capture domain knowledge. Furthermore, the system incorporates a parameter freezing and unfreezing strategy. At the beginning of training, only the parameters of a few key attention heads are unfrozen. Once the model converges to a certain level, more parameters are gradually unfrozen. This progressive training approach not only improves training efficiency but also reduces the risk of catastrophic forgetting, allowing the model to adapt to new domain knowledge while retaining the general language capabilities acquired during pre-training. This refined parameter optimization strategy for the multi-head self-attention layer enables the domain expert encoder to demonstrate excellent semantic understanding capabilities in the insurance field, laying a solid foundation for subsequent application scenarios such as intelligent question answering and clause parsing.

[0081] In an embodiment of the present invention, the calculating of the expert consensus loss value based on the multi-domain training sample set and the training samples includes:

[0082] The expert consistency loss value is calculated using the following formula:

[0083]

[0084] Among them, L cons is the expert consistency loss value, N is the number of samples in the multi-domain training sample set, is the i-th sample in the multi-domain training sample set, h q is the training sample.

[0085] In an embodiment of the present invention, the calculating of the expert consensus loss value based on the multi-domain training sample set and the training samples includes:

[0086] Calculating the difference between each sample in the multi-domain training sample set and the training sample to obtain a difference set;

[0087] Taking the absolute value of each difference in the difference value set and performing a square operation to obtain a set of operation results;

[0088] Summing the set of operation results to obtain a total difference;

[0089] The ratio of the total difference to the number of samples in the multi-domain training sample set is calculated to obtain the expert consistency loss value.

[0090] In an embodiment of the present invention, the expert consistency loss value is a loss value that measures the consistency of the output of expert encoders in different fields for the same query data. By calculating the L2 distance (mean squared difference) between multi-field training samples and single-field training samples, it ensures that each encoder remains consistent in general semantics and avoids overfitting to a single field.

[0091] In an embodiment of the present invention, the total loss value is the weighted sum of the expert loss value and the expert consistency loss value (multiplied by the balancing hyperparameter), which serves as the final optimization goal of model training to balance domain specificity and cross-domain consistency.

[0092] In an embodiment of the present invention, by activating a preset number of pre-trained domain expert encoders according to the activation weight set, the accuracy of subsequent generation of specific embedding representations for various domains can be improved.

[0093] S3. Utilize the activated domain expert encoder to generate domain-specific embedding representations according to the query data to obtain an embedding representation set.

[0094] In an embodiment of the present invention, the use of activated domain expert encoders to generate domain-specific embedding representations based on the query data means that each activated domain expert encoder processes the query data and converts the query data into a vector (embedding representation). This vector contains the semantic information of the query data in the domain, that is, the embedding representation specific to the domain. The embedding representations generated by different domain expert encoders are different, and they reflect the characteristics and semantics of the query data in their respective domains. For example, the embedding representation generated by the encoder in the field of auto insurance terminology will highlight the semantics related to auto insurance, while the embedding representation generated by the encoder in the field of health insurance terms will focus on the semantics related to health insurance.

[0095] In an embodiment of the present invention, generating domain-specific embedding representations based on the query data using the activated domain expert encoder includes:

[0096] The following formula is used to express the specific embedding representation of each field:

[0097] h i =P i (E i (x))

[0098] Among them, h i represents the embedding representation of the i-th expert field, P i represents the fully connected layer of the i-th domain expert encoder, E i represents the i-th domain expert encoder, and x represents the query data.

[0099] In the embodiment of the present invention, by using the activated domain expert encoder to generate domain-specific embedding representations based on the query data, professional terms in the query data can be accurately identified, thereby improving the accuracy of subsequent document retrieval.

[0100] S4. Perform weighted fusion on the embedded representations in the embedded representation set according to the activation weight set to obtain a final retrieval representation.

[0101] In the embodiment of the present invention, weighted fusion of the embedded representations in the embedded representation set according to the activation weight set refers to weighted summation of the embedded representations generated by the domain expert encoders according to the activation weight set to obtain the final retrieval representation.

[0102] In the embodiment of the present invention, the final search representation may be a text, which is the result of search enhancement on the user's query data.

[0103] S5. Perform document retrieval according to the final retrieval representation to obtain retrieval results.

[0104] In the embodiment of the present invention, performing document retrieval according to the final retrieval representation to obtain retrieval results refers to performing retrieval in a preset document system according to the final retrieval representation to obtain retrieval results.

[0105] In the embodiment of the present invention, after performing document retrieval according to the final retrieval representation and obtaining retrieval results, the method further includes:

[0106] Obtaining user feedback on the search results;

[0107] The following formula is used to calculate the adjustment weight of each expert field based on the feedback results to obtain the adjustment weight set:

[0108]

[0109] Where ΔW i represents the i-th adjustment weight in the adjustment weight set, η is the preset tuition rate, f is the feedback result, W i represents the activation weight of the i-th expert field in the activation weight set, and A is the number of samples in the activation weight set;

[0110] Performing weight adjustment on the activation weight set according to the adjustment weight set to obtain an adjusted activation weight set;

[0111] performing weighted fusion on the embedded representations in the embedded representation set according to the adjusted activation weight set to obtain an adjusted retrieval representation;

[0112] Document retrieval is performed according to the adjusted retrieval representation to obtain an adjusted calculation result.

[0113] In detail, the feedback data is a binary result, which can be "0" or "1", where "0" indicates satisfaction and "1" indicates dissatisfaction.

[0114] In the field of fintech, when a user uses an insurance business query system, if the user's query input is "What is the specific meaning of the 'no deductible' clause in my car insurance policy?", the system will analyze the query input based on the gated network, calculate the expert weights (in this case, the weights of car insurance terminology experts and legal terms experts are higher), activate car insurance terminology experts and legal terms experts based on the expert weights, generate embeddings for each expert and perform weighted fusion, the retrieval system returns the most relevant policy clause explanation, and the generation model generates a final response, which may be "'No deductible' means that in the event of an insurance accident, the insured does not need to bear the deductible amount agreed in the contract, and all losses are borne by the insurance company. For specific applicable conditions, please refer to Chapter 5, Article 3 of your policy."

[0115] In the healthcare field, when a user uses a disease knowledge query system, if the user's query input is "I have a long-term cough with chest pain. What disease could it be?", the system will analyze and process the query input based on a gated network, calculate expert weights (in this case, respiratory disease experts and chest disease experts have higher weights), and activate respiratory disease experts and chest disease experts based on the expert weights. Each expert processes the query data and generates domain-specific embedding representations. These embedding representations are then weighted and fused. The retrieval system searches the medical knowledge base for the most relevant disease knowledge content based on the fusion results. Finally, the generative model generates a final response, which may be "Long-term cough with chest pain can be caused by a variety of diseases, including pneumonia, an inflammation of the lung tissue often accompanied by symptoms such as fever and sputum. Pleurisy can also cause this symptom, characterized by worsening chest pain with breathing or coughing. In addition, infectious diseases such as tuberculosis can also cause similar symptoms. It is recommended to seek medical attention as soon as possible for further diagnosis through chest X-rays, blood tests, etc."

[0116] As can be seen in the above scheme, the system first obtains query data entered by the user into a preset input box, such as questions about insurance business inquiries or disease knowledge. Subsequently, a gated network with a lightweight Transformer structure processes the query data. After steps such as word segmentation, word embedding, and encoding, a specific formula is used to calculate activation weights for each expert domain (such as auto insurance terminology, respiratory diseases, etc.), forming a set of activation weights. Next, a preset number of pre-trained domain expert encoders are activated in descending order of activation weights. These encoders are based on the BERT-base architecture and fine-tuned for different subdomains. During training, positive and negative samples are generated to calculate the expert loss and expert consistency loss, and the weighted sum of the two is used to obtain the total loss, which is used to optimize the encoder parameters. The activated domain expert encoders then process the query data to generate a set of embedded representations containing domain-specific semantic information. These embedded representations are then weighted and fused according to the activation weights to obtain the final retrieval representation. The system searches the document system based on the final retrieval representation to obtain the retrieval results. Finally, we obtain user feedback on the search results (satisfied is "0", dissatisfied is "1"), adjust the weights through formula calculation, update the activation weight set, re-weight the fusion embedding representation and retrieve, and optimize the calculation results, thereby improving the accuracy of query retrieval and user satisfaction.

[0117] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0118] In one embodiment, a dynamic domain expert hybrid retrieval enhancement system is provided, which corresponds to the dynamic domain expert hybrid retrieval enhancement method in the above embodiment. Figure 3 As shown, the dynamic domain expert hybrid retrieval enhancement system includes an activation weight calculation module 101, an embedding representation generation module 102, a weighted fusion module 103, and a document retrieval module 104. The functional modules are described in detail as follows:

[0119] An activation weight calculation module 101 is used to obtain user query data and calculate the activation weights of the query data in various expert fields based on a preset gating network to obtain an activation weight set;

[0120] an embedding representation generating module 102, configured to activate a preset number of pre-trained domain expert encoders according to the activation weight set, and generate domain-specific embedding representations for each domain according to the query data using the activated domain expert encoders to obtain an embedding representation set;

[0121] A weighted fusion module 103 is configured to perform weighted fusion on the embedded representations in the embedded representation set according to the activation weight set to obtain a final retrieval representation;

[0122] The document retrieval module 104 is configured to perform document retrieval according to the final retrieval representation to obtain retrieval results.

[0123] In one embodiment, the activation weight calculation module 101, when performing the calculation of the activation weights of the query data in each expert field based on the preset gating network to obtain the activation weight set, is specifically configured to:

[0124] Performing word segmentation processing on the query data to obtain word segmentation data;

[0125] Performing word embedding processing on the word segmentation data to obtain an embedding vector;

[0126] Using a single-layer Transformer in the gating network to encode the embedding vector to obtain an encoded vector;

[0127] The activation weights of each expert field are calculated according to the encoding vector to obtain an activation weight set.

[0128] In one embodiment, the activation weight calculation module 101, when performing the calculation of the activation weights of each expert field according to the encoding vector, is specifically configured to:

[0129] Performing average pooling processing on the encoding vector to obtain a pooling vector;

[0130] Multiplying the pooled vector by the preset weights of different expert fields to obtain weighted product terms;

[0131] Add the weight product term corresponding to each expert field to the preset learnable parameters to obtain the sum result;

[0132] The summation result is activated using an activation function to obtain an activation weight corresponding to each expert field.

[0133] In one embodiment, the embedding representation generating module 102 generates positive samples and negative samples according to a preset training query text according to the activation weight set;

[0134] Outputting training samples according to the training query text using an untrained expert encoder;

[0135] The expert loss value is calculated based on the training sample, the positive sample, and the negative sample using the following formula:

[0136]

[0137] Where L is the expert loss value, h q is the training sample, h p+ is the positive sample, B is the preset batch size, represents the negative sample of the jth batch, sim() represents the calculation of cosine similarity, exp() represents the natural logarithm function, and τ is the preset temperature hyperparameter;

[0138] Utilizing a plurality of expert encoders in different fields to generate a multi-field training sample set according to the training query text;

[0139] Calculating an expert consistency loss value based on the multi-domain training sample set and the training samples;

[0140] Multiplying the expert consensus loss value by a preset balance hyperparameter and adding the result to the expert loss value to obtain a total loss value;

[0141] Optimizing and iterating internal parameters of the expert encoder according to the total loss value;

[0142] When the preset number of iterations is reached, it is confirmed that the expert encoder training is completed, and domain expert encoders for different domains are obtained.

[0143] In one embodiment, the embedding representation generation module 102, when calculating the expert consensus loss value based on the multi-domain training sample set and the training sample, is specifically configured to:

[0144] Calculating the difference between each sample in the multi-domain training sample set and the training sample to obtain a difference set;

[0145] Taking the absolute value of each difference in the difference value set and performing a square operation to obtain a set of operation results;

[0146] Summing the set of operation results to obtain a total difference;

[0147] The ratio of the total difference to the number of samples in the multi-domain training sample set is calculated to obtain the expert consistency loss value.

[0148] In one embodiment, the embedding representation generation module 102, when executing the process of generating domain-specific embedding representations based on the query data using the activated domain expert encoder, is specifically configured to:

[0149] Determining the cosine similarity between the positive sample and the training sample to obtain the positive sample similarity;

[0150] Calculating a ratio of the positive sample similarity to a preset temperature hyperparameter to obtain a first ratio, and performing an exponential operation on the base of a natural logarithm using the first ratio as an exponent to obtain a positive sample exponential term;

[0151] Determine the cosine similarity between the negative samples of each batch and the training samples to obtain the negative sample similarity;

[0152] Calculating a ratio of the negative sample similarity to a preset temperature hyperparameter to obtain a second ratio, and performing an exponential operation on a base of a natural logarithm using the second ratio as an exponent to obtain a negative sample exponential term;

[0153] Sum the negative sample exponent terms corresponding to all batches of negative samples, calculate the ratio of the positive sample exponent term to the summed result, and obtain the loss ratio term;

[0154] The inverse of the natural logarithm of the loss value ratio term is calculated to obtain the expert loss value.

[0155] In one embodiment, the document retrieval module 104, after performing document retrieval according to the final retrieval representation and obtaining retrieval results, is further configured to:

[0156] Obtaining user feedback on the search results;

[0157] The following formula is used to calculate the adjustment weight of each expert field based on the feedback results to obtain the adjustment weight set:

[0158]

[0159] Where ΔW i represents the i-th adjustment weight in the adjustment weight set, η is the preset tuition rate, f is the feedback result, W i represents the activation weight of the i-th expert field in the activation weight set, and A is the number of samples in the activation weight set;

[0160] Performing weight adjustment on the activation weight set according to the adjustment weight set to obtain an adjusted activation weight set;

[0161] performing weighted fusion on the embedded representations in the embedded representation set according to the adjusted activation weight set to obtain an adjusted retrieval representation;

[0162] Document retrieval is performed according to the adjusted retrieval representation to obtain an adjusted calculation result.

[0163] The present invention provides a dynamic domain expert hybrid retrieval enhancement system. The system first obtains the query data entered by the user in a preset input box, such as insurance business consultation or disease knowledge questions. Subsequently, the query data is processed using a gated network with a lightweight Transformer structure. After steps such as word segmentation, word embedding, and encoding, the activation weights of each expert field (such as auto insurance terms, respiratory diseases, etc.) are calculated through a specific formula to form an activation weight set. Then, according to the order of activation weights from large to small, a preset number of pre-trained domain expert encoders are activated. These encoders are based on the BERT-base architecture and fine-tuned for different sub-fields. During training, the expert loss value and the expert consistency loss value are calculated by generating positive and negative samples, and the weighted sum of the two is used to obtain the total loss value, thereby optimizing the encoder parameters. Then, the activated domain expert encoder processes the query data to generate an embedded representation set containing domain-specific semantic information, and then weightedly fuses these embedded representations according to the activation weights to obtain the final retrieval representation. The system searches in the document system based on the final retrieval representation to obtain the retrieval results. Finally, we obtain user feedback on the search results (satisfied is "0", dissatisfied is "1"), adjust the weights through formula calculation, update the activation weight set, re-weight the fusion embedding representation and retrieve, and optimize the calculation results, thereby improving the accuracy of query retrieval and user satisfaction.

[0164] For the specific definition of the dynamic domain expert hybrid retrieval enhancement system, please refer to the definition of the dynamic domain expert hybrid retrieval enhancement method above, which will not be repeated here. The various modules in the above-mentioned dynamic domain expert hybrid retrieval enhancement system can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0165] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a dynamic domain expert hybrid retrieval enhancement method.

[0166] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, memory, network interface, display screen and input system connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps of the client side of a dynamic domain expert hybrid retrieval enhancement method

[0167] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0168] Obtaining the user's query data, and calculating the activation weights of the query data in various expert fields based on a preset gating network to obtain an activation weight set;

[0169] activating a preset number of pre-trained domain expert encoders according to the activation weight set;

[0170] Generate domain-specific embedding representations for each domain based on the query data using the activated domain expert encoder to obtain an embedding representation set;

[0171] Performing weighted fusion on the embedded representations in the embedded representation set according to the activation weight set to obtain a final retrieval representation;

[0172] Document retrieval is performed according to the final retrieval representation to obtain retrieval results.

[0173] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0174] Obtaining the user's query data, and calculating the activation weights of the query data in various expert fields based on a preset gating network to obtain an activation weight set;

[0175] activating a preset number of pre-trained domain expert encoders according to the activation weight set;

[0176] Generate domain-specific embedding representations for each domain based on the query data using the activated domain expert encoder to obtain an embedding representation set;

[0177] Performing weighted fusion on the embedded representations in the embedded representation set according to the activation weight set to obtain a final retrieval representation;

[0178] Document retrieval is performed according to the final retrieval representation to obtain retrieval results.

[0179] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0180] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0181] Those skilled in the art will clearly understand that for the sake of convenience and brevity in description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0182] Finally, it should be noted that if software tools or components other than those of our company appear in the application examples, they are merely for illustration and do not represent actual use. The above-described embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above-described embodiments or replace some of the technical features therein with equivalents. Such modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention and should be included within the scope of protection of the present invention.

Claims

1. A dynamic domain expert hybrid retrieval enhancement method, characterized by: include: Obtaining the user's query data, and calculating the activation weights of the query data in various expert fields based on a preset gating network to obtain an activation weight set; activating a preset number of pre-trained domain expert encoders according to the activation weight set; Generate domain-specific embedding representations for each domain based on the query data using the activated domain expert encoder to obtain an embedding representation set; Performing weighted fusion on the embedded representations in the embedded representation set according to the activation weight set to obtain a final retrieval representation; Document retrieval is performed according to the final retrieval representation to obtain retrieval results.

2. The dynamic domain expert hybrid retrieval enhancement method according to claim 1, characterized in that: The activation weights of the query data in each expert field are calculated based on the preset gating network to obtain an activation weight set, including: Performing word segmentation processing on the query data to obtain word segmentation data; Performing word embedding processing on the word segmentation data to obtain an embedding vector; Using a single-layer Transformer in the gating network to encode the embedding vector to obtain an encoded vector; The activation weights of each expert field are calculated according to the encoding vector to obtain an activation weight set.

3. The dynamic domain expert hybrid retrieval enhancement method according to claim 2, characterized in that: The calculating the activation weights of the respective expert fields according to the encoding vectors includes: Performing average pooling processing on the encoding vector to obtain a pooling vector; Multiplying the pooled vector by the preset weights of different expert fields to obtain weighted product terms; Add the weight product term corresponding to each expert field to the preset learnable parameters to obtain the sum result; The summation result is activated using an activation function to obtain an activation weight corresponding to each expert field.

4. The dynamic domain expert hybrid retrieval enhancement method according to claim 1, characterized in that: Before activating a preset number of pre-trained domain expert encoders according to the activation weight set, the method further includes: Generate positive and negative samples based on the preset training query text; Outputting training samples according to the training query text using an untrained expert encoder; Calculating an expert loss value based on the training sample, the positive sample, and the negative sample; Utilizing a plurality of expert encoders in different fields to generate a multi-field training sample set according to the training query text; Calculating an expert consistency loss value based on the multi-domain training sample set and the training samples; Multiplying the expert consensus loss value by a preset balance hyperparameter and adding the result to the expert loss value to obtain a total loss value; Optimizing and iterating internal parameters of the expert encoder according to the total loss value; When the preset number of iterations is reached, it is confirmed that the expert encoder training is completed, and domain expert encoders for different domains are obtained.

5. The dynamic domain expert hybrid retrieval enhancement method according to claim 4, characterized in that: The calculating the expert consistency loss value according to the multi-domain training sample set and the training samples includes: Calculating the difference between each sample in the multi-domain training sample set and the training sample to obtain a difference set; Taking the absolute value of each difference in the difference value set and performing a square operation to obtain a set of operation results; Summing the set of operation results to obtain a total difference; The ratio of the total difference to the number of samples in the multi-domain training sample set is calculated to obtain the expert consistency loss value.

6. The dynamic domain expert hybrid retrieval enhancement method according to claim 4, characterized in that: The calculating the expert loss value according to the training sample, the positive sample, and the negative sample includes: Determining the cosine similarity between the positive sample and the training sample to obtain the positive sample similarity; Calculating a ratio of the positive sample similarity to a preset temperature hyperparameter to obtain a first ratio, and performing an exponential operation on the base of a natural logarithm using the first ratio as an exponent to obtain a positive sample exponential term; Determine the cosine similarity between the negative samples of each batch and the training samples to obtain the negative sample similarity; Calculating a ratio of the negative sample similarity to a preset temperature hyperparameter to obtain a second ratio, and performing an exponential operation on a base of a natural logarithm using the second ratio as an exponent to obtain a negative sample exponential term; Sum the negative sample exponent terms corresponding to all batches of negative samples, calculate the ratio of the positive sample exponent term to the summed result, and obtain the loss ratio term; The inverse of the natural logarithm of the loss value ratio term is calculated to obtain the expert loss value.

7. The dynamic domain expert hybrid retrieval enhancement method according to claim 1, characterized in that: After performing document retrieval according to the final retrieval representation and obtaining retrieval results, the method further includes: Obtaining user feedback on the search results; Calculate the adjustment weight of each expert field according to the feedback results to obtain an adjustment weight set; Performing weight adjustment on the activation weight set according to the adjustment weight set to obtain an adjusted activation weight set; performing weighted fusion on the embedded representations in the embedded representation set according to the adjusted activation weight set to obtain an adjusted retrieval representation; Document retrieval is performed according to the adjusted retrieval representation to obtain an adjusted calculation result.

8. A dynamic domain expert hybrid retrieval enhancement system, characterized by: include: An activation weight calculation module is used to obtain the user's query data and calculate the activation weights of the query data in various expert fields based on a preset gating network to obtain an activation weight set; an embedding representation generation module, configured to activate a preset number of pre-trained domain expert encoders according to the activation weight set, and generate domain-specific embedding representations for each domain according to the query data using the activated domain expert encoders to obtain an embedding representation set; A weighted fusion module, configured to perform weighted fusion on the embedded representations in the embedded representation set according to the activation weight set to obtain a final retrieval representation; The document retrieval module is used to perform document retrieval according to the final retrieval representation to obtain retrieval results.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the dynamic domain expert hybrid retrieval enhancement method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the dynamic domain expert hybrid retrieval enhancement method according to any one of claims 1 to 7 are implemented.

Images