Dynamic weight mixed domain retrieval method, system and equipment based on large model and medium
By supervising and fine-tuning the large language model and dynamically assigning weights, the limitations of fixed weight strategies in mixed-domain retrieval are overcome, achieving accurate adaptation of mixed-domain retrieval results and improving the accuracy and adaptability of retrieval.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-12
- Publication Date
- 2026-04-07
AI Technical Summary
Existing large language models employ a fixed-weight fusion strategy in mixed-domain retrieval, which cannot dynamically adjust the weight ratio of different knowledge sources according to the query intent. This results in limited relevance and accuracy of search results, especially in query scenarios involving professional domain knowledge and general knowledge.
By constructing a training dataset and introducing learnable domain weight parameters, the base large language model is supervised and fine-tuned to dynamically generate weights for private and public content. Combined with semantic similarity scores, dynamic threshold filtering is performed to achieve accurate adaptation of mixed domain retrieval results.
It significantly improves the accuracy and adaptability of mixed-domain retrieval, and can automatically optimize retrieval strategies based on query intent, thereby improving the quality and reliability of retrieval results.
Smart Images
Figure CN121808028A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of retrieval enhancement generation technology, specifically to a dynamic weighted mixed-domain retrieval method, system, device, and medium based on a large model. Background Technology
[0002] With the rapid development of natural language processing technology, large language models (hereinafter referred to as large models) are increasingly widely used in fields such as intelligent question answering and knowledge retrieval. Large language models have gained powerful language understanding and generation capabilities through training on massive amounts of data, providing important support for the implementation of artificial intelligence technology. However, large language model technology faces two significant challenges: first, the generated content may be "illusory"; second, the model's knowledge is difficult to cope with the needs of real-time or dynamically updated information.
[0003] To alleviate the aforementioned problems in large language models, two common technical approaches are employed to enhance their accuracy: Supervised Fine-Tuning (SFT) and Retrieval-Augmented Generation (RAG). Supervised Fine-Tuning improves the model's performance in specific domains through targeted training, while Retrieval-Augmented Generation enhances the model's knowledge coverage by integrating external knowledge bases with the model.
[0004] However, existing large language model retrieval technologies have significant shortcomings when handling mixed-domain retrieval (such as simultaneously searching private documents and publicly available internet data): data from different sources vary in terms of authority, timeliness, and domain relevance. Therefore, it is necessary to dynamically weigh the weights of each information source according to the query intent. However, traditional retrieval methods typically employ a fixed-weight strategy to fuse search results from different sources, failing to dynamically adjust the weight ratios of different knowledge sources based on the query intent. This fixed-weight fusion method is ill-suited to complex and ever-changing query scenarios, resulting in limitations on the relevance and accuracy of search results. Especially in query scenarios involving both specialized domain knowledge and general knowledge, fixed weight allocation often fails to meet users' requirements for the accuracy of search results. Summary of the Invention
[0005] To address the technical problem that existing hybrid domain retrieval methods based on large language models employ a fixed weight fusion strategy, which cannot dynamically adjust the weight ratio of different knowledge sources according to the query intent, thus limiting the relevance and accuracy of retrieval results, this application provides a dynamic weight hybrid domain retrieval method, system, device, and medium based on a large model. Through a dynamic weight allocation mechanism and weight-based threshold filtering, it achieves accurate adaptation of hybrid domain retrieval results, significantly improving retrieval accuracy and scenario adaptability.
[0006] Firstly, this application provides a dynamic weighted mixed-domain retrieval method based on a large model, comprising the following steps: S1. Construct a training dataset. The training dataset contains multiple data samples. Each data sample includes a user query, at least one piece of private content, and at least one piece of public content. Each data sample is labeled with a correlation tag between the user query and each piece of private and public content. Among them, private domain content is the content in the preset private domain knowledge base, and public content is the content in the public knowledge base; S2. Based on the training dataset, the base large language model is fine-tuned using a supervised fine-tuning method to obtain the fine-tuned base large language model; During the fine-tuning process, learnable domain weight parameters are introduced, enabling the base large language model to adaptively adjust the weights of private and public content based on the input user query. S3. Obtain the target user query and retrieve multiple candidate text segments from the hybrid knowledge base based on the target user query. The hybrid knowledge base includes a private knowledge base and a public knowledge base. S4. Input the target user query and all candidate text segments into the fine-tuned base language model. The fine-tuned base language model dynamically generates the private domain content weight α and the public content weight β, where α∈[0,1] and β∈[0,1]. S5. Calculate the semantic similarity score between each candidate text segment and the target user's query, and then perform dynamic threshold filtering on all candidate text segments, including: For candidate text segments belonging to private domain content, they are adopted only if their semantic similarity score is greater than (1-α); For candidate text segments belonging to public domain content, they are adopted only if their semantic similarity score is greater than (1-β); S6. Output all adopted candidate text segments as the final retrieval results for the target user's query.
[0007] It should be further noted that in step S1, the relevance tags for each piece of private and public content queried by the user include a first tag indicating complete relevance, a second tag indicating partial relevance, and a third tag indicating no relevance.
[0008] It should be further noted that in step S2, the base large language model is a pre-trained language model based on the Transformer architecture, including the Qwen series, ChatGLM series, or LLaMA series models, with model parameter scales ranging from 7B to 100B.
[0009] It should be further noted that step S2, which involves fine-tuning the base large language model using a supervised fine-tuning method, includes the following steps: The user query, private content, and public content of each data sample in the training dataset are concatenated to form the input sequence of that data sample; Input the data sample sequence into the base big language model and output the predicted relevance value between each piece of private or public content and the user query. The loss function value is calculated based on the difference between the predicted relevance value output by the model and the actual relevance tag of the corresponding private or public content. By simultaneously optimizing the parameters of the base language model and the domain weight parameters through the backpropagation algorithm, the base language model learns to adjust the weight allocation according to the query semantics.
[0010] It should be further noted that in step S2, the supervised fine-tuning adopts a cosine annealing learning rate scheduling strategy to dynamically adjust the learning rate during the training process, adopts an early stopping strategy based on the validation set F1 score to prevent overfitting, and uses label smoothing cross-entropy as the loss function, with the label smoothing factor set to 0.1.
[0011] It should be further noted that the hyperparameter settings during the supervised fine-tuning process include: The initial learning rate is set to 2e-5, and a cosine function is used to decay the learning rate. The number of training samples per batch is set to 16, and the gradient accumulation step is set to 4. The maximum number of training rounds is set to 5. Training is terminated early when the F1 score on the validation set no longer improves for 3 consecutive rounds.
[0012] It should be further explained that in step S3, multiple candidate text segments are retrieved from the hybrid knowledge base based on the target user query using semantic vector retrieval. Specifically, this includes: encoding the target user query into a query vector; encoding the text segments in the hybrid knowledge base into text vectors; calculating the cosine similarity between the query vector and each text vector; sorting the text segments from high to low according to the similarity score, and selecting the top K text segments as candidate text segments, where K is a preset positive integer, K∈[10,100].
[0013] It should be further noted that in step S4, the private content weight α and the public content weight β satisfy α + β = 1.
[0014] It should be further noted that in step S5, the semantic similarity score... The calculation formula is:
[0015] in, Indicates semantic similarity score; This represents the query vector obtained based on the target user's query encoding; This represents the candidate text vector obtained based on the encoding of the current candidate text segment; This represents the dot product operation; Indicates the L2 norm of the query vector; This represents the L2 norm of the candidate text vector.
[0016] Secondly, this application provides a dynamic weighted mixed-domain retrieval system based on a large model, used to implement the above-mentioned dynamic weighted mixed-domain retrieval method, including: The training dataset construction module is used to build the training dataset. The training dataset contains multiple data samples. Each data sample includes a user query, at least one piece of private content, and at least one piece of public content. Each data sample is labeled with a correlation tag between the user query and each piece of private and public content. The base large language model fine-tuning module is used to fine-tune the base large language model based on the training dataset using a supervised fine-tuning method, so as to obtain the fine-tuned base large language model; The candidate text fragment retrieval module is used to obtain the target user query and retrieve multiple candidate text fragments from the hybrid knowledge base based on the target user query; The weight generation module is used to input the target user query and all candidate text segments into the fine-tuned base language model. The fine-tuned base language model dynamically generates private content weights and public content weights. The candidate text segment filtering module is used to calculate the semantic similarity score between each candidate text segment and the target user's query, and then perform dynamic threshold filtering on all candidate text segments. The search results output module is used to output all the adopted candidate text segments as the final search results for the target user's query.
[0017] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described dynamic weighted mixed-domain retrieval method.
[0018] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described dynamic weighted mixed-domain retrieval method.
[0019] As can be seen from the above technical solutions, this application has the following advantages: 1. This application introduces learnable domain weight parameters, enabling the model to dynamically adjust the weight distribution of private and public content based on the semantic characteristics of user queries. This effectively overcomes the limitations of traditional fixed-weight fusion methods, allowing the system to automatically optimize retrieval strategies for different types of queries, and significantly improving the accuracy and adaptability of hybrid domain retrieval.
[0020] 2. This application achieves refined filtering of search results from different sources by setting a weight-based dynamic threshold filtering mechanism, which applies different filtering thresholds to private and public content respectively. It can automatically adjust the filtering criteria according to the weight allocation, effectively improving the quality of search results and reducing the impact of low-relevance content on the final results.
[0021] 3. This application combines weight learning and relevance judgment through end-to-end training, enabling the model to optimize both weight parameters and relevance judgment capabilities during fine-tuning. This enhances the system's adaptability to different application scenarios and provides a more reliable retrieval foundation for practical applications such as intelligent customer service and knowledge management. Attached Figure Description
[0022] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a flowchart of a dynamic weighted mixed-domain retrieval method based on a large model in one embodiment of this application.
[0024] Figure 2 This is a schematic block diagram of a dynamic weighted mixed-domain retrieval system based on a large model in one embodiment of this application.
[0025] Figure 3 This is a schematic diagram of the hardware structure of an electronic device in one embodiment of this application. Detailed Implementation
[0026] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0027] The dynamic weighted mixed-domain retrieval method of this application will be described in detail below. Specific details such as particular system structures and techniques are presented for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details.
[0028] In the dynamic weighted mixed-domain retrieval method involved in this application, the term "comprising" indicates the presence of the described feature, whole, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or sets thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.
[0029] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.
[0030] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.
[0031] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.
[0032] The dynamic weighted mixed-domain retrieval method provided in this application embodiment is executed by a computer device, and correspondingly, the dynamic weighted mixed-domain retrieval system based on a large model runs on the computer device.
[0033] Figure 1 This is a flowchart of a dynamic weighted mixed-domain retrieval method based on a large model, according to an embodiment of this application. Figure 1 The executing entity can be a dynamic weighted mixed-domain retrieval system. Depending on different requirements, the order of the steps in this flowchart can be changed, and some can be omitted.
[0034] like Figure 1 As shown, this dynamic weighted mixed-domain retrieval method based on a large model includes: Step S1: Construct a training dataset. The training dataset contains multiple data samples. Each data sample includes a user query, at least one private content item, and at least one public content item. Each data sample is labeled with a correlation tag between the user query and each private and public content item. Private domain content refers to content from a pre-defined private domain knowledge base, while public content refers to content from a public knowledge base.
[0035] By constructing a training dataset that includes user queries, private domain content, and public content, and annotating their relevance, a supervised training signal is provided for the model to distinguish knowledge sources from different domains.
[0036] In some specific embodiments, the relevance tags for each piece of private and public content query include a first tag indicating complete relevance, a second tag indicating partial relevance, and a third tag indicating no relevance.
[0037] By refining the relevance labels into three levels—fully relevant, partially relevant, and irrelevant—more granular supervision signals are provided for model training, enhancing the model's ability to discriminate the degree of text relevance.
[0038] Step S2: Based on the training dataset, the base large language model is fine-tuned using a supervised fine-tuning method to obtain the fine-tuned base large language model. During the fine-tuning process, learnable domain weight parameters are introduced, enabling the base large language model to adaptively adjust the weights of private and public content based on the input user query.
[0039] By supervising and fine-tuning the base large language model based on the training dataset and introducing learnable domain weight parameters, the model gains the ability to adaptively adjust weights according to query semantics, thus realizing dynamic weight allocation in hybrid domain retrieval.
[0040] In some specific embodiments, the base large language model is a pre-trained language model based on the Transformer architecture, including the Qwen series, ChatGLM series, or LLaMA series models, with model parameter sizes ranging from 7B to 100B.
[0041] By defining the base language model as a pre-trained model based on the Transformer architecture and specifying the range of parameter sizes, the model is ensured to have sufficient semantic understanding capabilities and computational efficiency, providing a reliable basic model support for hybrid domain retrieval.
[0042] In some specific embodiments, the steps of fine-tuning the base large language model using a supervised fine-tuning method include: The user query, private content, and public content of each data sample in the training dataset are concatenated to form the input sequence of that data sample; Input the data sample sequence into the base big language model and output the predicted relevance value between each piece of private or public content and the user query. The loss function value is calculated based on the difference between the predicted relevance value output by the model and the actual relevance tag of the corresponding private or public content. By simultaneously optimizing the parameters of the base language model and the domain weight parameters through the backpropagation algorithm, the base language model learns to adjust the weight allocation according to the query semantics.
[0043] By clearly defining the steps of supervised fine-tuning, including input sequence concatenation, correlation prediction output, loss calculation, and parameter optimization, a complete model training process was established, ensuring the effectiveness and stability of domain weight parameter learning.
[0044] In some specific embodiments, supervised fine-tuning uses a cosine annealing learning rate scheduling strategy to dynamically adjust the learning rate during the training process, adopts an early stopping strategy based on the validation set F1 score to prevent overfitting, and uses label smoothing cross-entropy as the loss function, with the label smoothing factor set to 0.1.
[0045] By employing optimization strategies such as cosine annealing learning rate scheduling, early stopping strategy, and label smoothing cross-entropy loss function, the stability and generalization ability of model training were improved, effectively preventing overfitting.
[0046] In some specific embodiments, the hyperparameter settings during the supervised fine-tuning process include: The initial learning rate is set to 2e-5, and a cosine function is used to decay the learning rate. The number of training samples per batch is set to 16, and the gradient accumulation step is set to 4. The maximum number of training rounds is set to 5. Training is terminated early when the F1 score on the validation set no longer improves for 3 consecutive rounds.
[0047] By setting specific hyperparameter configurations such as learning rate, batch size, and number of training rounds, clear parameter setting guidance is provided for model training, ensuring the repeatability of the training process and the consistency of the results.
[0048] Step S3: Obtain the target user query and retrieve multiple candidate text segments from the hybrid knowledge base based on the target user query. The hybrid knowledge base includes a private knowledge base and a public knowledge base.
[0049] By acquiring target user queries and retrieving candidate text segments from a hybrid knowledge base, sufficient candidate materials are provided for refined screening, ensuring the comprehensiveness and completeness of the retrieval process.
[0050] In some specific embodiments, multiple candidate text segments are retrieved from the hybrid knowledge base based on the target user query using semantic vector retrieval. Specifically, this includes: encoding the target user query into a query vector; encoding the text segments in the hybrid knowledge base into text vectors; calculating the cosine similarity between the query vector and each text vector; sorting the text segments from high to low according to their similarity scores, and selecting the top K text segments as candidate text segments, where K is a preset positive integer, K∈[10,100].
[0051] By employing semantic vector retrieval and limiting the range of retrieval quantities, the accuracy and efficiency of candidate text segment retrieval are ensured, providing high-quality input for refined filtering.
[0052] Step S4: Input the target user query and all candidate text segments into the fine-tuned base language model. The fine-tuned base language model dynamically generates private content weight α and public content weight β, where α∈[0,1] and β∈[0,1].
[0053] By inputting target user queries and candidate text segments into a finely tuned model and dynamically generating weights and semantic similarity scores, intelligent analysis and evaluation of retrieved content are achieved, providing a quantitative basis for accurate screening.
[0054] In some specific embodiments, the private content weight α and the public content weight β satisfy α + β = 1.
[0055] By constraining the weights α of private content and β of public content to sum to 1, the normalization and rationality of weight allocation are ensured, giving the weight allocation of different knowledge sources a clear mathematical meaning.
[0056] Step S5: Calculate the semantic similarity score between each candidate text segment and the target user's query, and then perform dynamic threshold filtering on all candidate text segments, including: For candidate text segments belonging to private domain content, they are adopted only if their semantic similarity score is greater than (1-α); For candidate text segments belonging to public domain content, they are adopted only if their semantic similarity score is greater than (1-β).
[0057] In some specific embodiments, in step S5, the semantic similarity score is... The calculation formula is:
[0058] in, Indicates semantic similarity score; This represents the query vector obtained based on the target user's query encoding; This represents the candidate text vector obtained based on the encoding of the current candidate text segment; This represents the dot product operation; Indicates the L2 norm of the query vector; This represents the L2 norm of the candidate text vector.
[0059] By adopting the cosine similarity calculation formula and clarifying the meaning of each parameter, an accurate and reliable semantic similarity quantification method is provided, laying a solid computational foundation for dynamic threshold screening.
[0060] Step S6: Output all the adopted candidate text segments as the final retrieval results of the target user's query.
[0061] By performing weight-based dynamic threshold filtering on candidate text segments and outputting the adopted candidate text segments, adaptive retrieval based on adjusting the filtering criteria according to the importance of different knowledge sources is achieved, thereby improving the accuracy and reliability of hybrid domain retrieval.
[0062] The following are embodiments of the dynamic weighted mixed-domain retrieval system based on a large model provided in this application. This dynamic weighted mixed-domain retrieval system based on a large model belongs to the same inventive concept as the dynamic weighted mixed-domain retrieval methods in the above embodiments. For details not described in detail in the embodiments of the dynamic weighted mixed-domain retrieval system, please refer to the embodiments of the dynamic weighted mixed-domain retrieval methods based on a large model described above.
[0063] like Figure 2 As shown, the dynamic weighted mixed-domain retrieval system based on a large model includes: The training dataset construction module is used to build the training dataset. The training dataset contains multiple data samples. Each data sample includes a user query, at least one piece of private content, and at least one piece of public content. Each data sample is labeled with a correlation tag between the user query and each piece of private and public content. The base large language model fine-tuning module is used to fine-tune the base large language model based on the training dataset using a supervised fine-tuning method, so as to obtain the fine-tuned base large language model; The candidate text fragment retrieval module is used to obtain the target user query and retrieve multiple candidate text fragments from the hybrid knowledge base based on the target user query; The weight generation module is used to input the target user query and all candidate text segments into the fine-tuned base language model. The fine-tuned base language model dynamically generates private content weights and public content weights. The candidate text segment filtering module is used to calculate the semantic similarity score between each candidate text segment and the target user's query, and then perform dynamic threshold filtering on all candidate text segments. The search results output module is used to output all the adopted candidate text segments as the final search results for the target user's query.
[0064] The dynamic weighted mixed-domain retrieval system in this embodiment is used to implement a dynamic weighted mixed-domain retrieval method based on a large model.
[0065] This application also provides an electronic device for implementing the various embodiments of this application. Figure 3 To illustrate the hardware structure of an electronic device according to various embodiments of this application, as shown in the following diagram... Figure 3 As shown, the electronic device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor.
[0066] Those skilled in the art will understand that the electronic device structure involved in the embodiments of this application does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0067] In embodiments of this application, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.
[0068] In this application embodiment, the processor can be implemented using at least one of an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a processor, a controller, a microcontroller, a microprocessor, or an electronic unit designed to perform the functions described herein. In some cases, such implementations can be implemented within a controller. For software implementations, implementations such as processes or functions can be implemented with separate software modules that allow the performance of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, and the software code can be stored in memory and executed by the controller.
[0069] In addition, the electronic device includes some functional modules not shown, which will not be described in detail here.
[0070] Those skilled in the art will understand that the various aspects of the electronic device provided in this application can be implemented as a system, method, or program product. Therefore, the various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."
[0071] This application also provides a storage medium storing a program product capable of implementing a dynamic weighted mixed-domain retrieval method based on a large model. In some possible implementations, various aspects of this application can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this application.
[0072] The storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0073] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A dynamic weighted mixed-domain retrieval method based on a large model, characterized in that, include: S1. Construct a training dataset. The training dataset contains multiple data samples. Each data sample includes a user query, at least one piece of private content, and at least one piece of public content. Each data sample is labeled with a correlation tag between the user query and each piece of private and public content. Among them, private domain content is the content in the preset private domain knowledge base, and public content is the content in the public knowledge base; S2. Based on the training dataset, the base large language model is fine-tuned using a supervised fine-tuning method to obtain the fine-tuned base large language model; During the fine-tuning process, learnable domain weight parameters are introduced, enabling the base large language model to adaptively adjust the weights of private and public content based on the input user query. S3. Obtain the target user query and retrieve multiple candidate text segments from the hybrid knowledge base based on the target user query. The hybrid knowledge base includes a private knowledge base and a public knowledge base. S4. Input the target user query and all candidate text segments into the fine-tuned base language model. The fine-tuned base language model dynamically generates the private domain content weight α and the public content weight β, where α∈[0,1] and β∈[0,1]. S5. Calculate the semantic similarity score between each candidate text segment and the target user's query, and then perform dynamic threshold filtering on all candidate text segments, including: For candidate text segments belonging to private domain content, they are adopted only if their semantic similarity score is greater than (1-α); For candidate text segments belonging to public domain content, they are adopted only if their semantic similarity score is greater than (1-β); S6. Output all adopted candidate text segments as the final retrieval results for the target user's query.
2. The dynamic weighted mixed-domain retrieval method as described in claim 1, characterized in that, In step S2, the base large language model is a pre-trained language model based on the Transformer architecture, including the Qwen series, ChatGLM series, or LLaMA series models, with model parameter scales ranging from 7B to 100B.
3. The dynamic weighted mixed-domain retrieval method as described in claim 1, characterized in that, In step S2, the steps of fine-tuning the base large language model using supervised fine-tuning methods include: The user query, private content, and public content of each data sample in the training dataset are concatenated to form the input sequence of that data sample; Input the data sample sequence into the base big language model and output the predicted relevance value between each piece of private or public content and the user query. The loss function value is calculated based on the difference between the predicted relevance value output by the model and the actual relevance tag of the corresponding private or public content. By simultaneously optimizing the parameters of the base language model and the domain weight parameters through the backpropagation algorithm, the base language model learns to adjust the weight allocation according to the query semantics.
4. The dynamic weighted mixed-domain retrieval method as described in claim 1, characterized in that, In step S2, supervised fine-tuning uses a cosine annealing learning rate scheduling strategy to dynamically adjust the learning rate during the training process, adopts an early stopping strategy based on the validation set F1 score to prevent overfitting, and uses label smoothing cross-entropy as the loss function, with the label smoothing factor set to 0.
1.
5. The dynamic weighted mixed-domain retrieval method as described in claim 1, characterized in that, In step S3, multiple candidate text segments are retrieved from the hybrid knowledge base based on the target user query using semantic vector retrieval. Specifically, this includes: encoding the target user query into a query vector; encoding the text segments in the hybrid knowledge base into text vectors; calculating the cosine similarity between the query vector and each text vector; sorting the text segments from high to low according to their similarity scores, and selecting the top K text segments as candidate text segments, where K is a preset positive integer, K∈[10,100].
6. The dynamic weighted mixed-domain retrieval method as described in claim 1, characterized in that, In step S4, the private content weight α and the public content weight β satisfy α + β = 1.
7. The dynamic weighted mixed-domain retrieval method as described in claim 1, characterized in that, In step S5, semantic similarity score The calculation formula is: in, Indicates semantic similarity score; This represents the query vector obtained based on the target user's query encoding; This represents the candidate text vector obtained based on the encoding of the current candidate text segment; This represents the dot product operation; Indicates the L2 norm of the query vector; This represents the L2 norm of the candidate text vector.
8. A dynamic weighted mixed-domain retrieval system based on a large model, characterized in that, To implement the dynamic weighted mixed-domain retrieval method as described in any one of claims 1-7, the method includes: The training dataset construction module is used to build the training dataset. The training dataset contains multiple data samples. Each data sample includes a user query, at least one piece of private content, and at least one piece of public content. Each data sample is labeled with a correlation tag between the user query and each piece of private and public content. The base large language model fine-tuning module is used to fine-tune the base large language model based on the training dataset using a supervised fine-tuning method, so as to obtain the fine-tuned base large language model. The candidate text fragment retrieval module is used to obtain the target user query and retrieve multiple candidate text fragments from the hybrid knowledge base based on the target user query; The weight generation module is used to input the target user query and all candidate text segments into the fine-tuned base language model. The fine-tuned base language model dynamically generates private content weights and public content weights. The candidate text segment filtering module is used to calculate the semantic similarity score between each candidate text segment and the target user's query, and then perform dynamic threshold filtering on all candidate text segments. The search results output module is used to output all the adopted candidate text segments as the final search results for the target user's query.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When a processor executes a computer program, it implements the steps of the dynamic weighted mixed-domain retrieval method as described in any one of claims 1-7.
10. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the dynamic weighted mixed-domain retrieval method as described in any one of claims 1-7.