Intelligent question answering method, device and readable storage medium based on lightweight fine-tuning of domain knowledge and cross-domain dynamic knowledge base

Abstract

The present invention proposes an intelligent question-answering method, device, and readable storage medium based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base. The method includes constructing a cross-domain dynamic knowledge base, adopting a hierarchical block strategy to process multi-domain heterogeneous data, and achieving dynamic expansion and incremental updates through hierarchical vector indexing (bottom-level knowledge block embedding + upper-level domain association matrix); designing a two-stage lightweight fine-tuning strategy, using a high-rank adapter to roughly align the domain semantic space in the pre-access phase, and switching to a low-rank adapter to optimize retrieval results in the post-access phase, significantly reducing computing resource consumption; and based on a domain relevance reranking model, integrating original similarity, term coverage, and historical matching to optimize retrieval results and enhance cross-domain knowledge relevance. Through the closed-loop collaboration of the knowledge base, fine-tuning model, and reranking, the present invention addresses the problems of poor domain migration capabilities, lagging knowledge updates, and high customization costs in traditional systems.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing (NLP) and machine learning technology, and specifically to cross-domain dynamic knowledge base management, lightweight fine-tuning of domain knowledge, large model optimization, and the construction of intelligent question-answering systems. Background Art

[0002] Traditional question-answering systems rely on rule templates or shallow semantic matching technology, making them difficult to handle complex cross-domain problems (such as those across multiple fields such as law, medicine, and finance). While existing large-model-based question-answering methods have generalization capabilities, they suffer from the following key flaws:

[0003] 1. Poor domain migration capabilities: A separate knowledge base must be built for each domain, which lacks dynamic scalability and requires repeated development for cross-domain scenarios.

[0004] 2. Knowledge update lag: Large models have fixed parameters and cannot integrate dynamic domain knowledge (such as emerging concepts or regulatory updates) in real time, resulting in inaccurate responses.

[0005] 3. High customization costs: Vertical domain adaptation requires full retraining of models and knowledge bases, resulting in exponentially increasing resource consumption.

[0006] 4. Insufficient cross-disciplinary collaboration: The retrieval mechanism has not established a multi-disciplinary network, resulting in knowledge blind spots or logical gaps when solving complex problems;

[0007] 5. Lack of dynamic perception: Unable to automatically identify changes in domain boundaries and lack the ability to adaptively adjust the strength of cross-domain knowledge correlation.

[0008] These problems significantly restrict the response accuracy and efficiency of cross-domain question-answering systems, and a resource-efficient and dynamically adaptive solution is urgently needed. Summary of the Invention

[0009] The embodiments of the present invention provide an intelligent question-answering method, device and readable storage medium based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base. They address the problems existing in current technologies in cross-domain professional knowledge question-answering, such as poor domain migration capabilities, delayed knowledge updating, high customization costs, insufficient cross-domain collaboration and lack of dynamic perception, which lead to low accuracy and efficiency in question-answering.

[0010] The core technology of this invention is to achieve efficient and accurate cross-domain intelligent question answering through the collaborative optimization of a cross-domain dynamic knowledge base, a two-stage lightweight fine-tuning strategy and a domain relevance reranking model.

[0011] In a first aspect, the present invention provides an intelligent question-answering method based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base, the method comprising the following steps:

[0012] S1. Build a cross-domain dynamic knowledge base, including:

[0013] S11. Collect heterogeneous data from multiple fields, segment structured data according to field rules, and divide unstructured data into blocks according to semantic paragraphs to form unified knowledge units;

[0014] S12. Generate vector representations of knowledge blocks through the embedding model and construct a hierarchical vector index, where the bottom layer stores the original knowledge block embeddings and the upper layer maintains a dynamically updated domain association matrix;

[0015] S2. Perform two-stage lightweight fine-tuning on the pre-trained large model, including:

[0016] S21, pre-access fine-tuning stage: before accessing the knowledge base, a high-rank adapter is used to align the domain semantic space;

[0017] S22, post-access fine-tuning phase: After the knowledge base is accessed, the low-rank adapter is switched to optimize the semantic consistency between the retrieval results and the model output;

[0018] S3. Generate answers based on domain relevance re-ranking, including:

[0019] S31. Search the knowledge base and obtain candidate knowledge blocks based on the user input question;

[0020] S32. Rerank the candidate knowledge blocks by using a scoring function that integrates the original similarity, domain-specific term coverage, and historical matching;

[0021] S33. Concatenate the reordered knowledge blocks and questions into enhanced input, and use the fine-tuned large model to generate answers.

[0022] Furthermore, in step S12, the hierarchical vector index is constructed in the following way:

[0023] The underlying vector index stores knowledge blocks embedded in the database, and uses a vector database to achieve efficient retrieval;

[0024] The upper-level domain association matrix dynamically maintains the strength of knowledge associations between domains and supports cascading adjustments during incremental updates.

[0025] Furthermore, in step S2, two-stage lightweight fine-tuning is achieved by:

[0026] The high-rank adapter rank value in the pre-access fine-tuning stage is 5%-10% of the model parameters and is used for coarse alignment in the domain semantic space;

[0027] The low-rank adapter rank value in the post-access fine-tuning stage is 1%-2% of the model parameters, which is used for fine-grained alignment of retrieval results with model outputs.

[0028] Furthermore, in step S3, the scoring function formula for domain relevance reranking is:

[0029]

[0030] in, Represents the fused scoring function; represents a candidate document; q represents a question; represents the concatenated input; α, β, and γ are preset weight coefficients, and α+β+γ=1; Represents the original semantic similarity between question q and knowledge block d; represents the number of domain-specific terms covered by knowledge block d; Represents the matching frequency of knowledge block d in historical answers.

[0031] Furthermore, α, β, and γ are 0.45, 0.4, and 0.15, respectively.

[0032] Furthermore, in step S1, the cross-domain dynamic knowledge base supports the following update mechanisms:

[0033] When a new knowledge block is added, the underlying vector index is automatically updated and the cascade adjustment of the domain association matrix is triggered;

[0034] Based on the changes in domain boundaries, the association strength between knowledge blocks in the domain association matrix is dynamically adjusted.

[0035] In a second aspect, the present invention provides an intelligent question-answering device based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base, comprising:

[0036] The knowledge base construction module is used to build a cross-domain dynamic knowledge base, including:

[0037] Collect heterogeneous data from multiple fields, segment structured data according to field rules, and divide unstructured data into blocks according to semantic paragraphs to form unified knowledge units;

[0038] Generate knowledge block vector representations through the embedding model and construct a hierarchical vector index, where the bottom layer stores the original knowledge block embeddings and the upper layer maintains a dynamically updated domain association matrix;

[0039] The domain fine-tuning module is used to perform two-stage lightweight fine-tuning on pre-trained large models, including:

[0040] Pre-access fine-tuning phase: Before accessing the knowledge base, a high-rank adapter is used to align the domain semantic space;

[0041] Post-access fine-tuning phase: After the knowledge base is accessed, the low-rank adapter is switched to optimize the semantic consistency between the retrieval results and the model output;

[0042] The retrieval enhancement module is used to generate answers based on domain relevance re-ranking, including:

[0043] According to the user input question, search the knowledge base and obtain candidate knowledge blocks;

[0044] Re-rank candidate knowledge blocks through a scoring function that combines original similarity, domain-specific term coverage, and historical matching;

[0045] The question-answer generation module is used to concatenate the reordered knowledge blocks and questions into enhanced input, and then use the fine-tuned large model to generate answers.

[0046] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned intelligent question-answering method based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base.

[0047] In a fourth aspect, the present invention provides a readable storage medium, which stores a computer program. The computer program includes a program code for controlling a process to execute a process. The process includes the above-mentioned intelligent question-answering method based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base.

[0048] The main contributions and innovations of the present invention are as follows:

[0049] 1. Dynamic cross-domain adaptation: Through a hierarchical block strategy and layered vector indexing, it supports the dynamic expansion and real-time updating of multi-domain knowledge, solving the problems of "poor domain migration capabilities" and "lagging knowledge updates." Experiments show that indicators such as Rouge-L-Recall improve by 15% to 24%.

[0050] 2. Low-cost and efficient fine-tuning: A two-stage lightweight fine-tuning strategy (pre-connection coarse alignment + post-connection fine-tuning) optimizes only a small number of adapter parameters, reducing computational costs by over 80% compared to traditional full-scale training. It also enhances semantic alignment accuracy through the use of a domain-specific vocabulary.

[0051] 3. Precision retrieval and collaborative optimization: The re-ranking model integrates semantic similarity, domain term coverage, and historical matching to screen highly relevant knowledge blocks. Combined with the large model, it generates enhanced input, significantly improving the logical coherence and accuracy of answers to complex questions.

[0052] 4. Systematic synergy: The dynamic knowledge base, lightweight fine-tuning, and re-ranking model form a closed loop. Ablation experiments show that the synergy of these modules improves Rouge-L-Recall by over 24%, verifying the irreplaceable nature of this technology combination.

[0053] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0055] Figure 1 is a flow chart of an intelligent question-answering method based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base according to an embodiment of the present invention;

[0056] Figure 2 is a detailed flow chart of step S2 of the intelligent question-answering method according to an embodiment of the present invention;

[0057] Figure 3 is a detailed flow chart of step S3 of the intelligent question-answering method according to an embodiment of the present invention;

[0058] Figure 4 is a system architecture diagram of a device according to an embodiment of the present invention;

[0059] Figure 5 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0061] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0062] Existing technologies in cross-domain professional knowledge question and answering have comprehensive defects such as poor domain adaptability, delayed knowledge updating, high computing cost, insufficient cross-domain collaboration and lack of dynamic perception mechanism.

[0063] Based on this, the present invention solves the problems existing in the prior art based on the collaborative optimization of a cross-domain dynamic knowledge base, a two-stage lightweight fine-tuning strategy and a domain relevance reranking model.

[0064] Example 1

[0065] The present invention aims to propose an intelligent question answering method based on lightweight fine-tuning of domain knowledge and cross-domain dynamic knowledge base. Figure 1 , the method comprises the following steps:

[0066] Step S1: Build a cross-domain dynamic knowledge base by collecting knowledge data from multiple fields and generate structured knowledge representation through embedding model;

[0067] In this embodiment, step S1 specifically includes:

[0068] Step 11: In view of the heterogeneous data characteristics in multiple fields, a hierarchical segmentation strategy is adopted: structured data such as legal provisions are segmented by clauses, and unstructured case data are segmented by semantic paragraphs to form knowledge units for unified processing;

[0069] For example, structured data (such as legal texts): divided by clauses to form independent knowledge units (such as Article 100 of the Civil Code).

[0070] Unstructured data (such as medical cases): Divide the data into chunks using a semantic paragraph segmentation tool (such as spaCy) to ensure that each chunk contains complete semantics.

[0071] Step 12: Use the text segmentation tool to divide the data into blocks, use the Transformer model (such as bge-large-zh-v1.5) to vectorize the entities and relationships in the knowledge base, and generate a knowledge embedding matrix , where d is the embedding dimension and n is the number of entities (e.g. 768);

[0072] Step 13: Construct an incrementally updateable hierarchical vector index. The bottom layer stores the original knowledge block embeddings, and the upper layer maintains the domain association matrix to support cascading adjustments during dynamic updates.

[0073] For example, a vector database (such as FAISS) is used to store the embedding of original knowledge blocks to support efficient similarity retrieval. The upper-level maintenance domain association matrix can be:

[0074]

[0075] Among them, m is the number of fields, and the strength of knowledge association between fields is dynamically maintained. For example, the co-occurrence frequency of knowledge blocks related to "medical malpractice liability" in the medical and legal fields is adjusted. The value of .

[0076] Step S2: Use domain knowledge lightweight fine-tuning technology to fine-tune the pre-trained large model, adapting the semantic features of the knowledge base through a low-rank matrix and combining domain-customized prompt words;

[0077] Step S3: Rerank the knowledge base search results based on the domain relevance reranking model, and perform search enhancement generation on the user input question in combination with the fine-tuned large model to output accurate answers;

[0078] Reference Figure 2 Step S2 shown uses the domain knowledge lightweight fine-tuning technology to fine-tune the pre-trained large model, and adapts the semantic features of the knowledge base through the low-rank matrix and combined with the domain customized prompt words, which further includes the following steps:

[0079] Step S21: Build a domain-customized prompt vocabulary to provide more accurate domain information for the domain knowledge lightweight fine-tuning adapter.

[0080] Step S22: Design a two-stage domain knowledge lightweight fine-tuning strategy:

[0081] Pre-access fine-tuning phase: Before accessing the knowledge base, a higher-rank adapter is used to perform rough alignment of the domain semantic space.

[0082] Post-access fine-tuning phase: After the knowledge base is accessed, a lower-rank adapter is switched to fine-tune the retrieval results;

[0083] Step S23: Optimize the domain knowledge lightweight fine-tuning adapter parameters through gradient descent, adopt a dual-objective optimization strategy, and jointly optimize the knowledge retrieval accuracy and question-answer generation loss to align the large model output with the knowledge base embedding semantics.

[0084] See also Figure 3 The present invention provides a detailed flow chart of step S3 of an efficient question-answering system based on lightweight fine-tuning of domain knowledge, knowledge base embedding, and large model fine-tuning, which further includes the following steps:

[0085] Step S31: The first search uses standard similarity calculation, and subsequent searches introduce the domain distribution characteristics of the previous re-ranking results as the historical answer matching degree;

[0086] Step S32: Design a domain-aware re-ranking model to perform domain-specific Q&A re-ranking based on the retrieved information. The re-ranking model's scoring function integrates the following: original semantic similarity + domain-specific term coverage + historical Q&A matching. The retrieved information is sorted in descending order by score. The specific formula is as follows:

[0087]

[0088] in, Represents the fused scoring function; represents a candidate document (paragraph); q represents a question; Represents the concatenated input 1; Represents the original semantic similarity between question q and knowledge block d; represents the number of domain-specific terms covered by knowledge block d; Indicates the matching frequency of knowledge block d in historical answers; 0.45, 0.4 and 0.15 are preset weight coefficients, which can be adjusted according to the field.

[0089] Step S33: The question and domain-specific prompts are combined into an enhanced input. This input is fed into the fine-tuned macro model to generate an answer, and then integrated with a macro model framework (such as Langchain) to enable web page interaction. For example, legal domain prompts include "According to Article X of the Criminal Law" and "Refer to Supreme People's Court Guiding Case #X" to improve domain adaptability.

[0090] In order to verify the technical effect of the present invention, the effectiveness of the present invention will be verified by algorithm comparison:

[0091] The legal question-and-answer dataset used in the embodiments of the present invention is data collected from the real world. These data consist of legal consultations and corresponding legal answers. The experiment uses Rouge-L, METEOR, and BLEU as experimental indicators, and uses the ChatLaw-13B large model as the large language model in the embodiments of the present invention, and compares it with LLaMA-13B, Qianwen 2.5-14B (Qwen-2.5B), and ChatGLM3-6B. LLaMA-13B is a large model launched by the American company Meta, Qwen-2.5B is a large model launched by Alibaba Group, ChatGLM3-6B is a large model launched by Zhipu AI, and ChatLaw-13B is a large model launched by the Peking University team based on Jiang Ziya Ziya Ziya-LLaMA-13B-v1 training, and the Ziya-LLaMA-13B-v1 large model is a large model pre-trained based on LLaMA-13B. The comparison results of the present invention are shown in Table 1:

[0092] Table 1

[0093]

[0094] By comparison, it was found that the method provided by the present invention has significant improvements in indicators such as Rouge-L, METEOR, and BLEU compared to the other three models that did not use the present invention. The reason for the good performance of the present invention is that the present invention constructs a cross-domain dynamic knowledge base, vectorizes domain text, structured data, and entity relationships, and stores them in an efficient vector database, supporting similarity retrieval and dynamic updates. At the same time, the domain knowledge lightweight fine-tuning technology is used to perform low-rank matrix adaptation and fine-tuning on the pre-trained large model, significantly reducing computational costs and achieving semantic alignment of the knowledge base. In addition, the domain relevance reranking model is used to optimize the reranking of knowledge base retrieval results, screen highly relevant knowledge blocks and splice them with user questions as enhanced input, and finally the fine-tuned large model generates accurate answers, and the results are verified by knowledge base entity linking. This method not only improves efficiency, but also ensures the accuracy and relevance of the answers. Therefore, the present invention shows obvious advantages in indicators such as Rouge-L, METEOR, and BLEU, and has excellent performance. In addition, to verify that the various methods of the present invention play an important role in the present invention, corresponding ablation experiments were also carried out, and the experimental results are shown in Table 2:

[0095] Table 2

[0096]

[0097] Through comparative analysis, it can be seen that the method provided by the present invention significantly improves the overall performance of the cross-domain knowledge question-answering system, especially under the synergistic effect of multiple technical modules. Among them, the cross-domain dynamic knowledge base has the most significant improvement on the system effect. It not only supports efficient similarity retrieval and dynamic update, but also enhances the system's adaptability to data from different fields. Although the improvement effect of the domain relevance reordering module is relatively weak, it still plays an important role in screening highly relevant knowledge blocks and further optimizes the quality of the retrieval results. In general, the various modules of the present invention show good synergistic effects, and when they work together, they can maximize the system performance. However, even if some modules of the present invention are used alone to implement specific functions, their core value can be reflected and do not deviate from the basic principles of the present invention.

[0098] Example 2

[0099] like Figure 4 As shown, based on the same concept, the present invention also proposes an intelligent question-answering device based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base, including:

[0100] Knowledge base construction module: used to collect, clean and store domain-related dynamic data, generate structured knowledge representation, and support efficient similarity retrieval and dynamic updates through vector databases;

[0101] Domain fine-tuning module: This module optimizes parameters by inserting domain knowledge into the attention layer of the pre-trained large model and fine-tuning the low-rank adapter to align the model output with the semantic features of the knowledge base.

[0102] Retrieval enhancement module: Based on the user input question, it retrieves the top-K relevant knowledge embeddings from the knowledge base and selects the top-M highly relevant knowledge blocks through the domain relevance reranking model;

[0103] Question and answer generation module: The filtered knowledge blocks are combined with user questions as enhanced input, the fine-tuned large model is used to generate accurate answers, and the final results are provided in combination with web page interactive functions.

[0104] For ease of understanding, the following is a supplementary explanation of professional terms that are not explained in detail in the present invention, combined with general definitions in the technical field and specific application scenarios of the present invention:

[0105] 1. Lightweight Fine-tuning

[0106] definition:

[0107] It refers to the technology of achieving domain adaptation by optimizing a small number of model parameters. The core idea is to insert a trainable low-rank adapter (such as LoRA) based on the pre-trained large model, freeze most of the original model parameters, and only update the adapter weights, thereby greatly reducing the computational cost.

[0108] Application of the present invention:

[0109] A two-stage strategy is adopted:

[0110] Pre-access stage: Use a higher rank adapter (e.g., rank = 64, with an optional rank value of 5%-10% of the model parameters) to quickly align the domain semantic space;

[0111] Post-access stage: Switch to a lower-rank adapter (such as rank = 16, the optional rank value is 1%-2% of the model parameters) to fine-tune the knowledge base retrieval results to achieve efficient adaptation of "rough alignment-fine optimization".

[0112] 2. Low-Rank Matrix

[0113] definition:

[0114] In matrix decomposition, a low-rank matrix is used to approximate a high-dimensional parameter matrix, reducing the number of trainable parameters. For example, LoRA technology significantly reduces the amount of fine-tuning computation by decomposing a large model weight matrix into the product of two low-rank matrices (W=W0+AB), where A and B are low-rank matrices.

[0115] Application of this invention: In the domain fine-tuning module, by inserting low-rank adapters (such as matrices A and B), only about 0.1% to 1% of the model parameters are optimized to achieve performance close to that of full fine-tuning, solving the high cost problem of traditional full retraining.

[0116] 3. Vector Database

[0117] definition:

[0118] A database dedicated to storing and retrieving vector data. It supports efficient high-dimensional vector similarity retrieval (such as cosine similarity and inner product), and is often used in vectorized representation retrieval scenarios in natural language processing.

[0119] Application of the present invention:

[0120] FAISS is used as the vector database to store the knowledge embedding matrix of the cross-domain dynamic knowledge base, supporting the retrieval of Top-K similar knowledge blocks in seconds, and realizing cascading adjustment during dynamic updates through a hierarchical index structure.

[0121] 4. Hierarchical Vector Index

[0122] definition:

[0123] A tree-like index structure, where the bottom layer stores the vector embedding of the original knowledge blocks, and the upper layer maintains the association matrix between domains, achieving efficient retrieval through layer-by-layer filtering.

[0124] Innovations of the present invention:

[0125] Bottom layer: stores the original embeddings of knowledge blocks in various fields (such as vectors of legal terms and medical cases);

[0126] Upper layer: Maintains the domain association matrix, records the semantic association strength of knowledge blocks in different domains, and supports index cascade updates when new domains are dynamically added (for example, when adding financial domain knowledge, only the upper-layer association matrix needs to be updated without reconstructing the underlying index).

[0127] 5. Dual Objective Optimization Strategy

[0128] definition:

[0129] An algorithmic strategy that simultaneously optimizes two independent objective functions aims to balance multi-tasking requirements.

[0130] Application of the present invention:

[0131] The objective function is:

[0132]

[0133] Among them, λ1 and λ2 are weight parameters, and the retrieval and generation tasks are optimized synchronously through gradient descent.

[0134] RetrievalLoss: Knowledge retrieval accuracy loss, ensuring that the model can accurately recall relevant knowledge from the knowledge base;

[0135] GenerationLoss: question-answer generation loss, optimizing the natural language fluency and logical coherence of answers;

[0136] By simultaneously optimizing the two loss terms through gradient descent, the output of the large model is made to conform to the facts of the knowledge base (accurate retrieval) and meet the generation quality requirements.

[0137] 6. Domain-Specific Prompt Library

[0138] definition:

[0139] A collection of prompt words built for vertical fields (such as law and medicine) contains domain-specific terms, question-and-answer templates, and logical guide words, which are used to guide large models to generate answers that comply with domain specifications.

[0140] The construction method of the present invention is:

[0141] Extract high-frequency terms (e.g., "contractual negligence" and "medical history") from legal texts, medical guidelines, and other professional literature;

[0142] Design a domain-specific prompt word format, such as "According to Article XX of the Civil Code, analyze the following case: [Case description]", to enhance the large model's sensitivity to domain semantics.

[0143] 7. Reranking Model

[0144] definition:

[0145] A model that performs secondary ranking on the initial search results, improving the relevance of the results through a more complex scoring function.

[0146] Technical features of the present invention:

[0147] Adopting a domain-aware architecture, it integrates three dimensions of scoring: semantic similarity, domain term coverage, and historical matching.

[0148] Scoring function formula:

[0149]

[0150] The sorting logic of "semantic basis-domain priority-historical assistance" is realized through weighted summation.

[0151] 8. Multi-Head Attention

[0152] definition:

[0153] The core component of the Transformer architecture captures the multi-dimensional semantic correlation of input data through parallel calculations of multiple independent attention heads.

[0154] Application of the present invention:

[0155] In the reranking model, a multi-head attention mechanism is used to model the interaction between user questions q and candidate documents \(c_i\): each attention head focuses on different semantic subspaces (such as legal terminology subspace and temporal relationship subspace); and outputs a fused domain-aware score to improve the parsing accuracy of complex problems (such as cross-domain compound problems).

[0156] Example 3

[0157] This embodiment also provides an electronic device, referring to Figure 5 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0158] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0159] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0160] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0161] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the intelligent question-answering methods based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base in the above embodiments.

[0162] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0163] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0164] The input / output device 408 is used to input or output information.

[0165] Example 4

[0166] This embodiment also provides a readable storage medium, which stores a computer program. The computer program includes program code for controlling a process to execute a process. The process includes an intelligent question-answering method based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base according to embodiment one.

[0167] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0168] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0169] The embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components that are configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, it should be noted at this point that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0170] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0171] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. An intelligent question-answering method based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base, characterized by: The following steps are involved: S1. Build a cross-domain dynamic knowledge base, including: S11. Collect heterogeneous data from multiple fields, segment structured data according to field rules, and divide unstructured data into blocks according to semantic paragraphs to form unified knowledge units; S12. Generate vector representations of knowledge blocks through the embedding model and construct a hierarchical vector index, where the bottom layer stores the original knowledge block embeddings and the upper layer maintains a dynamically updated domain association matrix; S2. Perform two-stage lightweight fine-tuning on the pre-trained large model, including: S21, pre-access fine-tuning stage: before accessing the knowledge base, a high-rank adapter is used to align the domain semantic space; S22, post-access fine-tuning phase: After the knowledge base is accessed, the low-rank adapter is switched to optimize the semantic consistency between the retrieval results and the model output; S3. Generate answers based on domain relevance re-ranking, including: S31. Search the knowledge base and obtain candidate knowledge blocks based on the user input question; S32. Rerank the candidate knowledge blocks by using a scoring function that integrates the original similarity, domain-specific term coverage, and historical matching; S33. Concatenate the reordered knowledge blocks and questions into enhanced input, and use the fine-tuned large model to generate answers.

2. The intelligent question-answering method according to claim 1, wherein: In step S12, the hierarchical vector index is constructed in the following manner: The underlying vector index stores knowledge blocks embedded in the database, and uses a vector database to achieve efficient retrieval; The upper-level domain association matrix dynamically maintains the strength of knowledge associations between domains and supports cascading adjustments during incremental updates.

3. The intelligent question-answering method according to claim 1, wherein: In step S2, the two-stage lightweight fine-tuning is achieved by: The high-rank adapter rank value in the pre-access fine-tuning stage is 5%-10% of the model parameters and is used for coarse alignment in the domain semantic space; The low-rank adapter rank value in the post-access fine-tuning stage is 1%-2% of the model parameters, which is used for fine-grained alignment of retrieval results with model outputs.

4. The intelligent question-answering method according to claim 1, wherein: In step S3, the scoring function formula for the domain relevance reranking is: in, Represents the fused scoring function; represents a candidate document; q represents a question; represents the concatenated input; α, β, and γ are preset weight coefficients, and α+β+γ=1; Represents the original semantic similarity between question q and knowledge block d; represents the number of domain-specific terms covered by knowledge block d; Represents the matching frequency of knowledge block d in historical answers.

5. The intelligent question-answering method according to claim 4, wherein: α, β, and γ are 0.45, 0.4, and 0.15 respectively.

6. The intelligent question-answering method according to any one of claims 1 to 5, wherein: In step S1, the cross-domain dynamic knowledge base supports the following update mechanisms: When a new knowledge block is added, the underlying vector index is automatically updated and the cascade adjustment of the domain association matrix is triggered; Based on the changes in domain boundaries, the association strength between knowledge blocks in the domain association matrix is dynamically adjusted.

7. An intelligent question-answering device based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base, characterized by: include: The knowledge base construction module is used to build a cross-domain dynamic knowledge base, including: Collect heterogeneous data from multiple fields, segment structured data according to field rules, and divide unstructured data into blocks according to semantic paragraphs to form unified knowledge units; Generate knowledge block vector representations through the embedding model and construct a hierarchical vector index, where the bottom layer stores the original knowledge block embeddings and the upper layer maintains a dynamically updated domain association matrix; The domain fine-tuning module is used to perform two-stage lightweight fine-tuning on pre-trained large models, including: Pre-access fine-tuning phase: Before accessing the knowledge base, a high-rank adapter is used to align the domain semantic space; Post-access fine-tuning phase: After the knowledge base is accessed, the low-rank adapter is switched to optimize the semantic consistency between the retrieval results and the model output; The retrieval enhancement module is used to generate answers based on domain relevance re-ranking, including: According to the user input question, search the knowledge base and obtain candidate knowledge blocks; Re-rank candidate knowledge blocks through a scoring function that combines original similarity, domain-specific term coverage, and historical matching; The question-answer generation module is used to concatenate the reordered knowledge blocks and questions into enhanced input, and then use the fine-tuned large model to generate answers.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the intelligent question-answering method based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes an intelligent question-answering method based on lightweight fine-tuning of domain knowledge and a cross-domain dynamic knowledge base according to any one of claims 1 to 6.

Images