Intelligent question answering method and device based on domain knowledge lightweight fine tuning and cross-domain dynamic knowledge base and readable storage medium thereof

Abstract

The invention provides an intelligent question and answer method and device based on domain knowledge lightweight fine tuning and a cross-domain dynamic knowledge base and a readable storage medium thereof.The method comprises the steps that the cross-domain dynamic knowledge base is constructed, and multi-domain heterogeneous data is processed through a grading and blocking strategy; dynamic expansion and incremental updating are realized through hierarchical vector indexing (bottom-layer knowledge block embedding + upper-layer field incidence matrix); a two-stage lightweight fine tuning strategy is designed, a high-rank adapter is used for roughly aligning a field semantic space in a pre-access stage, and a low-rank adapter is switched for optimizing a retrieval result in a post-access stage, so that the consumption of computing resources is remarkably reduced; and on the basis of a domain correlation reordering model, the retrieval result is optimized by fusing the original similarity, the term coverage and the historical matching degree, and the cross-domain knowledge correlation is improved. Through closed-loop cooperation of the knowledge base, the fine tuning model and reordering, the problems of poor migration ability, knowledge updating lagging, high customization cost and the like in the traditional system field are solved.

Description

Technical Field

[0001] The present invention relates to the technical fields of natural language processing (NLP) and machine learning, and specifically relates to cross-domain dynamic knowledge base management, domain knowledge lightweight fine-tuning, large model optimization, and intelligent question-answering system construction. Background Art

[0002] Traditional question-answering systems rely on rule templates or shallow semantic matching techniques and are difficult to handle cross-domain complex questions (such as multi-domain cross-scenarios in law, medicine, finance, etc.). Although existing question-answering methods based on large models have generalization capabilities, they have the following key defects: 1. Poor domain migration ability: It is necessary to build a knowledge base for each domain separately, lacking dynamic scalability, resulting in repeated development for cross-domain scenarios; 2. Lag in knowledge update: The parameters of the large model are fixed and cannot fuse domain dynamic knowledge (such as emerging concepts or regulatory updates) in real time, resulting in inaccurate responses; 3. High customization cost: Vertical domain adaptation requires full-scale retraining of the model and the knowledge base, and resource consumption increases exponentially; 4. Insufficient cross-domain collaboration: The retrieval mechanism does not establish a multi-domain association network, resulting in knowledge blind spots or logical breaks during the analysis of complex questions; 5. Lack of dynamic perception: It cannot automatically identify changes in domain boundaries and lacks the ability to adaptively adjust the knowledge association strength in cross-domains.

[0003] These problems significantly restrict the response accuracy and efficiency of cross-domain question-answering systems, and there is an urgent need for a resource-efficient and dynamically adaptable solution. Summary of the Invention

[0004] Embodiments of the present invention provide an intelligent question-answering method, device, and readable storage medium based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base, aiming at the problems existing in current technologies, such as poor domain migration ability, lag in knowledge update, high customization cost, insufficient cross-domain collaboration, and lack of dynamic perception in cross-domain professional knowledge question-answering, resulting in low question-answering accuracy and efficiency.

[0005] The core technology of the present invention mainly realizes efficient and accurate cross-domain intelligent question-answering through the collaborative optimization of a cross-domain dynamic knowledge base, a two-stage lightweight fine-tuning strategy, and a domain relevance re-ranking model.

[0006] In a first aspect, the present invention provides an intelligent question-answering method based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base, and the method includes the following steps: S1. Build a cross-domain dynamic knowledge base, including: S11. Collect heterogeneous data from multiple domains, split the structured data according to domain rules, and chunk the unstructured data by semantic paragraphs to form unified knowledge units; S12. Generate vector representations of knowledge chunks through an embedding model and construct a hierarchical vector index, where the bottom layer stores the original knowledge chunk embeddings and the upper layer maintains a dynamically updated domain association matrix; S2. Perform two-stage lightweight fine-tuning on the pre-trained large model, including: S21. Pre-connection fine-tuning stage: Before connecting to the knowledge base, use a high-rank adapter to align the domain semantic space; S22. Post-connection fine-tuning stage: After connecting to the knowledge base, switch to a low-rank adapter to optimize the semantic consistency between the retrieval results and the model output; S3. Generate answers based on domain relevance re-ranking, including: S31. According to the user's input question, retrieve the knowledge base and obtain candidate knowledge chunks; S32. Re-rank the candidate knowledge chunks through a scoring function that fuses the original similarity, domain-specific term coverage, and historical matching degree; S33. Concatenate the re-ranked knowledge chunks with the question to form an enhanced input, and input it into the fine-tuned large model to generate an answer.

[0007] Further, in step S12, the hierarchical vector index is constructed as follows: The bottom-layer vector index stores the knowledge chunk embeddings and uses a vector database to achieve efficient retrieval; The upper-layer domain association matrix dynamically maintains the knowledge association strength between domains and supports cascading adjustments during incremental updates.

[0008] Further, in step S2, the two-stage lightweight fine-tuning is implemented as follows: The rank value of the high-rank adapter in the pre-connection fine-tuning stage is 5%-10% of the model parameters and is used for rough alignment of the domain semantic space; The rank value of the low-rank adapter in the post-connection fine-tuning stage is 1%-2% of the model parameters and is used for fine-grained alignment between the retrieval results and the model output.

[0009] Further, in step S3, the formula for the scoring function of domain relevance re-ranking is:

[0010] where, represents the fused scoring function; represents the candidate document; q represents the question; represents the concatenated input; α, β, γ are preset weight coefficients respectively, and satisfy α + β + γ = 1; represents the original semantic similarity between the question q and the knowledge chunk d; Denote the number of domain-specific terms covered by the knowledge chunk d; Denote the matching frequency of the knowledge chunk d in historical answers.

[0011] Furthermore, α, β, and γ are 0.45, 0.4, and 0.15 respectively.

[0012] Furthermore, in step S1, the cross-domain dynamic knowledge base supports the following update mechanisms: When adding a new knowledge chunk, automatically update the underlying vector index and trigger a cascading adjustment of the domain association matrix; Based on the change of the domain boundary, dynamically adjust the association strength between knowledge chunks in the domain association matrix.

[0013] In a second aspect, the present invention provides an intelligent question-answering device based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base, including: A knowledge base construction module for constructing a cross-domain dynamic knowledge base, including: Collect multi-domain heterogeneous data, segment the structured data according to domain rules, and divide the unstructured data into semantic paragraphs to form unified knowledge units; Generate vector representations of knowledge chunks through an embedding model and construct a hierarchical vector index, where the underlying layer stores the original knowledge chunk embeddings and the upper layer maintains a dynamically updated domain association matrix; A domain fine-tuning module for performing two-stage lightweight fine-tuning on a pre-trained large model, including: Pre-access fine-tuning stage: Before the knowledge base is accessed, use a high-rank adapter to align the domain semantic space; Post-access fine-tuning stage: After the knowledge base is accessed, switch to a low-rank adapter to optimize the semantic consistency between the retrieval result and the model output; A retrieval enhancement module for generating an answer based on domain relevance re-ranking, including: According to the user input question, retrieve the knowledge base and obtain candidate knowledge chunks; Re-rank the candidate knowledge chunks through a scoring function that fuses the original similarity, domain-specific term coverage, and historical matching degree; A question-answering generation module for splicing the re-ranked knowledge chunks with the question into an enhanced input and inputting it into the fine-tuned large model to generate an answer.

[0014] In a third aspect, the present invention provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the above intelligent question-answering method based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base.

[0015] Fourthly, the present invention provides a readable storage medium storing a computer program, the computer program including program codes for controlling a process to execute the process, and the process including the intelligent question-answering method based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base as described above.

[0016] The main contributions and innovations of the present invention are as follows: 1. Cross-domain dynamic adaptation: Through the hierarchical and block-based strategy and hierarchical vector index, it supports the dynamic expansion and real-time update of multi-domain knowledge, solves the problems of "poor domain migration ability" and "lagging knowledge update", and experiments show that indicators such as Rouge-L-Recall are improved by 15% - 24%.

[0017] 2. Low-cost and high-efficiency fine-tuning: The two-stage lightweight fine-tuning strategy (pre-access rough alignment + post-access fine-tuning) only optimizes a small number of adapter parameters, reducing the computing cost by more than 80% compared with traditional full-scale training, and enhancing the semantic alignment accuracy through the domain prompt word library.

[0018] 3. Precise retrieval and collaborative optimization: The re-ranking model integrates semantic similarity, domain term coverage, and historical matching degree to screen highly relevant knowledge chunks, and combines the large model generation to enhance the input, significantly improving the logical coherence and answer accuracy of complex questions.

[0019] 4. Systematic collaborative synergy: The dynamic knowledge base, lightweight fine-tuning, and re-ranking model form a closed loop. Ablation experiments prove that the synergistic effect of each module improves Rouge-L-Recall by more than 24%, verifying the irreplaceability of the technology combination.

[0020] Details of one or more embodiments of the present invention are set forth in the following drawings and description to make other features, objects, and advantages of the present invention more concise and understandable. Description of the Drawings

[0021] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present invention, and the schematic embodiments of the present invention and their descriptions are used to explain the present invention, not to limit the present invention improperly. In the drawings: Figure 1 is a flowchart of the intelligent question-answering method based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base according to an embodiment of the present invention; Figure 2 is a detailed flowchart of step S2 of the intelligent question-answering method according to an embodiment of the present invention; Figure 3 is a detailed flowchart of step S3 of the intelligent question-answering method according to an embodiment of the present invention; Figure 4 is a system architecture diagram of the device according to an embodiment of the present invention; Figure 5 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Specific embodiments

[0022] Here, exemplary embodiments will be described in detail, and their examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0023] It should be noted that: In other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or fewer than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0024] The prior art has comprehensive defects in cross - domain professional knowledge Q&A, such as poor domain adaptability, lagging knowledge update, high computing cost, insufficient cross - domain collaboration, and lack of dynamic perception mechanism.

[0025] Based on this, the present invention solves the problems existing in the prior art through the collaborative optimization of a cross - domain dynamic knowledge base, a two - stage lightweight fine - tuning strategy, and a domain - relevance re - ranking model.

[0026] Embodiment 1 The present invention aims to propose an intelligent Q&A method based on domain - knowledge lightweight fine - tuning and a cross - domain dynamic knowledge base. Specifically, referring to Figure 1 , the method includes the following steps: Step S1: Construct a cross - domain dynamic knowledge base by collecting knowledge data from multiple domains, and generate a structured knowledge representation through an embedding model; In this embodiment, the specific content of step S1 includes: Step 11: For the characteristics of multi - domain heterogeneous data, adopt a hierarchical and block - based strategy: segment structured data such as legal provisions by clause, and divide unstructured data such as cases into semantic paragraphs to form knowledge units for unified processing; For example, structured data (such as legal provisions): segmented by clause to form independent knowledge units (such as Article 100 of the Civil Code).

[0027] Unstructured data (such as medical cases): It is divided by a semantic paragraph chunking tool (such as spaCy) to ensure that each chunk contains complete semantics.

[0028] Step 12: Chunk the data through a text segmentation tool, and use a Transformer model (such as bge-large-zh-v1.5) to vectorize the entities and relationships in the knowledge base to generate a knowledge embedding matrix , where d is the embedding dimension and n is the number of entities (such as 768); Step 13: Build a hierarchically vector index that can be incrementally updated. The bottom layer stores the original knowledge chunk embeddings, and the upper layer maintains the domain association matrix to support cascading adjustments during dynamic updates; For example, use a vector database (such as FAISS) to store the original knowledge chunk embeddings to support efficient similarity retrieval. The domain association matrix maintained by the upper layer can be:

[0029] where m is the number of domains, and the knowledge association strength between domains is dynamically maintained. For example, due to the co-occurrence frequency of knowledge chunks related to "medical accident liability" in the medical and legal domains, adjust the value.

[0030] Step S2: Fine-tune the pre-trained large model using the domain knowledge lightweight fine-tuning technology, and adapt to the semantic features of the knowledge base through a low-rank matrix combined with domain-specific prompt words; Step S3: Re-rank the retrieval results of the knowledge base based on the domain relevance model, and perform retrieval-enhanced generation on the user input question in combination with the fine-tuned large model to output accurate answers; Refer to Figure 2 As shown in the steps of S2, use the domain knowledge lightweight fine-tuning technology to fine-tune the pre-trained large model, and adapt to the semantic features of the knowledge base through a low-rank matrix combined with domain-specific prompt words. It further includes the following steps: Step S21: Build a domain-specific prompt word library to provide more accurate domain information for the domain knowledge lightweight fine-tuning adapter.

[0031] Step S22: Design a two-stage domain knowledge lightweight fine-tuning strategy: Pre-connection fine-tuning stage: Before the knowledge base is connected, use a higher-rank adapter to perform a rough alignment of the domain semantic space Post-connection fine-tuning stage: After the knowledge base is connected, switch to a lower-rank adapter for fine-tuning the retrieval results; Step S23: Optimize the parameters of the domain knowledge lightweight fine-tuning adapter through gradient descent, and adopt a dual-objective optimization strategy to jointly optimize the knowledge retrieval accuracy and the question-answering generation loss to align the output of the large model with the semantics of the knowledge base embedding.

[0032] See Figure 3 This is a detailed flowchart of step S3 of the efficient question-answering system provided by the present invention based on lightweight fine-tuning of domain knowledge, knowledge base embedding, and large model fine-tuning, which further includes the following steps: Step S31: The first retrieval uses standard similarity calculation, and the subsequent retrievals introduce the domain distribution characteristics of the previous re-ranking results as the historical answer matching degree; Step S32: Design a domain-aware re-ranking model, and perform domain-questioning exclusive scoring and re-ranking according to the information retrieved in the domain; among them, the scoring function of the re-ranking model is fused with: original semantic similarity + domain-specific term coverage + historical question-answer matching degree, and the retrieved information is sorted in descending order of scores. The specific formula is as follows:

[0033] Among them, represents the fused scoring function; represents the candidate document (paragraph); q represents the question; represents the concatenated input 1; represents the original semantic similarity between the question q and the knowledge block d; represents the number of domain-specific terms covered by the knowledge block d; represents the matching frequency of the knowledge block d in the historical answer; 0.45, 0.4, and 0.15 are preset weight coefficients, which can be adjusted according to the domain.

[0034] Step S33: Concatenate the question with domain-specific prompt words to form an enhanced input, input it into the fine-tuned large model to generate an answer, and combine the large model framework (such as Langchain) to achieve web interaction. For example, legal domain prompt words include "According to Article X of the Criminal Law" and "Refer to the Supreme People's Court Guiding Case #X", etc., to improve domain adaptability.

[0035] In order to verify the technical effects of the present invention, the effectiveness of the present invention will be verified through algorithm comparison: The legal and regulatory question-and-answer dataset used in the embodiments of the present invention is data collected from the real world. These data consist of legal consultations and corresponding legal answers. The experiment uses Rouge-L, METEOR, and BLEU as experimental indicators, and uses the ChatLaw-13B large model as the large language model in the embodiments of the present invention, and compares it with LLaMA-13B, Qianwen 2.5-14B (Qwen-2.5B), and ChatGLM3-6B, where LLaMA-13B is a large model launched by the American company Meta, Qwen-2.5B is a large model launched by Alibaba Group, ChatGLM3-6B is a large model launched by Zhipu AI, and ChatLaw-13B is a large model launched by the Peking University team based on Jiang Ziya Ziya Ziya-LLaMA-13B-v1 training, where the Ziya-LLaMA-13B-v1 large model is a large model pre-trained based on LLaMA-13B. The comparison results of the present invention are shown in Table 1: Table 1

[0036] By comparison, it is found that the method provided by the present invention has obvious improvements in Rouge-L, METEOR, and BLEU compared with the other three models that do not use the present invention. The present invention has good performance because: the present invention constructs a cross-domain dynamic knowledge base, vectorizes domain text, structured data, and entity relationships, and stores them in an efficient vector database, supporting similarity retrieval and dynamic update. At the same time, the domain knowledge lightweight fine-tuning technology is used to perform low-rank matrix adaptation and fine-tuning on the pre-trained large model, which significantly reduces the computational cost and realizes the semantic alignment of the knowledge base. In addition, the domain relevance re-ranking model is used to optimize the re-ranking of the knowledge base retrieval results, screen high-relevance knowledge blocks and splice them with user questions as enhanced input, and finally generate accurate answers from the fine-tuned large model, and complete the result verification through knowledge base entity linking. This method not only improves efficiency, but also ensures the accuracy and relevance of the answers. Therefore, the present invention shows obvious advantages in Rouge-L, METEOR, BLEU and other indicators, and has excellent performance. In addition, in order to verify that the various methods of the present invention play an important role in the present invention, corresponding ablation experiments were also carried out, and the experimental results are shown in Table 2: Table 2

[0037] Through comparative analysis, it can be seen that the method provided by the present invention significantly improves the overall performance of the cross-domain knowledge Q&A system, especially when the cooperation of multiple technical modules is involved, the performance is more prominent. Among them, the cross-domain dynamic knowledge base has the most significant improvement on the system effect. It not only supports efficient similarity retrieval and dynamic update, but also enhances the system's adaptability to data in different domains. Although the improvement effect of the domain relevance re-ranking module is relatively weak, it still plays an important role in screening highly relevant knowledge chunks, further optimizing the quality of the retrieval results. Generally speaking, there is a good synergistic effect among the various modules of the present invention. When they work together, the system performance can be maximized. However, even if some modules of the present invention are used alone to achieve specific functions, their core value can still be reflected, without departing from the basic principles of the present invention.

[0038] Embodiment 2 As Figure 4 shown, based on the same concept, the present invention also proposes an intelligent Q&A device based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base, including: Knowledge base construction module: used to collect, clean and store domain-related dynamic data, generate structured knowledge representations, and support efficient similarity retrieval and dynamic update through a vector database; Domain fine-tuning module: optimize the parameters by inserting a domain knowledge lightweight fine-tuning low-rank adapter into the attention layer of the pre-trained large model, so that the model output is aligned with the semantic features of the knowledge base; Retrieval enhancement module: based on the user input question, retrieve the top-K relevant knowledge embeddings from the knowledge base, and screen the top-M highly relevant knowledge chunks through a domain relevance re-ranking model; Q&A generation module: splice the screened knowledge chunks with the user question to form an enhanced input, input it into the fine-tuned large model to generate accurate answers, and provide the final result in combination with the web interaction function.

[0039] For easy understanding, the following is an explanatory supplement to the professional terms not detailed in the present invention, which is explained in combination with the general definition in the technical field and the specific application scenario of the present invention: 1. Lightweight Fine-tuning Definition: It refers to a technology for achieving domain adaptation by optimizing a small number of model parameters. The core idea is to insert a trainable low-rank adapter (such as LoRA) on the basis of the pre-trained large model, freeze most of the parameters of the original model, and only update the adapter weights, thereby greatly reducing the computational cost.

[0040] Application of the present invention: Adopt a two-stage strategy: Pre-access phase: Use a higher-rank adapter (e.g., rank = 64, and the optional rank value is 5% - 10% of the model parameters) to quickly align the domain semantic space; Post-access phase: Switch to a lower-rank adapter (e.g., rank = 16, and the optional rank value is 1% - 2% of the model parameters) to fine-tune the retrieval results of the knowledge base, achieving efficient adaptation of "coarse alignment - fine optimization".

[0041] 2. Low-Rank Matrix Definition: In matrix factorization, a low-rank matrix is used to approximate the high-dimensional parameter matrix, reducing the number of trainable parameters. For example, the LoRA technique decomposes the large model weight matrix into the product of two low-rank matrices (W = W0 + AB), where A and B are low-rank matrices, significantly reducing the fine-tuning computation volume.

[0042] Application of the present invention: In the domain fine-tuning module, by inserting low-rank adapters (such as matrices A and B), only about 0.1% - 1% of the model parameters need to be optimized to achieve performance close to that of full-scale fine-tuning, solving the high cost problem of traditional full-scale retraining.

[0043] 3. Vector Database Definition: A database specifically for storing and retrieving vector data, supporting efficient high-dimensional vector similarity retrieval (such as cosine similarity, inner product, etc.), and is commonly used in the retrieval scenario of vectorized representation in natural language processing.

[0044] Application of the present invention: Adopt FAISS as the vector database to store the knowledge embedding matrix of the cross-domain dynamic knowledge base, support retrieving the top-K similar knowledge chunks in seconds, and implement cascade adjustment during dynamic update through a hierarchical index structure.

[0045] 4. Hierarchical Vector Index Definition: A tree-like index structure, where the bottom layer stores the vector embeddings of the original knowledge chunks, and the upper layer maintains the association matrix between domains, achieving efficient retrieval through layer-by-layer filtering.

[0046] Innovation point of the present invention: Bottom layer: Store the original embeddings of knowledge chunks in each domain (such as vectors of legal provisions and medical cases); Upper layer: Maintain the domain association matrix, record the semantic association strength of knowledge chunks in different domains, and support index cascade update when dynamically adding a domain (such as when adding financial domain knowledge, only the upper layer association matrix needs to be updated, without reconstructing the bottom layer index).

[0047] 5. Dual Objective Optimization Strategy Definition: An algorithmic strategy that simultaneously optimizes two independent objective functions, aiming to balance multi-task requirements.

[0048] Application of the present invention: The objective functions are:

[0049] where λ1, λ2 are weight parameters, and the retrieval and generation tasks are synchronously optimized through gradient descent.

[0050] RetrievalLoss: The loss of knowledge retrieval accuracy, ensuring that the model can accurately recall relevant knowledge from the knowledge base; GenerationLoss: The loss of question-answer generation, optimizing the natural language fluency and logical coherence of the answers; By synchronously optimizing the two loss terms through gradient descent, the output of the large model not only conforms to the facts in the knowledge base (accurate retrieval) but also meets the requirements of generation quality.

[0051] 6. Domain-Specific Prompt Library Definition: A collection of prompts constructed for vertical domains (such as law, medicine), including domain-specific terms, question-answer templates, and logical guiding words, used to guide the large model to generate answers that conform to domain norms.

[0052] Construction method of the present invention: Extract high-frequency terms (such as "culpa in contrahendo", "chief complaint history") from professional documents such as legal provisions and medical guidelines; Design a domain-specific prompt format, such as "According to Article XX of the Civil Code, analyze the following case: [Case description]", to enhance the large model's sensitivity to domain semantics.

[0053] 7. Reranking Model Definition: A model that performs a secondary ranking on the initial retrieval results, improving the relevance of the results through a more complex scoring function.

[0054] Technical features of the present invention: Adopt a domain-aware architecture, integrating three dimensions of scoring: semantic similarity, domain term coverage, and historical matching degree; Scoring function formula:

[0055] Implement the sorting logic of "semantic basis - domain priority - historical assistance" through weighted summation.

[0056] 8. Multi-Head Attention Definition: The core component of the Transformer architecture, which captures multi-dimensional semantic associations of input data by parallel computing of multiple independent attention heads.

[0057] Application of the present invention: In the re-ranking model, the multi-head attention mechanism is used to model the interaction between the user question q and the candidate document \(c_i\): each attention head focuses on different semantic sub-spaces (such as legal term sub-space, time relation sub-space); the output is the fused domain-aware score, improving the parsing accuracy of complex questions (such as cross-domain composite questions).

[0058] Embodiment III This embodiment also provides an electronic device, referring to Figure 5 , including a memory 404 and a processor 402. A computer program is stored in the memory 404, and the processor 402 is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0059] Specifically, the above processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0060] Among them, the memory 404 may include a mass memory 404 for data or instructions. By way of example and not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 404 may include removable or non-removable (or fixed) media. Where appropriate, the memory 404 may be internal or external to the data processing device. In a particular embodiment, the memory 404 is non-volatile memory. In a particular embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these. Where appropriate, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0061] The memory 404 can be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402.

[0062] By reading and executing the computer program instructions stored in the memory 404, the processor 402 implements any one of the intelligent question-answering methods based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base in the above embodiments.

[0063] Optionally, the above electronic device may further include a transmission device 406 and an input / output device 408. Among them, the transmission device 406 is connected to the above processor 402, and the input / output device 408 is connected to the above processor 402.

[0064] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above network may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0065] The input / output device 408 is used to input or output information.

[0066] Embodiment Four This embodiment also provides a readable storage medium. The readable storage medium stores a computer program, and the computer program includes program code for controlling a process to execute the process. The process includes the intelligent question-answering method based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base according to Embodiment One.

[0067] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein.

[0068] Generally, various embodiments can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the present invention can be implemented in hardware, while other aspects can be implemented by firmware or software executed by a controller, a microprocessor, or other computing devices, but the present invention is not limited thereto. Although various aspects of the present invention can be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, as a non-limiting example, the blocks, devices, systems, technologies, or methods described herein can be implemented in hardware, software, firmware, dedicated circuits or logic, general hardware or a controller or other computing devices, or some combination thereof.

[0069] Embodiments of the present invention can be implemented by computer software, which is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. A computer software or program (also referred to as a program product), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer-executable components configured to perform the embodiments when the program runs. The one or more computer-executable components can be at least one software code or a part thereof. Additionally, in this regard, it should be noted that any box in the logical flow, as Figure 1 described, can represent a program step, or interconnected logic circuits, boxes, and functions, or a combination of program steps and logic circuits, boxes, and functions. The software can be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. The physical media are non-transitory media.

[0070] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope described in this specification.

[0071] The above embodiments only represent several implementation manners of the present invention. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. An intelligent question-answering method based on lightweight fine-tuning of domain knowledge and cross-domain dynamic knowledge base, characterized in that, It includes the following steps: S1. Construct a cross-domain dynamic knowledge base, including: S11. Collect heterogeneous data from multiple domains, segment the structured data according to domain rules, and chunk the unstructured data according to semantic paragraphs to form unified knowledge units; S12. Generate vector representations of knowledge chunks through an embedding model and construct a hierarchical vector index, where the bottom layer stores the original knowledge chunk embeddings and the upper layer maintains a dynamically updated domain association matrix; S2. Perform two-stage lightweight fine-tuning on the pre-trained large model, including: S21. Pre-connection fine-tuning stage: Before connecting to the knowledge base, use a high-rank adapter to align the domain semantic spaces; S22. Post-connection fine-tuning stage: After connecting to the knowledge base, switch to a low-rank adapter to optimize the semantic consistency between the retrieval results and the model output; S3. Generate answers based on domain relevance re-ranking, including: S31. According to the user's input question, retrieve the knowledge base and obtain candidate knowledge chunks; S32. Re-rank the candidate knowledge chunks through a scoring function that combines the original similarity, domain-specific term coverage, and historical matching degree; S33. Concatenate the re-ranked knowledge chunks with the question to form an enhanced input, and input it into the fine-tuned large model to generate an answer.

2. The intelligent question-answering method according to claim 1, wherein In step S12, the hierarchical vector index is constructed in the following way: The bottom-layer vector index stores the knowledge chunk embeddings and uses a vector database to achieve efficient retrieval; The upper-layer domain association matrix dynamically maintains the knowledge association strength between domains and supports cascading adjustments during incremental updates.

3. The intelligent question-answering method according to claim 1, wherein, In step S2, the two-stage lightweight fine-tuning is achieved in the following way: The rank value of the high-rank adapter in the pre-connection fine-tuning stage is 5%-10% of the model parameters and is used for rough alignment of the domain semantic spaces; The rank value of the low-rank adapter in the post-connection fine-tuning stage is 1%-2% of the model parameters and is used for fine-grained alignment between the retrieval results and the model output.

4. The intelligent question and answer method according to claim 1, characterized in that, In step S3, the formula for the scoring function of the domain relevance re-ranking is: Among them, represents the fused scoring function; represents the candidate document; q represents the question; represents the concatenated input; α, β, γ are preset weight coefficients respectively, and satisfy α + β + γ = 1; represents the original semantic similarity between the question q and the knowledge block d; represents the number of domain-specific terms covered by the knowledge block d; represents the matching frequency of the knowledge block d in the historical answers.

5. The intelligent question-answering method according to claim 4, wherein α, β, γ are 0.45, 0.4, 0.15 respectively.

6. The intelligent question-answering method according to any one of claims 1-5, characterized in that, In step S1, the cross-domain dynamic knowledge base supports the following update mechanisms: When adding new knowledge chunks, automatically update the bottom-layer vector index and trigger cascading adjustments of the domain association matrix; Based on changes in domain boundaries, dynamically adjust the association strength between knowledge chunks in the domain association matrix.

7. An intelligent question-answering device based on lightweight fine-tuning of domain knowledge and cross-domain dynamic knowledge base, characterized in that, It includes: A knowledge base construction module for constructing a cross-domain dynamic knowledge base, including: Collect heterogeneous data from multiple domains, segment the structured data according to domain rules, and chunk the unstructured data according to semantic paragraphs to form unified knowledge units; Generate vector representations of knowledge chunks through an embedding model and construct a hierarchical vector index, where the bottom layer stores the original knowledge chunk embeddings and the upper layer maintains a dynamically updated domain association matrix; A domain fine-tuning module for performing two-stage lightweight fine-tuning on the pre-trained large model, including: Pre-connection fine-tuning stage: Before connecting to the knowledge base, use a high-rank adapter to align the domain semantic spaces; Post-connection fine-tuning stage: After connecting to the knowledge base, switch to a low-rank adapter to optimize the semantic consistency between the retrieval results and the model output; A retrieval enhancement module for generating answers based on domain relevance re-ranking, including: According to the user's input question, retrieve the knowledge base and obtain candidate knowledge chunks; Re-rank the candidate knowledge chunks using a scoring function that combines the original similarity, domain-specific term coverage, and historical matching degree; A question-answering generation module, configured to splice the re-ranked knowledge chunks with the question to form an enhanced input, and input the enhanced input into the fine-tuned large model to generate an answer.

8. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to execute the intelligent question-answering method based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that, A computer program is stored in the readable storage medium, and the computer program includes program codes for controlling a process to execute the process, and the process includes the intelligent question-answering method based on domain knowledge lightweight fine-tuning and cross-domain dynamic knowledge base according to any one of claims 1 to 6.

Images