An intelligent question-answering system based on a large language model

Abstract

This invention discloses an intelligent question-answering system based on a large language model, belonging to the fields of natural language processing and information retrieval technology. The system utilizes retrieval-enhanced generation (RAG) and knowledge re-ranking techniques to improve the ability to process knowledge related to mechanical manufacturing process design, industry standards, and related specifications. The system mainly consists of four parts: the first part constructs a knowledge base covering mechanical manufacturing knowledge; the second part uses a LoRA+ fine-tuned Qwen2-7B model to ensure high-precision requirements such as process parameter calculation; the third part integrates RAG technology to effectively retrieve and process domain knowledge; and the fourth part introduces a re-ranking model to optimize the answer. This system significantly outperforms the traditional large language model LLM in BLEU-4 and ROUGE text matching evaluation metrics, verifying its reliability and professionalism in the mechanical manufacturing field, providing an effective solution for intelligent question answering in this field, and showing broad prospects.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and information retrieval technology, and more specifically, to an intelligent question-answering system based on a large language model. Background Technology

[0002] The machinery manufacturing industry is a pillar of modern industry, encompassing complex processes such as product design, material processing, and equipment assembly. It supports key industries like automotive, aerospace, and energy, and drives technological innovation and economic development. As manufacturing transforms towards intelligent and digital transformation, engineers and managers increasingly demand timely and accurate knowledge. However, the knowledge in this field is highly specialized, complex, and fragmented, distributed across technical documents, standards, and patent literature. Traditional knowledge acquisition methods, such as manual searches and document retrieval, are time-consuming and labor-intensive, failing to meet real-time needs and reducing work efficiency.

[0003] To improve knowledge acquisition efficiency, machinery manufacturing enterprises urgently need question-answering systems that can quickly and accurately answer specialized technical questions. Such systems must understand domain terminology and handle complex technical issues, providing high-quality answers to support design, manufacturing, and maintenance decisions. This has spurred research into vertical domain question-answering systems based on advanced natural language processing (NLP) technologies. In this context, while Large Language Models (LLMs) perform well in open-domain question answering, their accuracy and professional application in specialized fields such as machinery manufacturing face challenges.

[0004] While closed-source models such as GPT-3 and ChatGPT have demonstrated superior capabilities through pre-training and fine-tuning, their lack of transparency, controllability, and high resource requirements limit their application in high-security and customized environments. This study chooses to use open-source models to address the shortcomings of closed-source models. Open-source models can be flexibly deployed and customized according to specific needs, thus better adapting to the question-answering requirements of specialized domains.

[0005] To address the limitations of LLM in domain knowledge coverage, RAG technology emerged. RAG combines information retrieval and generative models, using a retrieval system to extract relevant document fragments from external knowledge bases and then processing them through LLM to generate more accurate and context-relevant answers. This approach improves the accuracy and relevance of answers to complex queries in specialized domains, but further optimization is needed in retrieval and generation integration, real-time performance, and domain knowledge understanding. Summary of the Invention

[0006] In view of the shortcomings of existing technologies, the purpose of this invention is to provide an intelligent question-answering system based on a large language model.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] An intelligent question-answering system based on a large language model includes a knowledge base for the field of mechanical manufacturing, which covers mechanical manufacturing process design, industry standards and related specifications;

[0009] The system optimizes the accuracy and professionalism of answers by combining retrieval enhancement generation (RAG) technology with re-ranking technology; the steps are as follows:

[0010] Step 1: Build a knowledge base containing diverse knowledge in the field of mechanical manufacturing, covering process design, material properties, and equipment operation information, to support answers to a wide range of questions in the field.

[0011] Step 2: Fine-tune various open-source large models and train them to meet the complexity and high precision requirements of the mechanical manufacturing field to ensure that the models have strong reasoning and computation capabilities. Among them, the Qwen2-7B model with LoRA+ fine-tuning has the best performance.

[0012] Step 3: Using RAG technology, effectively retrieve and process domain knowledge from the knowledge base, especially numerical tables and complex technical specifications, to obtain preliminary answers;

[0013] Step 4: Optimize the retrieved results using Re-ranking technology to generate more professional and accurate answers, ensuring that the output meets the requirements of the mechanical manufacturing field.

[0014] Preferably, in step 1, knowledge from mechanical manufacturing process design, industry standards, and related specifications is integrated to form a structured knowledge base, which is convenient for the system to call during the answering process; the data collection covers multiple important sources in the field of mechanical manufacturing process design, ensuring the advantages of the question-and-answer system in terms of professionalism and accuracy.

[0015] Preferably, in step 2, the large model is pre-trained and supervised fine-tuned using a specific dataset for the mechanical manufacturing field during the training process to improve the model's ability to understand and apply domain knowledge.

[0016] The Qwen2-7B model is an open-source large model. After comparing various open-source large models, it was found that the Qwen2-7B model with LoRA+ fine-tuning has the best performance in the field of mechanical manufacturing. The Qwen2-7B model has a large parameter scale and can handle complex tasks.

[0017] The core architecture of the large model is an autoregressive model based on Transformer; an autoregressive model is a sequence generation model that depends on previously generated elements when generating each element of the sequence; it has the ability to predict the i-th token based on the first i-1 tokens.

[0018] P(ω i ω1,ω2,…,ω i-1 ;θ)

[0019] During pre-training, the most traditional causal modeling method is used, and the model is optimized in an unsupervised manner by minimizing the log-likelihood of the following formula:

[0020]

[0021] Where ω1,ω2,…,ω i-1 It is an input sequence consisting of i words;

[0022] During the fine-tuning phase, each data sample includes an "instruction" field, which describes the specific requirements of the task; an "input" field, which provides the input content for the model; and an "output" field, which contains the target output that the model should generate.

[0023] LoRA is a method for fine-tuning large models. Instead of adjusting all model parameters, it adds a low-rank matrix called an adapter to the weights of a pre-trained model. This method can effectively adapt to new tasks while keeping computational costs low.

[0024] The original LoRA suffers from poor fine-tuning performance when dealing with wide models because it updates adapter matrices A and B using the same learning rate. This is because using the same learning rate in wide networks is detrimental to effective feature learning. To address this issue, the proposed method LoRA+ sets different learning rates for adapter matrices A and B while maintaining a fixed ratio. It suggests setting the learning rate of B to λ times that of A, where λ > 1, thereby improving feature learning efficiency. Through extensive experiments, LoRA+ demonstrates improved performance and fine-tuning speed while maintaining the same computational cost as LoRA. Using the LoRA+ fine-tuning method, and following the rule of thumb, setting λ = 2... 4 =16.

[0025] Preferably, in step 3, RAG technology uses relevance analysis of the user's input question to filter out the most relevant information from the knowledge base, thereby providing a preliminary basis for the answer;

[0026] RAG technology involves embedding models and vector databases; the embedding model used is the BGE M3e model, an open-source model that can handle input granularity at different levels, from "sentences" to "documents", and supports dense retrieval, lexical retrieval and multi-vector retrieval.

[0027] Dense retrieval is based on a text encoder that transforms the input query q into a hidden state H. q And the normalized hidden state marked with a special symbol "[CLS]" is used to represent the embedding representation of the query: e q =norm(H q [0]); Similarly, the embedded representation of paragraph p can be obtained: e p =norm(H p [0]); Therefore, the relevance score between the query and the paragraph is represented by two embeddings e p and e q Measured by the inner product: s dense ← <e p ,e q >;

[0028] The embeddings output in lexical retrieval are used to evaluate the importance of each term in order to perform lexical-level retrieval; for each term t in the query, its weight is calculated as follows: Among them, W lex ∈R d×1 This is a matrix used to map hidden states to floating-point numbers; if the same term t appears multiple times in the query, only its highest weight is retained; the term weights in paragraphs are also calculated in the same way; based on these term weights, the relevance score between the query and the paragraph is calculated by the joint importance of the co-occurring terms, denoted as q∩p:

[0029] Building upon dense retrieval, the multi-vector method uses complete output embeddings to represent queries and paragraphs: Among them W mul ∈R d×d It is a trainable projection matrix; and uses late-stage interaction to compute fine-grained relevance scores: N and M represent the length of the query and the paragraph, respectively;

[0030] The multifunctional nature of the BGE M3e model allows for a hybrid retrieval process; firstly, candidate results can be retrieved using each method separately; then, the final retrieval results are reordered based on a comprehensive relevance score. rank ←s dense +s lex +s mul ;

[0031] The training objective of the embedding model is to distinguish between positive and negative samples; for each retrieval method, it is desirable to assign higher scores to positive samples and lower scores to negative samples; therefore, the training process aims to minimize the InfoNCE loss, which has the general form of:

[0032]

[0033] Where p * p and p' represent the positive and negative samples of query q, respectively; s(·) is {s dense (·),s lex (·),s mul Any function in (·)};

[0034] The training objectives of different retrieval methods may conflict, and multi-objective training may be detrimental to the quality of embeddings. Therefore, based on the principle of ensemble learning, a unified training process is established on the basis of self-knowledge distillation, combining the predictions of different retrieval methods to provide more accurate relevance scores. The simplest form is to sum the different prediction scores:

[0035] s inter ←s dense +s lex +s mul

[0036] Previous research has shown that the training quality of embedding models can benefit from knowledge distillation, which utilizes fine-grained soft labels from another ranking model; the BGE M3e model uses ensemble scores. inter As a teacher, I modified the loss function for each retrieval method:

[0037] L' * ←-p(s inter )*logp(s * )

[0038] Where p(·) is the softmax activation; s * It is dense s lex and s mul Any one of them; the further integrated and normalized modified loss function is as follows:

[0039] L'←(L' dense +L' lex +L' mul ) / 3

[0040] Finally, the final loss function derived from knowledge distillation is obtained through a linear combination of L and L':

[0041] L final ←L+L'

[0042] After generating high-quality embedding vectors, the Faiss vector database is used to achieve efficient similarity search. Faiss is designed for processing large-scale vector datasets and can quickly retrieve the vectors most similar to the query vectors. Its core technologies include Product Quantization (PQ) and Inverted File System (IVF).

[0043] PQ compresses the original vector into a cluster core ID sequence by dividing the vector into multiple sub-vectors and clustering them. During the retrieval process, the query vector is also divided into corresponding sub-vectors, and the distance to the vector in the database is quickly calculated through table lookup. Compared with the traditional brute-force calculation, the PQ method calculates the distance between sub-vectors by looking up the table.

[0044] Although PQ has optimized the computation of vector retrieval, when the number of vectors in the database is extremely large, it is still necessary to traverse the entire database to calculate the distance, which affects retrieval efficiency. To address this, Faiss introduced IVF and further developed IVFPQ technology. The core idea of ​​IVFPQ is to pre-segment the vector space to lock the "region of interest" of the query vector and avoid global retrieval. In its implementation, Faiss first performs KMeans clustering on the vectors in the database to generate several cluster centers. When a query vector is input, it first calculates the distance between it and these cluster centers and only retrieves the vector subspace related to the nearest cluster center, thereby avoiding unnecessary full database computation.

[0045] Furthermore, Faiss introduces residual optimization, which converts all vectors into residuals with the cluster center when calculating the distance between the query vector and the subspace vectors. This further improves the accuracy of PQ calculation. Through this method, IVFPQ reduces the order of magnitude of the vectors to be calculated, significantly improving vector retrieval efficiency.

[0046] Preferably, in step 4, the Re-ranking technology significantly improves the accuracy and professionalism of the final answer through multi-level information fusion and optimization, especially in text matching evaluation indicators such as BLEU-4 and ROUGE.

[0047] The BGE M3e model transforms the input text into a fixed-dimensional vector representation, captures semantic information, and normalizes the vector to a length of 1 through L2 regularization so that cosine similarity can be calculated through dot product.

[0048] The regularized vectors are stored in the Faiss vector database and retrieved quickly using an efficient indexing algorithm. In the initial retrieval stage, the cosine similarity of dense vectors and keyword retrieval of sparse vectors are combined to usually recall the top K most relevant texts to the query. However, the correct answer may be ranked lower, causing LLM to process redundant information, reducing inference efficiency and increasing the probability of error. Therefore, further filtering through reordering is needed to improve the accuracy of relevant texts.

[0049] The re-ranking process utilizes the BERT model and a linear classifier for text matching, capturing more complex semantic details. Although the classifier has high computational complexity, the system achieves a balance between efficiency and accuracy by combining it with cosine similarity calculation in the initial retrieval. The system evaluates the quality of the generated answers using BLEU and ROUGE metrics.

[0050]

[0051]

[0052] BP is used to penalize candidate texts that are too short, p n The weight ω represents the precision of the n-gram, i.e., the proportion of n-grams appearing in the candidate text compared to the reference text; typically, for all n values, the weight ω... n Equal; n represents n-gram, i.e., gram n Size, Count match (gram n ) represents the maximum number of n-grams that appear together in both the generated answer and the reference answer sets.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. This invention proposes an innovative cross-technology integration framework, which for the first time combines LLM, RAG and reordering in the field of mechanical manufacturing, significantly improving the overall performance of question answering systems.

[0055] 2. This invention has created a Mechanical Manufacturing Process Design (MMPD) dataset, which optimizes the system’s performance in professional scenarios by integrating and fine-tuning domain knowledge.

[0056] 3. This invention proposes a collaborative mechanism for retrieval generation, ensuring that the generation model can fully utilize the relevant document information retrieved, thereby improving the accuracy and contextual consistency of the answer; at the same time, it successfully integrates a reordering engine to reorder the documents retrieved by RAG, further optimizing the quality of the final answer.

[0057] 4. This invention expands the application of LLM in vertical domain question-answering systems and verifies the feasibility of combining multiple technologies in industrial applications through experiments. Attached Figure Description

[0058] Figure 1 This invention presents a flowchart illustrating the process of optimizing responses in an intelligent question-answering system based on a large language model.

[0059] Figure 2 This invention presents a schematic diagram of the structure of an intelligent question-answering system based on a large language model.

[0060] Figure 3 This is a schematic diagram illustrating the construction method of the RAG proposed in this invention;

[0061] Figure 4 This is a schematic diagram of the large model fine-tuning proposed in this invention. Detailed Implementation

[0062] Reference Figures 1 to 4 .

[0063] The embodiments further illustrate the intelligent question-answering system based on a large language model proposed in this invention.

[0064] An intelligent question-answering system based on a large language model includes a knowledge base for the field of mechanical manufacturing, which covers mechanical manufacturing process design, industry standards and related specifications;

[0065] The system optimizes the accuracy and professionalism of answers by combining retrieval enhancement generation (RAG) technology with re-ranking technology; the steps are as follows:

[0066] Step 1: Build a knowledge base containing diverse knowledge in the field of mechanical manufacturing, covering process design, material properties, and equipment operation information, to support answers to a wide range of questions in the field.

[0067] Step 2: Fine-tune various open-source large models and train them to meet the complexity and high precision requirements of the mechanical manufacturing field to ensure that the models have strong reasoning and computation capabilities. Among them, the Qwen2-7B model with LoRA+ fine-tuning has the best performance.

[0068] Step 3: Using RAG technology, effectively retrieve and process domain knowledge from the knowledge base, especially numerical tables and complex technical specifications, to obtain preliminary answers;

[0069] Step 4: Optimize the retrieved results using Re-ranking technology to generate more professional and accurate answers, ensuring that the output meets the requirements of the mechanical manufacturing field.

[0070] In step 1, knowledge from mechanical manufacturing process design, industry standards and related specifications is integrated to form a structured knowledge base, which is convenient for the system to call during the solution process.

[0071] The data collection covers multiple important sources in the field of mechanical manufacturing process design, ensuring the advantages of the question-and-answer system in terms of professionalism and accuracy. Specific data sources include the following authoritative documents and standards: "Handbook of Mechanical Manufacturing Process Design", "Mechanical Manufacturing Technology", "Fundamentals of Mechanical Manufacturing Technology", "Mechanical Engineering Materials", "GBT4863-2008 Basic Terminology of Mechanical Manufacturing Process", "National Standard 1 for Digital Simulation of Mechanical Manufacturing: General Requirements", "National Standard 2 for Digital Simulation of Mechanical Manufacturing: Requirements for Production Line Planning and Layout Simulation", "National Standard 3 for Digital Simulation of Mechanical Manufacturing: Requirements for Assembly Workshop Logistics Simulation", and "National Standard 4 for Digital Simulation of Mechanical Manufacturing: Requirements for CNC Machining Process Simulation".

[0072] In step 2, the large model is pre-trained and supervised fine-tuned using a specific dataset for the mechanical manufacturing field during the training process to improve the model's ability to understand and apply domain knowledge.

[0073] The Qwen2-7B model is an open-source large model. After comparing various open-source large models, it was found that the Qwen2-7B model with LoRA+ fine-tuning has the best performance in the field of mechanical manufacturing. The Qwen2-7B model has a large parameter scale (7 billion parameters) and can handle complex tasks.

[0074] The core architecture of the large model is an autoregressive model based on Transformer; an autoregressive model is a sequence generation model that depends on previously generated elements when generating each element of the sequence; it has the ability to predict the i-th token based on the first i-1 tokens.

[0075] P(ω i ω1,ω2,…,ω i-1 ;θ)

[0076] During pre-training, the most traditional causal modeling method is used, and the model is optimized in an unsupervised manner by minimizing the log-likelihood of the following formula:

[0077]

[0078] Where ω1,ω2,…,ω i-1 It is an input sequence consisting of i words.

[0079] During the fine-tuning phase, each data sample includes an "instruction" field, which describes the specific requirements of the task; an "input" field, which provides the input content for the model; and an "output" field, which contains the target output that the model should generate. This structured format design can effectively guide the model's behavior in specific tasks, ensuring that the model can fully understand and generate high-quality responses when dealing with questions and answers in the field of mechanical manufacturing.

[0080] LoRA is a method for fine-tuning large models. Instead of adjusting all model parameters, it adds a low-rank matrix (called an adapter) to the weights of the pre-trained model. This method can effectively adapt to new tasks while keeping computational costs low.

[0081] The original LoRA suffers from poor fine-tuning performance when dealing with models with large embedding dimensions because it updates adapter matrices A and B using the same learning rate. This is because using the same learning rate in large-width networks is detrimental to effective feature learning. To address this issue, the proposed method LoRA+ sets different learning rates for adapter matrices A and B while maintaining a fixed ratio. It suggests setting the learning rate of B to λ times that of A, where λ > 1, thereby improving feature learning efficiency. Through extensive experiments, LoRA+ improves performance (by 1%-2%) and fine-tuning speed (by approximately 2x) while maintaining the same computational cost as LoRA. Using the LoRA+ fine-tuning method, and following the rule of thumb, let λ = 2... 4 =16.

[0082] In step 3, RAG technology uses relevance analysis of the user's input question to filter out the most relevant information from the knowledge base, thereby providing a preliminary basis for the answer.

[0083] RAG technology involves embedding models and vector databases; the embedding model used is the BGE M3e (BGE M3-Embedding) model, which is an open-source model that can handle different levels of input granularity from "sentences" to "documents" and supports dense retrieval, lexical retrieval and multi-vector retrieval.

[0084] Dense retrieval is based on a text encoder that transforms the input query q into a hidden state H. q And the normalized hidden state marked with a special symbol "[CLS]" is used to represent the embedding representation of the query: e q =norm(H q [0]); Similarly, the embedded representation of paragraph p can be obtained: e p =norm(Hp [0]); Therefore, the relevance score between the query and the paragraph is represented by two embeddings e p and e q Measured by the inner product: s dense ← <e p ,e q >

[0085] The embeddings output in lexical retrieval are used to evaluate the importance of each term in order to perform lexical-level retrieval; for each term t in the query (in this work, terms correspond to tokens), its weight is calculated as follows: Among them, W lex ∈R d×1 This is a matrix used to map hidden states to floating-point numbers; if the same term t appears multiple times in the query, only its highest weight is retained; the term weights in paragraphs are also calculated in the same way; based on these term weights, the relevance score between the query and the paragraph is calculated by the joint importance of the co-occurring terms, denoted as q∩p:

[0086] Building upon dense retrieval, the multi-vector method uses complete output embeddings to represent queries and paragraphs: Among them W mul ∈R d×d It is a trainable projection matrix; and uses late-stage interaction to compute fine-grained relevance scores: N and M represent the length of the query and the paragraph, respectively.

[0087] The multifunctional nature of the BGE M3e model allows for a hybrid retrieval process; firstly, candidate results can be retrieved using each method separately; then, the final retrieval results are reordered based on a comprehensive relevance score. rank ←s dense +s lex +s mul .

[0088] The training objective of the embedding model is to distinguish between positive and negative samples; for each retrieval method, it is desirable to assign higher scores to positive samples and lower scores to negative samples; therefore, the training process aims to minimize the InfoNCE loss, which has the general form of:

[0089]

[0090] Where p * p and p' represent the positive and negative samples of query q, respectively; s(·) is {s dense (·),s lex (·),s mulAny function in (·)}.

[0091] The training objectives of different retrieval methods may conflict, and multi-objective training may be detrimental to the quality of embeddings. Therefore, based on the principle of ensemble learning, a unified training process is established on the basis of self-knowledge distillation, combining the predictions of different retrieval methods to provide more accurate relevance scores. The simplest form is to sum the different prediction scores:

[0092] s inter ←s dense +s lex +s mul

[0093] Previous research has shown that the training quality of embedding models can benefit from knowledge distillation, which utilizes fine-grained soft labels from another ranking model; the BGE M3e model uses ensemble scores. inter As a teacher, I modified the loss function for each retrieval method:

[0094] L' * ←-p(s inter )*logp(s * )

[0095] Where p(·) is the softmax activation; s * It is dense s lex and s mul Any one of them; the further integrated and normalized modified loss function is as follows:

[0096] L'←(L' dense +L' lex +L' mul ) / 3

[0097] Finally, the final loss function derived from knowledge distillation is obtained through a linear combination of L and L':

[0098] L final ←L+L'

[0099] After generating high-quality embedding vectors, the Faiss vector database is used to achieve efficient similarity search. Faiss is designed for processing large-scale vector datasets and can quickly retrieve the vectors most similar to the query vector. Its core technologies include Product Quantization (PQ) and Inverted File System (IVF).

[0100] PQ compresses the original vector into a cluster core ID sequence by dividing it into multiple sub-vectors and clustering them. This process reduces dimensionality and preserves semantic information, greatly reducing computational complexity. During retrieval, the query vector is also divided into corresponding sub-vectors, and the distance to the vector in the database is quickly calculated through table lookup. Compared with traditional brute-force calculation, the PQ method effectively reduces computation time through table lookup and distance calculation between sub-vectors.

[0101] Although PQ has optimized the computation of vector retrieval, when the number of vectors in the database is extremely large, it is still necessary to traverse the entire database to calculate the distance, which affects retrieval efficiency. To address this, Faiss introduced IVF and further developed the IVFPQ technology. The core idea of ​​IVFPQ is to pre-segment the vector space to lock the "region of interest" of the query vector and avoid global retrieval. In its implementation, Faiss first performs KMeans clustering on the vectors in the database to generate several cluster centers. When a query vector is input, it first calculates the distance between it and these cluster centers and only retrieves the vector subspace related to the nearest cluster center, thereby avoiding unnecessary full database computation.

[0102] Furthermore, Faiss introduces residual optimization, which converts all vectors into residuals with the cluster center when calculating the distance between the query vector and the subspace vectors. This further improves the accuracy of PQ calculation. Through this method, IVFPQ reduces the order of magnitude of the vectors to be calculated, significantly improving vector retrieval efficiency.

[0103] In step 4, the Re-ranking technology significantly improves the accuracy and professionalism of the final answer through multi-level information fusion and optimization, especially in text matching evaluation metrics such as BLEU-4 and ROUGE.

[0104] The BGE M3e model captures semantic information by transforming input text into fixed-dimensional vector representations and normalizing the vectors to a length of 1 using L2 regularization, enabling the calculation of cosine similarity through dot product. The regularized vectors are stored in the Faiss vector database and retrieved quickly using an efficient indexing algorithm. The initial retrieval stage combines cosine similarity of dense vectors with keyword retrieval of sparse vectors, typically recalling the top K most relevant texts. However, in the initial retrieval results, texts containing the correct answer may be ranked lower, causing LLM to process a large amount of redundant information, reducing inference efficiency and increasing the probability of errors. Therefore, further filtering through re-ranking is necessary to improve the accuracy of the most relevant texts.

[0105] Re-ranking involves processing the recalled text using the BERT model and employing a linear classifier to perform a binary classification task to determine text matching. Compared to matching methods that solely rely on cosine similarity, re-ranking can capture more complex semantic details. This is because BERT's attention mechanism and feedforward neural network can extract richer features, and the classifier's accuracy is further improved through parameter tuning. Although the classifier has higher computational complexity, the system achieves a balance between efficiency and accuracy by combining it with cosine similarity calculation in the initial retrieval. By first using efficient cosine similarity for initial screening and then using re-ranking to improve accuracy, the overall performance of the system is significantly improved. The system evaluates the quality of the generated answers using BLEU and ROUGE metrics.

[0106]

[0107]

[0108] BP is used to penalize candidate texts that are too short, p n The weight ω represents the precision of the n-gram, i.e., the proportion of n-grams appearing in the candidate text compared to the reference text; typically, for all n values, the weight ω... n Equal; n represents n-gram, i.e., gram n Size, Count match (gram n ) represents the maximum number of n-grams that appear together in both the generated answer and the reference answer sets.

[0109] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An intelligent question-answering system based on a large language model, characterized in that, This includes a knowledge base for the field of mechanical manufacturing, which covers mechanical manufacturing process design, industry standards and related specifications; The system optimizes the accuracy and professionalism of answers by combining retrieval enhancement generation (RAG) technology with re-ranking technology; the steps are as follows: Step 1: Build a knowledge base containing diverse knowledge in the field of mechanical manufacturing, covering process design, material properties, and equipment operation information, to support answers to a wide range of questions in the field. Step 2: Fine-tune various open-source large models, train them to meet the complexity and high precision requirements of the mechanical manufacturing field, and determine the LoRA+ fine-tuned Qwen2-7B model with the best performance. Step 3: Using RAG technology, domain knowledge is effectively retrieved and processed from the knowledge base to obtain preliminary answers; the embedding model is BGE M3e; and the vector database is Faiss. Step 4: Optimize the retrieved results using Re-ranking technology; specifically, use the BERT model and a linear classifier to determine text matching.

2. The intelligent question-answering system based on a large language model according to claim 1, characterized in that, In step 1, knowledge from mechanical manufacturing process design, industry standards, and related specifications is integrated to form a structured knowledge base, which is convenient for the system to call during the answering process. The data collection covers multiple sources in the field of mechanical manufacturing process design, ensuring the advantages of the question-and-answer system in terms of professionalism and accuracy.

3. The intelligent question-answering system based on a large language model according to claim 2, characterized in that, In step 3, RAG technology uses relevance analysis of the user's input question to filter out the most relevant information from the knowledge base, thereby providing a preliminary basis for the answer. RAG technology involves embedding models and vector databases; the embedding model used is the BGE M3e model, an open-source model that can handle input granularity at different levels from "sentence" to "document", and supports dense retrieval, lexical retrieval and multi-vector retrieval. The BGE M3e model employs a hybrid approach; firstly, candidate results are retrieved using the three methods described above; then, the final retrieval results are reordered based on a comprehensive relevance score: s rank ←s dense +s lex +s mul ; The training objective of the embedding model is to distinguish between positive and negative samples; for each retrieval method, it is desirable to assign higher scores to positive samples and lower scores to negative samples; therefore, the training process aims to minimize the InfoNCE loss, which has the general form of: Where p* and p' represent the positive and negative samples of query q, respectively; s(·) is {s dense (·),s lex (·),s mul Any function in (·)}; Based on the principle of ensemble learning, the training process is unified on the basis of self-knowledge distillation, and the predictions of different retrieval methods are combined to provide more accurate relevance scores. The BGE M3e model uses integrated scores. inter As a teacher, I modified the loss function for each retrieval method: L' * ←-p(s inter )*logp(s * ) Where p(·) is the softmax activation; s * It is s dense s lex and s mul Any one of them; the further integrated and normalized modified loss function is as follows: The ←(The dense +The lex +The mul ) / 3 Finally, the final loss function derived from knowledge distillation is obtained through a linear combination of L and L': L final ←L+L' After generating high-quality embedding vectors, the Faiss vector database is used to achieve efficient similarity search; its core technologies include product quantization (PQ) and inverted index file system (IVFPQ). Faiss first performs KMeans clustering on the vectors in the library to generate several cluster centers. When a query vector is input, it first calculates the distance between the vector and these cluster centers, and only retrieves the vector subspace related to the nearest cluster center, thus avoiding unnecessary full library computation.

4. The intelligent question-answering system based on a large language model according to claim 3, characterized in that, In step 4, the Re-ranking technology employs multi-level information fusion and optimization; The BGE M3e model transforms the input text into a fixed-dimensional vector representation, captures semantic information, and normalizes the vector to a length of 1 through L2 regularization so that cosine similarity can be calculated through dot product. The regularized vectors are stored in the Faiss vector database, and the efficient indexing algorithm retrieves them quickly. In the initial retrieval stage, the cosine similarity of dense vectors and the keyword retrieval of sparse vectors are combined to recall the top K texts that are most relevant to the query. The re-ranking process utilizes the BERT model and a linear classifier for text matching, capturing more complex semantic details. The system evaluates the quality of the generated answers using BLEU and ROUGE metrics. BP is used to penalize candidate texts that are too short, p n The precision of the n-gram is represented by the proportion of n-grams appearing in the candidate text compared to the reference text; for all n values, the weight ω is... n Equal; n represents n-gram, i.e., gram n Size, Count match (gram n ) represents the maximum number of n-grams that appear together in both the generated answer and the reference answer sets.

Images