Large model file question-answering system based on two-parameter sparse regularization parameter fine tuning

Abstract

The invention is suitable for the technical field of archive management query, and particularly relates to a large model archive question-answering system based on two-parameter sparse regularization parameter fine tuning, and the system comprises a query generation module which is used for converting a question input by a user into semantic query data through natural language processing, and generating a semantic vector; the knowledge base retrieval module is used for screening out file contents queried by the user; the generation enhancement module is used for outputting answers; and the optimization module is used for expanding the semantic coverage range of the knowledge base, analyzing the picture content, preprocessing the OCR document and finely adjusting the large model. According to the large model file question-answering system based on the two-parameter sparse regularization parameter fine tuning, aiming at the common problems of messy codes, information loss and structure confusion in an OCR document, a qwen2-7b-struct model is adopted for intelligent preprocessing, through the model, the system can automatically clean redundant line feed characters and reconstruct paragraph structures, and therefore the efficiency is improved. Therefore, the logicality and continuity of the text are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of file management and query, and particularly relates to a large model file question-answering system based on dual-parameter sparse regularization parameter fine-tuning. Background Art

[0002] Driven by the digital wave, file management work is undergoing unprecedented changes. Due to relying on manual processing and paper storage, the traditional file management method not only has problems such as slow retrieval speed, high storage cost, and easy damage, but also is difficult to meet the needs of modern society for efficient and accurate information retrieval.

[0003] With the rapid development of artificial intelligence technology, especially the rise of large model technology represented by large language models (LLMs), a new solution has been provided for the digital and intelligent upgrade of file management. Through their powerful language understanding and generation capabilities, these models can extract valuable information from large-scale data, greatly improving the intelligent level of file management.

[0004] However, general large models still face the following challenges in specific applications: the problem of processing unstructured text: Many files contain mixed information of text and pictures. The picture part may involve important charts, handwritten records, or scanned documents, and the general large models have limited processing capabilities for picture information. Existing models cannot directly extract high-quality text or semantic information from pictures and effectively associate it with the text content, resulting in the omission of picture-related content in retrieval and question answering. Summary of the Invention

[0005] The purpose of the present invention is to provide a large model file question-answering system based on dual-parameter sparse regularization parameter fine-tuning, aiming to solve the problem that existing models cannot directly extract high-quality text or semantic information from pictures and effectively associate it with the text content, resulting in the omission of picture-related content in retrieval and question answering.

[0006] The present invention is implemented as follows. A large model file question-answering system based on dual-parameter sparse regularization parameter fine-tuning, the system includes:

[0007] A query generation module, configured to convert the question input by the user into semantic query data through natural language processing and generate a semantic vector;

[0008] A knowledge base retrieval module, configured to extract corresponding documents from the local knowledge base, perform retrieval based on semantic matching, and screen out the file content corresponding to the user's query;

[0009] A generation enhancement module, configured to use the retrieved document as context information, combine the context semantics with the user's query content, and output an answer;

[0010] Optimization module, used to expand the semantic coverage of the knowledge base, parse the content of pictures, preprocess OCR documents, and fine-tune large models.

[0011] Preferably, during the process of expanding the semantic coverage of the knowledge base, a domain-specific thesaurus is constructed, and synonym mapping relationships are generated through natural language processing tools, professional dictionaries, and industry glossaries. The user's query input generates an extended query set through synonym replacement and semantic expansion.

[0012] Preferably, it is characterized in that an expansion function is defined: Expand(Q) = {Q, Q (1) , Q (2) , …, Q (k)}

[0013] A series of semantic variants Q (k) of the original query Q are obtained.

[0014] Preferably, in the steps of parsing the content of pictures, in the picture text extraction link, a picture large model is used to extract text information and identify scene content from archival pictures. In the semantic parsing and mapping stage, the extracted text content and picture labels are further mapped to nodes and relationships in the knowledge graph to form a complete structured data representation. In the multimodal fusion stage, the picture information and relevant text content in the archives are embedded into a unified semantic space.

[0015] Preferably, the knowledge graph is represented by G = (V, E), where V = {v1, v2, …} represents knowledge nodes, and E = {e i,j} represents the relationships between nodes.

[0016] Preferably, during the preprocessing of OCR documents, in the model preprocessing stage, the qwen2-7b-instruct model is used to intelligently repair the original OCR text; in the redundancy deletion and structuring stage, the formatting of the text is optimized, and meaningless line breaks, blank lines, or duplicate content generated during the OCR extraction process are cleared; in the knowledge base optimization stage, the index system of the knowledge base is rebuilt based on the preprocessed text content.

[0017] Preferably, in the steps of fine-tuning the large model, sparse LoRA fine-tuning is adopted, and the mathematical model of sparse LoRA is:

[0018] Assume that the weight matrix that needs to be fine-tuned for the model is W ∈ R d×k , and the fine-tuned weight is expressed as: W LR = W + ΔW, ΔW = AB where, A ∈ R d×r , B ∈ R r×kReal matrices of d×r and r×k respectively, where d, r, and k are integers representing the number of rows and columns of the matrices, r << min(d, k), and ΔW is a low-rank fine-tuning matrix.

[0019] Preferably, a two-parameter L2,1 norm regularization is introduced to constrain the sparsity in ΔW, and its regularization objective function is:

[0020]

[0021] where λ A and λ B are regularization coefficients that control the sparsity intensity, and A j,j and B i,j are the elements in the i-th row and j-th column of matrix A and matrix B respectively.

[0022] Preferably, by combining the task objective loss and the regularization term, the overall optimization objective is:

[0023] L = Ltask + Lreg

[0024] where: Ltask is the cross-entropy loss function for the file Q&A task.

[0025] Preferably, the implementation steps of sparse LoRA fine-tuning include:

[0026] Initializing the low-rank matrices: randomly initialize matrices A and B, and set the sparse regularization coefficients λ A and λ B to a preset value;

[0027] Sparsifying matrix update: through iterative optimization, gradually sparsify the preset parameters of the matrix. Each iteration includes the following steps:

[0028] Calculating the gradients: according to the task loss function L, calculate the gradients of the elements A i,j and B i,j in matrices A and B and

[0029] Updating the elements of matrices A and B:

[0030]

[0031] where η is the learning rate;

[0032] Sparsifying matrix truncation: apply the sparsification rule to the updated matrix ΔW, and set the elements less than the threshold ∈ to zero: A i,j = 0 if |A i,j | < ∈; B i,j = 0 if |B i,j | < ∈;

[0033] Normalization operation: Ensure that the row norms of matrix ΔW are maintained within a preset range;

[0034] Convergence detection: Repeat the execution until the loss function L converges.

[0035] The large model archive question - answering system based on the fine - tuning of double - parameter sparse regularization parameters provided by the present invention addresses the common problems of garbled characters, information loss, and structural chaos in OCR documents. It uses the qwen2 - 7b - instruct model for intelligent pre - processing. Through this model, the system can automatically clean redundant line breaks, reconstruct the paragraph structure, thereby enhancing the logic and continuity of the text. At the same time, the model can also identify and repair garbled characters in the text, automatically fill in the missing content. This pre - processing process significantly improves the quality of unstructured documents in the knowledge base, making subsequent retrieval and generation more accurate. In addition, this method also optimizes the document segmentation and index construction to ensure the integrity of the retrieved content, overcoming the problems of excessive manual intervention and low efficiency in the traditional OCR pre - processing process. Brief Description of the Drawings

[0036] Figure 1 It is a flowchart of the operation of the large model archive question - answering system based on the fine - tuning of double - parameter sparse regularization parameters provided by the embodiments of the present invention. Detailed Embodiments

[0037] In order to make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] As Figure 1 shown, it is a flowchart of the operation of the large model archive question - answering system based on the fine - tuning of double - parameter sparse regularization parameters provided by the embodiments of the present invention. The system includes:

[0039] A query generation module, configured to convert the question input by the user into semantic query data through natural language processing and generate semantic vectors.

[0040] In this system, the query generation module is responsible for converting the question input by the user into a semantic query. Through natural language processing technology, the system can accurately analyze the query intention of the user and generate high - quality semantic vectors. These vectors provide support for subsequent knowledge base retrieval, ensuring the relevance and accuracy of the retrieval.

[0041] A knowledge base retrieval module, configured to extract corresponding documents from the local knowledge base, perform retrieval based on semantic matching, and filter out the content related to the user's query.

[0042] In this system, the knowledge base retrieval module is used to extract highly relevant documents from the local knowledge base. Based on the retrieval method of semantic matching, the system can quickly screen the file content closely related to the user's query, optimizing the accuracy and coverage of the results.

[0043] The generation enhancement module is used to take the retrieved documents as context information, combine the context semantics with the user's query content, and output answers.

[0044] In this system, the generation enhancement module uses the retrieved relevant documents as context information to further improve the accuracy and coherence of the large model's generated answers. By combining the context semantics with the user's query content, the system can output answers that better meet the actual needs.

[0045] The optimization module is used to expand the semantic coverage of the knowledge base, parse picture content, preprocess OCR documents, and fine-tune the large model.

[0046] In this system, the optimization module further improves the system performance through various technical means, including: expanding the semantic coverage of the knowledge base through synonym replacement, increasing the matching flexibility of queries; parsing the pictures in the files through the picture large model and converting them into available text information and knowledge graphs; intelligent preprocessing of OCR documents in the files, repairing and optimizing unstructured text, improving the integrity and quality of the knowledge base; fine-tuning the pre-trained large model through sparse LoRA fine-tuning, effectively reducing the computational cost and storage requirements of the model.

[0047] In the embodiments of the present invention, the construction of the thesaurus and the optimization of synonym replacement: To expand the semantic coverage of the knowledge base, the present invention constructs a domain-specific thesaurus to optimize the expansion ability of queries. First, using natural language processing tools such as NLTK and spaCy, combined with professional dictionaries and industry glossaries, a set of synonym mapping relationships containing domain-specific vocabulary is generated. This thesaurus provides basic support for subsequent semantic expansion. In specific applications, the user's query input will undergo the processing of synonym replacement and semantic expansion to generate a set of expanded query sets. For example, by defining the expansion function

[0048] Expand(Q) = {Q, Q (1) , Q (2) , …, Q (k)},

[0049] a series of semantic variants Q (k)These variants achieve multi-dimensional semantic expression of the query through synonym replacement. To ensure the effectiveness and relevance of the expansion results, the expanded query is further optimized through multiple rounds in combination with domain corpus, and invalid expansion variants are removed using semantic similarity filtering methods, thus ensuring the accuracy and applicability of the final query results. This method significantly improves the semantic coverage of the knowledge base and provides users with more comprehensive and accurate query services.

[0050] In the embodiment of the present invention, the working process of the system includes picture content processing and knowledge graph construction: for the picture data in the archives, a picture content processing and knowledge graph generation process is added. For the picture data in the archives, a systematic picture content processing and knowledge graph generation process is constructed to achieve effective structuring and semanticization of picture information. First, in the picture text extraction link, an advanced picture large model (such as BLIP) is used to extract text information and identify scene content from the archive pictures. For example, for an organizational chart, the model can extract the names of each department, position information, and hierarchical relationships, and output this information in text form. This process not only extracts the explicit text information in the picture but also captures the implicit semantics in the image in combination with scene recognition technology, such as the relationship between an icon and its association.

[0051] Next, in the semantic parsing and mapping stage, the extracted text content and picture labels are further mapped to the nodes and relationships in the knowledge graph to form a complete structured data representation. The knowledge graph can be represented by G=(V,E), where V={v1,v2,...} represents knowledge nodes and E={e i,j} represents the relationships between nodes. For example, for the information extracted from the organizational chart, "Sales Department" and "Technical Department" can be mapped to nodes v1 and v2, and "The Sales Department reports to the Technical Department" can be represented as the relationship e 1,2 between the nodes. This mapping can convert picture information into structured data that can be understood by a computer, laying a foundation for subsequent analysis and processing.

[0052] Finally, in the multi-modal fusion stage, the picture information and relevant text content in the archives are embedded into a unified semantic space. Through the unification of semantic representation, the accuracy of retrieval and question answering is further improved. For example, by combining the information extracted from the organizational chart with the descriptions in the archive text, questions such as "What department is the direct superior of the Sales Department?" or "Who is the person in charge of the Technical Department?" can be answered. Multi-modal fusion makes full use of the complementarity of the two information sources of pictures and text, providing higher accuracy and reliability for the intelligent processing of archives and knowledge services.

[0053] In one embodiment of the present invention, for the OCR document quality problem, the present invention adds an intelligent preprocessing process to effectively solve problems such as chaotic text structure, garbled characters, and missing content, and finally generate a formatted document and optimize the content of the knowledge base.

[0054] First, in the model preprocessing stage, the advanced qwen2-7b-instruct model is used to intelligently repair the original OCR text. This model can identify and repair garbled characters and missing content caused by OCR errors. For example, the original OCR document may extract such content: "Th1s is a t3st d0cum3nt w1th err0rs", and after being repaired by the model, it can be restored to the correct text: "This is a

[0055] test document with errors". In addition, for paragraphs with text misalignment or logical incoherence caused by typesetting errors, the model can also make reasonable adjustments through context reasoning to restore the normal logical order and semantic integrity of the text.

[0056] Next, in the redundancy deletion and structuring stage, the formatting process of the text is further optimized. A large number of meaningless line breaks, blank lines, or duplicate contents generated during the OCR extraction process are removed. For example, a certain text may be wrongly divided into multiple lines: "This is\na text\nwith broken lines.", and after preprocessing, it will be reconstructed as: "This is atext with broken lines.". At the same time, for lists, tables, or hierarchical structures in complex documents, the paragraphs are re-divided and typeset in a regular way to make the document more readable and logical.

[0057] Finally, in the knowledge base optimization stage, the index system of the knowledge base is rebuilt based on the preprocessed text content, thereby improving the accuracy of document retrieval. For example, after OCR extraction, a meeting record document has key information such as the meeting theme, participant list, and key decisions that cannot be effectively indexed due to garbled characters and paragraph chaos. After the preprocessing is completed, this information will be accurately extracted and structured and stored in the knowledge base, enabling users to quickly retrieve the target content through keyword or natural language queries.

[0058] Through this preprocessing process, not only is the text quality of the OCR document significantly improved, but also a foundation is laid for subsequent knowledge base applications, providing an efficient solution for intelligent document processing.

[0059] In a preferred embodiment of the present invention, in order to further improve the answer quality of the file large model, on the basis of the above processing, the present invention proposes a sparse LoRA fine-tuning technique. This technique is a lightweight fine-tuning method for large models, aiming to reduce the computational cost and storage requirements of model fine-tuning while improving the adaptability to specific tasks. Specifically, the present invention introduces a two-parameter L2,1 norm regularization on the basis of the traditional LoRA method to generate a sparse low-rank matrix, thereby improving the efficiency and sparsity of fine-tuning.

[0060] When fine-tuning large language models (such as GPT), traditional methods usually update large-scale full model parameters, which leads to the following problems: high computational overhead: the scale of model parameters is huge, and full model fine-tuning requires huge computational resources and storage space; redundant updates: some weights have little impact on specific tasks but are still updated, resulting in waste of resources; insufficient generalization ability: dense weight updates may lead to overfitting of the model on specific tasks. Therefore, the present invention combines the LoRA technique to make the fine-tuning process more efficient while maintaining the performance and generalization ability of the model by restricting the sparsity of the updated matrix.

[0061] The mathematical model of sparse LoRA is as follows:

[0062] Assume that the weight matrix of the model required is W ∈ R d×k , in the LoRA method, the fine-tuned weight is expressed as:

[0063] W LR = W + ΔW, ΔW = AB

[0064] where A ∈ R d×r , B ∈ R r×k are real matrices of d×r and r×k respectively, and d, r, and k are integers, representing the number of rows and columns of the matrices, and r << min(d,k). ΔW is a low-rank fine-tuning matrix, which greatly reduces the number of parameters to be updated.

[0065] To reduce unnecessary update amounts, the present invention introduces a two-parameter L2,1 norm regularization to constrain the sparsity in ΔW. Its regularization objective function is:

[0066]

[0067] where λ A and λ B are regularization coefficients that control the sparsity intensity, A i,j and B i,j are the elements of the i-th row and j-th column of matrix A and matrix B respectively. This regularization term sparsifies the L2 norm of each row, ensuring that non-zero values are only retained on key feature dimensions.

[0068] Combining the task objective loss and the regularization term, the overall optimization objective is:

[0069] L = Ltask + Lreg

[0070] Where: Ltask is the cross - entropy loss function for the file Q&A task, which is used to ensure the accuracy of the model's answers.

[0071] The implementation steps of sparse LoRA fine - tuning are as follows:

[0072] (1) Initialize the low - rank matrix

[0073] Randomly initialize matrices A and B, and set the sparse regularization coefficients λ A and λ B to a small value to balance model performance and sparsity;

[0074] (2) Sparse matrix update

[0075] Through iterative optimization, gradually sparsify the unimportant parameters of the matrix. Each iteration includes the following steps:

[0076] Calculate the gradient: Calculate the gradient according to the task loss function

[0077] Low - rank matrix update:

[0078]

[0079] Where η is the learning rate.

[0080] Sparse matrix truncation: Apply the sparsification rule to the updated matrix ΔW, and set the elements less than the threshold ∈ to zero: A i,j = 0 if |A i,j |<∈; B i,j = 0 if |B i,j |<∈.

[0081] Normalization operation: Ensure that the row norm of the matrix ΔW remains within a certain range to avoid the problems of gradient explosion or vanishing.

[0082] (3) Convergence detection: Repeat the above steps until the loss function L converges, that is, the change in the loss value is lower than the preset threshold.

[0083] Through sparse LoRA fine - tuning, the present invention has achieved the following performance improvements: 1. Improved computational efficiency: Compared with full - model fine - tuning, sparse LoRA only updates about 1% of the parameters, and the training speed is significantly improved; 2. Reduced memory occupancy: Due to the sparsification of the parameter matrix, the storage requirement is reduced, and it can run on resource - constrained devices. 3. No performance loss: In the knowledge - base retrieval - enhanced generation task, the Q&A performance of the sparse LoRA fine - tuned model is comparable to that of the full - parameter fine - tuned model, but with higher efficiency.

[0084] Through the combination of synonym replacement expansion, picture information extraction, and knowledge graph construction, the present invention achieves comprehensive coverage of multimodal archives. By using the synonym replacement technology, the semantic coverage of the knowledge base is significantly expanded, thus enhancing the diversity and accuracy of retrieval results. Through a picture large model (such as BLIP), key information in pictures, such as charts and handwritten records, is extracted and transformed into text embeddings for unified processing with the text content. Further, through the construction of the knowledge graph, the semantic relevance of archive information is made explicit, enabling the system to accurately understand the context relationship in the archive content. This multimodal information fusion method overcomes the problem of insufficient coverage of unimodal data in traditional methods, resulting in a wider retrieval coverage and more accurate results.

[0085] Aiming at the common problems of garbled characters, information loss, and structural chaos in OCR documents, the present invention adopts the qwen2-7b-instruct model for intelligent preprocessing. Through this model, the system can automatically clean redundant line breaks and reconstruct the paragraph structure, thereby enhancing the logic and continuity of the text. At the same time, the model can also identify and repair garbled characters in the text and automatically fill in the missing content. This preprocessing process significantly improves the quality of unstructured documents in the knowledge base, making subsequent retrieval and generation more accurate. In addition, this method also optimizes the document segmentation and index construction to ensure the integrity of the retrieved content, overcoming the problems of excessive manual intervention and low efficiency in the traditional OCR preprocessing process.

[0086] Based on the LoRA (Low-Rank Adaptation) technology, the present invention introduces the dual-parameter L2,1-norm regularization to further optimize the fine-tuning process. The traditional full-parameter fine-tuning method has a high computational cost and is prone to overfitting. However, the present invention enables the localization adaptation of large models by sparsifying the low-rank matrix, requiring only a very small number of parameters to be updated. Specifically, by regularizing the weight matrix with the L2,1 norm, the sparsity of the parameters can be effectively controlled, retaining important features and suppressing redundant features, thereby reducing the storage requirements and computational complexity of the model. For example, for domain-specific terms that are frequently used, the sparse LoRA method can assign a lower regularization intensity to ensure that these key features are retained, while for noisy data, a higher regularization intensity can be assigned to significantly weaken its impact. This method not only greatly reduces the amount of parameter updates but also performs equivalently to the full-parameter fine-tuning method in the question-and-answer generation task, overcoming the problems of high cost and low efficiency in traditional fine-tuning.

[0087] The embodiments described above merely represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation to the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the appended claims.

[0088] The above is only the preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A large model file question answering system based on dual-parameter sparse regularization parameter fine-tuning, characterized in that, The system includes: A query generation module, which is used to convert the questions input by users into semantic query data through natural language processing and generate semantic vectors; A knowledge base retrieval module, which is used to extract corresponding documents from the local knowledge base, retrieve based on semantic matching, and screen out the content related to the user's query profile; A generation enhancement module, which is used to use the retrieved documents as context information, combine the context semantics with the user's query content, and output answers; An optimization module, which is used to expand the semantic coverage of the knowledge base, parse the content of pictures, preprocess OCR documents, and fine-tune large models.

2. The large model file question answering system based on dual-parameter sparse regularization parameter fine-tuning according to claim 1, characterized in that During the process of expanding the semantic coverage of the knowledge base, a domain-specific thesaurus is constructed, and the synonym mapping relationship is generated through natural language processing tools, professional dictionaries, and industry glossaries. The user's query input generates an extended query set through synonym replacement and semantic expansion.

3. The large model file question-answering system based on dual-parameter sparse regularization parameter fine-tuning according to claim 2, wherein Define an extended function: Expand(Q) = {Q, Q (1) , Q (2) , …, Q (k)} Obtain a series of semantic variants Q of the original query Q (k) .

4. The large model file question-answering system based on dual-parameter sparse regularization parameter fine-tuning according to claim 1, characterized in that, In the steps of parsing the content of pictures, in the picture text extraction link, a picture large model is used to extract the text information and identify the scene content of the archive pictures. In the semantic parsing and mapping stage, the extracted text content and picture labels are further mapped to the nodes and relationships in the knowledge graph to form a complete structured data representation. In the multi-modal fusion stage, the picture information and the relevant text content in the archive are embedded into a unified semantic space.

5. The large model archive question-answering system based on dual-parameter sparse regularization parameter fine-tuning according to claim 3, wherein The knowledge graph is represented by G=(V, E), where V={v1, v2, …} represents knowledge nodes, and E={e i,j} represents the relationships between nodes.

6. The large model archive question-answering system based on dual-parameter sparse regularization parameter fine-tuning according to claim 1, characterized in that During the preprocessing of OCR documents, in the model preprocessing stage, the qwen2-7b-instruct model is used to intelligently repair the original OCR text; in the redundancy deletion and structuring stage, the formatting process of the text is optimized, and meaningless line breaks, blank lines, or duplicate content generated during the OCR extraction process are cleared; In the knowledge base optimization stage, the index system of the knowledge base is reconstructed based on the preprocessed text content.

7. The large model file Q&A system based on dual-parameter sparse regularization parameter fine-tuning according to claim 6, characterized in that In the steps of fine-tuning the large model, sparse LoRA fine-tuning is adopted, and the mathematical model of sparse LoRA is: Suppose it is necessary to fine-tune the model weight matrix \(W\in\mathbb{R}\) d×k The fine-tuned weights are represented as: W LR = W + ΔW, ΔW = AB where \(A\in\mathbb{R}\) d×r , \(B\in\mathbb{R}\) r×k is a low-rank real matrix, \(d\), \(r\), and \(k\) are integers representing the number of rows and columns of the matrix, \(r\ll\min(d,k)\), and \(\Delta W\) is a low-rank fine-tuning matrix.

8. The large model file Q&A system based on dual-parameter sparse regularization parameter fine-tuning according to claim 7, characterized in that, Introduce the double-parameter L2,1 norm regularization to constrain the sparsity in ΔW, and its regularization objective function Lreg is defined as: Among them, λ A and λ B are regularization coefficients used to control the sparsity intensity. A i,j and B i,j are the elements in the i-th row and j-th column of matrix A and matrix B respectively.

9. The large model archive question answering system based on dual-parameter sparse regularization parameter fine-tuning according to claim 8, characterized in that, Combining the task objective loss and the regularization term, the overall optimization objective is: L = Ltask + Lreg Where: Ltask is the cross-entropy loss function of the archive question-answering task.

10. The large model file question answering system based on dual-parameter sparse regularization parameter fine-tuning according to claim 7, characterized in that, The implementation steps of sparse LoRA fine-tuning include: Initialize the low-rank matrix: randomly initialize matrices A and B, and set the sparse regularization coefficients λ A and λ B to a preset value; Sparsification matrix update: Through iterative optimization, gradually sparsify the preset parameters of the matrix. Each round of iteration includes the following steps: Calculate the gradient: Based on the task loss function L, calculate the elements A i,j and B i,j in matrices A and Element update of matrix A and B: Where η is the learning rate; Sparse matrix truncation: Apply the sparsification rule to the updated matrix ΔW, setting elements less than the threshold ∈ to zero: A i,j = 0, if |A i,j | < ∈; B i,j = 0, if |B i,j | < ∈; Normalization operation: Ensure that the row norm of matrix ΔW remains within the preset range; Convergence detection: Repeat the execution until the loss function L converges.

Images