An optimization method for a fundamental large-scale model applied in the legal field

By integrating external knowledge bases and optimizing the LangChain+Streamlit+urban safety big data model, the accuracy and adaptability issues of human-computer interaction in the legal field were solved, enabling intelligent response services in urban safety and administrative review, and improving the model's response accuracy and robustness.

CN118153714BActive Publication Date: 2026-11-17TIANJIN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410339246.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-25
Publication Date
2026-11-17
Estimated Expiration
2044-03-25

AI Technical Summary

Technical Problem

Existing technologies struggle to provide highly accurate and adaptable human-computer interaction solutions in the legal field, especially in areas such as urban security and administrative review, as they cannot effectively handle complex legal language and rapidly changing legal environments.

Method used

We construct a large language model that integrates external knowledge bases, and deploy it locally using LangChain+Streamlit+city safety large model. We combine RLHF-human feedback reinforcement learning and sparse fine-tuning strategies to optimize question-answer pair knowledge retrieval and model parameters. We also utilize multi-level retrieval mechanisms and vector indexing technology for data processing and model fine-tuning.

Benefits of technology

It improves the model's response accuracy and robustness in urban safety and legal fields, enabling it to handle complex legal tasks, adapt to changing legal environments, and provide intelligent response services.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118153714B_ABST
    Figure CN118153714B_ABST
Patent Text Reader

Abstract

This invention discloses an optimization method for a fundamental large-scale model applied to the legal field. It employs a LangChain + Streamlit + urban safety large-scale model for localized deployment, providing intelligent response services. The main components include data collection, data processing, question-answer pair knowledge retrieval optimization, parameter fine-tuning, and model optimization. Fine-tuning the large-scale model improves its robustness in addressing legal and urban safety challenges. By adding new tasks and data to the model, it can better adapt to complex and ever-changing legal environments and urban safety challenges. The model is streamlined and optimized according to specific application scenarios to ensure efficient operation in the legal and urban safety fields.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the fields of deep learning and natural language processing, specifically relating to an optimization method for a foundational large-scale model applied to the legal field. The aim is to train models to understand and generate natural language, enabling them to perform various tasks such as dialogue, answering questions, writing text, and translating languages. Furthermore, we hope to train and optimize large-scale language models in specific domains (administrative review and urban security) or industries. Compared to general-purpose language models, vertical domain-specific large-scale models are more focused on knowledge and skills within a particular domain, possessing higher domain specialization and practicality. Background Technology

[0002] The legal field, due to its inherent importance and the requirement for accuracy, is one that necessitates specialized research and development of specific legal models. As a crucial component of urban safety and administrative review, law plays a key role in shaping society, regulating human interaction, and upholding justice. Legal professionals rely on accurate and up-to-date information to make informed decisions, interpret the law, and provide legal advice. The complexity of legal language, the nuanced interpretations, and the constantly evolving nature of legislation present unique challenges, requiring tailored solutions. Summary of the Invention

[0003] The purpose of this invention is to provide an optimization method for a fundamental large-scale model applied in the legal field. Based on the above problems, we construct a large language model integrating an external knowledge base to extract legal feature words from users' everyday language. This model identifies words with legal significance, thereby effectively recognizing and analyzing the legal context in user input. It is used to handle various scenarios, such as legal topic extraction, legal consultation, and question answering. Simultaneously, we employ a LangChain + Streamlit + urban safety large-scale model approach for localized deployment, providing intelligent response services for urban safety issues. Based on the effectiveness of actual applications (customer feedback), the model is optimized and adjusted to further improve its accuracy and reliability.

[0004] For a given paper dataset, we first collect relevant data, then clean the data to build question-answer pairs and a knowledge base, select an appropriate fine-tuning strategy (RLHF – Human Feedback Reinforcement Learning), define an appropriate loss function, set a series of fine-tuning parameters, train the model to minimize the loss term, update the parameters, and finally evaluate and optimize the fine-tuned model.

[0005] This invention specifically includes the following steps: an optimization method for a basic large model applied in the legal field, which adopts LangChain+Streamlit+urban security large model for localized deployment and provides intelligent response services;

[0006] Includes the following steps:

[0007] S1 Data Collection

[0008] S2 Data Processing: Organize and preprocess the collected data, including data filtering, data cleaning, knowledge base construction and knowledge retrieval, and further construct question-answer pairs;

[0009] The knowledge base integrates two construction techniques: inverted index and vector index; it adopts an efficient and redundant multi-level recall mechanism; it performs distributed parallel inference operations by splitting documents and using a vector model with multi-node elastic scaling capabilities; and finally, it uses the KNN algorithm for index preheating and construction.

[0010] S3 Question Answering Optimization for Knowledge Recall

[0011] S4 defines the goal and strategy: consider partial fine-tuning, as well as the level and scope of fine-tuning, and consider using a new dataset for fine-tuning, using RLHF-human feedback reinforcement learning as the strategy to complete the fine-tuning process.

[0012] The fine-tuning task is formulated as an RL problem, where the policy is a language model LM that accepts prompts and returns a series of texts or a probability distribution of texts. The reward function RM is a combination of a preference model and a policy transition constraint.

[0013] Specifically, this strategy model can be based on the generated partial sequence x0,x1,…,x k-1 Predict the next text unit x k The probability distribution, where 0≤k <n;

[0014] Given a vocabulary of length n, x0, x1, ..., xn, ... n-1 The joint probability can be represented by the language model ρ as: ρ(x0 … x n-1 )=∏ 0≤k<n ρ(x k |x0 … x k-1 )

[0015] Here, the input space X is defined as the set of all articles with a length not exceeding m, i.e., X = Σ ≤m The output space Y is the summation set of all elements of length n, i.e., Y = Σ n Therefore, the probability of generating summary y from article x is expressed as:

[0016] First, the policy is initialized as π = ρ. Then, the policy π is updated using the PPO algorithm, and the reward function is defined as r. The expected value of the reward can be expressed as:

[0017]

[0018] Learning loss:

[0019] The initial LM and the currently fine-tuned LM are given as inputs, and the output texts are obtained respectively. The text from the current policy is passed to the RM to obtain a scalar reward. The generated texts of the two models are compared to calculate the loss function for the difference, which is usually designed as a scaling of the (KL) divergence between the output word distribution sequences, i.e.:

[0020] r = r θ -βr KL

[0021] in This is used to penalize the RL strategy for generating models that deviate significantly from the initial model in each training batch, in order to prevent the model from deviating too much from the initial model, thereby maintaining the consistency and reasonableness of the model output;

[0022] S5 parameter fine-tuning training: Set hyperparameters and determine the hyperparameters during the fine-tuning process; initialize model parameters and initialize the parameters of the fine-tuned model according to the weights of the pre-trained model; for partial fine-tuning, only the parameters of the top layer or a few layers will be randomly initialized; perform fine-tuning training, train the model using the prepared dataset and fine-tuning strategy, and gradually adjust the model parameters to minimize the loss function according to the set hyperparameters and optimization algorithm during the training process.

[0023] S6 model tuning

[0024] After the fine-tuning phase, the final adjusted model is comprehensively evaluated using an independent test set. A sparse fine-tuning method is used to optimize the model performance. A set of parameter indices and an incremental array between them and the pre-trained values ​​are carefully designed to achieve the required parameter update density control.

[0025] Furthermore, the S3 question answering method described in this invention optimizes knowledge retrieval as follows: First, during the knowledge base construction process, a large-scale language model (LLM) is used to enhance and expand the existing knowledge corpus, generating new FAQ-style question answering data; at the same time, by combining two strategies—QQ question-question symmetric retrieval and QD question-document asymmetric retrieval—knowledge in the form of FAQs and original documents is unified, and its retrieval effect is optimized.

[0026] Next, the vector model will be fine-tuned using the expanded corpus sets of QQ and QD. In practical applications, new knowledge corpus and user feedback data will be continuously introduced to iteratively update the training set, and the vector model will be fine-tuned regularly. As training data accumulates, the model will be able to cover more diverse user questions.

[0027] A multi-path recall strategy is adopted, supplemented by Rerank technology to optimize the ranking of recall results.

[0028] Furthermore, in the sparse fine-tuning strategy of this invention, the active parameter index is updated incrementally, outdated indexes are removed, and new indexes are added alternately. The removal of indexes is based on the changes in parameter amplitude between training iterations, while the addition of new indexes is determined based on gradient information calculated instantaneously. Gradient accumulation technology is introduced to effectively reduce gradient variance.

[0029] Beneficial effects:

[0030] This method aims to address the challenges facing my country's current urban safety and risk assessment system, including an incomplete legal and standards framework, a lack of rigorous legal basis for risk assessment methods, and insufficient legal integration of urban safety management systems. By deeply integrating large-scale model calculation principles and complex systems legal thinking, it innovatively constructs a comprehensive quantitative assessment mechanism for urban safety risks with a legal foundation. By overcoming key legal and technological barriers, such as the legal collection and identification of urban safety risk information across time and space, the effective integration and application of cross-domain data and legal knowledge, and comprehensive assessment that dynamically adapts to changes in the legal environment, this business platform not only achieves intelligent and compliant operation of urban safety risk monitoring but also creatively constructs a new business model of "intelligent risk monitoring and a comprehensive overview of urban safety." This model strictly adheres to legal requirements, possesses replicability and universality for nationwide implementation, and provides strong technical support and decision-making assistance for advancing the rule of law in urban safety in my country.

[0031] Fine-tuning a large model can significantly improve its performance in tasks related to law and urban safety. It enhances the model's robustness in these areas. By adding new tasks and data, the model can better adapt to complex and ever-changing legal environments and urban safety challenges. The model can be streamlined and optimized according to specific application scenarios to ensure efficient operation in these fields. In this way, even complex tasks and data can be processed through a concise and effective model, providing better support and assistance for legal and urban management. Attached Figure Description

[0032] Figure 1 This provides a comprehensive legal model and fine-tuning framework for administrative reconsideration. Detailed Implementation

[0033] The following is combined Figure 1 The present invention will now be further described.

[0034] First, data collection was conducted. A large amount of raw legal data was collected, including legal news, social media content, and discussions from legal industry forums. These sources provided a variety of real-world legal texts, offering insights into a wide range of legal topics and discussions. The dataset was then built upon laws, regulations, and judicial interpretations to ensure comprehensive coverage of legal knowledge. This ensured that the dataset reflected the legal framework and provided accurate and up-to-date information. Real-world legal consultation data was also extracted: existing legal consultation datasets were utilized to retrieve real-world legal consultation data. This enabled the inclusion of real-world legal scenarios and issues frequently encountered by users, enriching the dataset with practical legal examples.

[0035] Secondly, data processing. The collected data is organized and preprocessed, including data filtering and cleaning. To address issues such as the model's lack of evidence and the presence of illusions in legal professional fields, a knowledge base is built and knowledge retrieval is performed to further construct question-answer pairs.

[0036] The knowledge base integrates two advanced construction technologies: inverted index and vector index, which play a decisive role in knowledge retrieval, a key step in the knowledge question-and-answer process.

[0037] Unlike ordinary document or log indexing methods, the vectorization process of knowledge relies heavily on the semantic understanding capabilities of deep learning models, especially for large-scale document collections of millions or even tens of millions of documents. In our design, we adopted an efficient and redundant multi-level recall mechanism, meaning that the corresponding knowledge entries could reach hundreds of millions in scale.

[0038] Given the massive scale of the offline computing task, the entire construction process must achieve high concurrency; otherwise, it will be unable to meet the demands of continuous knowledge content addition and iterative optimization of vector retrieval performance. To address this, we employ document-splitting parallel processing and utilize a vector model with multi-node elastic scaling capabilities (e.g., Endpoint / Endpoint server / client batch inference functionality) for distributed parallel inference operations. Finally, we use the KNN algorithm for index pre-warming and construction, effectively accelerating the knowledge base construction process based on vector data across large-scale document sets, ensuring both rapid and complete construction.

[0039] Question answering optimization for knowledge retrieval: In the process of building the knowledge base, we found that in the inverted index retrieval practice of the FAQ knowledge corpus in the domain, we discovered two common scoring biases through the above analysis methods: the IDF distortion problem of private domain documents and the lack of keyword matching.

[0040] Our proposed solution is summarized as follows: First, we enhance and expand the existing knowledge corpus using a large-scale language model (LLM) to generate new FAQ-style question-and-answer data. Simultaneously, we combine two strategies—QQ (question-to-question symmetric recall) and QD (question-to-document asymmetric recall)—to unify knowledge in the form of FAQs and original documents, and optimize their retrieval performance.

[0041] Next, we fine-tune the vector model using corpora augmented with QQ and QD. This significantly improves model performance on the existing training dataset and demonstrates some generalization ability on the test set. In practical applications, we can continuously introduce new knowledge corpora and user feedback data to iteratively update the training set and periodically fine-tune the vector model. As training data accumulates, the model will be able to cover more diverse user questions. To improve the breadth and accuracy of the knowledge question answering system, we will adopt a multi-path recall strategy, supplemented by Rerank technology to optimize the ranking of recall results, thereby fully utilizing the advantages of multi-source information.

[0042] We constructed the final data question-answer pairs as follows:

[0043] Question: How does the law protect the safety of urban residents?

[0044] Answer: The law protects the personal and property safety of urban residents by stipulating and enforcing various laws and regulations, such as the Criminal Law and the Public Security Administration Law. Furthermore, the law defines the responsibilities of law enforcement agencies such as the police, ensuring they can respond to various security threats promptly and effectively.

[0045] Question: What are the main contents of laws concerning urban safety?

[0046] Answer: Laws concerning urban safety cover multiple areas, including but not limited to criminal law, public security administration law, and traffic management law. These laws stipulate penalties and sanctions for various illegal acts, as well as relevant urban management regulations, such as building safety and environmental protection.

[0047] Next, we define the objectives and strategies: based on the question-answering task requirements and available computational resources, we select an appropriate fine-tuning strategy. We consider partial fine-tuning, as well as the level and scope of fine-tuning. We also consider fine-tuning using a newly added dataset, employing RLHF (Human Feedback Reinforcement Learning) as the strategy to complete the fine-tuning process.

[0048] RLHF – Human Feedback Reinforcement Learning

[0049] When we formalize the fine-tuning task as a reinforcement learning (RL) problem, the policy is first a language model (LM) that accepts a prompt and returns a sequence of texts (or a probability distribution of the texts). The action space of this policy is all the lexicons corresponding to the vocabulary of the language model (typically on the order of 50k), and the observation space is a relatively large sequence of possible input lexicons (vocabulary size ^ number of input tokens). The reward function (RM) is a combination of a preference model and policy transition constraints. Specifically, this policy model can adjust the response based on the generated partial sequence x0, x1, ..., x... k-1 Predict the next text unit x k The probability distribution, where 0≤k <n。

[0050] Given a vocabulary of length n, x0, x1, ..., xn, ... n-1 The joint probability can be represented by the language model ρ as:

[0051]

[0052] Here, the input space X is defined as the set of all articles with a length not exceeding m, i.e., X = Σ ≤m The output space Y is the summation set of all elements of length n, i.e., Y = Σ n Therefore, the conditional probability of generating a corresponding summary y∈Y from an article x∈X can be expressed as:

[0053] The initial policy π is equivalent to the original language model ρ, and then the proximal policy optimization (PPO) algorithm is used to progressively update the policy π. The reward function is defined as r, whose expected value represents the expectation of the joint distribution of articles x and corresponding policy-generated summaries y drawn from the sample dataset D under the current policy.

[0054]

[0055] Learning loss:

[0056] The initial LM and the currently fine-tuned LM are given as inputs, and the output texts are obtained respectively. The text from the current policy is passed to the RM to obtain a scalar reward. The generated texts of the two models are compared to calculate the loss function for the difference, which is usually designed as a scaling of the (KL) divergence between the output word distribution sequences, i.e.:

[0057] r = r θ -βr KL

[0058] in It quantifies the reinforcement learning strategy π RLThe probability distribution of generating text y given input x and the strategy π before fine-tuning. SFT Compared to the degree of change, this term is used to penalize the RL policy for generating significantly different versions of the initial model in each training batch, in order to prevent the model from deviating excessively from the initial model and thus maintain the consistency and reasonableness of the model output. Removing this penalty term might cause the model to be more inclined to try unreasonable or nonsensical strategies to obtain higher reward values, which could harm the model's performance and output quality.

[0059] Parameter fine-tuning training

[0060] Setting Hyperparameters: Determine the hyperparameters used in the fine-tuning process, such as the learning rate, batch size, and number of training epochs. The choice of these hyperparameters significantly impacts the performance and convergence speed of the fine-tuning. Initializing Model Parameters: Initialize the parameters of the fine-tuned model based on the weights of the pre-trained model. For partial fine-tuning, only the parameters of the top layer or a few layers are randomly initialized. Performing Fine-Tuning Training: Train the model using the prepared dataset and fine-tuning strategy. During training, gradually adjust the model parameters to minimize the loss function based on the set hyperparameters and optimization algorithm.

[0061] Model tuning

[0062] After the fine-tuning phase, we comprehensively evaluate the final adjusted model using an independent test set. Given the extremely large number of parameters involved in the fine-tuning process, traditional full-scale fine-tuning strategies face challenges. Therefore, we innovatively propose a sparse fine-tuning method to optimize model performance. This method achieves the desired parameter update density control by carefully designing a set of parameter indices and an incremental array between them and the pre-trained values.

[0063] In this sparse fine-tuning strategy, the updates of active parameter indices, the removal of outdated indices, and the growth of new indices are performed alternately. Index removal is based on the changes in parameter magnitudes between training iterations, while the addition of new indices is determined based on instantaneously calculated gradient information. To further enhance the stability and convergence of the algorithm, we introduce gradient accumulation techniques to effectively reduce gradient variance.

[0064] Although essentially employing sparse fine-tuning, this method cleverly extends to basic large language models, producing efficient models with dense update capabilities. This avoids tedious and complex sparse tensor operations while combining the SpIEL (Sparse Index Learning) strategy with advanced optimizers such as SM3. Considering the potential for high variance and memory consumption in mini-batch gradients for individual samples, leading to unreliable gradient estimation, we use momentum approximation as an alternative criterion for parameter growth decisions, effectively improving the model's generalization ability and fine-tuning efficiency.

Claims

1. An optimization method for a fundamental large-scale model applied in the legal field, characterized in that, It adopts the approach of LangChain + Streamlit + city security big model for local deployment, and provides intelligent response services; Includes the following steps: S1 Data Collection S2 Data Processing: The collected data is organized and preprocessed, including data filtering, data cleaning, knowledge base construction and knowledge retrieval, and further question-answer pairs are constructed. The knowledge base integrates two construction techniques: inverted index and vector index; it adopts an efficient and redundant multi-level recall mechanism; it performs distributed parallel inference operations by splitting documents and using a vector model with multi-node elastic scaling capabilities; and finally, it uses the KNN algorithm for index preheating and construction. S3 Question Answering Optimization for Knowledge Recall S4 defines the goal and strategy: consider partial fine-tuning, as well as the level and scope of fine-tuning, and consider using a new dataset for fine-tuning, using RLHF-human feedback reinforcement learning as the strategy to complete the fine-tuning process. The fine-tuning task is formulated as an RL problem, where the policy is a language model LM that accepts prompts and returns a series of texts or a probability distribution of texts. The reward function RM is a combination of a preference model and a policy transition constraint. Specifically, this strategy model can be based on the generated partial sequence x0, x1, ..., x... k-1 Predict the next text unit x k The probability distribution, where 0≤k <n; Let the vocabulary be ∑, then given a sequence x0, x1, ..., xn of length n... n-1 The joint probability can be represented by the language model ρ as: ρ(x0…x n-1 )=Π 0≤k<n ρ(x k |x0…x k-1 ) The input space X is defined as the set of all articles with a length not exceeding m, i.e., X = ∑ ≤m The output space Y is the summation set of all elements of length n, i.e., Y = ∑ n ; The probability of generating summary y from article x is expressed as: First, the policy is initialized as π = ρ. Then, the policy π is updated using the PPO algorithm, and the reward function is defined as r. The expected value of the reward can be expressed as: Learning loss: The input is given as an initial LM and the currently fine-tuned LM, and the output text is obtained from each. The text from the current policy is passed to the RM to obtain a scalar reward. The generated texts of the two models are compared to calculate the loss function for the difference, which is usually designed as a scaling of the KL divergence between the output word distribution sequences, i.e.: r=r θ -βr KL in This is used to penalize the RL strategy for generating models that deviate significantly from the initial model in each training batch, in order to prevent the model from deviating too much from the initial model, thereby maintaining the consistency and reasonableness of the model output; S5 parameter fine-tuning training: Set hyperparameters and determine the hyperparameters during the fine-tuning process; initialize model parameters and initialize the parameters of the fine-tuned model according to the weights of the pre-trained model; for partial fine-tuning, only the parameters of the top layer or a few layers will be randomly initialized; perform fine-tuning training, train the model using the prepared dataset and fine-tuning strategy, and gradually adjust the model parameters to minimize the loss function according to the set hyperparameters and optimization algorithm during the training process. S6 model tuning After the fine-tuning phase, the final adjusted model is comprehensively evaluated using an independent test set. A sparse fine-tuning method is used to optimize the model performance. A set of parameter indices and an incremental array between them and the pre-trained values ​​are designed to achieve the required parameter update density control.

2. The optimization method for a fundamental large-scale model applied in the legal field according to claim 1, characterized in that, The S3 question-answering optimization for knowledge retrieval is as follows: First, during the knowledge base construction process, the existing knowledge corpus is enhanced and expanded using a large-scale language model (LLM) to generate new FAQ-style question-answer data. At the same time, it combines two strategies: QQ question-question symmetric retrieval and QD question-document asymmetric retrieval, to achieve knowledge unification in the form of FAQs and original documents and optimize its retrieval performance. Next, the vector model will be fine-tuned using the expanded corpus sets of QQ and QD. In practical applications, new knowledge corpus and user feedback data will be continuously introduced to iteratively update the training set, and the vector model will be fine-tuned regularly. As training data accumulates, the model will be able to cover more diverse user questions. A multi-path recall strategy is adopted, supplemented by Rerank technology to optimize the ranking of recall results.

3. The optimization method for a fundamental large-scale model applied in the legal field according to claim 1, characterized in that, In the sparse fine-tuning strategy, the active parameter index is updated incrementally, outdated indexes are removed, and new indexes are added alternately. The removal of indexes is based on the changes in parameter magnitudes between training iterations, while the addition of new indexes is determined based on the gradient information calculated instantaneously. The gradient accumulation technique is introduced to effectively reduce the gradient variance.

Citation Information

Patent Citations

  • Retrieval method and device, electronic equipment and storage medium

    CN114428834A

  • Text semantic retrieval method in military scene

    CN116150335A