Intelligent bilingual legal question answering method and system based on language model

Abstract

The invention discloses an intelligent bilingual legal question answering method and system based on a language model. Bilingual legal information is collected from a multi-source channel for text cleaning processing to form a corpus set; vectorizing a corpus set by using an N-gram language model, carrying out semantic analysis on the corpus set by using a multilingual model, and adding a semantic analysis result vector into a bilingual law initial vector library; a mapping matrix is calculated for each vector in the library for spatial semantic mapping, and dynamic language alignment is realized by using a cross-language attention mechanism; and taking the Chinese and foreign legal semantic library as a large language model retrieval library, and outputting bilingual legal question and answer answers for legal questions by utilizing a large language model. The advanced cross-language alignment technology, the supervision alignment algorithm and the cross-language attention mechanism are utilized, texts of two languages are mapped to a unified semantic space, it is ensured that the same legal concept can accurately correspond to different languages, and the actual requirements of cross-language legal information retrieval and intelligent question answering are met.

Description

Technical Field

[0001] The present invention relates to the technical field of legal question answering, and in particular to an intelligent bilingual legal question answering method and system based on a language model and a computer storage medium. Background Art

[0002] In recent years, with the rapid development of natural language processing (NLP) and deep learning technologies, large-scale pre-trained language models have demonstrated outstanding performance in various text generation and question-answering tasks. As a highly specialized field, legal texts possess rigorous formatting, rich structured information, and complex semantic relationships. Traditional legal question-answering systems rely primarily on keyword retrieval and rule matching, which suffer from shortcomings such as slow response speed, low matching accuracy, and an inability to fully understand the deep semantics of legal texts. Meanwhile, the recent emergence of RAG (Retrieval-Augmented Generation) technology, which organically combines retrieval technology with generative models, provides a novel approach to solving these problems.

[0003] The core of RAG technology lies in utilizing a retrieval module to search for information relevant to user questions from a large document repository. The retrieved text, along with user input, is then fed into a large language model, ultimately generating accurate and authoritative answers. This approach retains the retrieval system's advantage in efficiently finding factual evidence while leveraging the generative model's deep capabilities in semantic understanding and natural language generation, effectively addressing the shortcomings of traditional systems in terms of legal expertise and accuracy. In the legal field, RAG can quickly locate legal texts, judicial interpretations, and relevant cases, providing users with targeted and well-founded answers. However, current legal question-and-answer systems mostly focus on information retrieval and question-answering in a single language. For example, when searching for Chinese legal information, bilingual search scenarios require translation from Chinese into English before conducting information retrieval. This results in inefficient legal text retrieval and a lengthy search cycle, failing to meet the needs of bilingual users.

[0004] Based on the above technical problems, the applicant proposed the technical solution of this application. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the existing technology, the present invention provides an intelligent bilingual legal question-and-answer method and system based on a language model. By constructing independent Chinese and German legal text vector libraries and performing cross-language alignment, the texts in the two languages ​​are mapped into a unified semantic space, ensuring that the same legal concepts can be accurately mapped in different languages. This creates an efficient, accurate, professional, and dynamically optimized legal question-and-answer platform to meet the practical needs of cross-language legal information retrieval and intelligent question-and-answer. In addition, the present invention can continuously adjust key parameters in accordance with user feedback, achieving continuous self-adaptation and intelligent optimization of the system.

[0006] To achieve the above objectives, the present invention discloses a bilingual legal question answering method based on a language model, comprising the following steps:

[0007] Collecting bilingual legal information from multiple source channels, performing text cleaning on the bilingual legal information, and obtaining a Chinese legal training set and a foreign legal training set;

[0008] Vectorizing the Chinese legal training set and the foreign legal training set using a Chinese N-gram language model and a foreign language N-gram language model constructed using a parameter adaptive algorithm, respectively, to obtain a Chinese legal initial database and a foreign language legal initial database;

[0009] Using a pre-trained multilingual model, semantic analysis is performed on the Chinese legal corpus and the foreign legal corpus, and semantic analysis result vectors are added to the Chinese legal initial vector library and the foreign legal initial vector library respectively;

[0010] For each vector in the Chinese legal initial library and the foreign legal initial library, a mapping matrix is ​​calculated and spatial semantic mapping is performed to obtain a Chinese and foreign legal semantic library. The Chinese and foreign legal semantic library aligns the vectors in the Chinese legal initial library and the vectors in the foreign legal initial library into the same semantic space, and uses a cross-language attention mechanism to achieve dynamic language alignment.

[0011] The Chinese and foreign legal semantic database is used as a retrieval database of a large language model, and the large language model is used to output bilingual legal question-and-answer answers to legal issues.

[0012] Preferably, before using the Chinese and foreign legal semantic database as a retrieval database for a large language model, the method further comprises:

[0013] Identify key information from legal texts and perform text extraction based on preset strategies to form a legal rule text collection, wherein the key information includes legal clause numbers, legal application texts, and legal entity texts;

[0014] The legal rule text collection is used to perform domain adaptive training on a large language model.

[0015] Preferably, the Chinese N-gram language model and the foreign language N-gram model language are constructed by the following steps:

[0016] Two N-gram models are constructed, and the two N-gram language models are used to process the Chinese legal data set and the foreign legal data respectively, so as to form Chinese data and foreign data.

[0017] Preferably, a parameter adaptive algorithm is used for N value selection, specifically: pre-setting a range of candidate N values, building a corresponding N-gram model for each candidate N value in the range, then using a validation set to calculate the perplexity of the N-gram model, using a candidate scoring function to score the perplexity, and taking the candidate N value corresponding to the N-gram model with the highest score as the optimal N value, and using the optimal N value to build an N-gram model for the corresponding language.

[0018] Preferably, the optimal N value of the Chinese N-gram model is 3, the foreign language N-gram model is a German N-gram model, and the optimal N value of the German N-gram model is 4.

[0019] Preferably, the multilingual model is an XLM-R model, and the spatial semantic mapping adopts a Procrustes analysis method.

[0020] The present invention also discloses an intelligent bilingual legal question-answering system based on a language model, comprising the following modules:

[0021] A collection module is used to collect bilingual legal information from multiple source channels, perform text cleaning on the bilingual legal information, and obtain a Chinese legal training set and a foreign legal training set;

[0022] A vectorization module, configured to vectorize the Chinese legal training set and the foreign legal training set respectively using a Chinese N-gram language model and a foreign language N-gram model constructed using a parameter adaptive algorithm, to obtain a Chinese legal initial library and a foreign legal initial library;

[0023] A semantic analysis module, configured to perform semantic analysis on the Chinese legal corpus and the foreign language French corpus using a pre-trained multilingual model, and to add semantic analysis result vectors to the Chinese legal initial vector library and the foreign language legal initial vector library respectively;

[0024] The semantic alignment module is used to calculate a mapping matrix for each vector in the Chinese legal initial library and the foreign legal initial library, perform spatial semantic mapping, and obtain a Chinese and foreign legal semantic library. The Chinese and foreign legal semantic library aligns the vectors in the Chinese legal initial library and the vectors in the foreign legal initial library into the same semantic space, and uses a cross-language attention mechanism to achieve dynamic language alignment; the model retrieval module is used to use the Chinese and foreign legal semantic library as a retrieval library for a large language model, and use the large language model to output bilingual legal question-and-answer answers to legal issues.

[0025] Preferably, the language model-based intelligent bilingual legal question-answering system further includes:

[0026] A text extraction module is used to identify key information from legal texts and perform text extraction according to a preset strategy before using the Chinese and foreign legal semantic library as a retrieval library for the large language model to form a legal rule text collection, wherein the key information includes the legal clause number, legal application text, and legal entity text; and use the legal rule text collection to perform domain adaptive training on the large language model.

[0027] The present invention also discloses an intelligent bilingual legal question-and-answer device based on a language model, which includes a processor, a memory, and a computer program stored in the memory and executable by the processor. When the computer program is executed by the processor, the steps of the above-mentioned intelligent bilingual legal question-and-answer method based on a language model are implemented.

[0028] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned language model-based intelligent bilingual legal question-answering method are implemented.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The language model-based intelligent bilingual legal question-answering method provided by the present invention constructs independent Chinese and German legal text vector libraries and performs cross-language alignment to map the texts in the two languages ​​into a unified semantic space. It also implements dynamic language alignment through bilingual cross-language attention mechanisms, etc., to enhance the alignment effect of a single static global mapping matrix, ensuring that the same legal concepts can be accurately mapped in different languages, and facilitating the construction of an efficient, accurate, professional and dynamically optimized legal question-answering platform to meet the actual needs of cross-language legal information retrieval and intelligent question-answering.

[0031] 2. This invention uses regular expressions, dependency syntax analysis, and named entity recognition to automatically extract key information such as clause numbers and legal applications from Chinese and German legal texts, and combines it with a legal field adaptive algorithm to correct legal terminology, thereby constructing a high-quality hybrid vector representation to ensure the professionalism and accuracy of legal question-and-answer answers.

[0032] 3. The present invention forms a comprehensive representation that combines local statistical features with global semantic understanding by splicing, weighting or attention fusion of n-gram-based statistical vectors and deep semantic vectors extracted from the language model, thereby improving the overall effect of retrieval and question answering.

[0033] 4. The present invention introduces a user feedback mechanism to continuously optimize key parameters of the present invention, such as n-gram model parameters, rule extraction and vector fusion strategy, to achieve continuous adaptation and intelligent optimization to cope with legal text updates and changes in user needs.

[0034] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the language model-based intelligent bilingual legal question-answering method of the present invention. DETAILED DESCRIPTION

[0036] In order to make the technical means, creative features, objectives and effects of the invention easier to understand, the invention is further described below with reference to specific diagrams. However, the invention is not limited to the following implementation cases.

[0037] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings in this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them. They are not used to limit the conditions under which the present invention can be implemented. Therefore, they have no substantive technical significance. Any modification of the structure, change in the proportion relationship or adjustment of the size should still fall within the scope of the technical content disclosed in the present invention without affecting the efficacy and purpose that can be achieved by the present invention.

[0038] "Large Language Model" refers to a large-scale language model (LLM), which is a language model with a large parameter scale. It is designed to be trained with large amounts of text data in order to understand and generate human language. The large language model is used in combination with RAG, and has achieved remarkable results in tasks such as information retrieval and dialogue generation. This technical approach has been widely used in the field of intelligent question answering. Legal texts include fixed structured information, but also require a deep understanding of the context. Through RAG technology, the most relevant regulations and precedents are extracted from the pre-built legal database, and then combined with the large model to generate answers, thereby achieving accurate matching and generation of legal information. However, existing legal question answering mostly focuses on information retrieval and question answering in a single language, and a high-quality bilingual legal question answering system has not yet appeared.

[0039] Large language models such as GPT, BERT, and their multilingual variants learn vast amounts of general knowledge during pre-training, resulting in strong contextual understanding capabilities. However, these models often require further fine-tuning in specialized domains. By performing transfer learning on legal texts, large language models can capture specialized legal information, improving the accuracy of legal knowledge question-and-answer systems. Furthermore, RAG technology, by combining search results with generative models, enables large models to maintain efficient responses even when faced with extremely long texts or knowledge-rich legal documents, while ensuring authoritative and professional answers.

[0040] like Figure 1 As shown, the first embodiment of the present invention discloses an intelligent bilingual legal question answering method based on a language model, comprising the following steps:

[0041] Step S1: Collect bilingual legal information from multiple source channels, perform text cleaning on the bilingual legal information, and obtain a Chinese legal corpus and a foreign legal corpus.

[0042] Specifically, Chinese and German legal provisions, regulations, judicial interpretations and related cases are collected from authoritative and legal legal databases, government websites and professional publications in China and Germany. The collected bilingual legal information is subjected to text cleaning processing respectively. The purpose of text cleaning processing is to pre-process the bilingual legal information and remove invalid text therein. For example, word segmentation, stop word removal and format standardization are adopted for Chinese. A special word segmenter and word form restoration are used for German to ensure the uniformity of data format. The special word segmenter used in this embodiment is Morphy. Morphy is a free, integrated German morphological processing tool with a large dictionary that can process more than 320,000 word forms. It has strong processing capabilities for German compound nouns and can perform operations such as word segmentation and context-sensitive word form restoration. The correct root of the fuzzy word form is determined by the output of part-of-speech tagging, providing a more comprehensive language preprocessing function. After text cleaning processing, a Chinese legal corpus and a foreign legal corpus are formed respectively for subsequent vectorization processing.

[0043] Step S2: vectorizing the Chinese legal corpus and the foreign legal corpus respectively using the Chinese N-gram language model and the foreign language N-gram language model constructed by the parameter adaptive algorithm to obtain an initial library of Chinese legal vectors and an initial library of foreign language legal vectors.

[0044] Specifically, due to the significant differences in linguistic patterns between Chinese and German legal texts, two N-gram models need to be constructed for separate model training. In this embodiment, the two N-gram language models are constructed using the Chinese legal training set and the foreign legal training set, respectively, to form a constructed Chinese N-gram language model and a foreign language N-gram language model. It should be noted that vectorization of the N-gram language model involves converting the text into sparse, discrete feature vectors through feature engineering (One-Hot, TF-IDF).

[0045] During the construction process of the Chinese N-gram language model and the foreign language N-gram language model, the value of N is determined by a parameter adaptive algorithm. Specifically, a range of candidate N values ​​is pre-set, and a corresponding N-gram model is constructed for each candidate N value in the range. The perplexity of the N-gram model is then calculated using a validation set, and the perplexity is scored using a candidate scoring function. The candidate N value corresponding to the N-gram language model with the best score is used as the optimal N value, and the N-gram language model corresponding to the optimal N value is used as the trained N-gram language model. In one example, the KenLM language model library is used to construct Chinese and German bilingual N-gram language models. The frequency of each N-gram language model in the text is counted, and the scores are compared. It is concluded that the optimal N value of the Chinese N-gram language model is 3, and the foreign language N-gram language model is a German N-gram model, and the optimal N value of the German N-gram language model is 4.

[0046] In this embodiment, the candidate N value range is set as (N = 2, 3, 4, 5). For each candidate N value, an N-gram model is constructed using the corpus. The model estimates the conditional probability of the next word based on the context. For example, for a sentence S = (w1, w2, ..., w L ), the N-gram model estimates the probability of the entire sentence as:

[0047]

[0048] In the subsequent verification phase, the reserved Sino-German legal verification set T is used to calculate the perplexity (PP) of the model. Assume that the verification set contains M sentences, T = {S1, S2, ..., S M}, for each sentence Si, calculate its probability P(Si) according to the above formula 1. The total number of words in all sentences in the validation set is recorded as:

[0049]

[0050] Among them L i It is sentence S i The number of words in , including boundary symbols.

[0051] The perplexity PP(T) is defined as the exponential form of the average negative log probability on the validation set, and the mathematical expression is:

[0052]

[0053] In the subsequent scoring stage, for each candidate N value, the corresponding perplexity PPn(T) is calculated using Formula 2. The specific calculation method is as follows:

[0054] First, for the candidate n value, the n-gram model is trained using the training corpus, and each sentence S in the validation set T is trained according to the model. i Calculate the probability P n (S i ).

[0055] Next, calculate the total log probability of the model on the validation set:

[0056]

[0057] Afterwards, the perplexity is calculated based on the total number of words N:

[0058]

[0059] The candidate scoring function set during scoring is:

[0060] S(n)=α·Perplexity(n)-β·Coverage(n)

[0061] Where: Perplexity(n) is the perplexity of the model corresponding to the candidate n value on the validation set; Coverage(n) is the coverage rate of the candidate model in correctly identifying legal phrases in the validation set; α and β are constants for adjusting the weights.

[0062] By training a corresponding n-gram model for each candidate n value and calculating the scoring function on a validation set, the optimal n value is automatically selected as the basis for subsequent cross-language vector alignment and hybrid representation, ensuring the most accurate statistical representation of legal texts and laying a solid foundation for the entire Chinese-German bilingual legal question-answering system. This adaptive selection of the optimal n value is achieved by calculating the scoring function for different candidate n values ​​on a reserved validation set, enabling the model to adaptively select statistical representations, ensuring that both Chinese and German legal texts receive the optimal n-gram model representations based on their respective language characteristics. Using perplexity to assess the model's predictive performance allows for rapid adjustment of model parameter settings in practical applications and has synergistic effects on subsequent deep semantic representation and cross-language alignment. During the scoring process, mathematical formulas are used to describe the definition and calculation of the scoring function, providing quantitative metrics for system parameter tuning and ensuring that the selected optimal n value effectively captures the linguistic regularities of legal texts.

[0063] Step S3: Use a pre-trained multilingual model to perform semantic analysis on the Chinese legal corpus and the foreign legal corpus respectively, and add the semantic analysis result vectors to the Chinese legal initial vector library and the foreign legal initial vector library respectively.

[0064] Specifically, the multilingual model is the XLM-R model, which is an upgraded version of the XLM (Cross-lingual Language Model). It is based on the improvement of the RoBERTa architecture and can improve cross-language migration capabilities through large-scale language pre-training and model optimization, especially in low-resource languages. The XLM-R model performs vector representation and semantic analysis on the Chinese legal training set and the foreign legal training set, so that the vectors of the Chinese and foreign legal provisions are stored in the Chinese vector library and the foreign vector library respectively, and the semantic analysis results are added to the Chinese legal initial library and the foreign legal initial library respectively. When performing vector representation, the XLM-R model can be used to vectorize a single text or batch text. When performing semantic analysis, the semantic similarity is calculated first, and then fine-tuned through text classification. After processing using the multilingual model, the Chinese legal initial library and the foreign legal initial library are further expanded with vector data.

[0065] Step S4: For each vector in the Chinese legal initial library and the foreign legal initial library, a mapping matrix is ​​calculated, and spatial semantic mapping is performed to obtain a Chinese and foreign legal semantic library. The Chinese and foreign legal semantic library aligns the vectors in the Chinese legal vector initial library and the vectors in the foreign legal vector initial library into the same semantic space, and uses a cross-language attention mechanism to achieve dynamic language alignment.

[0066] Specifically, the spatial semantic mapping adopts the Procrustes analysis method. Procrustes analysis realizes cross-language word vector alignment through simple linear transformation, provides a basic tool for multilingual NLP tasks, and is also a technical method for shape alignment or spatial mapping in mathematics and statistics. In this embodiment, Procrustes analysis is used to map one vector space to another vector space through linear transformation. The linear transformation adopts rotation, scaling and translation. After the cross-language vector alignment, the vector representations of Chinese and German are aligned to the same semantic space.

[0067] The n-gram vector set X of Chinese legal provisions zh and the n-gram vector set Y of German legal texts de As input, the output is the mapping matrix W obtained by Procrustes analysis, which can transform the Chinese vector X zh Mapped to the German vector Y de Similar spaces, making them as close as possible in semantics, that is, satisfying:

[0068]

[0069] The purpose of Procrustes analysis is to adjust the Chinese vectors through the mapping matrix W, so that the Chinese and German vectors are consistent in the same semantic space, thereby making cross-language retrieval and matching more accurate. In one example, the mapping matrix W is obtained through singular value decomposition (SVD), which converts the Chinese vectors to a space consistent with the German vectors.

[0070] In one example, a Chinese-German bilingual cross-lingual attention mechanism was created so that the Chinese and German vectors are not only aligned through a static mapping relationship, but also further enhance the semantic connection between the two languages ​​through dynamic weighting during the query and retrieval process.

[0071] For example, first construct Query, Key and Value values. For a Chinese legal text, use n-gram statistical vectors or deep semantic representation to generate a set of Query vectors Q zh , each n-gram or each word has a corresponding Query vector. Correspondingly, for the German legal text that is semantically similar to the Chinese text, the Key vector K is generated in the same way de and Value vector V de After that, cross-language attention calculation is performed. When calculating the attention score, for each Chinese query vector, the dot product between it and all German key vectors is calculated and then divided by the scaling factor. Get the attention score matrix A, that is Each element in this attention score matrix A reflects the semantic relevance between a local segment in Chinese and a local segment in German. Based on the attention score matrix A, normalization is performed to obtain the attention weight matrix B, that is, Here B ij represents the attention weight of the Chinese Query i to the German Key j. The next step is to generate a cross-language aligned representation. Using the weighted summation method, the normalized attention weight matrix B is weighted summed with the German Value vector, that is, O = BV de The resulting output O is a new representation that incorporates German information into the Chinese query representation. Through the cross-lingual attention mechanism and cross-lingual alignment, each Chinese vector receives a weighted representation based on its relevance to the German vector, making the representation of the Chinese text closer to its German counterpart in the semantic space.

[0072] In one example, a hybrid vector representation is added during spatial semantic mapping, where the aligned statistical vectors and deep semantic vectors are spliced, weightedly fused, or combined through attention fusion to form a unified hybrid vector representation. This allows the resulting Chinese and foreign legal semantic library to cover both local statistical features and global contextual information.

[0073] In one example, after step S4, steps S40 and S41 are further included to further enrich the Chinese and foreign legal semantic libraries through steps S40 and S41.

[0074] Step S40: identifying key information from the legal text according to a preset strategy and performing text extraction to form a legal rule text set, wherein the key information includes the legal clause number, legal application text, and legal entity text.

[0075] Specifically, legal texts typically have strict formats and fixed citation rules, such as "Article 5" or "According to Article 10 of the Civil Code." Predicting strategies based on these rules and conventions in legal texts can identify key information from legal texts and provide clear structured information for semantic representation and cross-language alignment.

[0076] In legal texts, rule conventions are usually expressed as strings in a specific format, such as the clause number "Article 5", "§5BGB", etc., the legal citation is "in accordance with the provisions of Article 10 of the Civil Code", the legal entity is "Civil Code", etc.

[0077] When performing legal text recognition, the legal text is defined as a sentence S = (m1, m2, ..., m L ), the sentence includes each word m in the text i Define the extraction function R to represent the extracted rules, R(S) is the set of rules extracted from the text S: R(S) = {clause number, legal reference, legal entity}.

[0078] Each rule r∈R(S) can be extracted by different techniques: r=Regex(S) or r=DependencyAnalysis(S) or r=NER(S). The legal rule text set obtained by step S40 is a rule set containing structured information.

[0079] The regular expression Regex is used to identify fixed format rules. For example, to match "Article 5" or "§5BGB", you can use the following regular expression:

[0080] Regex="Article (\d+)" or "§\s*(\d+)". This regular expression can extract the numbers in the article number.

[0081] Dependency Analysis is used to analyze the grammatical structure in text to help extract legal references or relationships. For example, when analyzing the sentence "According to Article 10 of the Civil Code," dependency analysis can help identify the relationship between "Civil Code" and "Article 10."

[0082] Named Entity Recognition (NER) is used to identify proprietary terms in legal texts, such as legal entities like the "Civil Code." Named Entity Recognition (NER) can identify and annotate these proprietary entities based on pre-trained dictionaries and rule sets.

[0083] Step S41: Use the legal rule text set to perform domain adaptive training on the large language model.

[0084] Specifically, the goal of domain adaptation is to make the model better adapt to the proprietary terminology and expressions in the Chinese and German legal fields. Since general pre-trained language models are mainly trained on general texts, the model may not be able to accurately understand and process the terminology and sentence patterns that are unique to the legal field. Transfer learning and fine-tuning in domain adaptation technology are needed to enable the model to better understand and process legal texts. The key technology of domain adaptation is transfer learning. First, a general pre-trained language model is used to fine-tune a large-scale Chinese and German legal corpus so that the model can capture the vocabulary, terminology, and syntactic structures that are unique to the legal field. Then, the model is further fine-tuned using a dictionary of domain-specific terminology (such as "Civil Code", "Contract", "Breach of Contract Liability", etc.) to enhance its sensitivity to legal terminology.

[0085] In one example, domain adaptation is also performed using mathematical modeling. Specifically, domain adaptation can optimize the model by minimizing the loss function. Suppose there is a training set {S1, S2, ..., SN} containing N legal texts (in Chinese or German), and each text S i The corresponding label is y i (For example, the classification of legal provisions, judgment results, etc.) The goal of domain adaptation is to minimize the following loss function by optimizing the model parameters θ

[0086] in, represents the loss function, f θ (S i ) is the model's response to the input text S under the parameter θ i The predicted output of , c represents the sample weight coefficient, which is used to make weighted adjustment to the loss term of a single sample.

[0087] Domain adaptation is usually achieved by fine-tuning the existing pre-trained model to obtain the fine-tuning parameter θ *, using a domain-specific corpus to tune the model to make it more adaptable to the characteristics of legal texts:

[0088]

[0089] The above is achieved by further training the pre-trained model on large-scale labeled data in the legal field so that the large language model can better understand legal terminology, norms and structures.

[0090] Through domain adaptation, it is possible to update its legal domain knowledge as new data is continuously received. With the release of new legal provisions and judicial interpretations, it is able to automatically use new training data and fine-tune the model to continuously improve the large language model's ability to understand and recognize new legal content. After the rule extraction in steps S40 and S41 is combined with domain adaptation, it is possible to automatically extract structured information such as clause numbers and legal citations from Chinese and German legal texts, providing a more accurate and professional text representation. Through domain adaptation, the model can understand and process proprietary terminology and expressions in the legal field, improving the system's ability to understand legal texts. Combining rule extraction and domain adaptation can accurately structure legal texts and enhance their professionalism, providing high-quality input for cross-language alignment, hybrid vector representation, and intelligent question and answer.

[0091] Step S5: Using the Chinese and foreign legal semantic database as a retrieval database for a large language model, and utilizing the large language model to output bilingual legal question-and-answer answers to legal issues.

[0092] Specifically, a large language model is used for intelligent question-answering and search. Users can enter queries in Chinese or German. The Chinese and foreign legal semantic database contains both Chinese and foreign language vector libraries, allowing rapid searches in different languages ​​using their respective language vector libraries. Cross-language alignment and hybrid representation enable cross-language matching, returning the corresponding legal texts in Chinese and German. If a direct match is insufficient, the large language model can be used to generate supplementary legal advice answers, ensuring comprehensive and professional responses.

[0093] A specific implementation example is as follows: Suppose a German investment company plans to conduct business in China, and the two parties sign a cross-border contract. Later in the contract execution process, a breach of contract arises, leading to a dispute between the two parties. The user enters a query in German into the German investment company's lawyer, for example, "Welche rechtlichen Consequenzen ergeben sich bei einem Vertragsbruch nach chinesischem Vertragsrecht?" ("According to Chinese Contract Law, what are the legal consequences of contract breach?"). The system supports both Chinese and German input, and the user chooses German as the language for this query. After answering the user's query, the system generates a comprehensive legal advice answer. This answer typically consists of two parts: 1. The German part details the German legal provisions, judicial interpretations, and practical advice related to the user's question; 2. The Chinese part introduces similar legal principles or Chinese legal provisions from similar cases. In this example, the Chinese answer might cite relevant provisions of the Contract Law and explain liability for breach of contract, compensation standards, and so on.

[0094] In one example, it also includes user feedback and continuous optimization. User feedback input is set to collect user evaluations in real time. Based on the user evaluations, active learning strategies are combined to adjust the n-gram model parameters and related parameters in domain adaptation according to the feedback dynamics, thereby forming a closed-loop automatic learning and achieving continuous optimization of retrieval and question-answering effects.

[0095] Explicit feedback, implicit feedback, and comprehensive feedback can be used in user feedback collection. Explicit feedback refers to users rating, commenting, or marking the returned answers as "satisfied," "partially satisfied," or "unsatisfied" after using the Sino-German Legal Q&A platform. Assume that after each Q&A session, the user gives a satisfaction score r in the range [0,1], where 1 indicates complete satisfaction and 0 indicates complete dissatisfaction. Implicit feedback automatically records user behavior data, such as query click-through rate, dwell time, and re-query frequency. These implicit feedback can be converted into an implicit satisfaction indicator s. For example, by counting the average dwell time t of users on the answer page (after normalization), we can obtain: where t min and t max are the set values ​​of the minimum and maximum dwell time, respectively. The comprehensive feedback is a weighted fusion of the explicit feedback r and the implicit feedback s to obtain the final feedback index F, F = λr + (1-λ)s, where λ∈[0,1] is the adjustment weight used to reflect the importance ratio of the two feedbacks.

[0096] Among the relevant parameters in domain adaptation, a loss function L is defined to quantify the gap between the prediction and the user's expectation. Assume that the answer output generated for a query is (For example, representing the quality score or relevance score of the answer), and the user's feedback indicator is F (ideal answer quality), then the loss function can be defined as the mean square error (MSE): Where N is the number of feedback samples. The lower the loss function, the closer the output is to the user's expectation.

[0097] In order to minimize the loss function L, the gradient descent method is used to update the model parameters θ (including n-gram model parameters, vector alignment weights, rule extraction parameters, hybrid representation layer parameters, etc.). The update formula is: where η is the rate of decline, represents the gradient of the loss function L with respect to the parameter θ.

[0098] In actual operation, user feedback is not only used for one-time parameter updates, but also for continuous optimization of the system through active learning mechanisms. The specific process is feedback data collection, sample screening, model fine-tuning and parameter update iteration. Feedback data collection is the regular collection of user feedback. And the corresponding question and answer output Sample screening is based on feedback values, selecting those whose feedback is below the expected threshold F i <θ min Samples with low satisfaction scores are the focus of optimization. Model fine-tuning utilizes low-satisfaction samples to retrain or fine-tune rule extraction, vector alignment, and hybrid representation, enabling the model to better capture the specialized features of legal texts and improve cross-language matching. Parameter update iteration involves re-injecting the fine-tuned parameters into the next round of feedback training, achieving continuous iterative optimization.

[0099] It should be noted that the continuous optimization of user feedback is closely integrated with intelligent question-answering retrieval, vector alignment, and rule extraction, and the ultimate goal is expressed as, θ * =arg min θ {L(θ)+Ω(θ)}, θ represents the set of model parameters, which may include the statistical parameters of the n-gram model, the vector alignment mapping matrix, the weights of rule extraction, and the parameters of the mixed vector fusion layer. L(θ) is a loss function built based on user feedback, usually using mean square error (MSE) or cross entropy loss, which quantifies the system prediction output. The difference between the expected user feedback F and the expected user feedback F. Ω(θ) is a regularization term used to prevent overfitting. This end-to-end joint training approach ensures that each update improves performance in legal question answering and cross-lingual retrieval tasks.

[0100] The large language model bilingual legal Q&A method in this embodiment is based on RAG. Unlike standard legal Q&A RAGs, it contains two bilingual vector libraries, one for Chinese and one for German. Independent Chinese and German legal text vector libraries are constructed and cross-language aligned, mapping the texts in both languages ​​into a unified semantic space. This ensures that the same legal concepts are accurately mapped across languages, thereby building an efficient, accurate, professional, and dynamically optimized legal Q&A platform to meet the practical needs of cross-language legal information retrieval and intelligent Q&A.

[0101] The second embodiment of the present invention discloses an intelligent bilingual legal question-answering system based on a language model, which is implemented using the method steps in the first embodiment. The intelligent bilingual legal question-answering system based on a language model includes an acquisition module, a vectorization module, a semantic analysis module, a semantic alignment module, a text extraction module and a model retrieval module.

[0102] A collection module is used to collect bilingual legal information from multiple source channels, perform text cleaning on the bilingual legal information, and obtain a Chinese legal training set and a foreign legal training set;

[0103] A vectorization module, configured to vectorize the Chinese legal training set and the foreign legal training set respectively using a Chinese N-gram language model and a foreign language N-gram model constructed using a parameter adaptive algorithm, to obtain a Chinese legal initial library and a foreign legal initial library;

[0104] A semantic analysis module, configured to perform semantic analysis on the Chinese legal corpus and the foreign language French corpus using a pre-trained multilingual model, and to add semantic analysis result vectors to the Chinese legal initial vector library and the foreign language legal initial vector library respectively;

[0105] The semantic alignment module calculates a mapping matrix for each vector in the Chinese and foreign language initial legal databases, performs spatial semantic mapping, and generates a Chinese and foreign legal semantic database. This database aligns the vectors in the Chinese and foreign language initial legal databases into the same semantic space and utilizes a cross-lingual attention mechanism to achieve dynamic language alignment. The model retrieval module uses the Chinese and foreign legal semantic databases as a retrieval database for a large language model, which then outputs bilingual legal question-and-answer answers to legal questions.

[0106] The text extraction module is used to identify key information from the legal text and perform text extraction according to a preset strategy before using the Chinese and foreign legal semantic library as a retrieval library for the large language model to form a legal rule text collection, wherein the key information includes the legal clause number, legal application text and legal entity text; and use the legal rule text collection to perform domain adaptive training on the large language model.

[0107] The model retrieval module is used to use the Chinese and foreign legal semantic database as a retrieval database of the large language model, and use the large language model to output bilingual legal question-and-answer answers to legal issues.

[0108] Since the first embodiment and this embodiment correspond to each other, this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and the technical effects achieved in the first embodiment can also be achieved in this embodiment. To reduce repetition, they will not be repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0109] A third embodiment of the present invention relates to a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction implements the steps of the method in the first embodiment when executed by a processor.

[0110] The preferred embodiments of the present invention have been described in detail above. It should be understood that numerous modifications and variations based on the concepts of the present invention are possible without inventive effort by those skilled in the art. Therefore, any technical solution that can be derived by one skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. An intelligent bilingual legal question answering method based on a language model, characterized by: The following steps are involved: Collecting bilingual legal information from multiple source channels, performing text cleaning on the bilingual legal information, and obtaining a Chinese legal corpus and a foreign legal corpus; Vectorizing the Chinese legal training set and the foreign legal training set using a Chinese N-gram language model and a foreign language N-gram language model constructed using a parameter adaptive algorithm, respectively, to obtain an initial library of Chinese legal vectors and an initial library of foreign language legal vectors; Using a pre-trained multilingual model, semantic analysis is performed on the Chinese legal corpus and the foreign legal corpus, and semantic analysis result vectors are added to the Chinese legal initial vector library and the foreign legal initial vector library respectively; For each vector in the Chinese legal initial library and the foreign legal initial library, a mapping matrix is ​​calculated and spatial semantic mapping is performed to obtain a Chinese and foreign legal semantic library. The Chinese and foreign legal semantic library aligns the vectors in the Chinese legal vector initial library and the vectors in the foreign legal vector initial library into the same semantic space and utilizes a cross-language attention mechanism to achieve dynamic language alignment; The Chinese and foreign legal semantic database is used as a retrieval database of a large language model, and the large language model is used to output bilingual legal question-and-answer answers to legal issues.

2. The intelligent bilingual legal question-answering method based on a language model according to claim 1 is characterized in that: Before using the Chinese and foreign legal semantic database as a retrieval database for a large language model, the method further includes: Identify key information from legal texts and perform text extraction based on preset strategies to form a legal rule text collection, wherein the key information includes legal clause numbers, legal application texts, and legal entity texts; The legal rule text collection is used to perform domain adaptive training on a large language model.

3. The intelligent bilingual legal question-answering method based on a language model according to claim 1 is characterized in that: The Chinese N-gram language model and the foreign language N-gram model language are constructed by the following steps: Two N-gram models are constructed, and the two N-gram language models are used to process the Chinese legal data set and the foreign legal data respectively, so as to form Chinese data and foreign data.

4. The intelligent bilingual legal question-answering method based on a language model according to claim 3 is characterized in that: A parameter adaptive algorithm is used to determine the value of N. Specifically, a range of candidate N values ​​is pre-set, and a corresponding N-gram model is constructed for each candidate N value in the range. The perplexity of the N-gram model is then calculated using a validation set. The perplexity is scored using a candidate scoring function, and the candidate N value corresponding to the N-gram model with the best score is taken as the optimal N value. The N-gram model of the corresponding language is then constructed using the optimal N value.

5. The intelligent bilingual legal question-answering method based on a language model according to claim 4 is characterized in that: The optimal N value of the Chinese N-gram model is 3, the foreign language N-gram model is the German N-gram model, and the optimal N value of the German N-gram model is 4.

6. The intelligent bilingual legal question-answering method based on a language model according to claim 1, characterized in that: The multilingual model is an XLM-R model, and the spatial semantic mapping adopts the Procrustes analysis method.

7. An intelligent bilingual legal question-answering system based on a language model, characterized by: Includes the following modules: A collection module is used to collect bilingual legal information from multiple source channels, perform text cleaning on the bilingual legal information, and obtain a Chinese legal training set and a foreign legal training set; A vectorization module, configured to vectorize the Chinese legal training set and the foreign legal training set respectively using a Chinese N-gram language model and a foreign language N-gram model constructed using a parameter adaptive algorithm, to obtain a Chinese legal initial library and a foreign legal initial library; A semantic analysis module, configured to perform semantic analysis on the Chinese legal corpus and the foreign language French corpus using a pre-trained multilingual model, and to add semantic analysis result vectors to the Chinese legal initial vector library and the foreign language legal initial vector library respectively; A semantic alignment module is configured to calculate a mapping matrix for each vector in the Chinese legal initial database and the foreign legal initial database, perform spatial semantic mapping, and obtain a Chinese and foreign legal semantic database. The Chinese and foreign legal semantic database aligns the vectors in the Chinese legal initial database and the vectors in the foreign legal initial database into the same semantic space and utilizes a cross-lingual attention mechanism to achieve dynamic language alignment. The model retrieval module is used to use the Chinese and foreign legal semantic database as a retrieval database of a large language model, and utilize the large language model to output bilingual legal question-and-answer answers to legal issues.

8. The intelligent bilingual legal question-answering system based on a language model according to claim 7 is characterized in that: Also includes: A text extraction module is used to identify key information from legal texts and perform text extraction according to a preset strategy before using the Chinese and foreign legal semantic library as a retrieval library for the large language model to form a legal rule text collection, wherein the key information includes the legal clause number, legal application text, and legal entity text; and use the legal rule text collection to perform domain adaptive training on the large language model.

9. An intelligent bilingual legal question-answering device based on a language model, characterized in that: The question-answering device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the language model-based intelligent bilingual legal question-answering method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the language model-based intelligent bilingual legal question-answering method according to any one of claims 1 to 6 are implemented.

Images