Intelligent legal question and answer method based on multi-modal large model

Abstract

The invention discloses an intelligent legal question-answering method based on a multi-modal large model. The method comprises the following steps: step 1, preprocessing multi-modal input; step 2, cross-modal feature extraction and alignment; step 3, performing multi-granularity attention fusion; step 4, legal logic enhanced decoding; and step 5, performing feedback type continuous learning. According to the method, through multi-modal input preprocessing and cross-modal feature extraction and alignment, legal document information is comprehensively captured, and the image-text logic mapping problem is solved; by designing an attention mechanism perceived by legal elements and constructing a legal provision reference map, the understanding of legal terms is deepened, and the answer accuracy is improved; in a decoding stage, an answer conforming to legal logic is output by embedding a legal case reasoning template library; and through a feedback type continuous learning mechanism, continuous optimization and updating of the system are realized, the system adapts to the change of the legal field, and the legal question-answering efficiency and the user experience are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence legal services, and in particular to a legal intelligent question-answering method based on a multimodal large model. Background Art

[0002] Traditional single-modal legal question-answering systems rely solely on text input, making it difficult to parse the semantic associations of non-text elements such as charts, formulas, and layouts in legal documents, resulting in semantic deviations in the answers to complex legal issues.

[0003] For example, when processing liquidated damages calculation tables in contract terms or judicial interpretation diagrams accompanying legal provisions, existing technologies are unable to establish a logical mapping relationship between images and texts, and lack a mechanism for in-depth understanding of professional terminology in the legal field.

[0004] In addition, traditional model training relies on manually labeled data, which makes it difficult to cover long-tail legal scenarios, and no dedicated architecture is designed for the causal reasoning needs of legal question answering. For this reason, a legal intelligent question answering method based on a multimodal large model is proposed. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a legal intelligent question answering method based on a multimodal large model to solve the problems raised in the above background technology.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a legal intelligent question-answering method based on a multimodal large model, comprising the following steps: Step 1: Multimodal input preprocessing: Perform semantic analysis on legal issues input by users to identify the modal types of text, images, and tables involved. Use specialized word segmentation tools in the legal field to segment the text and generate word vector sequences containing legal term weights. Use OCR technology to extract text content from image inputs, and simultaneously locate key legal symbols through object detection models. Step 2: Cross-modal feature extraction and alignment: Based on the structural features of legal documents, a library of image-text alignment templates is constructed, spatially mapping the text word vectors output in step 1 to the image region features. A dual-tower encoding structure is adopted to extract deep features through a legal text encoder and a visual encoder. The text encoder loads pre-trained legal corpus parameters, while the visual encoder uses a CNN network fine-tuned with legal document images. A contrastive learning loss function is introduced to force alignment of the feature distribution space of related modalities. Step 3: Multi-granularity attention fusion: Design an attention mechanism for legal element perception and dynamically weight the multimodal features output in step 2. Build a legal article citation graph. When a feature is detected involving an article association, activate the graph attention network for cross-article reasoning. Output a multimodal joint representation vector that integrates legal domain knowledge. The multimodal representation vector includes text, images, tables, formulas, and audio modalities. Step 4: Legal logic enhanced decoding: Embed a legal case reasoning template library in the decoder, match similar case paradigms based on the representation vector from step 3, use autoregressive generation combined with a legal terminology constraint dictionary to screen candidate answers, and output a structured answer that conforms to legal logic specifications. Step 5: Feedback-based continuous learning: Establish a user feedback interface to collect feedback on generated answers; build a legal knowledge distillation module to convert manually revised cases into incremental training data; regularly initiate a model fine-tuning process to update the legal terminology dictionary and reasoning template library; It uses the CRF algorithm, specifically the linear chain conditional random field model, and combines it with a legal field dictionary for word segmentation to improve the accuracy of word segmentation. For the image-text alignment template library, its construction method can be supplemented, that is, by collecting a large number of legal documents, statistically analyzing the positional relationship between the clause numbers and the explanatory text, and thus forming an alignment template to ensure the accurate correspondence between image and text information. For the legal case reasoning template library, a specific algorithm for case similarity calculation can be supplemented, that is, using cosine similarity to calculate the similarity between cases, and conducting a comprehensive evaluation based on the cause of the case, the focus of the dispute, and the court's opinion, to improve the accuracy of case reasoning. Through multimodal input preprocessing, cross-modal feature extraction and alignment, multi-granularity attention fusion, legal logic enhanced decoding and feedback-based continuous learning, comprehensive analysis and accurate answers to legal issues are achieved. Among them, the application of special word segmentation tools and image-text alignment template libraries in the legal field significantly improves the accuracy and efficiency of legal text processing; the design of legal case reasoning template libraries and legal logic enhanced decoding ensures the legal logic and standardization of answers; and the feedback-based continuous learning mechanism enables the system to be continuously optimized and updated to adapt to the constant changes in the legal field. These innovations jointly improve the performance and user experience of the legal intelligent question-answering system, providing strong support for its application in the legal field.

[0007] Preferably, the legal field-specific word segmentation tool in step 1 adopts a CRF algorithm and loads a dictionary library containing hundreds of thousands of legal terms to perform overall segmentation on the professional terms; The CRF model uses a feature template that includes part-of-speech, context, and legal terminology features. Combined with the dictionary building process, it forms an authoritative legal terminology database through data collection, term extraction, manual review, and dictionary construction. The dictionary database implements a dynamic update strategy, including regular updates, user feedback, and automatic crawling of new terms, to ensure the accuracy and timeliness of the word segmentation tool. The use of the CRF algorithm and a special word segmentation tool in the legal field, combined with a 100,000-level legal terminology dictionary, to perform overall segmentation of professional terms has significant advantages. First, the CRF algorithm can effectively capture the contextual information in the legal text. Combined with the legal terminology feature template, it can significantly improve the accuracy of word segmentation and ensure that legal terms are correctly identified. Secondly, the dictionary construction process is rigorous. Through data collection, term extraction, manual review and dictionary construction, an authoritative and accurate legal terminology database is formed, providing a solid foundation for word segmentation. Furthermore, the dynamic update strategy ensures that the dictionary keeps pace with the times, adapts to legal development and changes, and maintains the timeliness of the word segmentation tool. Finally, the overall segmentation method helps to maintain the integrity of legal terminology, avoid semantic ambiguity caused by word segmentation errors, and improve the accuracy and reliability of the legal intelligent question-answering system.

[0008] Preferably, the image-text alignment template library in step 2 includes common layout features of legal documents, such as statistical regularity of positional offsets between clause numbers and corresponding explanatory texts; This template library collects a large number of legal document samples, extracts the position coordinate data of the clause numbers and the corresponding explanatory texts, and calculates the statistical distribution patterns of their offsets (such as mean, variance, and spatial constraints) to form standardized layout templates. When processing new documents, the system automatically detects the position of the clause numbers based on the template matching algorithm and derives the explanatory text area based on the offset patterns, achieving precise spatial alignment of cross-modal graphic and text elements. By quantifying the positional offset patterns between clause numbers and explanatory texts, the association between graphic and text content can be quickly established, solving the efficiency bottleneck of traditional methods that rely on manual annotation or rule configuration. Secondly, the statistical law learning mechanism of the template library can adapt to the format differences of different legal documents (such as clause indentation and paragraph spacing), and improve the alignment robustness across document types. Furthermore, the design provides structured spatial constraints for subsequent multimodal feature fusion, making the semantic association between legal terms and visual elements (such as seals and signatures) more accurate, laying a reliable data foundation for legal logical reasoning. Finally, the template library can be automatically updated through continuous learning as the layout of legal documents evolves, ensuring that the system adapts to practical needs in the long term.

[0009] Preferably, the attention mechanism for legal element perception in step 3 generates attention weights through a pre-trained legal element classifier, and the legal element classifier can identify twenty types of legal elements including subject qualifications, rights and obligations, and liability for breach of contract; This mechanism constructs a legal element classifier through a pre-trained LSTM deep learning model. The training data uses a dataset of millions of annotated legal documents. The model structure includes a word embedding input layer, a bidirectional LSTM encoding layer, and a fully connected classification layer for twenty types of elements. During the attention fusion stage, the element probability vector output by the classifier is subjected to a Hadamard product operation with the multimodal feature matrix to achieve dynamic feature weighting based on the weights of legal elements, so that key legal elements receive higher weight allocations during feature fusion. Through the pre-training mechanism of the legal element classifier, the model can accurately identify twenty core legal elements such as subject qualifications, rights and obligations, effectively solve the problem of identifying professional terms in the legal field, and significantly improve the accuracy of the question-answering system's understanding of complex legal provisions; the dynamic attention allocation strategy based on element weights enables the model to automatically focus on key clauses when dealing with legal issues with multiple elements. For example, in contract dispute scenarios, it can simultaneously strengthen the correlation analysis between breach of contract liability clauses and jurisdiction clauses; the modular design supports independent iterative optimization of element classifiers. When legal provisions are revised, it is only necessary to update the classifier training data to complete the system upgrade, which greatly reduces the maintenance cost of the legal knowledge base and lays a technical foundation for building an evolvable legal intelligent system.

[0010] Preferably, the legal case reasoning template library in step 4 adopts a case similarity calculation algorithm, and the matching dimensions include case type, dispute focus and court opinion keywords; In the legal case reasoning template library, the case similarity calculation algorithm can use the cosine similarity algorithm to vectorize the case type, dispute focus, and court opinion keywords. The similarity between cases is measured by calculating the cosine value between vectors. At the same time, weighting can be performed based on the importance of each dimension to more accurately match similar cases. Through multi-dimensional matching, the accuracy of case matching can be significantly improved, ensuring that cases highly relevant to the current problem are found. Secondly, matching similar case paradigms helps to enhance the standardization of legal logic, making the output structured answers more in line with legal logic and adjudication standards. Finally, accurate case matching and standardized answers can improve user experience, enhance user trust and satisfaction with the legal intelligent question-answering system, and thus promote the widespread application and development of legal intelligent question-answering technology.

[0011] Preferably, the legal knowledge distillation module in step 5 includes an answer quality assessment submodule, which is trained based on legal expert annotated data and can automatically screen high-quality feedback samples; The legal knowledge distillation module converts manually revised cases into incremental training data, using deep learning technology for feature extraction and pattern recognition to optimize model performance. The answer quality assessment submodule is trained on data annotated by legal experts. By setting evaluation indicators such as accuracy, completeness, and logic, it automatically selects high-quality feedback samples for further optimization of model training. The introduction of the legal knowledge distillation module and the answer quality assessment sub-module has brought significant benefits to the legal intelligent question-answering system. First, these modules realize continuous learning and optimization of the model by integrating user feedback, enabling the system to continuously adapt to new legal issues and user needs, and improving the system's adaptability and flexibility. Secondly, the answer quality assessment sub-module ensures the data basis for model training. By screening high-quality feedback samples, it improves the system's legal logical judgment ability and case matching accuracy, providing users with more professional and accurate legal question-answering services. In addition, the legal knowledge distillation module generates incremental training data by manually revising cases, further optimizing the model performance, enabling the system to better understand and handle complex legal issues, and enhancing the system's reliability and user satisfaction. In summary, the introduction of these modules has significantly improved the overall performance of the legal intelligent question-answering system, providing users with more efficient and accurate legal services.

[0012] Preferably, the dual-tower coding structure in step 2 shares a legal entity link dictionary to ensure that the legal subject and object in different modal features point to the same entity; For specialized word segmentation tools in the legal field, a word segmentation model based on the Conditional Random Field (CRF) algorithm can be used and trained using legal text datasets to accurately identify legal terminology. The image-text alignment template library can analyze a large number of legal documents, extracting layout features such as the positional relationship between clause numbers and explanatory text, thereby constructing precise alignment templates. The legal case reasoning template library can use cosine similarity or Jaccard similarity algorithms to calculate the similarity between cases for efficient case matching. The design of a shared legal entity link dictionary with a dual-tower coding structure brings many benefits. First, it ensures that the legal subjects and objects in different modal features can accurately point to the same entity, thereby maintaining data consistency and accuracy, which is crucial for legal intelligent question-answering systems because the answers to legal questions often rely on an accurate understanding of the legal subjects and objects. Secondly, the design of a shared dictionary optimizes the processing flow, avoids duplication of work and waste of computing resources, and improves the processing efficiency of the system. In addition, this design also enhances the scalability and maintainability of the system, making it convenient to add new entities or update existing entities in the future by only one update in the dictionary. Finally, from the perspective of user experience, the shared dictionary ensures that the legal information obtained by users in different modalities is consistent and accurate, thereby improving user satisfaction and trust.

[0013] Preferably, when constructing the legal article citation graph in step 3, a community discovery algorithm based on the legal article co-occurrence network is used to automatically divide related article clusters; The CRF algorithm, in legal terminology segmentation, captures contextual dependencies between terms through model training and uses a 100,000-word dictionary for precise segmentation. Dictionary construction utilizes a semi-automated process, combined with expert annotation and automatic expansion, and regularly maintains timeliness through a legal document update mechanism. The image-text alignment template library covers common legal document formats, extracts format features through machine learning techniques (such as convolutional neural networks), and implements template matching in conjunction with a rule engine. The system supports online learning, automatically absorbs new document samples, and optimizes template matching accuracy and coverage. The legal element-aware attention mechanism utilizes a pre-trained legal element classifier (such as the BERT-Legal model) to generate attention weights. The classifier is fine-tuned using a large-scale legal text dataset, and performance is evaluated using multiple metrics such as accuracy, recall, and F1 score. It significantly improves the efficiency of article retrieval, enabling the system to quickly locate relevant articles when faced with complex legal issues, thereby quickly giving accurate answers. Secondly, by deeply exploring the internal connections between articles, the community discovery algorithm enhances the accuracy and comprehensiveness of legal reasoning, providing users with more authoritative and detailed legal interpretations. In addition, this method also has strong adaptability and scalability, and can be continuously optimized and improved with the updating and development of the legal system, ensuring that the legal intelligent question-answering system always remains at the cutting-edge level. In short, the construction of the legal article citation map not only improves the performance of the legal intelligent question-answering system, but also promotes the in-depth development of the legal informatization process, and provides strong support for research and application in the legal field.

[0014] Preferably, the legal term constraint dictionary in step 4 includes a dynamic update mechanism to adjust term weights by crawling the latest judicial interpretation documents; This mechanism uses a distributed crawler framework to regularly crawl authoritative legal databases, combines natural language processing technology to parse the paragraph structure of judicial documents, extract professional terms, and construct a temporal weight model. The model integrates the TF-IDF algorithm with the frequency of legal text citations to achieve dynamic adjustment of term weights, while also establishing a version management mechanism to ensure traceability of updates. Ensure the timeliness of the legal terminology database. By automatically capturing the latest judicial interpretations, the system always makes inferences based on the latest legal provisions, avoiding outdated answers due to legal amendments. Secondly, improve the accuracy of answers. The weight adjustment mechanism gives high-frequency citations or newly effective clauses a higher priority in answer generation, which is in line with judicial practice trends. Thirdly, enhance the system's adaptability. The temporal weight model can capture changes in the popularity of legal terms, making answers closer to actual legal application scenarios. Finally, update traceability is achieved through version management, providing an explanatory basis for legal question and answer results, which is in line with the closed-loop design requirements of "feedback continuous learning" in the patented technical solution, and overall improves the practical value and reliability of the legal intelligent question and answer system.

[0015] Preferably, the feedback-based continuous learning process in step 5 includes a privacy protection module, which desensitizes the user feedback data before storing it in the training set; Before user feedback data is stored in the training set, differential privacy technology is used to hash the user ID. Keyword replacement and semantic generalization based on the legal domain vocabulary are performed on the text feedback content. Homomorphic encryption protection is implemented for numerical feedback data. A dynamic desensitization rule engine is established to automatically match L2-L4 desensitization levels based on the feedback type. L4 desensitization converts the original data into a standardized expression template that complies with legal document specifications, ensuring a protection level of ≥95% against inversion attacks. The desensitization process is encapsulated in a trusted execution environment (TEE), and processing logs are stored on the blockchain to form a complete privacy compliance evidence chain. First, differential privacy and semantic generalization technologies achieve a balance between data availability and privacy, meeting the requirements of the "Personal Information Protection Law of the People's Republic of China" while retaining the key semantic features of legal question-and-answer scenarios. Second, the multi-level desensitization strategy takes into account the needs of different legal scenarios. L4-level standardized conversion enables training data to be directly connected to judicial document specifications, improving the accuracy of the legal language output by the model. The dynamic rule engine is combined with blockchain evidence to form a verifiable privacy compliance system, reducing the compliance risks of legal technology products. This mechanism enables the model to continuously absorb user feedback. The real case data after desensitization makes the answer logic closer to judicial practice. Combined with regular fine-tuning processes, the model accuracy is improved. The overall solution uses privacy computing technology to break the bottleneck of legal AI data, build a positive cycle of data security-model optimization-user experience, and provide a replicable technical path for the construction of smart justice.

[0016] In summary, compared with the prior art, the present invention provides a legal intelligent question-answering method based on a multimodal large model, which has the following beneficial effects: Through multimodal input preprocessing, this invention can simultaneously identify and process legal issues in multiple modalities, such as text, images, and tables, achieving the effect of comprehensively capturing the information in legal documents. Through cross-modal feature extraction and alignment, this invention constructs a graphic-text alignment template library and spatially maps text word vectors with image region features, effectively solving the problem that traditional systems cannot establish logical mapping relationships between images and text, enabling the system to more accurately understand the complex information in legal documents. This invention designs an attention mechanism for legal element perception, dynamically weights and fuses multimodal features, constructs a legal text citation graph, and activates a graph attention network for cross-text reasoning. This achieves a deeper understanding of legal terminology, improves answer accuracy, and outputs a multimodal joint representation vector that integrates legal knowledge. During the decoding phase, the invention embeds a legal case reasoning template library, matches similar case paradigms, and screens candidate answers using a legal terminology constraint dictionary. This achieves the effect of outputting structured answers that conform to legal logic norms, ensuring the legal logic and normativeness of the answers and improving the quality of legal question-answering. This invention uses a feedback-based continuous learning mechanism to establish a user feedback interface, collect correction opinions, and regularly initiate a model fine-tuning process to update the legal terminology dictionary and reasoning template library. This enables the system to be continuously optimized and updated to adapt to the ever-changing legal field, thereby improving the efficiency of legal Q&A and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a step-by-step diagram of the legal intelligent question-answering method based on a multimodal large model of the invention. DETAILED DESCRIPTION

[0018] This invention provides a technical solution, a legal intelligent question answering method based on a multimodal large model, please refer to Figure 1 , including the following steps: Step 1: Multimodal input preprocessing: Perform semantic analysis on legal issues input by users to identify the modal types of text, images, and tables involved. Use specialized word segmentation tools in the legal field to segment the text and generate word vector sequences containing legal term weights. Use OCR technology to extract text content from image inputs, and simultaneously locate key legal symbols through object detection models. Step 2: Cross-modal feature extraction and alignment: Based on the structural features of legal documents, a library of image-text alignment templates is constructed, spatially mapping the text word vectors output in step 1 to the image region features. A dual-tower encoding structure is adopted to extract deep features through a legal text encoder and a visual encoder. The text encoder loads pre-trained legal corpus parameters, while the visual encoder uses a CNN network fine-tuned with legal document images. A contrastive learning loss function is introduced to force alignment of the feature distribution space of related modalities. Step 3: Multi-granularity attention fusion: Design an attention mechanism for legal element perception and dynamically weight the multimodal features output in step 2. Build a legal article citation graph. When a feature is detected involving an article association, activate the graph attention network for cross-article reasoning. Output a multimodal joint representation vector that integrates legal domain knowledge. The multimodal representation vector includes text, images, tables, formulas, and audio modalities. Step 4: Legal logic enhanced decoding: Embed a legal case reasoning template library in the decoder, match similar case paradigms based on the representation vector from step 3, use autoregressive generation combined with a legal terminology constraint dictionary to screen candidate answers, and output a structured answer that conforms to legal logic specifications. Step 5: Feedback-based continuous learning: Establish a user feedback interface to collect feedback on generated answers; build a legal knowledge distillation module to convert manually revised cases into incremental training data; regularly initiate a model fine-tuning process to update the legal terminology dictionary and reasoning template library; It uses the CRF algorithm, specifically the linear chain conditional random field model, and combines it with a legal field dictionary for word segmentation to improve the accuracy of word segmentation. For the image-text alignment template library, its construction method can be supplemented, that is, by collecting a large number of legal documents, statistically analyzing the positional relationship between the clause numbers and the explanatory text, and thus forming an alignment template to ensure the accurate correspondence between image and text information. For the legal case reasoning template library, a specific algorithm for case similarity calculation can be supplemented, that is, using cosine similarity to calculate the similarity between cases, and conducting a comprehensive evaluation based on the cause of the case, the focus of the dispute, and the court's opinion, to improve the accuracy of case reasoning. Through multimodal input preprocessing, cross-modal feature extraction and alignment, multi-granularity attention fusion, legal logic enhanced decoding and feedback-based continuous learning, comprehensive analysis and accurate answers to legal issues are achieved. Among them, the application of special word segmentation tools and image-text alignment template libraries in the legal field significantly improves the accuracy and efficiency of legal text processing; the design of legal case reasoning template libraries and legal logic enhanced decoding ensures the legal logic and standardization of answers; and the feedback-based continuous learning mechanism enables the system to be continuously optimized and updated to adapt to the constant changes in the legal field. These innovations jointly improve the performance and user experience of the legal intelligent question-answering system, providing strong support for its application in the legal field.

[0019] See also Figure 1 The legal field-specific word segmentation tool in step 1 uses the CRF algorithm and loads a dictionary containing hundreds of thousands of legal terms to perform overall segmentation of professional terms; The CRF model uses a feature template that includes part-of-speech, context, and legal terminology features. Combined with the dictionary building process, it forms an authoritative legal terminology database through data collection, term extraction, manual review, and dictionary construction. The dictionary database implements a dynamic update strategy, including regular updates, user feedback, and automatic crawling of new terms, to ensure the accuracy and timeliness of the word segmentation tool. The use of the CRF algorithm and a special word segmentation tool in the legal field, combined with a 100,000-level legal terminology dictionary, to perform overall segmentation of professional terms has significant advantages. First, the CRF algorithm can effectively capture the contextual information in the legal text. Combined with the legal terminology feature template, it can significantly improve the accuracy of word segmentation and ensure that legal terms are correctly identified. Secondly, the dictionary construction process is rigorous. Through data collection, term extraction, manual review and dictionary construction, an authoritative and accurate legal terminology database is formed, providing a solid foundation for word segmentation. Furthermore, the dynamic update strategy ensures that the dictionary keeps pace with the times, adapts to legal development and changes, and maintains the timeliness of the word segmentation tool. Finally, the overall segmentation method helps to maintain the integrity of legal terminology, avoid semantic ambiguity caused by word segmentation errors, and improve the accuracy and reliability of the legal intelligent question-answering system.

[0020] See also Figure 1 ,The image-text alignment template library in step 2 contains common layout features of legal documents, such as the statistical ,rules of the position offset between the clause number and the corresponding ,explanatory text; This template library collects a large number of legal document samples, extracts the position coordinate data of the clause numbers and the corresponding explanatory texts, and calculates the statistical distribution patterns of their offsets (such as mean, variance, and spatial constraints) to form standardized layout templates. When processing new documents, the system automatically detects the position of the clause numbers based on the template matching algorithm and derives the explanatory text area based on the offset patterns, achieving precise spatial alignment of cross-modal graphic and text elements. By quantifying the positional offset patterns between clause numbers and explanatory texts, the association between graphic and text content can be quickly established, solving the efficiency bottleneck of traditional methods that rely on manual annotation or rule configuration. Secondly, the statistical law learning mechanism of the template library can adapt to the format differences of different legal documents (such as clause indentation and paragraph spacing), and improve the alignment robustness across document types. Furthermore, the design provides structured spatial constraints for subsequent multimodal feature fusion, making the semantic association between legal terms and visual elements (such as seals and signatures) more accurate, laying a reliable data foundation for legal logical reasoning. Finally, the template library can be automatically updated through continuous learning as the layout of legal documents evolves, ensuring that the system adapts to practical needs in the long term.

[0021] See also Figure 1 The attention mechanism of legal element perception in step 3 generates attention weights through the pre-trained legal element classifier. The legal element classifier can identify twenty types of legal elements including subject qualifications, rights and obligations, and breach of contract liability; This mechanism constructs a legal element classifier through a pre-trained LSTM deep learning model. The training data uses a dataset of millions of annotated legal documents. The model structure includes a word embedding input layer, a bidirectional LSTM encoding layer, and a fully connected classification layer for twenty types of elements. During the attention fusion stage, the element probability vector output by the classifier is subjected to a Hadamard product operation with the multimodal feature matrix to achieve dynamic feature weighting based on the weights of legal elements, so that key legal elements receive higher weight allocations during feature fusion. Through the pre-training mechanism of the legal element classifier, the model can accurately identify twenty core legal elements such as subject qualifications, rights and obligations, effectively solve the problem of identifying professional terms in the legal field, and significantly improve the accuracy of the question-answering system's understanding of complex legal provisions; the dynamic attention allocation strategy based on element weights enables the model to automatically focus on key clauses when dealing with legal issues with multiple elements. For example, in contract dispute scenarios, it can simultaneously strengthen the correlation analysis between breach of contract liability clauses and jurisdiction clauses; the modular design supports independent iterative optimization of element classifiers. When legal provisions are revised, it is only necessary to update the classifier training data to complete the system upgrade, which greatly reduces the maintenance cost of the legal knowledge base and lays a technical foundation for building an evolvable legal intelligent system.

[0022] See also Figure 1 ,The legal case reasoning template library in step 4 adopts a case similarity calculation algorithm, and the matching dimensions include case type, dispute focus and court opinion keywords; In the legal case reasoning template library, the case similarity calculation algorithm can use the cosine similarity algorithm to vectorize the case type, dispute focus, and court opinion keywords. The similarity between cases is measured by calculating the cosine value between vectors. At the same time, weighting can be performed based on the importance of each dimension to more accurately match similar cases. Through multi-dimensional matching, the accuracy of case matching can be significantly improved, ensuring that cases highly relevant to the current problem are found. Secondly, matching similar case paradigms helps to enhance the standardization of legal logic, making the output structured answers more in line with legal logic and adjudication standards. Finally, accurate case matching and standardized answers can improve user experience, enhance user trust and satisfaction with the legal intelligent question-answering system, and thus promote the widespread application and development of legal intelligent question-answering technology.

[0023] See also Figure 1 ,The legal knowledge distillation module in step five includes an answer quality ,assessment submodule, which is trained based on legal expert ,annotated data and can automatically screen high-quality feedback samples; The legal knowledge distillation module converts manually revised cases into incremental training data, using deep learning technology for feature extraction and pattern recognition to optimize model performance. The answer quality assessment submodule is trained on data annotated by legal experts. By setting evaluation indicators such as accuracy, completeness, and logic, it automatically selects high-quality feedback samples for further optimization of model training. The introduction of the legal knowledge distillation module and the answer quality assessment sub-module has brought significant benefits to the legal intelligent question-answering system. First, these modules realize continuous learning and optimization of the model by integrating user feedback, enabling the system to continuously adapt to new legal issues and user needs, and improving the system's adaptability and flexibility. Secondly, the answer quality assessment sub-module ensures the data basis for model training. By screening high-quality feedback samples, it improves the system's legal logical judgment ability and case matching accuracy, providing users with more professional and accurate legal question-answering services. In addition, the legal knowledge distillation module generates incremental training data by manually revising cases, further optimizing the model performance, enabling the system to better understand and handle complex legal issues, and enhancing the system's reliability and user satisfaction. In summary, the introduction of these modules has significantly improved the overall performance of the legal intelligent question-answering system, providing users with more efficient and accurate legal services.

[0024] See also Figure 1 ,The dual-tower encoding structure in step 2 shares the legal entity link dictionary, ensuring that the legal subject and object in different modal features point to the same entity; For specialized word segmentation tools in the legal field, a word segmentation model based on the Conditional Random Field (CRF) algorithm can be used and trained using legal text datasets to accurately identify legal terminology. The image-text alignment template library can analyze a large number of legal documents, extracting layout features such as the positional relationship between clause numbers and explanatory text, thereby constructing precise alignment templates. The legal case reasoning template library can use cosine similarity or Jaccard similarity algorithms to calculate the similarity between cases for efficient case matching. The design of a shared legal entity link dictionary with a dual-tower coding structure brings many benefits. First, it ensures that the legal subjects and objects in different modal features can accurately point to the same entity, thereby maintaining data consistency and accuracy, which is crucial for legal intelligent question-answering systems because the answers to legal questions often rely on an accurate understanding of the legal subjects and objects. Secondly, the design of a shared dictionary optimizes the processing flow, avoids duplication of work and waste of computing resources, and improves the processing efficiency of the system. In addition, this design also enhances the scalability and maintainability of the system, making it convenient to add new entities or update existing entities in the future by only one update in the dictionary. Finally, from the perspective of user experience, the shared dictionary ensures that the legal information obtained by users in different modalities is consistent and accurate, thereby improving user satisfaction and trust.

[0025] See also Figure 1 ,When constructing the legal article citation graph in step 3, a community discovery algorithm based on the ,co-occurrence network of legal articles is used to automatically divide ,the related article clusters; The CRF algorithm, in legal terminology segmentation, captures contextual dependencies between terms through model training and uses a 100,000-word dictionary for precise segmentation. Dictionary construction utilizes a semi-automated process, combined with expert annotation and automatic expansion, and regularly maintains timeliness through a legal document update mechanism. The image-text alignment template library covers common legal document formats, extracts format features through machine learning techniques (such as convolutional neural networks), and implements template matching in conjunction with a rule engine. The system supports online learning, automatically absorbs new document samples, and optimizes template matching accuracy and coverage. The legal element-aware attention mechanism utilizes a pre-trained legal element classifier (such as the BERT-Legal model) to generate attention weights. The classifier is fine-tuned using a large-scale legal text dataset, and performance is evaluated using multiple metrics such as accuracy, recall, and F1 score. It significantly improves the efficiency of article retrieval, enabling the system to quickly locate relevant articles when faced with complex legal issues, thereby quickly giving accurate answers. Secondly, by deeply exploring the internal connections between articles, the community discovery algorithm enhances the accuracy and comprehensiveness of legal reasoning, providing users with more authoritative and detailed legal interpretations. In addition, this method also has strong adaptability and scalability, and can be continuously optimized and improved with the updating and development of the legal system, ensuring that the legal intelligent question-answering system always remains at the cutting-edge level. In short, the construction of the legal article citation map not only improves the performance of the legal intelligent question-answering system, but also promotes the in-depth development of the legal informatization process, and provides strong support for research and application in the legal field.

[0026] See also Figure 1,The legal term constraint dictionary in step 4 contains a dynamic update mechanism, which ,adjusts term weights by crawling the latest judicial interpretation documents; This mechanism uses a distributed crawler framework to regularly crawl authoritative legal databases, combines natural language processing technology to parse the paragraph structure of judicial documents, extract professional terms, and construct a temporal weight model. The model integrates the TF-IDF algorithm with the frequency of legal text citations to achieve dynamic adjustment of term weights, while also establishing a version management mechanism to ensure traceability of updates. Ensure the timeliness of the legal terminology database. By automatically capturing the latest judicial interpretations, the system always makes inferences based on the latest legal provisions, avoiding outdated answers due to legal amendments. Secondly, improve the accuracy of answers. The weight adjustment mechanism gives high-frequency citations or newly effective clauses a higher priority in answer generation, which is in line with judicial practice trends. Thirdly, enhance the system's adaptability. The temporal weight model can capture changes in the popularity of legal terms, making answers closer to actual legal application scenarios. Finally, update traceability is achieved through version management, providing an explanatory basis for legal question and answer results, which is in line with the closed-loop design requirements of "feedback continuous learning" in the patented technical solution, and overall improves the practical value and reliability of the legal intelligent question and answer system.

[0027] See also Figure 1 ,The feedback continuous learning process in step five includes a privacy protection module, which desensitizes the user feedback data before storing it in the training set; Before user feedback data is stored in the training set, differential privacy technology is used to hash the user ID. Keyword replacement and semantic generalization based on the legal domain vocabulary are performed on the text feedback content. Homomorphic encryption protection is implemented for numerical feedback data. A dynamic desensitization rule engine is established to automatically match L2-L4 desensitization levels based on the feedback type. L4 desensitization converts the original data into a standardized expression template that complies with legal document specifications, ensuring a protection level of ≥95% against inversion attacks. The desensitization process is encapsulated in a trusted execution environment (TEE), and processing logs are stored on the blockchain to form a complete privacy compliance evidence chain. First, differential privacy and semantic generalization technologies achieve a balance between data availability and privacy, meeting the requirements of the "Personal Information Protection Law of the People's Republic of China" while retaining the key semantic features of legal question-and-answer scenarios. Second, the multi-level desensitization strategy takes into account the needs of different legal scenarios. L4-level standardized conversion enables training data to be directly connected to judicial document specifications, improving the accuracy of the legal language output by the model. The dynamic rule engine is combined with blockchain evidence to form a verifiable privacy compliance system, reducing the compliance risks of legal technology products. This mechanism enables the model to continuously absorb user feedback. The real case data after desensitization makes the answer logic closer to judicial practice. Combined with regular fine-tuning processes, the model accuracy is improved. The overall solution uses privacy computing technology to break the bottleneck of legal AI data, build a positive cycle of data security-model optimization-user experience, and provide a replicable technical path for the construction of smart justice.

[0028] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0029] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A legal intelligent question-answering method based on a multimodal large model, characterized by: The steps include: Step 1: Multimodal input preprocessing: Perform semantic analysis on legal issues input by users to identify the modal types of text, images, and tables involved. Use specialized word segmentation tools in the legal field to segment the text and generate word vector sequences containing legal term weights. Use OCR technology to extract text content from image inputs, and simultaneously locate key legal symbols through object detection models. Step 2: Cross-modal feature extraction and alignment: Based on the structural features of legal documents, a template library for image-text alignment is constructed to spatially map the text word vectors output in step 1 to the image region features. A dual-tower encoding structure is adopted to extract deep features through the legal text encoder and the visual encoder respectively; a contrastive learning loss function is introduced to force the alignment of the feature distribution space of related modalities; Step 3: Multi-granularity attention fusion: Design an attention mechanism for legal element perception and dynamically weight the multimodal features output in step 2. Build a legal article citation graph. When a feature is detected involving an article association, activate the graph attention network for cross-article reasoning. Output a multimodal joint representation vector that integrates legal domain knowledge. Step 4: Legal logic enhanced decoding: Embed a legal case reasoning template library in the decoder, match similar case paradigms based on the representation vector from step 3, use autoregressive generation combined with a legal terminology constraint dictionary to screen candidate answers, and output a structured answer that conforms to legal logic specifications. Step 5: Feedback-based continuous learning: Establish a user feedback interface to collect comments on the generated answers; build a legal knowledge distillation module to convert manually revised cases into incremental training data; regularly initiate the model fine-tuning process to update the legal terminology dictionary and reasoning template library.

2. The legal intelligent question-answering method based on a multimodal large model according to claim 1 is characterized by: The legal field-specific word segmentation tool in step 1 adopts the CRF algorithm and loads a dictionary library containing legal terms to perform overall segmentation on professional terms.

3. The legal intelligent question-answering method based on a multimodal large model according to claim 1 is characterized by: The image-text alignment template library in step 2 contains common layout features of legal documents, the text encoder loads pre-trained legal corpus parameters, and the visual encoder uses a CNN network fine-tuned by legal document images.

4. The legal intelligent question-answering method based on a multimodal large model according to claim 1 is characterized by: The legal element-aware attention mechanism in step three generates attention weights through a pre-trained legal element classifier.

5. The legal intelligent question answering method based on a multimodal large model according to claim 1 is characterized by: The legal case reasoning template library in step 4 adopts a case similarity calculation algorithm, and the matching dimensions include case type, dispute focus and court opinion keywords.

6. The legal intelligent question-answering method based on a multimodal large model according to claim 1 is characterized by: The legal knowledge distillation module in step five includes an answer quality assessment submodule, which is trained based on legal expert annotated data.

7. The legal intelligent question-answering method based on a multimodal large model according to claim 1 is characterized by: The dual-tower coding structure in step 2 shares a legal entity link dictionary.

8. The legal intelligent question-answering method based on a multimodal large model according to claim 1 is characterized by: When constructing the legal text citation graph in step three, a community discovery algorithm based on the legal text co-occurrence network is adopted.

9. The legal intelligent question answering method based on a multimodal large model according to claim 1 is characterized by: The legal term constraint dictionary in step 4 includes a dynamic update mechanism.

10. The legal intelligent question answering method based on a multimodal large model according to claim 1 is characterized by: The feedback-based continuous learning process in step five includes a privacy protection module.

Images